Hold piece is a feature of most newer Tetris games starting with The New Tetris.
The top left of the tetrion has a window called the hold space that displays a tetromino.
At any time starting when a tetromino enters the playfield until it locks, the player can press the Hold button on the controller to move the active tetromino into the hold space and move the tetromino that was in the hold space to the top of the playfield.
A tetromino moved into the hold space is unavailable for switching out until the tetromino that was moved out of the hold space locks.
What should I do
with this O tetromino?
Press the Hold button
O becomes available
after Z locks
Hold piece was introduced as part of the development of Square Tetris mode (introduced in The New Tetris; also appears in Tetris Worlds), making it easier for the player to assemble squares.
Its inclusion in standard Tetris is controversial, as some critics have claimed that it makes the game too easy.  
Most games rotate tetrominoes back to the starting orientation when moving them to the hold space. Tetris Worlds, on the other hand, does not (at least on GBA).
In Fliptop Tetris by Radica, the piece preview doubles as the hold space.
Initial hold from TGM3.
Arika's first TGM series games did not include hold piece.
Tetris The Grand Master 3 Terror-Instinct, on the other hand, was released during The Tetris Company's campaign to apply the Tetris Guideline to new games.
Both modes of TGM3 (Classic and World) include a feature called "initial hold system" or IHS, which allows the player to press and hold the Hold button during entry delay, and the tetromino will be swapped as soon as it enters the matrix, just as with IRS.
Lockjaw interprets a slightly different form of initial hold. At any time when no tetromino is falling (such as entry delay or line clear delay), the Hold button switches the hold piece with the next piece to come out.