A backrest adjustment device includes a bottom board, a support board movable relative to the bottom board, and two opposite linkages pivotally mounted between the support board and the bottom board to drive the support board to move relative to the bottom board to change the distance between the support board and the bottom board. Thus, the two linkages are pivoted to drive the support board to move relative to the bottom board to change the distance between the support board and the bottom board so as to adjust the position of the backrest relative to the seat.