A rotating structure for armrests includes a rotary plate disposed onto a support mounted to a holder extending upwardly on each of two sides of a chair. The support includes a receiving chamber formed on the top surface thereof and having a plurality of toothed recesses arranged around the external rim thereof. The rotary plate includes a pivotal member extending therefrom corresponding to the support and having a cutout fixed on one side thereof for receiving a slidable retaining tab which is biased by a spring. The retaining tab includes a number of toothed protrusions attached thereon and engaging with the toothed recesses. The bottom of a holding panel for placing a user's elbow may be assembled with the rotary plate, thereby obtaining a quick and secure positioning purpose and preventing making noise while in operation.