A polishing apparatus is suitable for use in polishing a periphery of a substrate, such as a semiconductor wafer. The polishing apparatus includes a holding section configured to hold a workpiece and a polishing head configured to bring a polishing tape into contact with the workpiece. The polishing apparatus also includes a supply reel configured to supply the polishing tape to the polishing head, a rewind reel configured to rewind the polishing tape that has contacted the workpiece, and a swinging mechanism configured to cause the polishing head to perform a swinging motion with its pivot at a predetermined point.