A method of manufacturing a semiconductor device includes forming on a semiconductor substrate, a plurality of multi-layered structures each including a first semiconductor layer and a second semiconductor layer that is deposited over the first semiconductor layer and has an etching rate smaller than the etching rate of the first semiconductor layer, forming a first trench through the first semiconductor layer and the second semiconductor layer, forming a support body on sidewalls of the first semiconductor layer and the second semiconductor layer in the first trench, forming a second trench that exposes through the second semiconductor layer, etching the first semiconductor layer via the second trench selectively, to form under the second semiconductor layer, a cavity resulting from removal of the first semiconductor layer, forming a buried insulating layer that is buried in the cavity, exposing a side surface of the deposited second semiconductor layer, forming a gate insulating film on the exposed side surface of the second semiconductor layer, forming a gate electrode over the side surface of the second semiconductor layer with intermediary of the gate insulating film there between, implementing first ion-implantation through the top surface of the second semiconductor layer, implementing second ion-implantation through the top surface of the second semiconductor layer.