Semiconductor devices and methods of forming the same include forming a restraint structure over a channel fin, having an opening that is smaller than a top surface of the channel fin. A top semiconductor structure is grown from the top surface of the channel fin, with lateral growth of the semiconductor structure being limited by the restraint structure.