A semiconductor device includes a semiconductor substrate and a fin positioned above the semiconductor substrate, wherein the fin includes a semiconductor material. Additionally, a ferroelectric high-k spacer covers sidewall surfaces of the fin and a non-ferroelectric high-k material layer covers the ferroelectric high-k spacer and the fin, wherein a portion of the non-ferroelectric high-k material layer is positioned on and in direct contact with the semiconductor material at the upper surface of the fin.