Structures for a non-volatile memory element and methods of forming a structure for a non-volatile memory element. A switching layer is positioned over a first electrode, and a dielectric layer is positioned over the switching layer. The dielectric layer includes an opening extending to the switching layer. A second electrode includes a portion in the opening in the dielectric layer. The portion of the second electrode is in contact with a first portion of the switching layer. The switching layer further includes a second portion positioned between the dielectric layer and the first electrode.