The Retentive network (RetNet) architecture is a foundational architecture for large language models proposed as an alternative to transformers.