Gemma (Google)

Gemma is a family of lightweight, open models from Google built using the same research and technology as the Gemini models. Gemma was developed by Google DeepMind and other teams across the company. The first two Gemma models (Gemma 2B and Gemma 7B) were released on February 21, 2024. Google has stated that Gemma 2B and Gemma 7B offer "best-in-class performance" compared to open models of the same size. Accompanying the model weights, Google also released tools to support developers using Gemma models including a responsible use guide. Users can fine-tune Gemma models on their own data to adapt to specific application needs, such as summarization or retrieval-augmented generation (RAG). Google plans to continue expanding the Gemma family of models, introducing new variants for different applications.

Based on the transformer decoder architecture, Gemma uses similar architectures, data, and training recipes as the Gemini model family. Gemma 2B and Gemma 7B were released with pre-trained and instruction-tuned variants. Google provided a Responsible Generative AI Toolkit, to provide users with guidance and tools for building safe applications using Gemma. The tool kit includes:

• Safety classification: We provide a novel methodology for building robust safety classifiers with minimal examples.
• Debugging: A model debugging tool helps you investigate Gemma's behavior and address potential issues.
• Guidance: You can access best practices for model builders based on Google’s experience in developing and deploying large language models.

Google also provides toolchains for inference and supervised fine-tuning (SFT) across major frameworks including JAX, PyTorch, and TensorFlow through native Keras 3.0. The pre-trained and instruction-tuned Gemma models are designed to run locally on the user's laptop or desktop or through Google Cloud with deployment on Vertex AI and Google Kubernetes Engine (GKE).

Gemma is designed to follow Google's AI principles with automated techniques used to filter out certain personal information from training sets. Google also used fine-tuning and reinforcement learning from human feedback (RLHF) to align the instruction-tuned models with responsible behaviors. Evaluations, including manual red-teaming, automated adversarial testing, and assessments of model capabilities for dangerous activities were conducted prior to the release of Gemma.

Gemma is a family of lightweight, open models from Google built using the same research and technology as the Gemini models. Gemma was developed by Google DeepMind and other teams across the company. The first two Gemma models (Gemma 2B and Gemma 7B) were released on February 21, 2024. Accompanying the model weights, Google also released tools to support developers using Gemma models including a responsible use guide.