PyTorch

PyTorch is an open-source machine learning framework designed to accelerate the path from research prototyping to production deployment. PyTorch was created to provide flexibility and speed during the development and implementation of deep-learning neural networks. Examples of deep learning software built on top of PyTorch include Tesla's Autopilot, Uber’s Pyro, HuggingFace’s Transformers, PyTorch Lightning, and Catalyst.

PyTorch began development at Facebook (now Meta) in 2016. In September 2022, PyTorch moved to the Linux Foundation as a top-level project under the name PyTorch Foundation. Members and the governing board of the PyTorch Foundation include Meta, Amazon Web Services (AWS), Google Cloud, AMD, Microsoft Azure, and NVIDIA.

What is PyTorch?

PyTorch is based on the Python programming language and Torch, an open-source machine learning library, written in the Lua scripting language, used for creating deep neural networks. PyTorch is pythonic in nature—it follows a coding style that uses Python's unique features to write readable code. It enables developers to run and test a portion of code in real time instead of waiting for the entire program to be written. PyTorch supports over 200 different mathematical operations. The framework simplifies the creation of artificial neural network models and is mainly used by data scientists for research and artificial intelligence (AI) applications. PyTorch is released under a modified BSD license.

PyTorch is an optimized tensor library for deep learning that uses GPUs and CPUs to greatly accelerate computation speed. It is a Python-based package that provides two high-level features: tensor computation (like NumPy) with strong GPU acceleration and deep neural networks built on a tape-based autograd system. PyTorch provides a wide variety of tensor routines to accelerate and fit scientific computation needs, such as slicing, indexing, mathematical operations, linear algebra, and reductions.

PyTorch Structure

PyTorch was developed by Facebook’s AI Research lab (FAIR), which is now Meta. PyTorch development began in 2016 as an internship project by Adam Paszke while working under one of Torch's core developers, Soumith Chintala. PyTorch's original authors were its founder Adam Paszke and Soumith Chintala, as well as Sam Gross and Gregory Chanan.

The initial group of Meta AI researchers aimed to create a single, standardized interface for their end-to-end workflows while fixing the time-consuming research-to-production pipeline of the AI field. They experimented with machine learning frameworks such as Theano and Torch as well as advanced concepts from Lua Torch, Chainer, and HIPS Autograd. The team released the PyTorch beta to the public in January 2017.

The framework became popular among AI researchers, and Facebook announced plans for a new version, PyTorch 1.0, on Day 2 of F8 (Facebook’s annual developer’s conference) in May 2018. PyTorch 1.0 was released at the NeurIPS conference on December 7, 2018. The new version of the framework allowed developers to experiment rapidly and transition to graph-based modes for deployment.

On September 12, 2022, PyTorch moved to the Linux Foundation as a top-level project under the name PyTorch Foundation with a governing board of leaders, including AMD, AWS, Google Cloud, Meta, Microsoft Azure, and NVIDIA. The creation of the PyTorch Foundation aims to ensure business decisions are made in a transparent and open manner by a diverse group of members as well as improving the project's technical governance. The Linux Foundation was chosen due to its experience hosting large multi-stakeholder open-source projects. The PyTorch Foundation acts as a steward for the technology and supports PyTorch through conferences, training courses, and other initiatives. Its mission is to drive the adoption of AI tooling through an ecosystem of open-source, vendor-neutral projects with PyTorch. The foundation also focuses on the business and product marketing of PyTorch. The transition will not entail any changes to PyTorch’s code and core project, including its separate technical governance structure.

At the time of the move to the Linux Foundation, Pytorch had over 2,400 contributors and had been used as the basis for nearly 154,000 projects, becoming one of the primary platforms for AI research. Over 80 percent of researchers submitting work at major ML conferences, such as NeurIPS or ICML, utilize Pytorch. While Meta is the largest contributor to Pytorch, many companies have made foundational investments, including AMD, Amazon Web Services (AWS), Google Cloud, HuggingFace, Lightning AI, Microsoft Azure, Nvidia, and others.