Weaviate

Weaviate is a company developing an open-source, low-latency vector database supporting a range of media types (text, images, etc.). The company's flagship product allows users to store and retrieve data objects based on semantic properties by indexing them with vectors. Weaviate can be used stand-alone (using pre-existing vectors) or with a variety of modules to perform the vectorization and extend core capabilities. With Weaviate, users can perform the following:

• Index and search billions of data objects using their own vectors or one of the database's vectorization modules
• Combine search techniques, including vector search, keyword searches, or structured filtering
• Improve search results using LLM models for new search experiences, such as Q&A over datasets

Weaviate applies a class property structure with a vector representing each data object. This allows users to connect data objects (similar to a traditional graph) and search for objects in vector space. Data can be added to Weaviate through the RESTful API end-points and access data using the GraphQL interface. The database's vector indexing mechanism is modular, and the available plugin is the Hierarchical Navigable Small World (HNSW) multilayered graph.

Weaviate offers potential user benefits, including those below:

• Improving the quality of search results by searching user data semantically
• Text and image similarity searches with out-of-the-box machine learning models
• Combining vector and scalar search with low-latency
• Scaling machine learning models to production
• Classifying large datasets in near real-time

Typical Weaviate use cases are semantic search, image search, similarity search, anomaly detection, power recommendation engines, e-commerce search, data classification in ERP systems, automated data harmonization, and cybersecurity threat analysis.

Headquartered in Amsterdam, Netherlands, Weaviate was founded in 2019 by Bob van Luijt (CEO) and Etienne Dilocker (CTO). Van Luijt entered Weaviate into a start-up accelerator program in the Netherlands in 2018. During the program, he built a team around Weaviate to get the software to production and create a business model around the open-source project. This start-up became SeMI Technologies, short for Semantic Machine Insights. At the start of the program was a traditional graph, using the semantic, NLP element as a feature rather than the core architecture.

Weaviate has raised $67.6M in three funding rounds. Investors include Index Ventures, Battery Ventures, New Enterprise Associates, Cortical Ventures, Zetta Venture Partners, ING Ventures, GTM-fund, Scale Asia Ventures, and Alex van Leeuwen.

Weaviate aims to provide software engineers with a machine-learning-first database for their applications. It includes a range of features:

• Fast queries—Weaviate can perform nearest neighbor (NN) searches of millions of objects in less than 100ms.
• Multi-modal data ingestion—Model inference (e.g., Transformers) can access data (text, images, etc.) with Weaviate managing the process of vectorizing data.
• Combining vector and scalar search—Weaviate facilitates combined vector and scalar searches.
• Real-time search—Weaviate lets users search through data even as it is being imported or updated.
• Horizontal scalability—Weaviate scales are based on user needs, e.g., maximum ingestion, largest possible dataset size, and maximum queries per second.
• Reduced costs—Large datasets do not need to be kept entirely in-memory in Weaviate, and available memory can be used to increase the speed of queries. This allows for a speed/cost trade-off to suit different use cases.
• Graph-link connections—Arbitrary connections are made between objects in a graph-like fashion to resemble real-life connections between data points. Those connections can be traversed using GraphQL.

Weaviate integrations allow users to pick from various well-known neural search frameworks. Integrations include those below:

• Auto-GPT—using Weaviate as a memory back end
• Cohere—using Cohere embeddings with Weaviate
• DocArray—using Weaviate as a document store in DocArray
• Haystack—using Weaviate as a document store in Haystack
• Hugging Face—using Hugging Face models with Weaviate
• LangChain—using Weaviate as a memory backend for LangChain
• LlamaIndex—using Weaviate as a memory backend for LlamaIndex
• OpenAI—using Weaviate as a memory backend for ChatGPT and using OpenAI embeddings with Weaviate