DataStax looks to help enterprises escape RAG ‘Hell’ with AI tools update 

Retrieval Augmented Generation (RAG) is key to enterprise usage of generative AI, but it’s not as easy as just simply connecting a Large Language Model (LLM) to a database.

DataStax is looking to help solve the challenge of enabling RAG for enterprise production deployments, with series of technologies announced today. DataStax is perhaps best known for its commercially supported version of the Apache Cassandra database, known as a DataStax Astra DB. In the last year, DataStax has increasingly focussed on enabling gen AI and specifically RAG, adding vector database search support alongside a data API to build gen AI RAG apps. 

Now DataStax is pushing further into enterprise RAG, with the release of Langflow 1.0 for building RAG and AI agent workflows. The company is also out with a new release of Vectorize which provides different vector embedding models. On top of it all is RAGStack 1.0 which combines a series of tools and technologies to help enterprise production deployments.

According to DataStax CPO Ed Anuff, the basics of RAG architecture is deceptively simple, but getting actual enterprise grade efficiency is a challenge many organizations now face.

“A lot of companies right now are in RAG Hell,” Anuff told VentureBeat.

Anuff explained that RAG Hell refers to the challenges companies face when they start importing full, live datasets into a RAG application after an initial proof of concept. Initially the results are good, but then 2 out of 5 times the results become terrible. Anuf emphasized that the goal with DataStax’s product updates is to help enterprises break out of RAG hell and get applications into production.

Building RAG enterprise AI apps with Langflow

Back on April 4, DataStax acquired Langflow, which provides an intuitive user interface and tools that work on top of the open source LangChain technology.

Anuff explained that Langflow makes it easy to build chat-based and other RAG-based applications visually without coding. The big update today is that Langflow 1.0 is now generally available as an open source tool. DataStax has also expanded the library of components that can be wired together visually in Langflow, including better integration with other DataStax products. Additionally, DataStax Langflow is a new managed cloud version for enterprises.

Anuff explained that as part of this release, Langflow’s execution engine is now Turing complete, allowing for more sophisticated logic flows and conditionals to be built.  Part of that completeness comes from the integration of enhanced branching and decision points for AI workflows. Branching points allow an application workflow to split or branch based on certain conditions, like if/else logic. Decision points allow an application to dynamically change the context or data passed to a model based on factors like chat history or a user’s previous actions. According to the Anuff, these types of branching and decision capabilities in Langflow lead to better user experiences in applications like conversational agents.

“You can now create very sophisticated logic flows, things like conditionals and so on,” Anuff said. “The end result is you get not just better relevancy, but you get better interactions.”

Vectors and unstructured data are key to RAG enterprise AI apps

At the heart of RAG are vector embeddings that are managed inside of a vector database. As it turns out, the model chosen to create the vector embedding, which is a numerical representation of data, really matters.

With its Vectorize technology, DataStax is now enabling its users to choose from a range of embedding models to best suit their specific datasets. The supported embedding providers include: Azure OpenAI, Hugging Face, Jina AI, Mistral AI, NVIDIA NeMo, OpenAI, Upstage AI and Voyage AI.

“These different embedding models all have different areas where they’ve been optimized or different trade offs,” Anuff said. “So you can really pick which one is the best for your dataset.”

To further improve RAG accuracy for enterprise deployment, DataStax now has a partnership with unstructured.io. The company’s technology helps to provide structure to unstructured content before it is vectorized. Anuf said that addition can help to provide an even more granular level of accuracy and precision.

RAGStack debuts ColBERT for even better Enterprise RAG relevance

Bringing it all together is the DataStax RAGStack 1.0 release. 

RAGStack 1.0 is an enterprise-oriented framework that bundles various AI ecosystem components alongside DataStax’s proprietary offerings. A key new addition in RAGStack 1.0 is ColBERT (Contextualized BERT Representations for Retrieval), which is a recall algorithm for RAG applications.

Anuff explained that ColBERT allows for deeper context matching and better relevancy.

“It’s not like searching for a needle in a haystack,” Anuff said. “With ColBERT, it’s  searching for a needle in a pile of needle shaped objects and you’re looking for the precise one.”