The 95% Breakthrough: Ditching RAG for MAG in Wix's AI-Powered Data Discovery

Wix Engineering
Jul 9
4 min read

Before You Read

If you haven’t read our previous article - "Solving Data Discovery at Scale: How Wix Uses RAG and Multi-Agent Systems to Find the Right Data Fast" on how we used RAG and multi-agent systems in Anna, we highly recommend you check that out first. It explains the background of how we tackled data discovery challenges with semantic search and embeddings. This new article builds on that foundation and shows how we evolved our system to use MAG.

Introduction

Data discovery has always been a critical challenge at Wix, especially as our data landscape expanded to include hundreds of tables and thousands of dimensions across multiple domains. Initially, we tackled this challenge using a Retrieval-Augmented Generation (RAG) approach, enabling users to ask natural language questions and receive structured answers by leveraging semantic search.

However, with recent advancements in Large Language Models (LLMs)—including expanded token limits, enhanced reasoning, and improved handling of complex contexts—we saw an opportunity to evolve our asset selection phase by transitioning from RAG to Model Assisted Generation (MAG).

This shift represents a significant milestone in our journey to build an intuitive, intelligent, and accurate data discovery platform for everyone at Wix.

Recap of the Old System

In our previous system, the heart of the data discovery process lay in Anna, our multi-agent AI assistant. Anna leveraged RAG to bridge the gap between user questions and our vast data model:

The idea was to use RAG and semantic search to cherry pick the right data assets in order to answer the users natural language questions.

However, RAG also came with challenges. Similar-sounding but business-irrelevant dimensions sometimes surfaced. Variations in metadata quality and domain-specific language occasionally led to misleading results.

Moreover, embeddings alone couldn’t capture the nuanced relationships between tables and dimensions, leaving gaps in reasoning.

What is MAG?

“MAG” stands for "Model Assisted Generation”. It is not a standardized term in the machine learning literature - it’s more of an engineering pattern. It refers to using LLM’s retrieval capabilities in complex systems.As technology evolves, large language models (LLMs) have become increasingly adept at passing what’s known as the "Needle in a Haystack" test.

In the context of artificial intelligence, it’s about assessing how well an LLM can retrieve specific information from a large amount of text within its context window.It challenges the model’s ability to navigate a vast sea of data and pinpoint the exact piece of information needed, effectively putting its reasoning and retrieval skills to the test.The “Needle in a Haystack” test is extremely similar to the data discovery challenge.

New models with larger context windows and improved reasoning capabilities can now process large data models and successfully cherry pick data assets based on the user input.

We put all this hypothesis to the test.We fed Gemini 2.5 pro with our largest data model , containing thousands of dimensions. Then we started asking the LLM to pick the right dimensions to answer users business questions from our evaluation set. And the results were AMAZINGWe improved dimensions selection from ~83% to ~95% success rate.

The New MAG Process

When a user asks a business question, Anna’s (our data discovery AI assistant) Assets Selection Agent gathers all relevant metadata from the data model, this includes schemas, descriptions, tags… and injects it into a prompt.

This prompt is fed to the LLM along with a system message that instructs the model to analyze the question and select the top k best dimensions or metrics to answer the question.

The model reasons over all this information together, identifying the most relevant assets while explaining its reasoning if necessary.

The rest of the agentic flow stayed untouched. The DPG Agent, responsible to form the question in a structured way, is fed those top results, choosing the best match. it also acts as a critic, making sure the Assets Selection Agent retrieves relevant assets.

This helps avoid hallucinations. The rest of the system stays untouched. The autonomy of each Agent in Anna’s flow allowed us to change the asset selection method from RAG to MAG without having to change the entire agentic flow.

Benefits of the MAG Approach

This new approach offers several key advantages:

Improved Accuracy: By reasoning holistically, the model avoids selecting misleading or loosely relevant dimensions.
Contextual Awareness: The assistant understands the relationships between dimensions, tables, and business context, leading to more relevant answers.
Scalability: MAG reduces reliance on pre-generated embeddings, making the system more adaptable as new data models emerge.
User Confidence: with reasoning and critic in place, the assurance of wrong assets selections reduced significantly

But with a few disadvantages as well:Using large context and advanced LLM models is both expensive and slow.This results in much higher costs as well as users fastrations due to the slower response time.

Challenges and Next Steps

Our initial experiments with MAG are promising: Feedback from users is positive as there are fewer instances of incorrect or incomplete answers. Accuracy in selecting dimensions jumped from 83% (in RAG) to around 95%.

While MAG represents a major leap forward, challenges remain: Token limits still exist, as data models expend, we might get close to those limitsMaintaining consistent results over time (especially across multiple sessions) is still an area we’re actively working on.

To reduce costs and improve speed, we are evaluating if a part of the system has become redundant. The full data model, when fed to the asset selection agent, might enable the selection of all necessary assets simultaneously, thereby making the process of breaking down complex questions into simpler ones redundant.Prompt chasing techniques might improve those kpis as well.

Conclusion

Moving from RAG to MAG marks a pivotal shift in our data discovery platform at Wix. By leveraging LLMs’ improved reasoning capabilities and expanded token limits, we’ve built a system that is more accurate and context-aware.

We’re excited about the potential this unlocks for our users, empowering them to navigate our complex data landscape with greater confidence and ease.

As we continue to refine and evolve this approach, we look forward to sharing more insights and results from our journey.