When Generative AI & RAG Are Wrong for eCommerce

Since the advent of generative artificial intelligence (GenAI), the world of eCommerce has quickly embraced the technology in many ways. From leveraging GenAI to drive conversions, reduce returns, or maximize customer Lifetime Value (LTV), GenAI and Retrieval-Augmented Generation (RAG) have moved from experiment to critical component in record time. Many eCommerce teams already use LLMs to power AI shopping assistants, help author product descriptions, or identify patterns in customer feedback. An area that is still largely misunderstood, however, is how Generative AI and RAG can help – or hinder – the world of core analytics for eCommerce practitioners.

A recent survey, summarized in AI in eCommerce Statistics 2025, indicates strong intent to adopt AI to deliver highly personalized experiences and virtual agents, both among the most common initiatives. The report also notes gaps between expectations and actual ROI as projects progress from content to quantitative insights. During the same period, technical work on LLMs has shown consistent and systemic limitations in hallucination, table reasoning, and multi-table joins, particularly when these models are deployed in roles more suited to statistical engines.

In this post, we examine the role of GenAI and RAG, identify areas where they can deliver real value for eCommerce companies, where they struggle with the depth of analysis, and how products such as dotData’s Feature Factory and Insight can complement them. The ultimate goal is not to diminish the value of GenAI, but to identify distinct roles in which each core technology delivers the most outstanding value: GenAI to drive the conversational layer, and automated feature discovery to uncover, quantify, and operationalize the business drivers critical to optimizing business performance.

• GenAI and RAG add value in eCommerce by enhancing conversational customer experiences and generating content.
• LLMs often struggle with structured multi-table analytics and reliable numerical reasoning.
• eCommerce leaders can overextend GenAI into analytical use cases, where it fails to deliver statistically grounded insights.
• It’s important to distinguish between ideal use cases for GenAI vs. specialized analytics engines to sustain long-term ROI.

The Value of RAG and Generative AI in eCommerce

The Customer Experience Agent

GenAI has demonstrated practical value in the eCommerce industry. AI tools such as chatbots and virtual shopping assistants answer sizing questions, allowing customers to visualize how products would look on them, or clarify return and refund policies, improving customer satisfaction and experience. Online shoppers who interact with AI chatbots often convert at higher rates than those who do not. Automated agents can recover a meaningful fraction of otherwise abandoned carts by addressing last-minute objections or resolving confusion.

The Content Creation Accelerator

GenAI has also helped retailers accelerate content creation. LLMs help merchandising teams draft or localize product descriptions, build SEO variations, or content generation aligned with brand guidelines for long-tail catalogs. The shift to GenAI can significantly reduce copy production time and enable faster SKU onboarding, particularly when integrating GenAI with translation and localization processes. Marketing operations teams use GenAI for highly personalized content creation for segmentation campaigns, generating social ad variants and personalized landing pages for A/B testing.

The Unstructured Data Analyst

A third success of GenAI in eCommerce is in the analysis of unstructured data. LLMs are particularly well-suited to parsing immense volumes of text from reviews, social media comments, and support call transcripts to identify and flag recurring themes and shifts in consumer sentiment or confidence. Text analysis using LLM models can identify frequent complaints, such as “inconsistent sizing for brand X,” and emerging interests, which can then feed into product and merchandising decisions. Although the output of LLM-based text analysis is not a statistical model, it nevertheless provides critical directional signals more quickly than manual text review.

The value of GenAI is clear when the primary input and output are language. Models can understand customer inquiries, generate contextually relevant answers, and synthesize scattered bits of information into easily understandable summaries and action items. For merchandising and marketing teams in eCommerce, the combination of LLMs justifies initial investments even before deeper analytics are in place.

• GenAI capabilities are exceptionally well aligned with conversational commerce and customer engagement, where language is the primary interface.
• Content-skewed workflows, such as product descriptions and campaign copy authoring, can benefit from GenAI’s ability to scale writing and localization.
• The ability of LLMs to understand and uncover patterns from unstructured text turns reviews and social data into themes and narrative insights.
• The key takeaway is that the strengths pertain to understanding and generating text, rather than to statistical modeling of structured data.

When GenAI Fails with Deep eCommerce Analytics

The promise of GenAI for analytics seems obvious. At first glance, it’s enticing to believe that an LLM-based system, often paired with Retrieval-Augmented Generation (RAG), could answer sophisticated questions like “What combination of behaviors predicts repeat purchases?” or “Which shoppers are more likely to return this order?” The core challenge, however, is that answering these questions—and numerous others like them—depends on precise analysis and reasoning over structured, multi-table data, an area well outside the original scope of language-based models such as LLMs and RAG.

In fact, research on LLM hallucination emphasizes that these models generate output based on learned language patterns rather than guaranteeing factual or (more importantly) numerical correctness. While RAG is designed to inject grounding information, the underlying process still optimizes for next-token prediction, which explains its propensity to produce confident, unsupported claims even when retrieval guardrails are in place. More recent research argues that the hallucination challenge is likely to persist as a characteristic of LLMs, not a bug that disappears with scale and further development.

This challenge of hallucinations becomes even more apparent when LLMs (and by extension, RAG-backed systems) interact with complex tabular data. Evaluations of LLM performance in table reasoning show that LLM and RAG models mishandle schemas with multiple related tables, leading to errors such as double counting, missing join conditions, and misinterpretation of one-to-many and many-to-many relationships. A technical article by the engineering team at Snowflake explains that “join hallucinations” in LLM-generated SQL can lead to incorrect aggregations and KPI inflation as schemas grow more intricate, and outlines mitigations to constrain model output.

Even when those structural issues can be controlled, GenAI and RAG still lack a systematic approach to feature and signal discovery. As in many industries, analytical work in eCommerce relies on building and testing numerous candidate variables, such as recency and frequency windows, discount patterns, device behaviors, and cross-category browsing – to name a few. Reviews of LLM applications for eCommerce suggest that, while models can assist with text-heavy tasks, they do not outperform specialized methods for structured prediction tasks, such as churn or purchase likelihood. The truth is that bespoke models often remain stronger for niche quantitative tasks. The missing component is an engine that explores the space of all possible features, evaluates them against a clear, well-understood metric, and tracks them over time.

• LLM hallucinations are inherent to how these models generate language, not anomalies, and RAG does not eliminate them.
• Complex eCommerce data schemas with multiple tables and relationships expose weaknesses in LLM-based SQL generation, leading to analytical errors like “join hallucinations.”
• Neither GenAI nor RAG automates the discovery of statistically significant patterns or the systematic exploration of the drivers of critical performance metrics such as churn or return on investment.
• GenAI and RAG cannot be treated as a complete “one size fits all” tool to replace statistically reliable analytics.

The Role for GenAI and RAG in Analytics

Like all experts, when GenAI is focused on a task it handles well, it becomes a critical component of an analytics environment rather than a problematic substitute. GenAI should be viewed as a layer that makes analytics more accessible and actionable, rather than a system designed to produce analytics from scratch.

Conversational interfaces allow line-of-business experts in merchandising, marketing, and ecommerce operations to ask natural-language questions about performance and receive explanations grounded in existing dashboards and analytical models. GenAI can translate complex reports and charts into easy-to-digest English-language explanations and insights, based on curated historical data. This approach positions GenAI to do what it does best: interpret questions, retrieve relevant context, and generate coherent summaries.

GenAI can also be useful to generate hypotheses. Models can suggest potential strategies informed by broader industry patterns, which teams can then test against in-house customer data. For example, a model might suggest exploring customer behavior when they mix full-price and discounted items in the same basket. The idea provided is not a final strategic answer, but it points analytics teams towards patterns worth validating with more rigorous methods.

Finally, GenAI can help translate complex analytical output into easily understandable communication. Once a specialized system identifies a high-impact microsegment, GenAI could be used to create multiple versions of messaging, email campaigns, or internal explanations tailored to different audiences. The analytical backbone remains sound, but GenAI ensures that the insight remains trapped in technical documentation.

• GenAI can add the most value when positioned as the narrative and interaction layer atop trusted analytics processes.
• Conversational access to KPIs and signals can help non-technical users consume insights.
• GenAI can propose hypotheses and messaging variations for analytic teams to test and refine.
• LLMs are most effective at explaining and deploying insights, not at discovering them from raw tables.

What Automated Signal Discovery Brings to Bear

The task of answering “why” questions in eCommerce means leveraging a system that can ingest structurally complex datasets and automatically explore all possible combinations – known as a “feature space.” Systems like dotData Feature Factory operate in that role by turning multi-table relational data into a rich set of candidate signals – aka “features” – that can be evaluated, reused, and deployed.

Feature Factory connects to data warehouses and lakehouses and reads multiple related tables, handling basic yet critical preparation tasks such as type inference, value normalization, and handling missing values. Once table relationships are established, Feature Factory generates numerous potential signals across numeric, categorical, temporal, behavioral, and relational dimensions. Examples include rolling counts and averages across time horizons, ratio features to capture discount intensity, cross-category interaction measures, and session-level behaviors aggregated to customer demographic data.

What we are describing is what, in the field of data science, is traditionally known as “Feature Engineering.” The critical difference between manual feature engineering and Feature Factory lies in the scale, consistency, and repetition of feature engineering. Feature Factory scores each discovered feature against a predefined target, such as “Product returned within 30 days of purchase,” and tracks metrics for each discovered signal, including predictive strength, stability, and redundancy, in an internal analytics repository. This repository enables teams to identify which signals emerge as strong drivers across diverse problems and datasets, thereby preventing duplication of effort across projects. Signal (also known as ‘feature’) definitions can be exported as SQL code or integrated into downstream pipelines, thereby simplifying deployment to production, dashboards, or experimental workflows.

For an eCommerce retailer (eTailer), this means questions like “what differentiates high-LTV customers in beauty” can be answered by allowing Feature Factory to explore literally thousands of candidate signals across tables like orders, browsing history, marketing campaigns, and product attributes to then provide focus for the human effort in interpreting the strongest ones, rather than having to hand-code and test each potential idea individually. The result is a deeper, more accurate delivery of the promise of a data-driven understanding of behavior that can be trusted for financial planning, assortment decisions, retention strategies, and nearly any business decision.

• Systems such as Feature Factory automate large-scale feature generation and evaluation for multi-table eCommerce data.
• An internal analytics repository preserves feature definitions and metrics, improving reuse and governance.
• Analytics teams can move from hand-crafting individual feature tests to reviewing ranked lists of candidates with quantified business impacts.
• The systemic, statistical approach provides a backbone that GenAI alone cannot supply.

From Features to Micro-Segments

Identifying strong feature candidates is only part of the journey. Business impact comes from taking those signals and combining them into narrow, but high-impact, customer segments – micro-segments – of data that imply actions for merchandising, marketing, and CX teams. dotData Insight is the bridge between the raw power of Feature Factory’s analytical discovery and day-to-day data-driven decision making.

dotData Insight can ingest the Feature-Factory-produced catalog of signals and present it through an easy-to-use interface. Instead of raw data tables or complex reports, users see “drivers” that describe specific conditions associated with the movement of a particular KPI. For example, a driver might describe a pattern, such as “customers who purchased product X within the last 60 days,” and present the group’s conversion rate, comparing it with a baseline. The interface can surface both lift (the relative change in KPI) and coverage (the portion of the population the signal impacts), helping subject matter experts judge whether a given driver is interesting but niche or broad enough to count.

dotData Insight also identifies threshold ranges that have the most significant impact on the KPI. The automatic identification of the optimal threshold eliminates the guesswork associated with variable binning and clarifies where behavior changes meaningfully. Importantly, Insight allows users to combine – or “stack” – signals together giving subject matter experts the ability to combine two, three or as many high-impact drives as they wish to evaluate, in seconds, how the intersection of signals impacts the business: What share of customers it represents, how its KPI differs from the average, and what the potential uplift might be if the segment is targeted effectively. 

In an apparel eCommerce example, one micro-segment might provide insight into “first-time customers who buy two sizes of the same item during late night sessions and who arrived from a deep-discount email campaign.” dotData Insight can quantify that this micro-segment, while statistically small, has a substantially higher return rate than average. From here, teams can design preventive measures such as more prominent fit guidance, size recommendation widgets, or tailored post-purchase emails to encourage exchanges rather than refunds.

• dotData Insight transforms technical features into understandable business drivers linked to KPIs, with clear lift and coverage indicators.
• Automatic thresholding helps business users discover where continuous variables begin to change outcomes.
• Driver stacking enables rapid exploration of micro-segments without code, backed by statistical evidence.
• Feature Factory and dotData Insight together create a workflow that transforms raw data into actionable insights.

GenAI Plus dotData in Action

Combining the interpretive strength of GenAI with the statistical rigor of Feature Factory and dotData Insight yields a powerful combination. Consider a retailer seeking to reduce footwear returns without affecting overall conversion.

First, Feature Factory ingests historical order, product, browsing session, and campaign data to generate and score a wide range of candidate features against the “returned in 30 days” target. The product might surface strong signals around multi-sized orders, discounts, prior category exploration, time of day, and device type during browsing sessions.

Next, dotData Insight interprets those signals and, with the help of GenerativeAI, translates them into human-readable drivers, allowing analysts and even line-of-business users to stack them to identify the highest-risk microsegments. Analysts might find that first-time footwear buyers, those who shop late at night on mobile devices, those who purchase at high discounts, and those who order multiple sizes of the same SKU have higher return rates than the average. With the backing of statistically sound micro-segments, GenAI can then be leveraged to help the organization target impacted clients:

• A tailored on-site message might appear when such shoppers add items to the cart, emphasizing sizing guides or a “try it at home” program.
• Follow-up emails could be drafted for the target audience to encourage exchanges and educate them on sizing, aligned with brand and tone guidelines.
• Internal executive summaries can be prepared to explain the segment, the drivers behind its behavior, and the rationale for proposed interventions.

As new data arrive, Feature Factory and Insight can rerun discovery so that segments and drivers adapt to seasonality, assortment changes, and promotional calendars. GenAI continues to operate as the communication and experience layer on top, reusing the updated segment definitions to keep messaging aligned with current behavior.

• A hybrid approach allows Feature Factory and Insight to handle discovery and segmentation, while GenAI focuses on messaging and explanation.
• Micro-segments defined in Insight give GenAI a precise context to create tailored customer experiences rather than vague personas.
• The loop from data to drivers to micro-segments to communication can be refreshed frequently as behavior changes.
• This division of labor respects the architectural strengths and limits of each component in the stack.

What eCommerce Leaders Should Do Next?

For leaders overseeing digital revenue, analytics, or CX, the practical option is to match tools to their strengths rather than expecting a single system to do everything. Short-term gains from GenAI are real, but long-term competitiveness will depend on whether organizations can also industrialize discovery and segmentation.

One priority is to formalize the analytical backbone for KPI-critical questions. This means defining clear targets such as conversion, repeat purchase, returns, churn, margin, and investing in automated feature discovery and driver analysis for each, rather than relying on ad hoc, manual feature engineering. Platforms such as dotData Feature Factory and Insight help compress this cycle from months to days or hours, while preserving transparency about what drives outcomes and how those drivers were identified.​

At the same time, organizations can refine their GenAI deployments. Instead of asking LLMs to infer patterns directly from raw tables, teams can route GenAI to the tasks of explanation, hypothesis framing, and multi-channel messaging for segments that have already been validated quantitatively. When that separation is in place, the risk of hallucinated “insights” decreases, and stakeholders gain greater confidence that key decisions rest on sound statistical foundations.

Key Takeaways

• Treat GenAI as one layer in the stack, not the entire analytics solution.
• Use automated feature discovery and driver analysis to underpin critical KPI decisions.
• Reserve GenAI for interpretation, communication, and experiment design, leveraging validated segments and drivers.
• Align governance and measurement so stakeholders can trace campaign and CX decisions back to concrete analytical evidence.

Frequently Asked Questions (FAQs)

Q: Where does GenAI deliver the most value in eCommerce?

A. Generative AI capabilities are ideal for text-based and repetitive tasks. For example, AI-powered chatbots and online shopping assistants for mobile shoppers, accelerating the production of product descriptions and marketing campaign copy, as well as improving operational efficiency and analyzing unstructured data like customer reviews to spot recurring themes and sentiment shifts.

Q: Why do LLMs and RAG struggle with deep eCommerce analytics?

A. RAG and Generative AI eCommerce brands use for analytics struggle with more strategic tasks because they are optimized for generating language patterns, not for spotting precise statistically-derived patterns in structured data. The tendency of LLMs to “hallucinate,” their difficulty handling large multi-table schemas, and their limited ability to discover systematic patterns make them unsuitable for KPI analysis.

Q: How do specialized tools like dotData Feature Factory and Insight complement GenAI?

A. Products like Feature Factory and Insight provide the statistical “backbone” to automate the discovery of statistically strong features from complex multi-table data. In the same light, dotData Insight then transforms the features into understandable, actionable business drivers and micro-segments, demonstrating clear lift against critical KPIs.

Q: What is the recommended strategy for a complete GenAI and Analytics solution?

A. The ideal approach to leveraging AI for analytics is to employ a hybrid model that uses automated feature discovery and driver analysis to identify statistically significant and sound micro-segments, coupled with GenAI custom solutions to accelerate and enhance the narrative and communication strategy, making results easier to digest, framing business hypotheses, and assisting in the creation of multi-channel messaging and a tailored client experience.

Q: What is the most critical takeaway for eCommerce leaders regarding GenAI and analytics?

A. To have a competitive advantage in AI implementation, eCommerce leaders should match tools to precise strengths. Formalize the analytical backbone for KPI-critical questions such as churn or product returns through automated signal-discovery platforms, while using Generative AI solutions for interpretation to automate repetitive tasks and add a deeper communication layer at the top, not as the entire analytics solution.