The integration of artificial intelligence into routine business analytics has promised a future of instantaneous insights, automated spreadsheet reviews, and frictionless decision-making. However, a recent controlled experiment examining how AI models handle everyday data analysis tasks reveals a sobering reality. Even advanced language models utilizing standard data science libraries like Pandas and SciPy frequently generate subtle, highly deceptive errors that standard code execution checks fail to catch.
Conducted using OpenAI’s fast-pass GPT-5.6 Terra and unhurried runs of GPT-5.6 Luna, the experiment tested an AI model on three small datasets and the routine questions business teams ask on any given week. The inquiry ranged from calculating average delivery times and identifying top-performing regions to counting athletes within a specific file. When researchers introduced a review pass—handing the model its own answers back with instructions to verify everything for an executive-level presentation—the results were alarming. One review pass managed to catch a wrong row count while simultaneously placing a checkmark next to a conclusion that was entirely backwards. Another review pass invented a correction out of thin air, transforming a correct answer into a flawed one.
These findings underscore a critical vulnerability in modern business intelligence. The mechanical execution of data analysis by AI is often exceptionally strong. Models can parse dates, write valid SQL and Pandas code, and occasionally produce sharp diagnostic observations that surpass standard human reporting. Yet, the errors they introduce do not stem from broken code or syntax exceptions. Instead, they arise from deep contextual misinterpretations that evade automated software testing entirely.
The Underlying Data and Structural Complexities
To test the limits of automated analysis, the experiment relied on three distinct datasets frequently used in professional data science evaluations. The first dataset, tracking shipment data, contained forty rows representing forty individual customer orders placed over a three-week period in January 2024. Each entry included dates for when the order was placed, when it shipped, and when it was delivered. A closer examination of this dataset revealed that eighteen out of the forty orders had never shipped and never arrived, remaining entirely in an unresolved, pending state.
The second dataset focused on regional sales figures spanning multiple years across various global markets. Intended to represent a clean grain of one row per region per year, the dataset harbored structural flaws invisible to basic column header checks. Several region-year combinations contained multiple rows, including duplicate entries and conflicting financial figures for the same time period. Furthermore, geographical coverage was uneven, varying wildly from regions with fifteen years of tracked history to others with only a single year of recorded data.
The third dataset cataloged Olympic athletes and events, structured with one row per athlete per event. While containing hundreds of rows, the file featured significant gaps, notably in biographical details like athlete height, where numerous entries were left entirely blank. These structural imperfections across all three files created the precise conditions under which AI models routinely stumble, demonstrating that raw data is rarely as clean as an automated query assumes.
Measuring the Wrong Metric Without Code Failures
The first major error category uncovered in the experiment involved a fundamental mismatch between the business question asked and the metric calculated by the AI model. When working with the shipment tracking data, researchers asked for the average delivery time. The model returned an output stating an average delivery time of 2.6 days, supported by a clean snippet of Pandas code that subtracted the shipped date from the delivered date.
While the code executed cleanly and produced a mathematically valid result, it answered an entirely different operational question. A customer waiting for a package experiences the timeline from the moment of checkout to the moment of delivery, known as order-to-door time. By calculating the ship-to-door timeline, the model measured warehouse fulfillment efficiency rather than customer wait times. When calculated correctly from the order date, the true average delivery time was 6.09 days—more than double the AI-reported figure.
This class of mistake presents a severe risk to corporate decision-making because no software bug exists to flag it. The code runs without throwing exceptions, the syntax is valid Pandas, and the output calculates exactly what the chosen columns dictate. The error lives exclusively in the selection of columns, meaning no automated test or type check will ever signal a warning. Catching such an error requires reading the initial business question and cross-referencing it directly against the column names used in the calculation, a manual verification step that automated workflows routinely bypass.
Fabricated Numbers and Unexecuted Code
Another troubling phenomenon observed during the testing phase involved AI models presenting figures that no underlying code had ever computed. In several instances, the generated text summaries included statistics and metrics completely detached from the execution logs of the session. For instance, a shipment tracking run correctly identified that twenty-two orders had delivery dates while eighteen remained pending, yet the accompanying summary text claimed that only twenty-two of fifty orders had delivery dates, leaving twenty-eight in transit. The numbers fifty and twenty-eight appeared nowhere in the code or the output console.
What makes these fabricated statements particularly hazardous is their internal arithmetic consistency. Because fifty minus twenty-two equals twenty-eight, a human reader verifying the text mentally finds no immediate mathematical contradiction, even though the underlying file only contained forty total rows.
The regional sales analysis suffered an even more pronounced failure of this nature. Asked to identify the best-performing region, the model reported a specific multi-million-dollar sales figure and claimed it represented nearly a third of all regional revenue, complete with an attached currency symbol for a dataset that carried no explicit units. An audit of the session logs revealed that the model had executed zero code whatsoever, simply printing a code snippet and inventing plausible-looking numbers beneath it. Even when models did successfully execute their queries, they frequently contradicted their own findings in the surrounding prose, blending measured statistical outputs with entirely speculative figures in the same paragraph.
Reading Trends From Incomplete Data
Temporal analysis proved to be another frequent stumbling block for the automated analytics pipeline. When asked whether shipping performance was accelerating or decelerating over time, the AI model’s fast pass concluded that deliveries were speeding up, citing a drop in average delivery days from 3.2 days in the first week to just 1.0 day in the third week.
While the individual numbers extracted from the database were mathematically correct, the analytical conclusion was entirely backwards. Because the dataset concluded on January 21, orders placed during the third week had only experienced a fraction of the time to complete compared to orders placed in the first week. The slow orders from the third week simply had not arrived yet, meaning the only shipments captured in that week’s average were the ones that happened to be exceptionally fast.
As a result, the model interpreted a data artifact caused by incomplete observation windows as a genuine operational improvement. Unresolved orders had climbed from twenty percent to seventy percent across the observed weeks, artificially dragging down the average delivery duration. Interestingly, when given the exact same file without strict time pressure, an unhurried model run caught the censoring bias unprompted, issuing a direct warning that the apparent performance boost was an illusion.
The Perils of Silent Data Dropping
Evaluating the Olympic athlete dataset for correlations between physical stature and medal success exposed yet another hidden trap concerning missing values. When asked if height helps an athlete win a medal, the model executed a comparison between medalists and non-medalists, concluding that height barely mattered because medalists were on average a mere fraction of a centimeter taller.
While the arithmetic was technically sound, the underlying Pandas operations had silently dropped two hundred and twenty-six blank height entries without notifying the user. The analysis ran on a mere fraction of the dataset, shrinking the sample size by nearly two thirds. Furthermore, a deeper statistical examination of the missing values revealed that the presence or absence of a recorded height was a powerful predictor of success in its own right. Recorded heights were heavily concentrated in more recent Olympic Games where medal rates were vastly higher, while historical records frequently omitted physical measurements entirely. By quietly discarding the blanks, the model analyzed a heavily skewed subset of the data and presented a misleading conclusion.
The Limits of Self-Correction and Verification
To determine whether artificial intelligence could reliably audit its own analytical output, researchers fed the flawed first-pass answers into clean sessions, instructing the models to rigorously verify every number for an executive presentation. Rather than catching systemic errors, the self-review passes often introduced new complications.
In some instances, the review pass successfully corrected obvious discrepancies, such as repairing mismatched row counts and adjusting fabricated totals. However, these same review sessions routinely rubber-stamped flawed logic, approving delivery time metrics that answered the wrong operational questions and validating delivery trends manufactured by missing data. In other cases, the review pass went further astray by ignoring original analytical defects, inventing entirely new statistical means that no single grouping of the data could produce, and reversing valid initial conclusions into newly minted errors delivered with absolute confidence.
Ultimately, these experiments highlight a fundamental boundary in automated data science. While artificial intelligence excels at the mechanical execution of syntax and the rapid parsing of structured information, the context that breathes meaning into numbers remains entirely outside its automated grasp. Whether navigating unrecorded heights, incomplete time horizons, or unexecuted code blocks, human oversight remains an indispensable safeguard in the modern analytical workflow.