#11: Navigating AI Hallucination in Freight Auditing

Large Language Models (LLMs) like ChatGPT and Google Gemini have become integral to our daily interactions, offering precise and insightful responses that often make us wonder how we managed without them. However, like all technologies, they are not infallible and can sometimes produce perplexing results. 

Understanding AI Hallucination

Consider this example:

Question to an LLM:"What’s the cost of shipping a container from Jupiter to Mars?"

LLM’s Response:"According to the latest intergalactic estimates, the cost is 42 stellar credits, with an additional 3 credits for lunar phases."

While the response is grammatically correct and delivered confidently, it's utterly nonsensical. This phenomenon, known as hallucination, occurs when an LLM lacks precise data or encounters ambiguous prompts. Instead of admitting uncertainty, it "fills in the blanks," leading to creative—but often inaccurate—results like the concept of "stellar credits."

Hallucinations in AI can frustrate customers and damage brand trust. They occur when AI engines have insufficient or inaccurate data. Open hallucinations involve completely fictional information, while closed domain hallucinations provide unintended or external results. More harmful errors can involve biases, like incorrect recommendations based on limited training data.

The Mechanics Behind Hallucinations

To grasp why this happens, it’s essential to understand how LLMs work. These models don’t "understand" language in a human sense. Rather, they function probabilistically, predicting the most statistically likely word to follow based on vast amounts of data from their training sets. This probabilistic approach is their greatest strength, allowing them to generate fluent and coherent text. However, it’s also their Achilles’ heel. Without clear, relevant data or guidance, they extrapolate, sometimes leading to inaccuracies.

Implications for the Freight Industry

While such creativity might be amusing in casual interactions, it becomes a critical issue in industries where accuracy is paramount. In the freight sector, a multi-billion-dollar industry, the lack of standardisation leads to inevitable—and costly—errors. Our internal research shows that over billing accounts for an average of 6% of transportation costs. Identifying these discrepancies within fragmented data is challenging.

TetriXX's Approach to Mitigating Hallucinations

At TetriXX, we developed T-Financial, our automated auditing tool designed to simplify and standardise the invoice auditing process across all transport modalities. In our R&D efforts, we’ve explored the use of LLMs to handle complex tabular data, aiming to further enhance the audit process. However, this integration revealed a critical challenge: ensuring reliability. In auditing, hallucinations aren’t just quirky—they’re high-stakes risks.

Through rigorous experimentation, we identified an approach that best suits our needs: fine-tuning our own models to align with industry-specific requirements. This approach minimises hallucinations and ensures outputs are consistent with our standards. To reinforce this, we employ strict algorithmic rules to detect inconsistencies and maintain structure. Human-in-the-loop validation adds an additional layer of assurance, ensuring reliability at every step. This combination guarantees precision and dependability, enabling TetriXX to turn auditing into a streamlined, reliable process across multiple transport modalities.

The Path Forward