Why Generative AI Apps Quality Often Sucks and What to Do About It

How to get from PoCs to tested high-quality applications in production

The generative AI hype has rolled through the business world in the past two years. This technology can make business process executions more efficient, reduce wait time, and reduce process defects. Some interfaces like ChatGPT make interacting with an LLM easy and accessible. Anyone with experience using a chat application can effortlessly type a query, and ChatGPT will always generate a response. Yet the quality and suitability for the intended use of your generated content may vary. This is especially true for enterprises that want to use generative AI technology in their business operations.

I have spoken to countless managers and entrepreneurs who failed in their endeavors because they could not get high-quality generative AI applications to production and get reusable results from a non-deterministic model. On the other hand, I have also built more than three dozen AI applications and have realized one common misconception when people think about quality for generative AI applications: They think it is all about how powerful your underlying model is. But this is only 30% of the full story.

But there are dozens of techniques, patterns, and architectures that help create impactful LLM-based applications of the quality that businesses desire. Different foundation models, fine-tuned models, architectures with retrieval augmented generation (RAG) and advanced processing pipelines are just the tip of the iceberg.

This article shows how we can qualitatively and quantitatively evaluate generative AI applications in the context of concrete business processes. We will not stop at generic benchmarks but introduce approaches to evaluating applications with generative AI. After a quick analysis of generative AI applications and their business processes, we will look into the following questions:

• In what context do we need to evaluate generative AI applications to assess their end-to-end quality and utility in enterprise applications?
• When in the development life cycle of applications with generative AI, do we use different approaches for evaluation, and what are the objectives?
• How do we use different metrics in isolation and production to select, monitor and improve the quality of generative AI applications?

This overview will give us an end-to-end evaluation framework for generative AI applications in enterprise scenarios that I call the PEEL (performance evaluation for enterprise LLM applications). Based on the conceptual framework created in this article, we will introduce an implementation concept as an addition to the entAIngine Test Bed module as part of the entAIngine platform.

1. Background: Business Processes and Generative AI

An organization lives by its business processes. Everything in a company can be a business process, such as customer support, software development, and operations processes. Generative AI can improve our business processes by making them faster and more efficient, reducing wait time and improving the outcome quality of our processes. Yet, we can further divide each process activity that uses generative AI even more.

Processes for generative AI applications. © 2025, Marcel Müller

The illustration shows the start of a simple business that a telecommunications companys customer support agent must go through. Every time a new customer support request comes in, the customer support agent has to give it a priority-level. When the work items on their list come to the point that the request has priority, the customer support agents must find the correct answer and write an answer email. Afterward, they need to send the email to the customers and wait for a reply, and they iterate until the request is solved.

We can use a generative AI workflow to make the 'find and write answer' activity more efficient. Yet, this activity is often not a single call to ChatGPT or another LLM but a collection of different tasks. In our example, the telco company has built a pipeline using the entAIngine process platform that consists of the following steps.

• Extract the question and generate a query to the vector database. The example company has a vector database as knowledge for retrieval augmented generation (RAG). We need to extract the essence of the customers question from their request email to have the best query and find the sections in the knowledge base that are semantically as close as possible to the question.
• Find context in the knowledge base. The semantic search activity is the next step in our process. Retrieval-reranking structures are often used to get the top k context chunks relevant to the query and sort them with an LLM. This step aims to retrieve the correct context information to generate the best answer possible.
• Use context to generate an answer. This step orchestrates a large language model using a prompt and the selected context as input to the prompt.
• Write an answer email. The final step transforms the pre-formulated answer into a formal email with the correct intro and ending to the message in the companys desired tone and complexity.

The execution of processes like this is called the orchestration of an advanced LLM workflow. There are dozens of other orchestration architectures in enterprise contexts. Using a chat interface that uses the current prompt and the chat history is also a simple type of orchestration. Yet, for reproducible enterprise workflows with sensitive company data, using a simple chat orchestration is not enough in many cases, and advanced workflows like those shown above are needed.

Thus, when we evaluate complex processes for generative AI orchestrations in enterprise scenarios, looking purely at the capabilities of a foundational (or fine-tuned) model is, in many cases, just the start. The following section will dive deeper into what context and orchestration we need to evaluate generative AI applications.

2. Concept

The following sections introduce the core concepts for our approach.

My team has built the entAIngine platform that is, in that sense, quite unique in that it enables low-code generation of applications with generative AI tasks that are not necessarily a chatbot application. We have also implemented the following approach on entAIngine. If you want to try it out, message me. Or, if you want to build your own testbed functionality, feel free to get inspiration from the concept below.

2.1. Context and Orchestration of Performance Evaluation for Generative AI Applications

When evaluating the performance of generative AI applications in their orchestrations, we have the following choices: We can evaluate a foundational model in isolation, a fine-tuned model or either of those options as part of a larger orchestration, including several calls to different models and RAG. This has the following implications.

Context and orchestration for LLM-based applications. © Marcel Müller, 2025

The MMLU (Massive Multitask Language Understanding) benchmark measures an LLMs natural language understanding across 57 tasks covering various subjects, from STEM to humanities. It includes 15,908 questions from elementary to advanced levels. MMLU is ideal for comprehensive knowledge assessment. Its broad coverage helps identify deficiencies, but limited construction details and errors may affect reliability.