To Build a GenAI-Powered Business App, You Need More Than an LLM

Layers of intelligence that bring together internal and external capabilities. Layers of intelligence include a staggering array of capabilities, including general-purpose, specialized, and embedded genAI models and tools from suppliers as well as your own knowledge models. For example, consider the layers that capture your knowledge for a customer service chatbot or give your sales teams tools to incorporate your products, value proposition, and competitor battle cards into their pitch presentations. You should also incorporate capabilities from your ML and AI models, software, and more.

Input and output control gates to protect people, the firm, and the models themselves. Input gates turn away bad requests and turn vague requests into answerable prompts. An HR chatbot for employees shouldn't be generating software code, but it should be great at answering detailed benefits questions. Input gates also keep models safe by rejecting requests that could lead them astray. Output gates of many kinds validate outputs for security, regulatory compliance, nonsense checks, bad-content filters, brand compliance, and more — before releasing the response (often with a person as the final decision-maker).

Application pipes to prompt, embed, and orchestrate capabilities to generate responses. You'll use a long-established development technique called pipes to use APIs to tap all the resources from your layers of intelligence, each prompted in turn. The orchestration comes from making sure that the API calls flow seamlessly from end to end, just like steps in a production line on a factory floor. This API-centric, loosely coupled architecture does the heavy lifting. It also provides the platform to implement your dashboards and capabilities to monitor, manage, and operate the application.

Testing and learning loops to test apps, monitor results, and make adjustments. Feedback loops are the fourth part in the architecture. They are vital to establishing validity, trust, and confidence in the application. GenAI-powered apps are pulsing, vibrating organisms. Just like a power plant or Jurassic Park, they need constant care. You'll not only test the heck out of a genAI-powered application before turning it on, but you'll also watch it and retest it continuously to make sure it doesn't get out of line. You'll monitor for quality, performance, costs, and drift.