Glossary
AI Orchestration Glossary
Plain-language definitions for the terms that come up when you build and operate production AI systems.
AI Orchestration
AI orchestration is the coordination and management of AI models, systems and integrations across a workflow or application.
AI orchestration is the discipline of coordinating multiple AI models, tools, data sources and human approvals so they behave as a single reliable system. An orchestration layer handles scheduling, context-sharing, retries, cost controls and audit — the operational concerns that turn one-off prompts into a production service.
Source: IBM — What is AI orchestration?
Related: agentic aimulti agent systemllm
Agentic AI
Agentic AI is software that pursues a goal autonomously by planning, calling tools, observing outcomes and replanning until the goal is met.
Unlike a chatbot, an agentic system does not just respond — it acts. It decomposes a goal into steps, picks tools, executes them, observes results, and either continues or hands off to a human. Production agentic systems require an orchestration layer to manage cost, safety and reliability.
Multi-Agent System
A multi-agent system is an AI application in which several specialized agents collaborate, hand off tasks and share context to reach a shared goal.
Multi-agent systems trade single-model breadth for specialized depth: a planner agent decomposes work, retrieval agents gather context, executor agents call tools, and reviewer agents check outputs. The hardest engineering problem is not model choice — it is shared context and deterministic handoff.
LLM (Large Language Model)
A large language model is a neural network trained on broad text data to predict and generate natural language, code and structured outputs.
LLMs are the inference primitive most modern AI systems are built on. They are useful but stateless, unreliable and expensive — orchestration, retrieval and tool use are what make them production-grade.
Related: ragtool usefunction calling
RAG (Retrieval-Augmented Generation)
RAG is a pattern that grounds an LLM's answer in retrieved documents fetched at query time, instead of relying solely on the model's training data.
RAG reduces hallucinations and lets a model answer over private or freshly updated data. A typical RAG pipeline embeds documents into a vector store, retrieves the top-k chunks for a query, and passes them to the LLM as context. RAG is one of the most reliable wins in production AI.
Related: llmvector databaseembeddings
Embeddings
Embeddings are numerical vector representations of text, images or audio that place semantically similar items close together in vector space.
Embeddings power semantic search, retrieval, clustering and recommendation. They are the substrate underneath RAG, deduplication and any "find similar" feature.
Related: ragvector database
Vector Database
A vector database is a store optimized for nearest-neighbor search over high-dimensional embeddings, enabling fast semantic retrieval at scale.
Vector databases are the retrieval engine behind RAG and semantic search. Production deployments care about recall, latency at high QPS, filterability by metadata, and operational maturity.
Related: embeddingsrag
Tool Use
Tool use is the capability of an LLM to call external functions — APIs, databases, code interpreters — to take actions or fetch fresh data.
Tool use turns an LLM from a text generator into an actor. Modern providers expose structured function-calling APIs; orchestration platforms add scheduling, retries, scopes and audit on top.
Related: function callingagentic ai
Function Calling
Function calling is a structured API pattern where an LLM returns a typed function name and arguments for the host application to execute.
Function calling is the contract between an LLM and the rest of the system. Done well, it is auditable, deterministic and safe. Done badly, it is the source of most agentic failures.
Related: tool useagentic ai
MCP (Model Context Protocol)
MCP is an open protocol that standardizes how LLM applications connect to external tools, data sources and prompts.
MCP defines a uniform way for any LLM client to discover and invoke tools exposed by any server. It reduces N×M integration work to N+M, and is rapidly becoming the default interop layer for agentic systems.
Related: tool usefunction calling
Voice Agent
A voice agent is an AI system that holds a real-time spoken conversation by combining streaming STT, an LLM and streaming TTS over a telephony or WebRTC layer.
Production voice agents are bottlenecked by latency, not intelligence. The total budget for a natural-sounding call is roughly 700ms round-trip across capture, transcription, inference, synthesis and network.
Related: sttttsai call center
STT (Speech-to-Text)
Speech-to-text converts audio into transcribed text, usually streaming partial results so downstream models can start processing before the speaker finishes.
STT quality is measured in Word Error Rate (WER) per language and per dialect. In low-resource languages like Hebrew and Arabic, dialect coverage matters more than headline WER.
Related: voice agenttts
- ProductTadiran: Secure Enterprise Voice AI Platform (On-Premise)
- ProductHebrew AI – Speech to Insight
- PostBeyond the IVR: What It Actually Takes to Build Real-Time Voice AI
- PostBeyond the Chatbot: The Engineering of Real-Time Voice Orchestration
- PostVoice AI in Contact Centers: STT/TTS Architecture for Hebrew and Arabic
TTS (Text-to-Speech)
Text-to-speech synthesizes natural-sounding spoken audio from text, ideally streaming so the first audio chunk arrives within ~150ms.
Modern TTS sounds natural in English; quality in Hebrew, Arabic and other RTL languages still varies significantly between providers. Streaming TTS is essential for natural-feeling phone conversations.
Related: voice agentstt
AI Call Center
An AI call center is a contact-center stack in which AI voice agents handle inbound or outbound calls end-to-end and escalate to humans with structured context.
AI call centers replace decision-tree IVR with natural-language voice agents that can read CRMs, take actions and route intelligently. The architecture-defining choice is whether telephony is self-managed or outsourced.
Related: voice agentivr
IVR (Interactive Voice Response)
IVR is the classic decision-tree phone menu system being replaced by agentic voice layers that understand natural language and take action mid-call.
"Press 1 for sales" was designed for a world where speech recognition was unreliable. Agentic voice layers are now the dominant replacement pattern for new enterprise deployments.
Related: voice agentai call center
Human-in-the-Loop
Human-in-the-loop is a design pattern where an AI system pauses for human approval at defined checkpoints before taking high-stakes actions.
Well-designed human-in-the-loop pauses only the affected branch, not the whole workflow. Done well it preserves throughput while adding safety.
Related: agentic aiai orchestration
Context Graph
A context graph stores messages, files, tool calls and outputs as connected nodes so downstream agents inherit full lineage instead of re-prompting from raw chat.
Replacing chat-log context with a graph is the single biggest reliability win in multi-agent systems. It enables least-context dispatch, episodic memory and deterministic replay.
Related: multi agent systemai orchestration
Prompt Engineering
Prompt engineering is the practice of designing inputs that reliably elicit a desired output from an LLM.
Prompt engineering is a starting tool, not a destination. In production systems most of the work moves from prompts into retrieval, tool design, evals and orchestration.
Evals
Evals are automated tests that score an AI system's outputs against fixed inputs to catch regressions before they reach users.
Evals are to LLM applications what unit tests are to traditional software. Without them, every model upgrade is a gamble.
Related: llmai orchestration
Fine-Tuning
Fine-tuning continues training a pre-trained model on a smaller domain-specific dataset to specialize its behavior.
Fine-tuning is powerful but expensive and brittle. For most production needs, retrieval + careful prompting + tool use beats fine-tuning on cost and maintainability.
GPU / VRAM
VRAM is the on-GPU memory that holds model weights and the key-value cache; it is the binding constraint for LLM inference at scale.
Cost-effective inference comes from matching model size, context length and concurrency to GPU VRAM, and routing batch workloads to cheaper spot capacity.
Related: llmai orchestration