N NexChron

Inference

Definition The process of running a trained AI model on new input data to produce a prediction or output, as opposed to training. Inference is what happens every time you send a message to a chatbot or submit an image for analysis.

In Depth

Inference latency and throughput are critical production engineering concerns. Optimization techniques include quantization (reducing weight precision), pruning, speculative decoding, batching, and hardware-specific compilation. As models grow larger, inference costs increasingly dominate total AI infrastructure spend, driving demand for specialized inference chips and edge deployment.

Browse more terms

AI Agent AI Alignment Agentic AI Artificial General Intelligence Artificial Superintelligence Attention Mechanism AutoML Benchmark Bias (AI) Chain-of-Thought Computer Vision Deepfake Diffusion Model Edge AI Embedding Federated Learning Few-Shot Learning Fine-Tuning Function Calling GPT