AI Engineering Glossary

A prompting strategy that forces the model to articulate its intermediate reasoning steps before providing a final answer.

A mechanism used during autoregressive generation to store previously computed Keys and Values, preventing redundant calculations.

A highly efficient fine-tuning technique that freezes the base model weights and trains a small set of injected low-rank matrices.

A neural network architecture that utilizes multiple specialized sub-networks ("experts"), routing tokens only to the most relevant ones.

A framework that grounds an LLM by fetching external, up-to-date information from a database before generating a response.