Optimize generative AI systems and model performance

When a generative AI system underperforms, this domain hands you exactly two levers, and reaching for the wrong one is the classic exam trap. At 10 to 15 percent it is one of the exam's two lightest domains, tied with the quality-and-observability domain at the same 10 to 15 percent, but it is where the measurement those domains produce finally turns into a fix. Retrieval-augmented generation (RAG) controls what the model can see: it grounds each answer on current, source-of-truth documents fetched at query time, so it is the lever for knowledge that is missing, wrong, or changing. Fine-tuning controls how the model behaves: it adjusts the model's weights to teach a consistent style, format, or task and to shorten prompts, but it does not reliably inject fresh or frequently changing facts.Microsoft Foundry fine-tuning considerations The rule is therefore diagnostic, not a matter of taste: a symptom of missing or out-of-date information is a retrieval problem, while a symptom where the facts are right but the tone, structure, or task behavior is wrong is a fine-tuning problem. The two compose rather than compete: fine-tune the behavior, retrieve the facts.

Read this page as a map, then follow the two subtopics in order; each owns one lever and carries the exact numbers, schemas, and evaluation orders this page only points to. Optimize RAG Performance and Accuracy owns the retrieval lever: splitting documents into chunks before an embedding model turns each chunk into a vector, choosing the embedding model and its output dimensions (a choice you make by selection, because Azure OpenAI embedding models cannot themselves be fine-tuned), the vector-search strategies (fast approximate Hierarchical Navigable Small World (HNSW) versus exact but non-scaling exhaustive k-nearest-neighbors (KNN)), the k parameter and the preview per-query similarity threshold, and hybrid search that runs a vector query and a BM25 keyword query in parallel, merges them with Reciprocal Rank Fusion (RRF), and reranks the top results with the semantic ranker.Vector Relevance and RankingHybrid Search OverviewSemantic Ranking Overview Advanced Fine-Tuning and Model Customization owns the model lever: the three methods Azure OpenAI in Foundry Models offers, supervised fine-tuning (SFT), direct preference optimization (DPO), and reinforcement fine-tuning (RFT); each method's JSON Lines (JSONL) training schema; manufacturing synthetic or distilled training data when you have no production traffic; the handful of hyperparameters you tune; and matching the deployment tier to the stage so a hosted model does not bill you while idle.Customize a model with fine-tuning

Both levers reward the same instinct: do not tune by intuition, tune by measurement, and change one thing at a time. On the retrieval side that means altering a single variable (chunk size, k, embedding model or dimensions, hybrid on or off, the semantic ranker on or off) and scoring every variant on one shared evaluation dataset, so the comparison is valid. On the fine-tuning side it means watching training loss during the run but judging success on a separate held-out validation set, then confirming the customized model actually beats the base model before you promote it. Two corollaries the exam leans on. First, reach for the cheaper lever before the expensive one: prompt engineering or better retrieval before you fine-tune, and better chunking or retrieval tuning before a larger embedding model. Second, stop paying for capacity you are not using: a fine-tuned model is stored for free, but a Standard deployment bills per hour whenever it exists, even while idle, so you delete it to stop the meter, whereas a Developer-tier deployment carries no hourly fee and auto-deletes within 24 hours.