Domain 2 of 5 · Chapter 3 of 4

Deploy Models for Production

Unlock the complete study guide + 1,040 practice questions across 16 full exams.

Bundled into the existing Operationalizing Machine Learning and Generative AI Solutions premium course — no separate purchase.

14-day money-back guarantee — no questions asked.

Included in this chapter:

  • The endpoint and deployment model
  • Online hosts and batch endpoints compared
  • Author the deployment: MLflow no-code vs score.py
  • Authentication modes for online endpoints
  • Safe rollout, rollback, and autoscaling
  • Exam patterns: pick the endpoint and the fix

Endpoint options for production model serving

CapabilityManaged online endpointKubernetes online endpointBatch endpoint
Inference modeReal-time, synchronousReal-time, synchronousAsynchronous job
Invoked asHTTPS scoring requestHTTPS scoring requestJob over a file dataset
Deployments per endpointMultiple, with traffic splitMultiple, with traffic splitSingle default deployment
Traffic mirroringSupportedUnsupportedNot applicable
InfrastructureAzure-managed (OS patch, node recovery)You manage the cluster and OSProvisioned per job, scale-to-zero when idle
ScalingAzure Monitor autoscale adds nodesReplicas within a fixed clusterParallel across the compute cluster
Entra token (aad_token) authSupportedNot supported (key or aml_token)Microsoft Entra (user/service principal)

Decision tree

Low-latency real-timeresponse needed?NoYesBatch endpointasync job, file datasetsSelf-manage Kubernetes?edge / on-prem / infra controlNoYesKubernetes online endpointself-managed, no mirroringReleasing a newmodel version?YesNoBlue/green rolloutmanaged endpoint0%, 10%, 100%; rollbackManaged online endpointsingle deployment100% trafficAlways: autoscale addresses load, not scoring errors.Diagnose failures with get-logs; recover by rolling back traffic.
Cheat sheet

Cheat sheet

  • Managed online endpoints serve low-latency real-time inference
  • Endpoint and deployment are separate objects
  • Autoscale a managed online deployment through Azure Monitor autoscale rules
  • Managed online endpoints support key, AML-token, and Entra-token auth modes
  • Custom-model deployments need a score.py plus environment; MLflow deploys no-code
  • Kubernetes online endpoints give infrastructure control but lose traffic mirroring and out-of-box Azure Monitor monitoring
  • Batch endpoints run asynchronous large-volume scoring
  • Batch deployment settings control scoring parallelism
  • Invoke to test, get-logs to troubleshoot a deployment
  • Local endpoints validate a deployment before the cloud
  • Blue/green rollout shifts traffic percentage gradually
  • Rollback is an instant traffic shift to the prior deployment
  • Mirror traffic shadow-tests a candidate deployment

Unlock with Premium — includes all practice exams and the complete study guide.

References

  1. Online endpoints for real-time inference - Azure Machine Learning
  2. What are batch endpoints? - Azure Machine Learning
  3. Autoscale online endpoints - Azure Machine Learning
  4. Deploy MLflow models to real-time endpoints - Azure Machine Learning
  5. Deploy Machine Learning Models to Online Endpoints - Azure Machine Learning
  6. CLI (v2) batch deployment YAML schema - Azure Machine Learning
  7. Authenticate Clients for Online Endpoints - Azure Machine Learning
  8. Safe rollout for online endpoints - Azure Machine Learning