Provisioning & Scaling

• Choose provisioning by traffic shape, not by data size
• Attach auto scaling to a real-time endpoint instead of pinning instance count
• Use target tracking as the default endpoint scaling policy
• Track SageMakerVariantInvocationsPerInstance for load-proportional scaling
• Use step scaling when you must scale an endpoint out from zero
• A real-time endpoint with auto scaling never drops below its minimum count
• Cooldown periods default to 300 seconds for scale-in and scale-out
• Add provisioned concurrency to keep a serverless endpoint warm for predictable bursts
• Serverless endpoints cap memory at 6 GB and max 200 concurrency
• Use managed spot training to cut training cost up to 90%
• Spot is for training jobs, never for real-time inference endpoints
• Use SageMaker hosting for managed serving; ECS or EKS only when you own orchestration
• Put an endpoint in your VPC with VpcConfig and at least two subnets
• A VPC endpoint reaches S3 with a gateway endpoint, other services with interface endpoints
• Set EnableNetworkIsolation to cut all outbound calls from the container
• Both CloudFormation and CDK provision infrastructure as code with rollback on error
• Share values across stacks with CloudFormation exports and Fn::ImportValue
• Use scheduled scaling for demand that follows the clock
• Let Karpenter right-size GPU nodes from nvidia.com/gpu pod requests
• Enable SMDDP distributed training via the estimator's distribution parameter
• Set compute, VPC, volume encryption, and timeout on a boto3 create_training_job call
• Reference SageMaker resource names with Fn::GetAtt so CloudFormation orders creation
• Wire SageMaker endpoint auto scaling in CloudFormation with ScalableTarget + ScalingPolicy

Unlock with Premium — includes all practice exams and the complete study guide.