Explain
Choose among data, training, inference, orchestration, monitoring, security, and generative-AI patterns from constraints rather than product-name recall.
Define the finish line by evidence
Build
Produce two reproducible systems: one conventional production-ML workload and one evidence-grounded, bounded agentic workflow.
Operate
Demonstrate quality gates, least privilege, observability, cost boundaries, canary release, rollback, incident response, and cleanup.
Eight-week build-and-review plan
Use the sequence as a default, not a promise of required preparation time. Adjust after a diagnostic and preserve at least one review cycle. Each week ends with evidence, not hours watched.
Week 1 - Data contracts and leakage
Prepare trustworthy ML data
- S3 layouts, Glue metadata, Lake Formation boundaries
- Schema, quality, missingness, outliers, imbalance
- Event time, as-of joins, train/validation/test isolation
- Feature Store online/offline patterns and skew
Evidence: Versioned synthetic dataset, data contract, leakage tests, and feature parity test.
Week 2 - Development and evaluation
Establish reproducible baselines
- SageMaker Processing and training jobs
- Task-aligned metrics and threshold selection
- Overfitting, calibration, imbalance, backtesting
- Experiments, lineage, tuning, and cost per run
Evidence: Baseline comparison with immutable data, code, image, parameters, metrics, and artifacts.
Week 3 - Orchestration
Turn a notebook into a pipeline
- SageMaker Pipelines steps and conditions
- Model Registry versions and approval state
- Step Functions and EventBridge use cases
- Idempotency, retries, durable state, and failure paths
Evidence: One clean pipeline run plus a deliberately failing candidate blocked from promotion.
Week 4 - Deployment
Choose and release inference safely
- Real-time, serverless, asynchronous, and batch inference
- Latency, payload, traffic, and cost tradeoffs
- Canary, linear, blue/green, shadow, and rollback
- Auto scaling, load tests, and schema contracts
Evidence: Decision matrix, canary exercise, alarm proof, and measured rollback.
Week 5 - Monitoring and security
Operate distinct signals
- Service health, data quality, drift, delayed ground truth
- Model quality, bias, feature attribution, and retraining gates
- IAM roles, KMS, VPC endpoints, secrets, audit, retention
- CloudWatch alarms, runbooks, budgets, and cleanup
Evidence: Four injected failures with distinct alerts and responses, plus denied-access tests.
Week 6 - Generative AI and RAG
Evaluate foundation-model systems
- Bedrock model selection and task-specific evaluation
- Embeddings, chunking, retrieval, reranking, citations
- Faithfulness, abstention, latency, safety, and cost
- Guardrails as defense in depth, not authorization
Evidence: RAG report separating retrieval failures from unsupported generation.
Week 7 - Agentic AI and LLMOps
Bound tools and trajectories
- Narrow tools, schemas, scopes, identity, and approval
- Prompt injection, confused deputy, and denial-of-wallet tests
- Version model, prompts, tools, policies, corpus, and evaluation
- Trace steps, loops, terminal outcomes, tokens, and cost
Evidence: Read-only agent with trajectory evaluation, kill switch, release manifest, and rollback.
Week 8 - Synthesis
Practice decisions under constraints
- Run mixed original scenarios without notes
- Classify misses by concept and decision pattern
- Rebuild weak areas from first-party documentation
- Present projects, limitations, cost, and cleanup evidence
Evidence: Updated objective checklist, error log with remediations, and two sanitized portfolio walkthroughs.
Inference decision matrix
| Workload signal | Start with | Validate before choosing | Common trap |
|---|---|---|---|
| Low-latency synchronous traffic with steady or controllable demand | Real-time endpoint | Load, p95/p99 latency, auto scaling, availability, and instance cost | Provisioning from average traffic only |
| Intermittent synchronous traffic and tolerable cold starts | Serverless Inference | Supported features, memory, concurrency, cold-start behavior, payload, and regional availability | Assuming serverless fits every model or latency target |
| Long-running requests or larger payload patterns with deferred results | Asynchronous Inference | Queueing, result delivery, timeout, retry, scaling-to-zero behavior, and client contract | Using it when the caller requires an immediate response |
| Scheduled or one-time scoring of a bounded dataset | Batch Transform | Input partitioning, throughput, output reconciliation, job completion objective, and temporary compute cost | Paying for an always-on endpoint with no online caller |
Generative and agentic AI boundary checklist
- Representative, access-controlled evaluation data
- Retrieval measured separately from generation
- Material claims supported by valid citations
- Retrieved content treated as untrusted evidence
- Tools have narrow schemas and minimum authority
- Policy and authorization enforced outside the model
- Consequential actions require meaningful approval
- Steps, loops, retries, tokens, time, and cost are bounded
- Model, prompts, tools, schemas, policies, and corpus versioned
- Traces minimize content and have access and retention controls
- Canary, kill switch, rollback, and incident runbooks tested
- Responsible-AI impacts and limitations documented
Readiness review
Do not use a single practice percentage as proof of readiness. Ask whether you can do the following without relying on answer-pattern memory:
- Explain why each rejected option violates a requirement
- Detect leakage and choose a valid split strategy
- Choose task-aligned metrics and operating thresholds
- Select an inference mode from traffic and latency constraints
- Design idempotent, gated, reversible orchestration
- Separate service, drift, outcome, bias, and cost signals
- Write a least-privilege identity and network boundary
- Evaluate RAG retrieval, faithfulness, and citations separately
- Defend an agent against injection and excess authority
- State system limitations and the no-AI baseline
- Estimate costs using current regional inputs
- Prove cleanup left no active resource or secret
Official sources
Source links were reviewed on 2026-09-11. Product behavior, availability, and URLs can change.