ML Container Creator

A CLI tool that creates SageMaker-compatible Docker containers for deploying ML models using the Bring Your Own Container (BYOC) paradigm.

Note: This is a pre-release (0.x). APIs may change between minor versions. Weekly releases are planned until v1.

Supported Configurations

| Architecture | Model Servers | Use Case | |---|---|---| | HTTP (traditional ML) | Flask, FastAPI | sklearn, XGBoost, TensorFlow | | Transformers (LLMs) | vLLM, SGLang, TensorRT-LLM, DJL/LMI | HuggingFace models, JumpStart, S3 | | Triton | FIL, ONNX, Python, TensorRT-LLM, vLLM | Multi-framework serving | | Diffusors | vLLM | Image generation models |

| Deployment Target | Description | |---|---| | Real-Time Inference | SageMaker real-time endpoints | | Async Inference | SageMaker async endpoints with S3 output | | Batch Transform | SageMaker batch processing | | HyperPod EKS | Kubernetes-based deployment |

Quick Start

Install from npm

npm install -g @aws/ml-container-creator

Or use without installing (npx)

npx @aws/ml-container-creator --help

Or install from source

git clone https://github.com/awslabs/ml-container-creator.git
cd ml-container-creator
npm install && npm link

Bootstrap AWS infrastructure (one-time)

ml-container-creator bootstrap

Sets up an IAM execution role, ECR repository, optional S3 buckets, and optional CI Integration Harness for automated testing. Configuration is saved to ~/.ml-container-creator/config.json.

Generate a project

# Interactive
ml-container-creator

# Non-interactive
ml-container-creator my-model \
  --deployment-config=transformers-vllm \
  --model-name=openai/gpt-oss-20b \
  --instance-type=ml.g6.12xlarge \
  --region=us-east-1 \
  --skip-prompts

Build, push, deploy

./do/build        # Build Docker image
./do/push         # Push to Amazon ECR
./do/deploy       # Deploy to SageMaker
./do/test         # Test the endpoint

Documentation

Full documentation is available at awslabs.github.io/ml-container-creator.

• Getting Started — Installation and walkthroughs
• Configuration — CLI flags, env vars, config files, MCP servers
• Deployment Guide — All deployment targets and lifecycle scripts
• CI Integration — Automated lifecycle testing for all deployment configurations
• Examples — Framework-specific walkthroughs
• Troubleshooting — Common issues and solutions

Prerequisites

| Tool | Version | Purpose | |---|---|---| | Node.js | 24+ | Runs the CLI | | Docker | 20+ | Container builds | | AWS CLI | 2+ | AWS resource management |

Contributing

See CONTRIBUTING.md for guidelines.

Security

See CONTRIBUTING.md for reporting security issues.

License

Apache-2.0. See LICENSE.