Model Serving with NVIDIA Triton Inference Server
Serve models from multiple frameworks at scale with Triton: model repositories, dynamic batching, ensembles and monitoring.
- ✓ Instructor-led live sessions, online or in Bengaluru
- ✓ Hands-on labs in every module
- ✓ Capstone project with trainer review
- ✓ Curriculum customisable for corporate teams
- ✓ Certificate of completion
Next batch: Wed, 16 Dec 2026 · 24 hours · 3 days
Model Serving with NVIDIA Triton Inference Server
Duration: 24 hours · 3 days
Next batch: Wed, 16 Dec 2026
Format: Online live · Bengaluru classroom
Modules: 7 · Topics: 34
Level: Intermediate to Advanced
Fee: ₹10,999
Trusted by leading experts
Upcoming batches
Open batches for individuals; corporate batches on your dates.
| Programme | Starts | Mode | Schedule | Duration | |
|---|---|---|---|---|---|
| Model Serving with NVIDIA Triton Inference Server | Wed, 16 Dec 2026 | Online live | Weekday | 3 days | Enquire |
| Model Serving with NVIDIA Triton Inference Server | Sat, 30 Jan 2027 | Online live | Weekend (Sat) | 4 Saturdays | Enquire |
| Model Serving with NVIDIA Triton Inference Server | Mon, 08 Mar 2027 | Bengaluru classroom | Weekday | 3 days | Enquire |
Soaring demand, and where it takes you
As organisations put more models into production, platform engineers who can run efficient, multi-framework inference services are needed.
Become an expert
Roles this programme prepares you for:
Become a MLOps Engineer
Deploys and operates model serving platforms.
- Explain Triton's architecture and model repository
- Serve models with multiple backends
- Configure dynamic batching and concurrent execution
About the course
Serve models at scale with NVIDIA Triton Inference Server: organise model repositories, use backends for TensorRT, PyTorch, ONNX Runtime and Python, configure dynamic batching and concurrent execution, build ensembles, tune with Perf Analyzer and Model Analyzer, and run on Kubernetes with metrics.
Serve models from multiple frameworks at scale with Triton: model repositories, dynamic batching, ensembles and monitoring.
Why should I attend?
MLOps and platform engineers
ML engineers deploying models
DevOps engineers supporting AI teams
Architects designing serving platforms
Prerequisites
Python; Docker; basic ML model knowledge; Kubernetes helpful
Level
Intermediate to Advanced
Delivery modes, duration and fee
Online live
Instructor-led virtual classroom; weekday (full-day) or weekend (Saturday) batches
Bengaluru classroom
In-person batches at our Bengaluru training venue
On-site at your office
Private corporate batch anywhere in India, scheduled around your teams
Blended
Live sessions plus guided self-paced labs and mentor check-ins
Duration: 24 hours · 3 days
Course fee (open batch): ₹10,999
Corporate / team batches: per-participant or flat batch pricing on request, based on group size, format and customisation.
What will you learn
- Explain Triton's architecture and model repository
- Serve models with multiple backends
- Configure dynamic batching and concurrent execution
- Build model ensembles and pipelines
- Tune performance with Perf Analyzer and Model Analyzer
- Operate Triton on Kubernetes with Prometheus metrics
Skills you will gain
Serving
Performance
Operations
Tools and platforms covered
PyTorch
ONNX
Python
Docker
Kubernetes
PrometheusCourse curriculum
M01 Triton architecture
- Model repository layout
- Model configuration
- HTTP and gRPC APIs
- Versioning policies
- Model control modes
🧪 Hands-on: Serve a first model
M02 Backends
- TensorRT, PyTorch and ONNX Runtime
- Python backend
- Choosing a backend
- Custom backends overview
- vLLM and TensorRT-LLM backends
🧪 Hands-on: Serve models from three frameworks
M03 Throughput features
- Dynamic batching
- Concurrent model instances
- Sequence batching
- Priority and queue policies
- Response caching
🧪 Hands-on: Tune batching settings
M04 Ensembles
- Pre- and post-processing
- Ensemble models
- Business logic scripting
- BLS patterns
- Debugging ensembles
🧪 Hands-on: Build an ensemble pipeline
M05 Performance tuning
- Perf Analyzer
- Model Analyzer
- Reading results
- Configuration search
- Cost per inference
🧪 Hands-on: Find the best configuration
M06 Production operations
- Kubernetes deployment
- Prometheus metrics
- Autoscaling
- Health and readiness probes
- Rolling upgrades
🧪 Hands-on: Deploy and monitor on Kubernetes
M07 Capstone: production inference service
- Multi-model repository
- Tuning
- Kubernetes deployment
- Load test report
🧪 Hands-on: Deploy and load-test a multi-model Triton service
Hands-on projects
Multi-model serving pipeline
A tuned Triton deployment serving an ensemble with monitoring dashboards.
Why train with us
Built around your stack
Labs and examples adapted to the tools, cloud and data your team actually uses.
Practice-first
Every module ends in a lab, and the course ends with a reviewed capstone.
Practitioner-led
Delivered by Karthick Selvam and vetted specialist trainers from our network.
Visible progress
Attendance, lab completion and assessment results shared with your L&D team.
Certificate of completion
Participants who complete the labs and capstone receive a Learning Dots certificate of completion for Model Serving with NVIDIA Triton Inference Server. [confirm certificate wording / sample image]
Learner reviews
“[Verified learner review for this course]”
“[Verified learner review for this course]”
Meet your trainer
Karthick Selvam
Founder · Data Engineering Corporate Trainer, Bengaluru
[Trainer bio, experience and areas of expertise]
Frequently asked questions
How does Triton relate to NIM?
NIM packages models as ready-made microservices; Triton is general-purpose serving software you configure yourself.
Can the curriculum be customised for our team?
Yes. Modules, depth, duration and labs can be tailored to your stack and goals.
What delivery formats are available?
Online live instructor-led sessions, Bengaluru classroom, or on-site at your office.
How do we enquire or book?
Use the enquiry form on this page or the Enquire button next to any batch.
What does it cost?
The open-batch fee is ₹10,999 per participant plus GST. Corporate batches are priced on group size, format and customisation.
How long is the course?
24 hours · 3 days. Corporate batches can be split into half-day sessions.
A closer look
About this course
Serve models at scale with NVIDIA Triton Inference Server: organise model repositories, use backends for TensorRT, PyTorch, ONNX Runtime and Python, configure dynamic batching and concurrent execution, build ensembles, tune with Perf Analyzer and Model Analyzer, and run on Kubernetes with metrics.
What is Model Serving with NVIDIA Triton Inference Server?
Serve models from multiple frameworks at scale with Triton: model repositories, dynamic batching, ensembles and monitoring.
Who this course is for
MLOps and platform engineers, ML engineers deploying models, DevOps engineers supporting AI teams, Architects designing serving platforms.
What you will be able to do
Explain Triton's architecture and model repository. Serve models with multiple backends. Configure dynamic batching and concurrent execution. Build model ensembles and pipelines. Tune performance with Perf Analyzer and Model Analyzer. Operate Triton on Kubernetes with Prometheus metrics.
Career paths
Roles this course supports include MLOps Engineer, ML Platform Engineer, Inference Engineer. As organisations put more models into production, platform engineers who can run efficient, multi-framework inference services are needed.
NVIDIA & GPU Computing guides
What Is CUDA?
CUDA is NVIDIA's parallel computing platform and programming model for running general-purpose code on NVIDIA GPUs.
What Is NVIDIA RAPIDS?
RAPIDS is a suite of open-source libraries from NVIDIA for GPU-accelerated data science and analytics.
What Are NVIDIA NIM and NeMo?
NIM and NeMo are parts of NVIDIA's software stack for generative AI: NIM for serving models, NeMo for building and customising them.
TensorRT, TensorRT-LLM and Triton Explained
Three NVIDIA technologies for fast, scalable AI inference.