Inference Optimisation with TensorRT & TensorRT-LLM
Make models faster and cheaper to serve: TensorRT for deep learning inference and TensorRT-LLM for large language models.
- ✓ Instructor-led live sessions, online or in Bengaluru
- ✓ Hands-on labs in every module
- ✓ Capstone project with trainer review
- ✓ Curriculum customisable for corporate teams
- ✓ Certificate of completion
Next batch: Sat, 28 Nov 2026 · 24 hours · 3 days
Inference Optimisation with TensorRT & TensorRT-LLM
Duration: 24 hours · 3 days
Next batch: Sat, 28 Nov 2026
Format: Online live · Bengaluru classroom
Modules: 7 · Topics: 34
Level: Advanced
Fee: ₹10,999
Trusted by leading experts
Upcoming batches
Open batches for individuals; corporate batches on your dates.
| Programme | Starts | Mode | Schedule | Duration | |
|---|---|---|---|---|---|
| Inference Optimisation with TensorRT & TensorRT-LLM | Sat, 28 Nov 2026 | Online live | Weekend (Sat) | 4 Saturdays | Enquire |
| Inference Optimisation with TensorRT & TensorRT-LLM | Wed, 13 Jan 2027 | Online live | Weekday | 3 days | Enquire |
| Inference Optimisation with TensorRT & TensorRT-LLM | Sat, 20 Feb 2027 | Online live | Weekend (Sat) | 4 Saturdays | Enquire |
Soaring demand, and where it takes you
Inference cost and latency are now central concerns for AI products, so optimisation skills are increasingly valuable.
Become an expert
Roles this programme prepares you for:
Become a Inference Optimisation Engineer
Tunes models and runtimes for latency and throughput.
- Explain latency, throughput and batching trade-offs
- Build TensorRT engines from ONNX models
- Apply FP16, INT8 and FP8 precision with accuracy checks
About the course
Make models faster and cheaper to serve on NVIDIA GPUs. Participants optimise deep learning models with TensorRT, apply reduced precision and quantisation, and use TensorRT-LLM for LLM inference with in-flight batching, paged KV cache and multi-GPU parallelism, benchmarking each change.
Make models faster and cheaper to serve: TensorRT for deep learning inference and TensorRT-LLM for large language models.
Why should I attend?
ML and inference engineers
MLOps and platform engineers
Performance engineers
Technical leads responsible for AI costs
Prerequisites
Python; PyTorch; GPU and deep-learning basics
Level
Advanced
Delivery modes, duration and fee
Online live
Instructor-led virtual classroom; weekday (full-day) or weekend (Saturday) batches
Bengaluru classroom
In-person batches at our Bengaluru training venue
On-site at your office
Private corporate batch anywhere in India, scheduled around your teams
Blended
Live sessions plus guided self-paced labs and mentor check-ins
Duration: 24 hours · 3 days
Course fee (open batch): ₹10,999
Corporate / team batches: per-participant or flat batch pricing on request, based on group size, format and customisation.
What will you learn
- Explain latency, throughput and batching trade-offs
- Build TensorRT engines from ONNX models
- Apply FP16, INT8 and FP8 precision with accuracy checks
- Optimise LLM serving with TensorRT-LLM
- Use parallelism across multiple GPUs
- Benchmark and compare configurations rigorously
Skills you will gain
Optimisation
LLM serving
Measurement
Tools and platforms covered
ONNX
PyTorch
Python
DockerCourse curriculum
M01 Inference performance
- Latency vs throughput
- Batching
- Cost per request
- Throughput-latency curves
- SLO-driven targets
🧪 Hands-on: Baseline a PyTorch model
M02 TensorRT basics
- ONNX export
- Building engines
- Layer fusion
- Dynamic shapes
- Plugins
🧪 Hands-on: Build and run a TensorRT engine
M03 Precision and quantisation
- FP16, INT8, FP8
- Calibration
- Accuracy trade-offs
- Weight-only vs activation quantisation
- Quality evaluation after quantisation
🧪 Hands-on: Quantise and validate
M04 TensorRT-LLM
- Building LLM engines
- In-flight batching
- Paged KV cache
- Speculative decoding
- LoRA serving
🧪 Hands-on: Serve an LLM with TensorRT-LLM
M05 Scaling
- Tensor and pipeline parallelism
- Multi-GPU serving
- Memory planning
- Memory budgets for KV cache
- Choosing GPU counts
🧪 Hands-on: Run a multi-GPU configuration
M06 Benchmarking
- Benchmark design
- Interpreting results
- Reporting
- Load generation tools
- Reporting for stakeholders
🧪 Hands-on: Produce an optimisation report
M07 Capstone: optimise an LLM deployment
- Baseline
- Engine builds
- Quantisation experiments
- Final report
🧪 Hands-on: Deliver a benchmark report with recommended configuration
Certification preparation
This course covers topics that can help participants prepare for the NVIDIA-Certified Professional: Generative AI LLMs (NCP-GENL) exam, which is run by NVIDIA. Check the current exam guide on nvidia.com. Passing is not guaranteed and the exam fee is not included unless stated. [confirm]
Hands-on projects
LLM optimisation report
Optimise an LLM serving setup and document latency, throughput and accuracy trade-offs.
Why train with us
Built around your stack
Labs and examples adapted to the tools, cloud and data your team actually uses.
Practice-first
Every module ends in a lab, and the course ends with a reviewed capstone.
Practitioner-led
Delivered by Karthick Selvam and vetted specialist trainers from our network.
Visible progress
Attendance, lab completion and assessment results shared with your L&D team.
Certificate of completion
Participants who complete the labs and capstone receive a Learning Dots certificate of completion for Inference Optimisation with TensorRT & TensorRT-LLM. [confirm certificate wording / sample image]
Learner reviews
“[Verified learner review for this course]”
“[Verified learner review for this course]”
Meet your trainer
Karthick Selvam
Founder · Data Engineering Corporate Trainer, Bengaluru
[Trainer bio, experience and areas of expertise]
Frequently asked questions
What hardware do labs need?
Recent NVIDIA GPUs; some precision modes need specific GPU generations.
Can the curriculum be customised for our team?
Yes. Modules, depth, duration and labs can be tailored to your stack and goals.
What delivery formats are available?
Online live instructor-led sessions, Bengaluru classroom, or on-site at your office.
How do we enquire or book?
Use the enquiry form on this page or the Enquire button next to any batch.
What does it cost?
The open-batch fee is ₹10,999 per participant plus GST. Corporate batches are priced on group size, format and customisation.
How long is the course?
24 hours · 3 days. Corporate batches can be split into half-day sessions.
A closer look
About this course
Make models faster and cheaper to serve on NVIDIA GPUs. Participants optimise deep learning models with TensorRT, apply reduced precision and quantisation, and use TensorRT-LLM for LLM inference with in-flight batching, paged KV cache and multi-GPU parallelism, benchmarking each change.
What is Inference Optimisation with TensorRT & TensorRT-LLM?
Make models faster and cheaper to serve: TensorRT for deep learning inference and TensorRT-LLM for large language models.
Who this course is for
ML and inference engineers, MLOps and platform engineers, Performance engineers, Technical leads responsible for AI costs.
What you will be able to do
Explain latency, throughput and batching trade-offs. Build TensorRT engines from ONNX models. Apply FP16, INT8 and FP8 precision with accuracy checks. Optimise LLM serving with TensorRT-LLM. Use parallelism across multiple GPUs. Benchmark and compare configurations rigorously.
Career paths
Roles this course supports include Inference Optimisation Engineer, ML Systems Engineer, AI Infrastructure Engineer. Inference cost and latency are now central concerns for AI products, so optimisation skills are increasingly valuable.
NVIDIA & GPU Computing guides
What Is CUDA?
CUDA is NVIDIA's parallel computing platform and programming model for running general-purpose code on NVIDIA GPUs.
What Is NVIDIA RAPIDS?
RAPIDS is a suite of open-source libraries from NVIDIA for GPU-accelerated data science and analytics.
What Are NVIDIA NIM and NeMo?
NIM and NeMo are parts of NVIDIA's software stack for generative AI: NIM for serving models, NeMo for building and customising them.
TensorRT, TensorRT-LLM and Triton Explained
Three NVIDIA technologies for fast, scalable AI inference.