A trained model only creates value once other systems and people can use it. Model serving is the step that turns a saved model file into a reliable service that accepts requests and returns predictions, either on demand (online inference) or over large datasets on a schedule (batch inference).
You'll start by wrapping a model in a REST API with FastAPI and packaging it with Docker so it runs the same everywhere. From there you'll look at purpose-built serving tools such as BentoML and NVIDIA Triton Inference Server, and finish by deploying a containerised model service to Kubernetes so it can scale.
Once a model is live, it needs somewhere solid to run. The next module, Cloud & Infrastructure as Code, covers provisioning that infrastructure in a repeatable way.
Take a trained XGBoost model out of a notebook, expose it as an API with FastAPI, and package it in a Docker container.
Build a FastAPI prediction service, containerise it with Docker, and deploy it to Heroku so anyone can call it.
A step-by-step tutorial on using BentoML to package, serve, and deploy a machine learning model as a production-ready API.
An end-to-end walkthrough of serving an object detection model with NVIDIA Triton Inference Server.
Build a FastAPI service, package it with Docker, and deploy it on a Kubernetes cluster step by step.