Production

Last post in the Ollama production series: test your FastAPI wrapper without a model, run the tests in CI, add health checks, watch the right status codes, and rotate keys and upgrade without downtime.

Follow-up to the Ollama security post: cap simultaneous requests, share one rate limit across workers with Redis, survive a Redis outage, and encrypt the hop between your load balancer and workers. Tested with a local Redis and Nginx.

Part three of the Ollama production series: keep Ollama private, add API key authentication and TLS, cap what clients can send, and stop leaking errors. Tested FastAPI code included.

This follow-up post builds on the previous article about creating a FastAPI wrapper for Ollama models. It explores what’s needed to move from a dev-friendly API to a more production-grade service, including API authentication, rate limiting, request validation, load balancing, and monitoring. FastAPI performance a concern? Read on to find out more!

This series of follow-up posts builds on the original article about creating a FastAPI wrapper for Ollama models. These posts explore what’s needed to move from a dev-friendly API to a more production-grade service, including API authentication, rate limiting, request validation, load balancing, and monitoring.