Reducing the Size of Docker Images Serving Large Language Models (part 2)

How to reduce the size of a “small” Docker image by another 10%

Michał Marcińczuk, Ph.D.
Towards Data Science
7 min readMay 8, 2024

--

Generated by Runway for the prompt: There are two containers on board the ship, one large and the other small. They are bright, vivid, and realistic in color.

Introduction

This is a continuation of the topic of reducing the size of Docker images serving large language models. In my previous story [1], I presented how to reduce the size of a docker image…

--

--

Head of Statistical AI @ Samural Labs. I'm working on neuro-symbolic AI, NLP, ML, LLM, Python