2 things: First, you can (and should) replace your `pip install` with `uv pip install` for instant speed boost. This matters even for Docker builds.
Second, you can use uv to build and install to a separate venv in a Docker container and then, thanks to the wonders of multistage Docker builds, copy that venv to a new container and have a fully working minimal image in no time, with almost no effort.