So your trusted base is the Python runtime and maybe a few choice libraries that actually have engaged package management. Sadly, it is a trap to think you can put things on the internet as a small player by leveraging lots of third-party stuff, when you don't have the resources to actually maintain that large dependency base.
This also may mean you want a base OS like Fedora and not some crusty "enterprise" variant that tries to bundle a frozen Python from 15 years ago. This is getting a little better in recent years, with some enterprise platforms having optional package streams for parallel install of newer runtime versions, etc.
Edit to add: on the Fedora-derived platforms, pip doesn't really interfere much anymore. The platform python-* RPMs go into /usr/lib/... while pip installs into /usr/local/lib/... or your ~/.local/lib/... hierarchy. The only mysterious behavior you get is if you manage to install in both places and then you have to understand the path search order. Just like if you do strange things with user and virtual environments.
Basically once you leave the realm of the 5-10 most popular building blocks in an area (which OS packagers do distribute), you're still left with plenty of packages that have 10ks+ of users, sane maintainer/security/ownership models, and code that would be very difficult to just copy-paste a la stackoverflow (e.g. native/compiled deps with complex wheel build tooling).
What isn't really tenable for a small team is to drag in scores to hundreds of different third-party packages and rely on them all when you have no staff allocated to really evaluate and monitor so much upstream stuff.