I always thought CMake was good enough. I use FetchContent for all external dependencies and git submodules + add_directory for all internal dependencies. It took me a while to figure out that this was the simplest thing that works reliably though. I have used vcpkg and conan extensively and have completely given up on them.
I don't use C++ modules, because afaik they are still mostly broken. Without modules clangd works wonderfully and has been working wonderfully for a few years. I really, really like it. I think it can be done! I am missing reflection though, but if I would really want to use it, I'd probably just use clang -ast-dump instead of the new reflection functionality.
> I quite like modern C++, so I really tried to make it work for gamedev, but I think you can't really do it.
What exactly do you mean? What parts of modern C++ did not work for you?
> You can use all the modern C++ features that are not in the standard library though!
> I just decided to not use any of the C++ standard library (I just use a few C headers)
So what do you do with C++ that C alone couldn't do?
What I do like and use is overloading, references, templates, concepts, lambdas, enum classes, user defined literals, constexpr, operator overloading for math (!), little syntax stuff like structured binding declarations, "auto" and range based for. I also made my own little fmt (like https://riki.house/fmt). C++ is a much nicer language than C imo, even without the standard library.