It's just too complex for its level of assurance and its blast radius.
To achieve tolerable security you can either have very little code and thoroughly review it all (the seL4 approach), you can write it in a way the computer can prevent most of your bugs (e.g. write in Rust), or you can run it in some kind of blastproof container. Linux has none of the above.
With how many people and organizations use Linux it's security is considered "tolerable"
Most of the software needed to provide a useful desktop Linux never runs on servers, so we cannot conclude that it is safe to use desktop Linux just because Linux is a popular server OS.
Also, at organizations like Google with a good reputation for security, most of the Linux installs are connected to internal networks that are quite difficult to break into from the open internet. The Linux installs at Google that are connected directly to the open internet are very carefully configured to minimize attack surface.
And Google has an internal team of Linux kernel maintainers that do not share most of their work outside Google.