The OS absolutely controls functionality of the baseband, it would be a legal nightmare if it just didn't actually respect proper radio settings. But from a design perspective, it doesn't actually matter that much at all I feel. Any practical and secure design for a cellular phone has to assume the network is totally compromised and hostile, and so it can never be in the trusted base. From this perspective, adding the radio chip and its software to that category is not actually a huge leap. Even today vendors who own all the IP on a bus will still insert IOMMUs, etc inbetween components. It isn't dissimilar to game consoles; they assume that anything outside their die is compromised and hostile, and if you start from that premise, you can still make it very secure even when your threat model includes hostile components on the PCB.

Radio firmware is probably somewhat broken in practice (what isn't these days) but ultimately the openness stuff is a red herring IMO as far as security is concerned, you either assume it's compromised or not.

The userspace OS needs to be where control of any radio (bluetooth, wi-fi, cellular, nfc, uwb etc.) components happens, otherwise it would be difficult to be compliant in terms of regulatory requirements like a working Flight Mode.

Baseband firmware/processors are still making strides in terms of defending against attacks so I don't think we are at the point where backdoors are needed. More likely that undocumented AT commands, plausibly deniable bugdoors and existing memory corruption vulnerabilities are already convenient reality. However, part of that reality is also that once the cellular modem is exploited OTA it is still a massive task to traverse IOMMU restrictions and get device owner data/secrets from the userspace OS. It doesn't even grant visibility into application traffic which has been aggressively TLS encrypted since the mid 2010s.

Graphene doesn't expect to have control of the RF. It makes all the RF stuff happen on the other side of a firewall.

In modern phones, doesn't the main chip control the baseband chip rather than the other way? What is the threat model?

Obviously since phone calls and text messages are unencrypted the baseband chip could send them elsewhere. But your internet communication should already be encrypted before it hits the baseband chip. So is the threat model Motorola has intentionally introduced a backdoor to leak data from the main chip to the baseband chip?