There's also the grey area of remote kill. It should be required to be disclosed up front and the company should be required to put up a deposit with the FTC for a simple you-can-live server. If the company shuts down the FTC's copy is spun up and anything that hasn't been killed continues to operate.
Why on earth can't I just send the command directly to my garage door opener over my LAN? That should be the simplest mode of operation possible. I only need Internet connectivity if I somehow want to close my garage door from miles away!
Because of NAT. Your door opener isn't connected to the internet, it's connected to your router, which uses network address translation to make all your devices share a single internet connection. So your door opener can send outgoing messages no problem, and the NAT router will route those outside, and then route responses to those messages back to the opener, but there's no (easy) way for a device outside your home network to access your opener. Of course, your phone or PC is on your home network, but only when you're at home. Presumably, you also want to be able to control your door opener when you're not at home, and at those times, you have to get through the NAT. Doing that requires the company's server, or something like Tailscale which sets up a virtual private network (VPN) between your opener and selected devices. But even Tailscale requires some type of server on the internet to work, even though the main traffic goes direct.
Basically, what you're asking for could have been a reality if 1) we had IPv6 (or just not IPv4 with its very limited number of unique addresses), and 2) we didn't have to worry about security so we could feel safe putting random crappy IoT devices directly on the internet and not worrying about hackers opening our garage doors or houses so thieves could enter and take stuff.