But surely the point is that the proxy key is for your local network, it ain't got no value on the interwebz.
Precisely. You absolutely have to ensure that random agents can't join your local network, which means you need a deterministic orchestrator or an AI orchestrator that only spins up a handful of vetted agents.