Anything that it has access to it can potentially share.

The solution is, like you are already alluding to, to not give it access. But that requires a level of isolation that folks normally don't do. To the contrary, the AI agent ads to non-tech folks emphasize it acting like a personal assistant which by its nature needs access to data you don't normally want public. Emails, photos, files with written notes, etc.