- promising monogamy to someone who loves you, and then breaking that promise, is bad and provides negative evidence about your character
and
- if you're the leader of a very powerful entity, your character is relevant even to people who don't know you personally.
There's also just the fact that when someone is famous, and it emerges publicly that they've broken some norm, there's not really a fully neutral option available to us. We can condemn their behaviour (reinforcing the norm), defend it (weakening the norm), or ignore it (also somewhat weakening the norm, especially if we make a point of ignoring it and suggest that others are silly for caring).
Obviously it's somewhat complicated when subordinates and Chinese spies are involved, and I'm not condoning infidelity, but IMO infidelity is a reason for a spouse to be justifiably outraged, not the public at large.