Agreed.

I hold Control and double-tap b for managing the remote session, then everything else is the same.

Granted, I'm not a power user, so there may be numbers that get frustrating. I could imagine complex splits getting confusing (I don't use splits at all).