What people think of as a very strong password often has <100 "security bits" of entropy.

Keys are usually at least 128 bits of entropy, and it's easy to take them up to 256.

Yes, both are a shared secret, but password auth - even with challenge-response - is weaker than key auth exactly because key auth provides more guess resistance.