Sure, but "must never leave the HSM" is much easier to audit for and much more difficult to screw up. This entire thing (allegedly) happened because of faulty logic; categorically disallowing as much logic as possible seems like the straightforward fix to me.
If transfer were to be allowed I'd think it should be entirely facilitated by the hardware itself and that capability audited as part of the approval process. (I assume the HSM used in this context is already subject to approval?)