Exporting keys under wrap isn't the problem. Importing new keys not generated by an HSM would be, if that's what happened and the HSM didn't actually generate the ridiculous keys. Since the HSM shouldn't be capable of generating such weak keys, it's likely they generated them externally somehow (typing by hand? all decimal digits being in increasing order is unlikely even for rand(255)-style insecure RNG calls) and imported them to the HSM.
Sure, but "must never leave the HSM" is much easier to audit for and much more difficult to screw up. This entire thing (allegedly) happened because of faulty logic; categorically disallowing as much logic as possible seems like the straightforward fix to me.
If transfer were to be allowed I'd think it should be entirely facilitated by the hardware itself and that capability audited as part of the approval process. (I assume the HSM used in this context is already subject to approval?)