It's a broad statement and can mean different things depending where you are in that chain.
You have design and compliance. Compliance is what you are allowed to do (laws). Design is how you construct your applications to understand how it will impact the people directly or indirectly.
Then you have the accountability, transparency, auditing and reproducing. Understanding why the model worked the way it did. Models can go wrong, but if you have the details of how it went wrong and who is at fault, it can protect people who use it.
Then there is alignment, which is a higher longer goal.
Your comment about security questions is a matter of ethics as well.
If you say apply it to medical, is it ethical to allow a model to give medical advice, knowing that it can be wrong. Most people would say no, but by doing so you are denying people who can't afford medical advice, so there has to be a balance. Most companies err on the side of not getting sued.
I wouldn't be so sure, most people I've met seem to be all too eager to suggest what worked for them (or they have heard is supposed to work) in conversations about medical issues, so I can't really see the public generally categorizing advice or guidance being given in non-professional setting or by non-experts as unethical. The taxpayers representatives having to bear the costs of the outcomes (at the very least as reductions of tax revenue) I can see being of a different opinion though.