Hacker News new | past | comments | ask | show | jobs | submit
Organisationally it never works to have a group whose only job is to say no to some other group. The incentives are diametrically opposed and as a structure it can’t last.

If you had an AI company and want it to be ethical you have to find a way to make ethics everyone’s responsibility, and have the consequences of poor ethics bite the people who make those bad decisions. If you just outsource it to the ethics group what happens is

1)everyone else thinks they don’t need to worry about ethics

2)the ethics group need to justify their existence so introduce a bunch of guidelines that everyone initially thinks are reasonable but over time people think are increasingly out of touch

3) The ethics group start to “make difficult calls” and say no to things. Initially everyone supports this and feels like the system is working as it should but over time everyone starts to just see them as an obstacle to work around

4)everyone else starts to try to work around what the ethics group says

5)The ethics group grows powerless and disconnected. The people who work around them “get things done” so get promoted etc whereas they only visibly put roadblocks in peoples’ way, so they get sidelined.

6)Eventually they get disbanded with some corporate announcement thanking them for their hard work, thought leadership etc. All that has been achieved is a lot of wasted time and bad blood.

If this were true, food companies wouldn’t have a Regulatory team. Except every single one of them does, and they function similarly to how you describe, but without dissolving. They’re treated as a constraint that must be checked before major projects can advance. Sometimes they’re the longest-lead item on a new project and if you don’t get them involved early, your project can sink at a late stage after great expense.

The only difference is AI isn’t regulated, so they just have a vague “ethics” department with no real teeth because it’s essentially PR and has no legal consequences to back up their stance.

loading story #49270595
loading story #49271741
loading story #49271911
loading story #49281802
loading story #49271910
loading story #49271637
loading story #49271591
loading story #49270565
loading story #49271664
loading story #49270357
loading story #49270581
I get what you are saying but that’s exactly how security and compliance work. And I don’t think amount of people who want to be ethical in an organization ia that different than people who want to build secure software (but possibly no one gives a shit compliance other than it’s something that needs to be done)
loading story #49268397
loading story #49268750
loading story #49268939
loading story #49268541
loading story #49268557
loading story #49269317
loading story #49268359
loading story #49272280
loading story #49268268
loading story #49268246
loading story #49269636
loading story #49268584
In a situation where it doesn’t happen this way, the ethics team grows in power and staffing over time until it destroys the company, because ultimately all commercial activity can be construed as less than perfectly ethical at some level, or it morphs into an elaborate legalese department that masks unethical practices in a cloak of justification.

Either way, you need third party regulation and then a department dedicated to ethical compliance, or you end up with corporate cancer.

I say this as a person with a knee-jerk anti-regulation reflex.

Operations was like this so we added Ops to the Dev. DevOps.

Security was like this so we added Sec to the DevOps. DevSecOps.

Ethics is now the problem, so we’re going to add Eth to the chain. DevEthSecOps.

Legal and compliance is now the final hurdle to shipping fast. AI tools will allow us to implement DevEthSecLawOps at scale no doubt.

> whose only job is to say no to some other group

I suspect that people will miss just how precise you've been there.

Your proposal - if I understand correctly - is not just to spread the "ethics team" onto everyone (which is correct), but also to have someone in a dedicated role. However, they need to have not _just_ that role, but also some skin in the game.

loading story #49268729
Absolutely - though the flip side of this is if something is everybody responsibility - it's nobodies.

There are two things to successful organisations structure and people, you are focusing only on the structure.

What you need for somebody responsible for health and safety, ethics, security, or compliance is a person who has courage and is willing to take calculated risks.

However these roles do have a tendency to attract the risk adverse, or make them risk adverse if you punish risks that go wrong too harshly.

And a key critical success factor is leadership from the top - companies have personalities and leaders in the organisation are very important in shaping that.

Hard disagree, as someone whose main job is telling developers "no". If it's everybody's responsibility it is in fact nobody's responsibility.
loading story #49271581
I don't get why in the comments this gets compared with regulations and security teams...

Say no to regulations and security --> Fine, forced correction, fewer customers, shutdown.

Say no to ethics --> money (at least for AI companies)

Why is this so hard to understand?

It is good PR to have what is largely ceremonial role that oversees an entire system, whose powers are in practice useless or completely defanged.

I’m thinking of the role of the King or Queen of England, for example, or any country’s president. They have the power to disband parliament if it is deemed unfit for government, or reject a law that has been approved by the senate/Lords, yet in practice they are not expected to disagree with the ‘will of the people’ so to speak.

Same applies to the head of an ethics department on a business which is obviously unethical (or they wouldn’t need one).

Isn't this the exact setup for every compliance, security, or other regulatory group in any company? To say no then "here's what's acceptable that's close" to guide the business or entity?

Perhaps "ethics" is a more slippery slope, but the approach is well trodden.

It's a choice of company culture from leadership on whether the company acks the suggestions or ignores them. And when they get ignored or drive little impact for the cost, then teams disband. Which is also true for any team at any company.

Isn’t ethics incompatible with the nature of a commercial enterprise? Now, if OpenAI was a non-profit… as it was intended originally.
Shall we get rid of the FAA and the FDA then? Lets just have all pfizer employees make ethics their own responsibility.

What your describing is the specific case of a company so paralyzed by short-term thinking that they see regulation as a burden. Some maturity in the organization would allow reframing this to be less antagonistic.

loading story #49268883
loading story #49268900
> Organisationally it never works to have a group whose only job is to say no to some other group.

That is only true where the pressure to be [the thing that is inconvenient to the other group] is not from a source that can cause massive problems if non-compliance is spotted. When the blocking group is legally mandated or otherwise really has teeth or is defending the company against an external regulator with teeth, then it works better (though obviously not perfectly). Think legal and compliance in banking realms, where the company significantly fined and the people breaking the rules could be sacked & blackballed (though sometimes not the people ordering them to break the rules!) when something bad is noticed. That is quite different to an ethics officer in a company like OpenAI where the position is basically there for PR purposes (“look plebs, we care about doing the right thing, honest, we got a manager with a small team dedicated to it” and “look [government body], we are regulating ourselves, do you really need to spend time looking too?”) and therefore has no real teeth directly or indirectly especially as fault for non-compliance might not be easy to attribute.

Well, not when the whole objective of the company is to find ways to say they are ethical while knowing they are not
yup, like risk managers in investment banks with their own trading desks.

just ngmi

General Counsel has entered the chat
{"deleted":true,"id":49272345,"parent":49268172,"time":1786542321,"type":"comment"}
this is a strong reminder of the character of Toby from The Office.... :-)