Hacker News new | past | comments | ask | show | jobs | submit
fhfncjcc | on: Ten advances in mathematics and theoretical computer science
The models are frequently getting worse at items that they aren’t being benchmarked for — and that’s happening more and more over time! Other people in other fields aren’t idiots, they are accurately perceiving the fact that these models are being hyper optimized for our industry, and are becoming less capable in other domains over time. Models of the same scale are massively worse at writing a broad variety of styles of prose than their equivalent from two years ago. (Models of increased scale are a mixed bag.)

Maybe you’re the one who needs breaking out of your cached beliefs.

Legend2440 | parent | next
Proof?

In my experience modern models are better at all tasks than models from two years ago, especially complex multi-step tasks.

nostrebored | root | parent | next
For customer support I don't think models have gotten better since gpt-4.1. The class of small models, with limited to no reasoning, that need to handle a complex issue with a touch of empathy, has not improved much.

I think most are actually worth, as agentic harnesses seem to optimize for solving poorly described problems rather than following complex procedures as written. In other words, instruction following maximizing models seem to make worse free-form agents, but they're really all that some domains need.

jstummbillig | root | parent
I understand the point (I don't agree with it; tool calling has gotten much better/reliable and that is very important for customer support) but consider: If you can get same for a lot less, that's an improvement. If we found a way to supply fresh water and electricity for -90% cost after 2 years, that would be fantastic.

You can do many more things, when stuff is cheaper, even if the stuff were otherwise unchanged.

alightsoul | root | parent | next
you are working on coding. they are working on things like "creative writing" remember that gpt 4o was popular among those who had ai as a romantic partnet?
Marha01 | root | parent | next
> remember that gpt 4o was popular among those who had ai as a romantic partner

I suspect GPT 5.6 would be even better at it, if given the same sycophantic system prompt and lack of guardrails.

QwenGlazer9000 | root | parent
Dude it's not a system prompt, it's the training.
whimsicalism | root | parent | next
gpt4o & associated parasociality is considered an alignment failure and is actively trained out of the model, so that is a terrible example of regression
michaelmrose | root | parent | next
sycophancy

It wasn't "better" it was better at kissing your ass which matches what a lot of people want in a partner.

Legend2440 | root | parent
Well that's on purpose lol. OpenAI does not want you falling in love with their chatbot and have been deliberately training it to be less romantic.
vablings | root | parent
There have been several cases of suicide and self-harm related to 4o, AI psychosis is a real risk and will probably be in the DSM
criddell | root | parent
Do any of the big AI companies have a model that are good at tasks that require learning?

For example, every day people teach teenagers how to drive and with only dozens of hours of practice, they are on the road.

whimsicalism | root | parent
is this not essentially what ARC-AGI-3 is? i agree that in-context/continual learning is somewhere the models are still mostly weak at
Chance-Device | parent
So your answer is: ignore the progress, it’s not really happening, actually it’s getting worse.

That’s not a credible position, but there isn’t anything that I or anyone else can say to someone who simply doesn’t want to believe something.

cmdli | root | parent
It sounds like they are making a clear argument: models are getting worse for certain domains even while they are getting better at others.

I don't know if I agree with that but it doesn't seem like an irrational claim and does seem credible to me.

Chance-Device | root | parent
I just don’t think it’s true, or is significant enough to matter to the direction of travel of AI even if there were something to it. It’s another cope post being lobbed at the idea of AI going somewhere and I’m sick of them.