Hacker News new | past | comments | ask | show | jobs | submit
hopa | on: Postgres 19 is late because so many more bugs are being found now
I feel like AI is amazing for read only “search” tasks (query this, look for bugs, find vulnerabilities) but it’s not quite as good for “write” tasks like “add this feature”. Search tasks are easily verified, code changes are much harder to verify
loading story #50335434
bheztio | parent | next
"Add this feature" is often too vague, unless it's yet another button to a list of buttons. If you want your feature in a particular way, you need to specify it. If you're not even sure what you want, it's granted you won't be satisfied by the result, so you need to spend time figuring that out first. In other words, they're definitely great for write tasks too, but it does require more effort from the operator to formulate the problem.
codebje | root | parent | next
That problem is as old as commercial programming; it doesn't matter that now your actors are prompt engineer and LLM, the same story can be told with manager and programmer.
berkes | root | parent
Indeed. And this is also why I've never once believed the "no-code" would replace developers. And why I see a clear role for software (prompt-, context- ?) engineers at least for a while.

I've worked for CEOs and managers that would insist on ridiculous ideas. Like that time a CEO (let's call him Cedric) came in with a printed powerpoint slide that had a screenshot of the website with a "login form" drawn on top of it.

Cedric: "I need you guys to put this onto that scrunt board thing and make this before friday".

Me (tech lead at that time): "But what features would users need login for? Where or whom needs the profile data? We'd need to work this out in detail and plan it. Certainly not even this month".

Cedric: "I don't know, it's for you to figure out". and then "It can't be hard, it's just two squares one for a username one for a password" (he forgot about the submit-button, the third "square", he didn't even fathom the idea that something has to lie "behind" the "squares").

I recall this anecdote whenever I feel anxious about whether soon "stakeholders" can just chat to an AI agent and have it build features, rather than hire me to do so. Sure, a lot of them will be able to. But there's always Cedrics. Those who not only fail to communicate their ideas in a way that even an AI agent would understand, they lack any understanding of what tradeoffs of their requested features are.

The Cedric's would be happy with an AI agent that complies and starts churning away. An agent that blindly accepts any request and builds feature upon feature. Making their product worse in every sense: unusable, unmaintainable, debt-ridden, security-nightmares.

Their agent will build features that are plain illegal. Or features that users hate, or confuse them. It will oblige requests that drive users away, or changes parts that users/integrations rely on. Their AI agent will gladly add a "form" to extract all PII from their database, but on request, will remove the authorization from it. Because "you are absolutely right. If we don't link to the page from the menu, no-one can reach it".

I'm not making the last one up. Another "Cedric" alike recently hired me to look at his vibe-coded app because his go-to developer gave up and dropped the project. It had loads of such extractors, dashboards and overviews. Online. Unauthenticated. His claude, however, did add the pages to the sitemap, so a customer found their private data on google and contacted "Cedric", after which he asked me.

No, AI is getting smarter, but as long as the "Cedrics" remain as stupid as ever, I don't fear for my job.

loading story #50335468
Forgeties79 | root | parent
This feels like just another variation of “it’s not the tool it’s the user,” which is too frequently leveraged during any discussion about AI. It makes it impossible to honestly assess the quality of the tool. You can always just say “people are doing it wrong.”
aleksiy123 | root | parent | next
Idk, I think it pretty clear to me that

“Find the issues in this code” vs “write this feature” are different problems.

The first one is easier because the feedback loop is llm to code/test and trivially scalable.

The second one is llm to human which is going to be slower because it can’t read your mind.

The tighter and more deterministic the loop the more llms excel.

8note | root | parent | next
its certainly benchmarkable.

make a bunch of "build the feature" vs "heres a detailed spec" prompts, and see which one does better.

if you take out the AI boogieman, and instead go with "next token prediction" then "people are doing it wrong" is much more reasonable

int3 | root | parent
I mean that's what you would expect from a new and rapidly-evolving tool. Being able to identify the kinds of work where people typically use it sub-optimally is a useful thing though!
simonw | parent | next
I'm not sure about that. The most common error for a search task is omission, and that's really hard to spot - how do you verify that some answer that should be there wasn't included?

For things like "add this feature" you can verify by trying out the feature and seeing if it works. Of course, that won't tell you if there are edge-case bugs or security holes or other design flaws, but you can get to a positive confirmation of the happy case pretty quickly.

27183 | root | parent
> The most common error for a search task is omission, and that's really hard to spot

NP-Hard, even[1]!

[1] https://www.i-programmer.info/news/112-theory/3830-physics-i...

whattheheckheck | root | parent
We're never solving nphard. Just like we'll never extract all of the coal/oil/natural gas from earth
ipdashc | parent | next
I've slowly been coming around to this viewpoint as well. It's interesting that there's so much focus on using AI to write code (which, don't get me wrong, does work well but makes you lose track of what you're building so quickly) and seemingly less on analyzing code, finding bugs, etc
mihaelm | root | parent | next
The planning, which can also be done in concert with AI, before getting the AI to write the code helps with not losing track of what you're building. In my current workflow, I never tell AI to just code X, I always approach it in such a way to leave space for research and breaking it up into multiple units of work (tickets) if the scope of the feature or idea calls for it.
kibwen | root | parent
I wouldn't say it's "interesting", rather that it's a natural result of incentive structures. If the AI labs try to sell their product as being good at finding bugs, then their expected valuation is the entire market cap of the existing static analysis industry, which is not very high. If they instead sell their product as being good at writing software, then their expected valuation is the "market cap" of the code-writing industry, a.k.a. the total spent on software development labor, which is very high. It doesn't matter what's true, only what promises to make them the most money.
MBCook | parent | next
But it DIDNT “write that feature”. They made it extremely clear on the delay announcement the code was written without AI.

AI’s only part of this was helping review things and finding bugs. More bugs, bugs that would have been found later, whatever.

Postgres did not have a slop issue and it’s wrong for people to insinuate it.

r_lee | parent | next
I mean the transformer model came out of Google, and it's literally a query thing so yeah.

any kind of pattern matching / transformation is going to be an ideal task for LLMs

jeremyjh | root | parent
It is not in any way a “query thing” in the same manner as PageRank, it was based on prior work in deep learning and it is irrelevant that Deepmind was purchased by the same company that was behind Google Search.
vatsachak | parent | next
What? AI can't easily implement features. Just write a spec, ask if to expand it and then ask it to implement it
xyzsparetimexyz | parent | next
No offense but this is not exactly a groundbreaking statement. We've known about this verifiable tasks thing for a while.
hoppp | parent
If the entire project is developed with AI it can add features effectively but for large old codebases it becomes an issue