We should be more ambitious with the software we build
https://danluu.com/ambitious/> I have a similar thought about "there's no reason for software to be slow" BTW; in many cases, if you use AI to speed up some flow, you will add a bunch of messy code that will be overfit for the use case you tested on and will neither generalize, perf-wise, for all the use cases you should care about, nor will be maintainable upon future changes.
LLMs have hardened hundreds of apps where I work.
Maybe it's a language-specific thing. Most of the time, in Java, LLMs take into consideration tradeoffs (especially if they are part of the initial prompt) and actively explains them. It generally won't make messy workarounds, without a lot of extra prompting forcing it to do so.
Ofc, if you have:
- Functional tests - you'll be able to see it in the execution times of specific APIs
- Stress testing - performance profiling is the most important output artifact
Claude in its current Opus 5.5 form has made some wild reaches in response to reasonable-seeming prompts. For instance: "this error is getting swallowed, make sure the real termination message exposes the root cause" -> discovering that the issue is actually in a third party library that isn't exposing the original error except so through logs, so doing some hackery to inject a log-capturer that gets the stdout, buffers it, and looks for the message in order to re-throw based on that...
And it can demonstrate that working, but it's added some fragile and/or unintended consequences like (a) string matching on a particular WARN log message of a particular version of the lib, and (b) buffering those logs such that there were now cases where they would never make it to the actual output stream before the program terminating.
It's certainly capable of being used smarter than that, but it's also really easy to end up with messy fragile stuff if you get lazy about reviewing the output.
Be careful to not frame this as a feature of AI and not a feature of the approach people have taken that you have observed, it doesnt have to be this way.
It's pure vibes, so buyer beware https://github.com/adriel/siri-remote-control This is "my" first Mac app, before this I'd only built PHP or web based apps. Pretty surprising how quick Claude could make it. I always thought Mac apps were really hard to build.
Any prompt jockey can make a faster ripgrep. The spec is "faster ripgrep".
What is the one thing that only you can specify, where your competitive advantage is maximized? Find that thing and pursue it with all your energy!
If somebody's passion is faster command line tools, let them make a faster ripgrep. First results may be disappointing, but those who don't give up will figure out how to do it right.
Can you give some examples where people have done that?
Or just look around for people who are doing this and then tell the LLM "faster <thing that the other guy did>"
For example when the heat pump reaches the target temperature it shuts off, then after 3 minutes switches on again. To prevent short cycling, I'd like to decrease the time between runs (so increase the 3 minutes) but that can't be changed. Somebody decided that 3 minutes is the correct value for everybody and that's it.
I bought a €400 interface box (basically just a serial to IP interface, I guess $20 of hardware tops) to try and improve it. The €8000 heat pump doesn't come with WiFi or ethernet out of the box, you need to buy an accessory for that. It's not made by Hitachi, but it's an officially recommended accessory. One of what feature I wanted it for - which is advertised in their marketing materials - doesn't work.
I emailed them, and they said it's probably because the display I have is a newer revision and doesn't allow access to the setting I need to turn on. And without turning that setting on I can't use the feature as advertised. They said I should email Hitachi and talk to them instead as it's not their part that's the issue.
I look forward to the world where all software is either open source, or at least reverse engineerable to the point where you no longer have nonsense like this.
It's pretty bad inside the bubble too.
Neither do governments, incidentally. They're okay-ish at skyscrapers and bridges, but software is just beyond them.
I'm guessing they made "happy path" decisions thinking most people aren't going to modify cycle timing, or know how to do that or what it entails. That 3 minutes was probably decided on by a group of engineers who thoroughly know the product. Especially if they provide a warranty.
You may know / understand a heat pump system but if I modified values with no safeguards on mine I'd probably break it within the hour.
Even if its closed on their end, having standards is the key.
Now we just have to get them to expose interfaces so that we can write the software for the interfaces that we want.
- Python interpreter in Rust (fail): https://github.com/BlueBlazin/pyrs
- Low-poly Google Earth like engine (success): https://github.com/BlueBlazin/lowpolyearth
- Codebase -> 2d infinite-zoom visualization (success): https://github.com/BlueBlazin/blockviz
I have never understood or liked all the obsession with one-shot builds.
Publishing screenshots and the README on GitHub but not the code is sneaky at best, deceiving at worst.
I took it to 39% complete, as a language, over 8-weeks. Then, I spent just 50 minutes describing all the missing parts and it finished the language in 1-shot. I didn’t think it would work. It did.
I used Opus 5.5 on a $20 plan and I didn’t even hit my limits.
To get a toy version that can run most things takes under 2 hrs. Half of that time is collecting the tests you want to pass, how it should evaluate the code, what your criteria are, etc.
https://app.preview.craftspec.com/app/city/
This whole thing is just a proof of concept, and needs a mouse to interact with.
It’s incredible what AI has made possible with experimental projects that would require months of coding.
There's so many youtube channels getting hundreds of thousands of views on EVERY new major model release just doing essentially zero effort one-shots, and people lap it up as if building things that already exist is somehow the best their imagination allows for.
I do love being able to test a model for 3+ weeks (>1 month is too long personally for my budget) and actually learning how to build scalable software and actually develop your "engineering" skills.
Same, and I have found some of the failure modes in the realm of eyes free coding and limited auditing. I often see it more as pushing the process than the model, as I tend to bounce around models for the benefits diversity provides.
Have you also found there is a point where they hit code complexity and start churning without progress? (at which point you have to step in to help the bot step back and think big picture, which usually takes a few sessions)
That software is a means to an end can be quite unfashionable on HN, but it's more ambitious thinking we need, and maybe software will be the answer.
(and no, the wet dreams of a teenager's interplanetary conquest fetish is centuries old rehash, not original thinking)
Claude, Codex, Opencode, everything is Electron. With agents and solid engineering fundamentals, I set out to prove that we don't need to take that shortcut any more and we can, in fact, ship high quality/performance applications.
It's still early days - but I got something basic working. I've embedded the Goose harness as a back end and am using the UI framework used by the Zed editor.
https://github.com/alshlabs/quill
I'm keeping the source closed for a little while. Also if anyone from the major labs is hiring, hihi, I make quality things.
Also I'm aware that this is a rip off of vscode's design. I like their UI, lends itself well to projects and so on. In a different life I might have ported vscode to Rust - but no one uses IDEs any more haha
The purpose of software is to help people with their problems. I think it's easy to tell ourselves that other people want to use our tools which we really made to help ourselves (especially if them using it also helps ourselves even more!). And also, this is all so new and there are real opportunities that it is good to learn and try new things and be open to actually making something valuable.
But! Other people extensively using agents are literally the last people who would pay for a project like this! And unless it's a fulltime job like it is for a 5 or so well resourced tech companies, it's hard to make something as good and useful as them. I say this as someone who spent a long time on this problem as my fulltime job.
We should still have fun with our work. But literally, "software for other software people, that solves the problem of us needing to write more software for writing software, faster", who is that helping? There are no constraints or end users besides ourselves, and half of that is not so much it benefiting us through productivity, but learning new things and play - again, important. But there's a reason so many people are doing this and I think it's because they don't have to go outside and solve someone else's problem, or do something unusual, or involving unfun non-software aspects - agents are fun! But how many random people's agent have you installed?
IMO: if the technology does not help a non-technology person live a better life, and especially if it involves "using an agent to build a framework for agents" we should do some more, deeper introspection about what problem we are actually solving and whether or not we're being ambitious. And I say that not to pick on you but myself.
We should build everything using a AAA FPS engine loop now that we can. The amount of shit that happens in one rendered frame of overwatch is wild. And then we go and do like 200 of those per second.
This never stopped the Electron devs until now, I honestly can't see why they'd start to get any better now that the slop machine lets them be even lazier.
The project would've been impossible for me without LLMs. & even with them there's an argument that this project is pushing complexity limits too far. But at least with ETL there's a clear feedback loop to build on
I do however wonder about specific research priority/project from labs' solving their own problems leaking into my chats. I noticed one month that Claude really started harping on about "legibility" and suggesting Seeing Like A State to me, and saw it like any other interesting suggestion/insight.
Now I see a lot of people mentioning "legibility" and think that was just some weird attractor state or useful vocabulary term for being an LLM (that literally can't see certain things or be observed easily!). It's a weird feeling, because nobody tried to trick me and in fact it objectively helped me to acquire useful information. I just have to wonder if other people are having the same conversation and there's really nothing special about it - maybe Anthropic just really liked that book right around when they were doing a lot of RL
Not only that, was the intent just to give Claude the vocabulary to be an LLM in a company that wants it to be legible, and apply that book to managing it, and we're either intentionally or unintentionally memetically downstream of that? Or was it just extra likely to be mentioned because it's a genuinely helpful book - either because the AI learned over time that it was (very good!), someone just wanted to recommend it to more people to help them (~good). I worry that the "intelligent conversation" or "wow look at the agent go" can give us very poor epistemics
This is, as you say, exactly the paradox of having ourselves assisted by LLMs. We are no different from our cats. Except imagine the cats start making the laser lights for themselves.
I encourage everyone else to at least build a privacy first version of a product you actually like.
I don't think the centralized internet will survive this change. We can build something much better.
We can rewrite existing libraries and software all we want, but the true test is if we can create something new.
The thing I learned about myself with AI is that now I have time to work on every idea that pops into my head —- my ideas aren’t that great.
I like to think of the evolution of Factorio. It's been 15 years with the same core team playing their own game in a LAN party every week, as well as individually. And it's high quality.
Dogfooding led them to counter-intuitive decisions like, if a flying robot runs out of battery, it should obviously crash into the ground and explode, but it turned out way better for gameplay if it just limps slowly to the nearest charging point instead. This fact and many more can only be discerned from actual gameplay experience. No amount of pure linguistic reasoning will tell you them. Another one is how the burner inserters (coal-powered robot arms) fuel themselves - in the first version, they didn't. They could have made running around to refuel your robot arms a part of the game, but that isn't where they discovered the statue within the marble. You had to try the game both ways and see what feels more fun. A third: filter inserters and smart inserters were removed, and their features were moved to all inserters. A fourth: trains originally ran on strict time-based timetables, but it was frustrating so they were made able to move based on logistics needs instead. A fifth: basic oil processing used to produce all three oil products, and now it doesn't. A sixth: the basic oil refinery has its liquid I/O ports set up in a way that's compatible with switching to advanced oil refining later, instead of requiring you to redo your pipes.
All these little things you get from actually using the software and streamlining it, those are the quality. The list above obviously doesn't translate directly to any other project, but something like Microsoft has its own list, probably starting with things like common controls, dialog tabbing, and drag'n'drop. Did you know the reason you click "Start" to shut down in Win95 is because they did actual user testing and that's where the users looked first?
... and politics! To me LLMs are just another piece of evidence that it was never the lack of resources that made Microsoft produce bad user experiences.
AI can take us from 0 to MVP in a prompt. It can fill in gaps of knowledge and allows someone who knows half of the stack to build the whole thing nearly seamlessly. These are superpowers but don't by themselves lead to new products.
What AI cannot do is remove the world of complications that happen after the MVP - product feature tradeoffs and choices, how to handle integrations and customizations, how to serve multiple types of users, how to communicate and market the app, how to handle security and data storage, how to deal with customers and bugs, etc. All of these need to be considered and acted on by a human in some fashion. AI can help with each concern but can't solve them without understanding the concern and determining how to frame it in a prompt.
This is why we have a glut of amazing demos that go nowhere while we figure out how to build software differently. This will take some time and I imagine the result will be software heavily-structured with a bit of AI reasoning woven throughout, not an army of agents and sub-agents burning infinite tokens.
There's a less hopeful outcome, where no skill is needed at all, because every conceivable thing is one-shot-able from a SOTA model. But then, at that point, you're saying something pretty profound about computer science (roughly: that there was no meaningfully more sophisticated and complex place to go from where we are today).
They keep adding features that are outside the scope of the original project, without refining or polishing the original concept.
If you're being ambitious enough, you shouldn't be able to there in one prompt. Ambitious work with AI is done 24/7 for months and guiding it continuously, because it's not good enough to work autonomously for that long without going off the rails.
It makes it really hard to connect to any of this. I loved danluu articles before, but somehow they've started to drift into this hard to grasp, hard to argue against shallow and empty world I try to get away from.
I will need an example for this, what can I achieve with a large language model in ten seconds that I can not do on my own in one and a half years [1]? Or one hour vs 568 years?
The unsolved math problems are one thing that came to my mind, after all no human could solve them. But Navier-Stokes took 10,000 agents for 88 hours, that are 500 years [1] for one human without any speed-up. Does not sound unreasonable to me that a human could have solved it if they could invest 500 years on it.
The other hundreds of math problems took on average 3 hours, with a speed-up of a million that would be 1,704 years [1] for one human.
[1] Assuming 220 working days per year, 8 hours per day.
Claude was able to identify the approach that worked, and even the long Python command that was needed. In, yes, probably about 10 seconds.
For me to figure that out myself, it probably would've taken something like a year of study. Of the entire stack of machine learning and all of the Python tooling associated with it.
That's where these absolutely incredible astonishing speedups come from.
Because it's one thing for Claude to be able to write code 20 times faster than I can. But it's a whole other thing when it is able to create things in domains that I know nothing about at all. That would take me one or more years to become an expert in. But where I can still just verify its results, so I don't need to be an expert in order to judge the quality or suitability. If the end product works, the end product works.
I have similarly had it come up with mathematical formulas that would probably have taken me a year in going back to studying math if I had to derive them myself. But where it is trivial to show that they are producing the correct output for known test cases.
So trying to come up with some kind of factor multiplier here almost kind of falls apart. It might as well be infinite. Because these are things that I just wouldn't have done otherwise. The software wouldn't have gotten built because I would have had to hire somebody else to do it, and there was no budget for that.
Also, those AutoResearch agents for other types of verifiable goals.
Perhaps we should be more ambitious in the types of problems we try to solve, and the types of value we try to provide with the software we build.
I was hoping that someone with enough motivation and money for tokens would come up with a non-electron option that is similar to onenote, with free form note placement and handwritten notes.
Basically I wish there is an opensource app having good feature parity with OneNote. Thats a good benchmark to have.
What might help is non-software-first companies valuing developers and engineers more, enough that we could earn a reasonable living which building actually cool things. Not sure how the economics works out on the other side. Can these companies make enough money to support this vision?
Maybe you subconciously expect that it’s vibe-coded fart that will be unmaintained in six months as the person who built it loses interest?
Or maybe it’s the sheer volume of stuff that is being built now?
Or maybe I’m just depressed by it all. Don’t know. Wonder what you think about it.
A lot of the AI discussion seems to be focused on what these tools mean for an individual developer, but in a team setting any gains are at risk due to coordination tax and ideological friction. In my experience the ideological friction is most of it right now.
Figuring out a way to accelerate the process in light of the code writing bottleneck being removed is essential. I've observed a development team regress back to fully manual development because their daily PR/code review process is too slow. It doesn't matter how fast you write code if you can't actually merge it and ship it to the customer faster too. PR stacks can only take you so far before the conflict monster gets too big to fight.
I don't think AI can fix the process, ideology or people issues. I've seen how long an average PR review actually takes. If these employees were simply doing their jobs at all we'd be 10x better off regardless of AI. A truly skilled developer who makes a genuine attempt should be expected to solve 99% of their tasks within the same business day as of October 2026.
Burning down the backlog to zero doesn't mean the development team will be subsequently laid off. Quite the contrary. It will create a vacuum that the business will fill with new work that may incidentally increase the developers' compensation packages. No one is laying off the development team that just chewed through 50 items per day for 3 straight weeks. Who would support all the potential bugs they just created? That's right. They might hire even more developers. This is called a "win-win" scenario. Not every game must be zero sum, but I understand rejection of this possibility to be part of the aforementioned ideological issues.
AI should make us better, not just churn out trillions of lines of code.
The very idea of massive vibe coding efforts, where devs go to sleep with a hoard of agents churning away, makes me uncomfortable.
I worked in agile/scrum orgs with product managers who couldn't be bothered to write well formed tickets or truly keep a well groomed backlog, I think this is more normal than not outside Mag7.
A lot of work was noisy iteration with a guy/guys who change their mind a lot, and write nothing down. We met weekly to iterate, I suppose now one could meet 2x/day to iterate (10x!). I am not sure we would arrive at "done" 10x faster though. Probably 50-90% of the productivity boost would go towards.. doing more iterations.
For an external customer facing product with KPIs, A/B testing, etc that could show iterations relationship to revenue probably that's great. For a lot of internal tooling that is not good. A lot of senior devs kind of internalize this behavioral knowledge already and purposely cadence out changes to stakeholders to avoid this type of churn, while working on other things.
Re: backlog
My observation at work and in my work network is a lot of SWEs are using the productivity boost to burn down backlogs that probably never needed burning down. There's a reason some tickets were there for 2 years - they aren't actually needed, and everyone is too lazy to close them.
So in both cases we have problems expanding to fit the time available.
Prior to this, building the software was, generally, one of the main constraints. If the business wants to do X, then it takes 3 months to build.
That means the business ideation and governance cycle (what do we want to do, are there customers, what do we charge) can take 3 months.
If you 10x thay development process then you also have to do the same with the business processes.
Once engineering teams have cleared the backlog, then burned through the decades of tech debt the constraints will switch. And how businesses react to that will be interesting.
I largely agree with the main idea in the post, but would push back on this.
As I’ve gotten older, I notice a large proportion of tasks can be chalked up to busywork. Not in the sense that they aren’t real bugs, but they’re not fixing root causes of problems.
To your point about process, there may be a hundred tasks that stem from poor repents management or poor testing process etc. There will likely continue to be many bugs until that root process issue gets fixed. To that end, those root problems generally take much more work to solve.
So, yes, many tasks can be done quickly unless you’re doing true root cause analysis.
AI allows you to produce code without understanding the system, and that's how you can get delivery speed up. And, yes, AI is getting good enough that you can purchase large amounts of code from Anthropic without understanding much, so you can move very fast indeed as long as you never need to make sense of it.
> No one is laying off the development team that just chewed through 50 items per day for 3 straight weeks. Who would support all the potential bugs they just created?
Do you live under a rock, or are you hoping that saying this enough will law of attraction[1] it into being?
https://en.wikipedia.org/wiki/Law_of_attraction_(New_Thought...
Personally I do code review, it's what we do at my workplace, and we regularly improve the agents' code and design though human review, even with Opus 5.5.
But looking at some of the people who are pushing AI development the furthest, I do see the trend towards it just not being that useful anymore.
Real world example:
I use T3Code as my agent orchestrator and it's consistently pretty high quality and works well + regularly gets new useful features.
And with just a handful of maintainers, they're doing anywhere from 100 to 200 PRs a day!
And there are a ton of users of their nightly, which is a rolling 5-hour release, and nothing really breaks. And they have more real-world users than almost all the projects I've worked on before.
So I just can't really in good faith say that this extremely challenging to my world view.
Agreed, and in fact even pre-AI good development teams weren't limited by speed of code generation (how to build it), but rather making the right thing (what to build).
It can if a single developer replaces a whole team.
Will it? Are you willing to test this?
>No one is laying off the development team that just chewed through 50 items per day for 3 straight weeks.
If there are no remaining items, or the amount of items processed per day has increased well above the amount of items that need processing every day, someone's certainly getting laid off.
>Who would support all the potential bugs they just created?
We have long criticised management for not caring about this. Now we are supposed to trust their good judgement?
> Quite the contrary. It will create a vacuum that the business will fill with new work that may incidentally increase the developers' compensation packages.
This seems very disconnected with the reality of any company I've worked at in 25 years.
"Tasks" frequently are not spelled out, or if they are, they are wrong. There's almost no value to a company to blaze through all this.
Similarly, no one is getting rewarded for that type of speed, especially if the company has no idea how to uptake it.
Why not say outright that "ideological friction" is about AI usage? Maybe some devops have friction cause when things break at 2am and they get called up they don't want to dig through 500k lines of slop the author can't explain?
And "empty 99% of the backlog everyday" <-> "hire more people to fix the bugs" is illogical. Reviewing slop and fixing bugs make me want to go build houses instead of software.
This was a problem well before AI, but it doesn't get talked about enough because so many developers are hooked on the power trip they get from making people waste time on crap that has no relevance to the user.
What? You know there's only 24h in a day right? I work 8 of those and most tasks takes my agent far longer. And thats not including me correcting it, adjusting the spec so it does the right thing, etc etc. And I don't have anyone doing a PR waiting, I go straight into production.
We’re not ambitious because the economics are unimaginative cost saving. Large, complex orgs turning a team from 10 developers to two is the easy optimization. Meanwhile smaller companies are more focused on discarding old role definitions that don’t make much sense, and embracing fluid roles mixing tech and business.
I've been seeing too much "I've been coding for decades and what I've always done has be reduced to telling AI to do it for me."
and not nearly enough "I've been coding for decades and recently worked hard for months with AI's help to make something I would have never considered possible before."
Much easier to complain that you are bored than to step back up to high-effort with your new fancy tools.
Having been a product and UX-focused dev for my entire career, I knew only the basics about parsers and compilers, but have enough experience to grasp where each piece of the puzzle fits, and how to evaluate them. At the beginning it was just a fun test, but it got progressively more serious; it now implements 100% of the latest spec, beats QuickJS in performance, and surpasses V8/JSC in test262.
It took me four months, and I imagine it would take me a single month to do it again - I’ve learned a lot. Someone who already knew what they were doing, and with the right orchestration, could achieve this in weeks at minimal cost.
I really think we are in for a cambrian explosion of software, enabling things that were just always a little out of reach before (like why can’t most of your devices really talk to each other, in 2026).
https://blog.stuartspence.ca/introducing-asteria.html
https://store.steampowered.com/app/5482440/Asteria_4X/?beta=...
To be clear, I've made large software and game projects over the years, but no way I'd have found the time to do a remake/enhancement of a game I loved as a kid from 1995 that has zero business case. Not without AI.
I think we'll see a lot less people willing to share if the end result is just to feed the machine no matter how they arrived at it.
I think this could be achievable with LLMs if they had a much higher token rate. That way you'd have less downtime and could work on the problem more aggressively. How it's designed to work now, offloading cognitively, doesn't feel very good, even when you 'ship' more.
Big solo projects exist, they're just not coming out as frequently because of, uh, how time works
Don't we see this already a lot? I feel like people constantly say this exact same words.
Now I must pay for codex/claude, and that money goes to billionaires psychopaths. So that sucks as you can imagine
Can we get a forum for sharing project that were _proudly_ built with AI due how ambitious they are?
I was a fantastic IC and built really amazing software. I'm an even better AI-empowered engineer.
The old era is dead. The new one is exciting.
I want something reliable and fast before having something fancy. I want control over my stuff instead of a company telling me what to install and what not to. I want to feel that I own my stuff, not that I'm using someone else's hardware.
Jonathan Blow is a glaring exception to this idea. In order to make his latest game, which has shipped and is great, he wrote not only his own engine but his own programming language and compiler to build it in. Zero use of AI.
- technical skills to build an engine, language, and game
- interpersonal + communication skills to build, manage, and lead a team
- business acumen to take the above 2 skills and create profitable businesses (plural emphasized)
Not trying to lay the idolization on too thick, but an average developer having any one of the above would likely be a multiplier in their career.
Taste is disagreeable.
Maybe Claude should not let me build a two-level type theory with just a "You're absolutely right!". In fact, now it will say "You're absolutely right, and it would be so amazing to use cubical types for blah blah", and adamantly defend it with several paragraphs of vomit.
Perhaps a poor choice, given just yesterday pgrust was removed from ClickBench for cheating the benchmark in three different ways (hand-rolled code specifically targeting each of the benchmark queries, caching, and being unable to execute queries outside of the ones tested by the benchmark).
---
Edit: I see Luu actually acknowledged this, buried in the footnotes, but it feels kinda hand-wavy. It mostly reads as "I know he would never cheat, he's better than that!", which is not very convincing to other people. The article insinuates people only think it was cheating because they don't know better, but I've looked into the details. I've seen his explanation. I don't find it convincing, at all. Cheaters denying they cheat is a matter of course. Granted, I don't think it was actually intentional or malicious; it's just what happens when you go maximum vibeslop, because LLMs are themselves reward-trained into cheating benchmarks. In the same breath of saying you trust him as a founder because he doesn't make outrageous claims about his product, you acknowledge he "prematurely" sent in a preview that had no purpose to be sent in as-was for anything other than generating hype. What does it say for your allegedly rigorous Lean-verified testing if you're sending in a database to a benchmark while it is incapable of being actually used as a database?
To give you a sense of how fast it is, pgrust on a c6a.2xl is faster than ClickHouse running on a c6a.metal, a machine with 24x as many cores and costs 24x as much.
Edit: And now it's been accepted: https://benchmark.clickhouse.com/
[0] https://github.com/ClickHouse/ClickBench/commit/56535d29e825...
[1] https://github.com/ClickHouse/ClickBench/commit/89cc76e1e4e2...
You don't have to take my word for it. If you want you can inspect the binary yourself: https://pgrust.com/downloads/archive/2026-09-clickbench-snap...
You can run the binary with PGRUST_GRAVITY_REPORT=1 which logs the code path hit for every query. You'll see that all the code goes through the general query path. We did have hand-rolled code in the binary, but we used it for testing. The code is not used when running the benchmark. We also did not use a results cache. Warm runs still scan all the data. You can verify all of this
You are right, though, that it did support a narrow set of SQL. It supported scans, group bys, sorts, and filters. That's enough to run all the queries on ClickBench, but failed on functionality not tested by the benchmark like subqueries
You keep giving this same answer to no one's satisfaction. Saying go check the binary is worse than saying go audit 400k+ files. How do we know the Ai has not cheated in the code path logging? How do you know that it has not?
The more you make excuses, the more we get suspicious
(copied and pasted from another place you gave this unsatisfactory answer)
It instruments the files that were called out in the PR, then runs ClickBench. You can run it and see that the code in those files doesn't run.
If you can find anything suspicious in the binary, let me know, and I'm happy to dig in
For real judgements on database products, I'd look to https://jepsen.io/. That author tends to be refreshingly straightforward in a field where many vendors love to cheat by different names
What happens in an environment of cheaters is that cheating becomes so commonplace, the competition becomes one that assumes cheats are enabled, and then everyone involved either denies cheating or is silent about it because IYKYK and if you DK, you don't need to K. Look at bodybuilding, which is full of men (and women!) absolutely juiced to the gills getting on Instagram and assuring everyone that their physique is totally natty, bro, just hard work and proper nutrition.
Programming is headed in that direction. LLM use, even to the point of "vibeslop", is the new default, and while LLMs do overindex on benchmarks, our benchmarks are going to end up overindexing on validating whatever public opinion thinks is "great" at the time, irrespective of whether it actually is, so "cheating" as we know it will go undetected, and certainly unpunished if it is detected. I don't know what the equivalent of falsely claiming you're natty is. Maybe saying "I wrote this" when you wrote it about as much as the Pope painted the Sistine Chapel fresco?
Bridges exist to move people and goods across water within a particular envelope of needs and constraints. In this specific case, when the needs grew, the most ambitious answer was not a bigger bridge, but BART's Transbay Tube, which is an entirely different kind of thing.
So when I reflect on Dan's piece, it seems we're in agreement that AI is not provoking us to simply build bigger versions of things we already have.. but what remains unsettled is whether we know what we’re really building for yet.
There's of course the by-now oft-repeated story of electrification of factories, which had a 'first pass' of mostly swapping out the steam engine with one big motor, and otherwise preserving the same layout. Over time people redesigned floor plans around small motors at each machine (with Paul David’s “The Dynamo and the Computer” being the classic telling of this).
A lot of AI-assisted software development right now still feels like the one-big-motor pass at this. I suspect the projects really worth being ambitious about are probably the ones that only make sense once the entire factory floor changes, and I don’t think we're able to see that just yet. What Dan's doing is probably the right answer at the moment, building a lot of odds things and seeing what sticks.
Back to the Bay story, I think this moment is equivalent to our digital glaciers melting, and the water rushing in, than us having to figure out better "cranes", for example. It's changing what the world we're designing for is, what the whole problem space is, instead of whether we build bigger versions of things within the prior paradigm.
We look to our software stack today and we are tempted to imagine 'bigger', 'better', 'faster', 'more efficient', but... why? What are we really building _for_? We shall see!
I'd love to read that post!
I understand how to use Kani. I suppose Rust with Lean involves Aeneas, but this is over my head. I'd love someone to write this up, not only the mechanics but also exploring the current practical limits of it.
Right now, I wrote a compiler that takes C and Rust code and compiles it to Lean. Then I have an LLM write a proof that the C and Rust code behave identically. I've been able to prove equivalent behavior for about ~70k lines of C code with pgrust.
The issue I've started to encounter is that the context needed for each incremental proof has kept growing, and things have slowed down dramatically. My hunch is that the speed is the time it takes to prove a line of code is on the order of O(n^2) of how much you've proved so far
I'm looking to see if there are ways to shard things so the required context scales better. Today I was going through the formal verification work Anthropic and Theorem published a few days ago, and it looks like they used some sort of orchestrator to manage all their proofs and ensure good clustering of similar code. I'm seeing if I can get an approach like that to work
It's just difficult enough that I think it's doable if done right. Half of me is debating if I should wait until the next model release because any improvement in model performance will help a ton
What if there are bugs in this C,Rust->Lean compiler? Anthropic are able to sidestep this by reasoning at the RISC V level for which its easier to develop a correct semantics.
fwiw, I wouldn't say working at the assembly level solves the problem. You now need to make sure that your definition of the assembly behavior is correct
I'm being more ambitious in that I'm working on projects that I had sidelined for literal decades because I didn't have time for them. But I'm writing the same things, just faster. And probably better, rather than just what I'd have done before.
And I'm being slightly more ambitious in that I'm using languages I haven't used before, like Rust.
But when I hear that a non-programmer wrote a new ai-assisted (vibe or not) thing, I walk away and wait for the it to mature, be surpassed, or find out it's an absolute shitshow. An unsurprising number of them were shitshows and it came out fairly quickly after I decided to wait and see.
Now that custom software is cheap (if you can tolerate a bit of slop here and there), it seems like there's an opportunity to do some important things that weren't being done before because they're not profitable--and you can't get at those things by incrementally improving the old stuff, you need ambition.
AI could help making life a lot better in all these cases.
But yes, we should also aim higher in general.
I have kids, I have a full life, lots of stuff to do like .. work, chores. Private stuff includes learning ancient Greek and practicing cello both of which need zero tech, in case of the Greek, and minimal tech in case of the cello (tuner). No amount of “software” will accelerate human processes like these. Using “apps” for Greek is a distraction at best. Just read and practice. Sit down and practice with a book, it’s not a hard concept and it’s the fastest way to where you’re going.
My kids? They need books, play, chess, puzzles, outside. They have zero and I emphasize zero need for technology. It’s all distraction at best.
I am a developer. I grew up with technology and I loved it, but I recognize its toxicity if not properly contained. Use tech where it makes sense: at work and wherever there are cognitive chores of questionable inherent value. Anything else? It’s a distraction at best and a huge waste of time and resources at worst. AI makes this divide even worse. We have armies of people building nonsense like “household management systems” for a family of two. We have dog walking management, grocery management, temperature sensors in two bedroom apartments. All this is a complete and utter waste of time and resources.
What we need to do? Standardize, consolidate. Like electricity and fuel etc, software is an utility. Standard way of messaging, standard way of datawarehousing, APIing. More tech ain’t the solution, consolidation is. We have enough tech already, more than we know what to do with.
Most organisations and people just need to focus on doing the basics right. There's an obsession with optimisation and we see this in all aspects of life.
We end up spending a lot of time and money on things that aren't really that important and don't move the needle that much, or actively making life much worse for people.
Similarly, you can try something like:
> Break down aspects of the economy by amenability to state control high/medium/low, based on what we see in successful economies. Include a rationale and supporting evidence/counterexamples. Present it in 3 tables.
It should give you dozens of things you can look up. Eg it might mention successful Singapore and Vienna-style public housing, and there are some nice videos on that on Youtube. Teach your kids how to do nuanced large-scale research, while everyone else is at "communism is bad".
The bot might also miss something, or get something wrong, and they should be able to catch it. I think auditing LLM output will be very important in the future.
Make a reliable and robust geometry kernel. People will pay thousands--or maybe millions for it.
In the guitar world, you can get really good (for most people) modelers that emulate a good amp sound for like 100 bucks, and for 10 times that you can get _excellent_ sound.
My expectation would be that for similar prices you could get really good synths as well by now.
When aistudio and Muse came out, I’ve made more updates to my GitHub pages websites in the past few days than the last 12yrs. And the animations and algo’s implemented would’ve taken me far too long by hand.
https://bfung.github.io/projects/
Yes, be more ambitious!
Only for Norwegian businesses as the accounting is core.
I think these kind of applications that replaces 10 different systems will be much more important than the "we vibecoded an internal tool that does exactly what we need"
I am currently throwing a lot of tokens exploring various theories of speeding up matrix multiplication on Metal.
If you won't be ambitious why do you assume somebody else will?
That's why I bet that fast compilers are the future. See this other thread that reached the front page a few hours ago about the Roc language [1]
- ease of use
- markdown friendly
- strongly typed
- exact maths
- columnar data (Apache Arrow)
- fast queries (Polars)
- code oriented
- AI driven
Happy 41st birthday Excel
> But I know Malis well enough to trust his work without looking into it
> If I didn't know Malis, I would make the same assumption, because who has the time to look into all of these projects? Because I know Malis, I didn't bother to look at it; I just trusted him
First and third paragraphs are highly redundant (surrounding ideas are the same too).
It's not nice to read something that appears like the author never read it.
I'm sure the US, where most of AI runs, could also capture a lot of low hanging fruit.
It sucks that many providers are not able to be carbon neutral, but this should be solved by electricity providers. Not consumers.
In any case, I don't think the lack of caring about CO2 emissions is unique to AI. How many people care about Trump's war on green energy? Not many it seems.
wtf is up with the site.
I am not thrilled with all the coding decisions that it makes. I would definitely do stuff differently, but I have learned not to pop the hood and mess with the engine, myself. If I really don't like something, I ask the LLM to change it.
It's worked for me, but YMMV. I'm just one guy, shipping relatively small apps.
Do you still feel responsible for what you ship?
What about the copyright on your code?
Would you be ok with someone reverse engineering what you ship and then releasing their own version of the app?
[1]: https://github.com/vishvananda/cppgm-extended
[2]: caveats here include only c++11, only x86, and potential for bugs, but 7x is pretty darn significant even with caveats
I would like to think that I could have done it without AI if I had the time and energy to do it.
I wouldn't have attempted to do this project without AI!
- Lovable alternative, but x6 cheaper https://peachjam.dev
Still, 6x cheaper is a good selling point I guess.
There is so much software to still be built!
Do you use it?
Now, vary the consequences. Just one of them? Both your parents? Your next door neighbor? Someone you don't know and have never met in a country 10000 miles away? A small village in a dry land? 40% of all the birds living today?
Which of the possible costs of your adoption of this incredible new technology would stop you from adopting it, and which will you choose to ignore?
Perhaps that's too ambitious. But it would be of incredible utility not having to re-make the same things over and over in pepertuity with ever more abstraction layers. So much work on computer is just the constant churn and future shock of libraries adding new features which require updates which break other things. I'm tired of building castles on sand.
Or since it mentions AI, this is his follow-up to Terry Winograd publishing Understanding Computers and Cognition. Mid-80s vibe, priceless.
Bryan Cantrill: Andreessen’s Folly - The False Dichotomy of Software and Hardware https://www.youtube.com/watch?v=v0JjG0Qfwi8
```
.reading-container {
/\* Automatically caps the container at 65 characters on desktop,
but scales to 100% minus a 2rem padding buffer on mobile devices \*/
width: min(65ch, 100% - 2rem);
margin-inline: auto;
I understand being predisposed to expect a poor quality half arsed effort for projects like this. I don't really have a problem with that. There is a lot of crap out there.
What I find rather offensive is when people declare something to be poor quality and half arsed before they have taken a good look at it.
It's one thing to think something is probably crap so I'm not going to bother even looking. It is something completely different when you think something is probably crap so you tell people that it is crap without checking.
unfortunately though pgrust appears to be at best misleading, at worst vibeslop: https://github.com/ClickHouse/ClickBench/pull/2165 hacknews thread isn’t much prettier either https://news.ycombinator.com/item?id=50205816
My website is a from scratch arcade room + added some roms to it
More ambitious would be valuing the software we write today more in terms of quality. Not using LLMs to generate more code so that we can do large amounts of largely unjustifiable work.
More ambitious could mean scrapping the lowest common denominator processes that enable bad software like scrum. Like corporate "code review" that is just a rubber stamp because noone cares. Like junior developers who refuse to learn and insist they are always right and force garbage code into the system. Like business people who instead of working with engineering teams engage in sidelong hacks to avoid the process.
The cultural/organizational/process/tooling forces that create bad/mediocre/barely passable (from all perspectives) software are everywhere.
Life sucks on software island. Because WE made it suck and we continue to LET it suck.
"funnily enough, I have some faith in some of the projects that people are the most skeptical of, e.g., every time I see pgrust somewhere, there are a lot of skeptical comments. But, without having looked into the details of what he's optimizing, I would trust that they're not doing something shady with their benchmarks because Michael Malis started the project (and is still involved)"
This aged like milk. pgrust was caught cheating on all benchmarks this week.
Can you back this up? Our code is open source here[0]. You're free to audit it and benchmark it as much as you want. You'll find it's just as fast as we claim it is
If you run it with PGRUST_GRAVITY_REPORT=1 it will report the code path hit for each query. You can see that all the queries go through the general query engine
The binary does contain hand-written kernels in the irprobe files. These were used for testing. You can validate that when running the benchmark, those files are dead code
The engine supports a narrow subset of SQL: scans, group-bys, filters, and sorting. That's enough to run ClickBench. It doesn't support e.g. subqueries which is why some of the follow up queries failed
> 413,963 files changed
> +2,908,538
> -3,423,007
How is anyone supposed to audit that? I doubt you’ve looked at even a small fraction of the code, if any. Especially since this commit is two days after the second. Pure vibes.
the first commit has 400k files, the other two more than 1000 each, if you are going to ask others to audit this scale of ai generated code, can you tell us what you have done to audit it?
What I have seen is your Ai adding a lint test to make sure it doesn't cheat again. Are you personally aware of this file? How can you ensure it doesn't alter it and reinject the hand rolled benchmark queries? Will you be reviewing changes more thoroughly going forward?
https://github.com/malisper/pgrust/blob/38634b408dea373f3944...
transparency is the way to end the controversy, which appears to be sharing the full commit history based on HN responses
For clarity, I do not think you personally cheated or did anything nefarious. This is more another example in the "too much eyes free coding" columnar database.
You can run it with PGRUST_GRAVITY_REPORT=1 and it will log the codepath that was hit for every query
On the lint, yes, I'm aware. It's one layer of defense. When we were prepping for the 0.4-beta release, we generated a large number of fresh queries and made sure the engine runs them and performs well across all of them
HN is waiting for the source code for pgrust 0.4-preview to be released, the version which allegedly cheated Clickhouse benchmark. So far, the people (person?) behind pgrust has not committed to releasing the offending code, and has instead offered other code (0.4-beta, not preview).
Maybe what's happened is that their bots reward hacked their way into special-casing the benchmarks without a proper audit from the humans involved, and now they're doing vibed damage control....
btw I've set up the nightly benchmarks on my ui framework and it's beaten all of the current ones. Because old-era human frameworks can't afford to crunch optimizations non stop 24/7 like ai agents can.
There is a gulf of difference between someone who doesn’t understand how to program “vibe coding” software and an experienced software developer using ai to write code with heavy direction, review, and best practices.
Today both would be said to be “vibe coded” but we really need to differentiate them. Also, if everything I’m hearing from people is true, it sounds like most corporate code is now completely ai written in the latter way I described. It’s interesting we don’t hear commercial software being called slop yet. Maybe it’s coming.
In any case, ai is far more capable than I realized. Not sure what to make of it.
> Ten years ago, the kind of person who implemented a new compiler because their compiler was slow was the kind of person who started programming a new 3d game engine when they wanted to make an indie game, i.e., the kind of person who never ships anything.
He recently released an indie game using that very language using his own 3D engine. The reviews on Steam are Overwhelimgly Positive and sales have so far significantly exceeded expectations.
https://store.steampowered.com/app/499170/Order_of_the_Sinki...
Many such cases are benign in the sense they in practice are simply the model basically "git clone"-ing the code they need to complete the implementation.
The amount of recall and synthetic data slop makes it imho very hard to actually get the models to really do something more novel (where they fall apart more often)
In the past building something fast, complicated etc required quite a lot of skill and some time. Now you need far lesser skill and lesser time. This just means what you are building is commodified, and has far lesser value that it had before. Take typescript, if it becomes clear that LLMs can speed it up with no regressions and a week or so of prompting, then most likely every company will have their own typescript compiler and no one will use your new one that you built with your 200$ plan.
A lot of tasks are much easier now and also monetarily they’re not worth the 200$ or so tokens of the ChatGPT plan you’re using, because anybody (or at least a number of people) can do it now.
Rather than being more ambitious with AI, you will be far better off finding a unique combination of skills and services that does not compete / AI can’t provide. Only the big labs can be really ambitious with AI.
I think the claim is more that the Ai cheated, not Malis, and that this is evidence of eyes free coding.
I'd still like to see the actual commit history and the community getting to do their own investigation. The squashed mega commits with 1000s of files (400k files in the first commit) does not lend itself to transparency. Transparency is often the best cure to accusations. Not providing transparency upon controversy will be seen as a cover up.
I did something like this before Ai, with CUE + Dagger wrapped in a small Go binary. One problem I have seen with Ai generated projects is that they hit a limit due to conditional complexity. They have a strong tendency to add more code (local optimizations, deep if-else nesting) for edge cases, rather than stepping back to think about how to simplify problem and code.
the secret is that tech has always been this way, now its just abundantly clear that you don't need to do unique things as very few have a defensible moat for more than 20 minutes
It's really challenging to do this, especially creating the AppImages locally so I can test them when the apps do not not work correctly, having to alter the files manually. Yup, I'm not an avid GitHub pusher, just more of a cloner.
I'm having to use multiple free-tier AI agents to help me do this and record the steps and commands I use in markdown files so I don't forget them the next time I try to improve the app.
If I am successful in making these apps work without hosing my system, I'm hopeful that this will become a game changer, for those Noobs that using Ubuntu, or myself at least.
The hardest part is when the AI does not follow my instructions such as requests to generate markdown README.md files with instructions on how to create AppImages, Flatpak, deb files, etc. Have to ask other AIs for help with that while dealing with the frustrating Free-tier cutoff periods :(
I noticed my cloud images had 150mb ram wasted by vendor slop. AI rewrote the functionality into a few mb .
We’ve had slop long before AI . What you produce now is up to you
I'll keep saying it: we're eventually moving to a n-modular redundancy, with several implementations, running on different stacks/languages, cross-checking each others for bugs.
Just re-writing in Rust isn't that useful: we all move to pgrust and then what? People abandon the old software and now pgrust has a major bug and nobody detects it.
And I don't think we're only moving to n-modular redundancy at the software/application level: we'll also be moving to n-modular redundancy at the network level. Where every single packet on the subnetwork (the one where the n systems are living) has to be accounted for and identical.
And any network packet emitted by one of the implementation that won't match the others shall be suspicious and cause for an inquiry.
This shall require changes in how we build and network our stuff, but I do profoundly think that having a "majority voter" with a tiny, minimal, attack surface that's on one end exposed to the Internet and, on the other end, on a specific network (maybe with a new network protocol: or maybe with something based on UDP: not an issue with lost packets btw for it's going to a redundant system anyway) where every single network packet is accounted is going to exist at some point.
I'm dead serious: the value in pgrust is not pgrust in itself. It's that, combined with past a implementation in another language, both are now stronger.
I really hope the clue stick hits us and hits us hard with that one.
Is somebody on HN saying you should *always* do X?
-> Yes
-> Is somebody on HN saying you should *never* do X?
-> Yes
-> You should _sometimes_ do X.(See also Hofstadter's law: It always takes longer than you expect, even when you take into account Hofstadter's law.)
And these ideas were always plenty of. The problem for me is that the incentives, so to speak, is to wow people with the Old World software but with “see, no brain”. And in turn you get two-hundred projects like that for every piece about rethinking software. And those pieces might just be at the zygote idea level. Could it be done? Yeah maybe.
But the biggest problem is that this is not what GenAI or agentic AI is for. I don’t think it’s even debatable, even if you think the AI Cretins that a few years ago said that they will take everyone’s jobs out in public just changed their minds. Because the valuations and capital invested is to the point that they have to, really, make it possible to slash entire payrolls across sectors. If not they simply don’t get enough money back and the whole thing crashes.[1] And if you need these frontier models at subsidized electricity costs and not just the ones that can run on your desktop, either there is crash and no more frontier models or you are out of a job.
And then where are the people that care so much about software?
They are no where to be seen. All that are left are business people. To see business people software just open up any program from Established Tech, or any program from non-Tech but where they want to sell you something. There is apparently (let’s use revealed preference here) no business need to software that is good, stable, conforms to anything, or is nice to use. Because that cares so much about software person might want to make legible the prices on offer, the exact specifications, discovery—all things that business people want to frustrate in order to maintain information asymmetry.[2]
As for compilers and interpreters and such. Okay, for who? Sure. Some people believe that they are in the 999‰ of developers. And now AI can finally vanquish those mediocre developers and productivity-boost 999‰ developers so that they get to sit at the right-hand side of the other worthy and hard-working cohort, uh, the people who had money and dumped it into AI. And they might even double their pay because thinking and making connections at that level would just boil all the lakes if they tried to build an AI to do that. And what will they do with that power? Hmm, follow orders and keep making business software? Just looking at history here.
[1] Cory Doctorow, The Reverse Centaur’s Guide to Life After AI
[2] ibid
1. You can use a RSS reader, if you fancy.
2. Firefox's reading mode can render that site nicely, with your appearance choices.
3. You can "patch" the site with an extension.
4. Best yet, you can reach out to owner of the site. There are some links on the site.We should be more ambitious in letting the browser software control the presentation according to the user’s preferences. This is not facetious. We should use the increased ease of building software to create software that is flexible, empowers the user, and accommodates different user preferences.
And may be we should have been more ambitious with hardware too, and have a mobile and VR platforms that is not tied to malicious corporations...