I've had a strangely asymmetric experience where frontier models are failing apparently basic tasks like making changes to a Pong clone without breaking it, while the same models are successfully designing and implementing multiplayer servers with rollback netcode!
I think it has to do with what they can and cannot verify (i.e. they can't actually play pong to see if they broke it), but I'm not sure.
(Also happy to hear anyone else's experiences on this matter!)
First, some background: At the start, I used Sonnet, but after a couple weeks, I'd switched to mostly Opus. I've used Fable when available. For any significant new feature or work, I use /brainstorming (superpowers plug-in). For Vestiges, I haven't written any design documentation per se. I'll start off a /brainstorming with a hefty prompt, maybe 5-15 lines. Throughout the brainstorming steps, I'll think of additional ideas/details about the feature, etc. I'll just add the change/addition to my answer to whatever question it had asked and it seems to handle that fine.
Sometimes I catch things in the subsequent design review process and it handles new guidance well there, too. Typically, I've left some details underspecified and Claude fills in the gaps -- often quite well, but I have to pay attention. The details that it decides later in the process (the implementation steps) are often solid, but when they're not, it's easy to iteratively fix later. Most of the time when Claude asks for clarity on a detail, its recommended option is what I wanted.
It's been rare for Claude to do something that just didn't work (but it has happened occasionally). I have found that it's not good at imagining the player's perspective; it doesn't consider the player's cognitive load for UX considerations, for example. That seems to be consistent with your observation about what they can and cannot verify. A recent instance was when I added a tutorial (which was much more scripted than regular gameplay, providing a more specific experience); I ultimately had to get very directly involved to get to the quality I wanted.
Vestiges doesn't have any gameplay involving timing like Pong does. I could imagine that sort of gameplay being harder for it to get right.
(I hope that covers some of what you were seeking!)
quackwave.com
Hasn't really been stress tested yet (this is the first time I'm mentioning it publicly), but it's been fun playing with friends & family and iterating.
Party game platform, think "jackbox but no host screen" - just need a web browser open on phone/tablet/PC.
Your comment matches my experience: it has to do what can and cannot be verified. As an example, it was much easier to have AI write a large e2e test using Playwright (then add test cases and expand) than to assume it'll correctly fix bugs without guidelines like this. Also, the human loop is still important in things like screenshot verification - but the frontier models are getting even better here so I'm not sure for how much longer this will be true. The ratio of test code to production code is a bit over 2:1 right now.