Hacker News new | past | comments | ask | show | jobs | submit
For my specific niche (medical imaging) all current models still suck. The amount of expert knowledge required to understand the data and display it in the right way - probably never was in the training set.

We have this one performance-critical 3D reconstruction engine part, that just just has to go FAST through billions of voxels. From time to time we try to improve it, by just a bit. I have probably wasted at least 2 full days with various models trying out their suggestions for optimizations and benchmarking on real-world data. NONE produced an improvement. And the suggested changes look promising programming-wise, but all failed with real-world data.

These models just always want to help. Even if there is just no way to go, they will try to suggest something, just for the sake of it. I would just like the model to say "I do not know", or "This is also the best thing that I can come up with"... Niche/expert positions are still safe IMHO.

On the other hand - for writing REST with some simple business logic - it's a real time saver.

Did you feed back the results of the tests / benchmark to the model?

I’m presuming you have a very robust test framework / benchmark setup etc?

I’m presuming you fed the model the baseline results of that setup as a starting point ?