Hacker News new | past | comments | ask | show | jobs | submit
praccu | on: Launch HN: Discovered Materials (YC P26) – AI agents to discover new materials
Cool stuff. I first worked on ML for exploratory synthesis in 2012, and am still in related areas.

Once you have the experimental loop running, I suspect it will be quite difficult to hill climb on this task.

There will be some improvements you can make to the harness, but I suspect you'll be doing a lot of human in the loop review and providing feedback that goes back into the harness instructions.

I know it's fashionable to imagine automating the whole process, but everything I've seen is that the only systems that succeed are the ones that are augmenting an expert.

advaith08 | parent
yeah, the tricky thing about the experimental loop is that: 1. its very difficult to do it in a reproducible manner (the same experiment done twice often gives different results due to small undocumented changes) 2. its expensive to do at scale.

Both of these properties make it hard to hill climb on experiment. What's worked for us so far is precisely what you said - having human experts review and provide feedback. we distil their reviews into rubrics, and have LLMs act as proxy experts using these rubrics. We expect the models will hill climb using this approach, and will reach (close to) human expert level by doing this.