Yeah, that's an interesting case study. It's pretty common practice to do something like llm-as-a-judge if your programming agentic systems.

I think those are different from pure "natural langyage programming" though because there's normally code you've written that's calling out to the LLM, so it's within the context of written code.

I think this, and lots of other comments are so interesting. There are a lot of different increasingly complex ways LLM outputs and coding are being tied together by people, that I hadn't really considered.