This morning I asked GPT which models and settings I should use to inspect my VM dashboard with computer control. It said: - GPT-6 Astra at Medium - GPT-6.1 Sol at Medium (alt) - use Luna only when the route was already known.
But I had already completed the task twice with Luna and it was 5.6! It navigated through a few pages of my dashboard, pulled up the 30-day usage chart, analyzed my CPU and RAM, and explained why I didn’t need an upgrade. I’d give Luna 5.6 an A on this task.
Maybe OpenAI should give us a “Tiptronic” selector: recommend the model and reasoning level based on the prompt, while still letting users override it.
What are you talking about? If it's for some slop you vibe check it. If it's for a serious effort you write an eval. Same as it has been for the last 3 years.