Nvidia – open-source auto-ontology
https://github.com/NVIDIA/auto-ontologyI guess it all comes back to the fact that they don't have a clear division between data they have learned about the world, instructions and data under analysis.
You can ask them only to consider the 5 facts in front of them, but they won't do that, and cannot do that.
Depending on how you combine imperfect systems, you can lower or increase error rates.
> data they have learned about the world, instructions and data under analysis.
I don't think there really is a clear distinction between these things though.
And I don't mean this in a gotcha way - these things apply to people too. Not that that means they're like people, but we have successfully built enormous systems out of fallible people, and other imperfect things.
The goal is rarely "eliminate all errors" but "have a thing that's useful, given a possibility of things going wrong".
Computer systems, though, can have that distinction. If you read data supplied by the user, and then process it via behavior in code, and process that result against administratively-set data in the database, the separation between user-supplied data, administratively-configured data, and instructions (code) is extremely clear. Even if you are doing something that could be nondeterministic, e.g. incorporating randomness or doing sentiment analysis or whatnot.
I don't think the oft-repeated "AI is fallible just like humans are" applies here. Computer systems are an area where that separation can be achieved. We know this because we've achieved it a lot. LLMs, powerful as they are, are a massive step back in that regard. Their benefits might outweigh that compromise, but it is, as they say, "worth naming" anyway.
Well, in principle yes one system can compensate for another, in practice, errors tend to compound and combine in interesting ways, not go away, particularly in systems like this which are unpredictable and generate slightly different responses each time without human intervention. They are not reliable in the way we're used to considering and don't work well without supervision from careful humans at present.
I don't think there really is a clear distinction between these things though.
Yes your second point is an interesting one - the divisions between these things are arbitrarily imposed on the world, but we do have these distinctions for a reason, even if they are not completely objective.
I do think this is a significant problem specific to LLMs though (that I at least see when using them) - we don't have a way to constrain them in what knowledge they consider. This can be a bonus (they are good at connecting faraway bits of maths in interesting ways), but it can be a significant problem when asking for a summary of a data - confabulations and data from outside the context easily leak into the output of the LLM. That is undesirable behaviour and we have no reliable way to constrain it.
I'm not sure that this particular effort is going to help because it doesn't change or constrain the behaviour of the actual machines and the behaviour is the problem, not the data they are fed.
You can give it instructions and 5k tokens later it will do exact opposite you told it to.
I think the 'reliability add' is from how it's presented back to the user. If you can trust the agent to at least say what tool it called, then you can mark a chart as being the canonical output of "get_projected_profit" tool vs "get_realised_profit" tool and everyone knows it hasn't come down to the current dice roll of randomness what the agent itself did this time.
Has training them to do it been tried and proven to fail?
Nvidia, I get you're all-in on AI, but please at least find a human to write some copy. It's both overly verbose and entirely lacking of important info at the same time.
How about we start by open-sourcing the NVIDIA user-space libraries: libcuda, libcudart, libOpenCL, libnvtx, as well as nvidia-smi and the CUDA and OpenCL profilers.