OTOH once you as an outsider know that sometimes the AVP is lying to you about where the wearer is looking why would you ever trust it?

For example, you could then use the AVP to stare at people and then claim afterwards you were doing no such thing.