Hacker News new | past | comments | ask | show | jobs | submit
> so if you are traveling, it will suggest best ways to travel, best hotel, etc.

The scary part: they are already doing that. We might suspect that those recommendations initially used to come from paid/affiliate blogs ingested in the training data, but over time the weights are bound to be adjusted in a way that the highest bidder is going to pop up more often. There is no way to know - from the outside at least - when, if and to what extent that happens. And it all happens under the guise of plausible deniability.

Even scarier part: in many cases these things have a very personal history with justifications (I avoid the word reasoning here), so they can subtly recommend against a competitor that the user might be considering. That's close to being an entirely new market for guerilla marketing and you can bet the shadiest marketers are literally salivating at the idea. "Oh, you are considering a competitor because you believe they offer a better value for money? Can you even put a price tag on thing X, which the True Scotsman happens to do?"

This isn’t how deep learning works. You can’t just “adjust weights” for some random user/product.

I feel like even otherwise intelligent people these days think these chatbots are Westworld-like programmable AIs and not pieces of shit that barely run or work. There is no tech monolith that’s getting advanced and gaining new capabilities. There are some very smart people who have switched from building ad recommenders or autonomous vehicles to building KV caches and reinforcement learning systems, and then in a different department there are the same people who built ads systems at whatever big tech company that will build the same shit at OAI etc.

You don't need to adjust the weights. Just have it query a vector database of current ad campaigns to find a PROMPT.md to inject when the context is relevant. e.g. user is talking about camping -> lookup ad campaign documents relevant to camping (e.g. with embeddings) -> inject prompt about the campaign. This is all basically obvious if you've been using SKILL.md for agents at work.