Of course instructing agent is programming; the agent is an interpreter (albeit in some cases nondeterministic, but that's true for most programming languages that have a function to get current time) of the natural language. How exactly it is interpreted is given by the weights, the context, and possibly the random seed.

But even if we write in Python like foo.bar(), the interpretation depends on what is in module foo and function bar, how they are defined contextually.

You can try to find some difference but most modern AI agents come more close to a language interpreter than not, if the actual instructions are given.

OTOH, giving instructions to people is not programming. People do not consist of binary code that is reliably executed; they are biological organisms subject to environmental pressures such as being tired. They have their own consciousness and will, and can refuse to follow the instructions. LLMs are programs; they cannot decide not to respond.