I have a few playlists on Spotify. I never had it add any track to it. Been using it for years.
Does it actually happen?
You can enable it. Either through 'smart shuffle' or by looking at the suggested added songs.
There is also 'song radio', discover weekly, and automated playlist based on mood or Genre.
I presume those playlists based on mood and Genre is what these songs have been added to.