My suspicion is still that this has to do with losing insight in the little specific details of the matter and general understanding.

An area where I realized that was feature engineering: Early ML systems had handcrafted features that were fed into the model. There were relatively arbitrary and the number of features you could reasonably generate that way was tiny, compared to modern systems - but it gave you some understanding what input the model got exactly and you could use it to clear up some failure modes, or be certain that the model learned something that could not possibly make sense.

Then the idea was to automate feature generation. What's not to like? Except that in practice, the automated features simply seem to become part of the blackbox and are not available anymore for understanding.