In my experience, until you implement the math of a NN in numpy or something similar, it's easy to feel like a library is abstracting something away from you. Many people in ML focused academia stand on those libraries. They may conceptually understand chain rule but do not manually implement the math. The same feeling exists for encryption. There is a culture of shying away from rolling out your own ops, so it can easily feel like your understanding is segmented to some degree.