I also have a high spec M1 and was delaying upgrading, but eventually I needed more RAM.

My M1 is still fast and enjoyable, but I regret to inform you that the M5 actually does feel faster in day to day use. I thought for sure I wouldn’t notice, but the difference is there.

When I do large builds the difference is also bigger than I was expecting.