Firefox remembers your zoom settings on each site:

https://support.mozilla.org/en-US/kb/font-size-and-zoom-incr...

So no, you don’t have to set the default.