Password isn't used directly as key. The key is derived with PBKDF.
>The key is derived with PBKDF.
Which doesn't do shit even if you're using argon with 32 GB memory usage or whatever. NSA is always going to be able to out-compute you. Not to mention you need a difficulty level that's usable for daily use, but the NSA can brute-force at their leisure, possibly decades in the future.
The NSA is only going to spend so much time trying to crack my specific password on 100 billion dollars of hardware. If I have a 6 word password (barely harder than XKCD) (72 bits), .1 second local hashing, and their computer is ten billion times faster than mine ($10 per my-computer-equivalent), in a month they'll get through 0.006% of the search space.
The "decades into the future" part is a much bigger threat than their ability to out-compute me in the short term.