Also removing other housing restrictions that ostensibly were put in there by constituents is a valid reason for constituents to oppose AH. They get called NIMBYs for this but if the local populace wanted more high density development then the density limits wouldnt be there to be excepted by AH
Like I said, the “it” in “it doesn’t need to be “affordable”” seemed like it was referencing the previous “Build more housing”, so situations in which nothing is built are different. If your original intent was that not all housing policy should be about affordable housing, then we agree. But I do think it's an important part of the solution.
If people didn't want housing there, it wouldn't be built. If they didn't want the exemptions to be codified, then they wouldn't be.
The only way your statement makes sense is if you restrict "local" to a sufficiently small subset of the people (a town? A block? One single address?), but in that case, a greater number of people within a greater definition of "local" seem to disagree.
If the state gifts a locality power to impose zoning restrictions, then the state can usually alter (or withdraw) that gift when it stops being beneficial to the people of the state, even if a small subset of those people living in that one locality don't like it.