Instead, what they are, are collections of mail delivery points. The "full" 5+4 zip code represents one mail delivery route that a postman follows to deliver mail. And the "fuller" 5+4+2 zip code has the last two digits of the street number.
So couple issues with trying to make polygons from zip codes:
1. There is no guarantee that a given zip code has a finite area. There are several zip codes that exist at a single street address. So... is that a point? Is it the footprint of the building where the mail is delivered? Maybe a hexagon with some minimal side length? Examples: Several IRS return addresses, and some large hospitals.
2. There is no guarantee that a given zip code has a contiguous polygon representation. It depends on the algorithm you use to generate polygons from address points.
3. There is no guarantee that a given zip code has a fixed geospatial representation. Example: the armed forces zip codes for delivering mail to overseas personnel on deployment. The AFPO zips represent "wherever the troops happen to be today". How do you represent that as a polygon?
This is why the USPO has long resisted publishing polygons. It doesn't make sense at the conceptual level, so there are a bunch of weird edge cases you wouldn't expect if you naively treat them as polygons.
ALSO: The census does publish zip polygons. Some of those have zero people living in them (like the hospitals and IRS locations), so you can get weird div-by-0 effects if you aren't careful.
APO/FPO is a good counterexample, though even these presumably have domestic delivery points where the postal service hands off mail to the military (not that their locations should matter to postal customers).
Same goes for DPO addresses, I assume USPS doesn't actually deliver to embassies overseas (or at least does so under contract with the State Department if it does, and again consolidates deliveries at one or more domestic locations).
I came across all this when building maps for sales territories defined by ZIP codes. I needed to have a simple, clean partitioning of the US for this use case, and ZIP codes are not that. So I took all ZIP codes as points and generated a Voronoi diagram to yield clean, non-overlapping polygons that covered every point in the country. That was plenty good for the purposes of doing visualizations of the US split into tens of territories.