A word is the unit of addressing. A 36-bit machine has 36 bits of data stored at address 1, and another 36 bits at address 2, and so forth. This is inconvenient for text processing. You have to do a lot of shifting and masking. There's a bit of hardware help on some machines. UNIVAC hardware allowed accessing one-sixth of a word (6 bits), or one-quarter of a word (8 bits), or one-third of a word (12 bits), or a half of a word (18 bits). You had to select sixth-word mode (old) or quarter-word mode (new) as a machine state.

Such machines are not byte-addressable. They have partial word accesses, instead.

Machines have been built with 4, 8, 12, 16, 24, 32, 36, 48, 56, 60, and 64 bit word lengths.

Many "scientific" computers were built with 36-bit words and a 36-bit arithmetic unit. This started with the IBM 701 (1952), although an FPU came later, and continued through the IBM 7094. The byte-oriented IBM System/360 machines replaced those, and made byte-addressable architecture the standard. UNIVAC followed along with the UNIVAC 1103 (1953), which continued through the 1103A and 1105 vacuum tube machines, the later transistorized machines 1107 and 1108, and well into the 21st century. Unisys will still sell you a 36-bit machine, although it's really an emulator running on Intel Xeon CPUs.

The main argument for 36 bits was that 36-bit floats have four more bits of precision, or one more decimal digit, than 32-bit floats. 1 bit of sign, 8 bits of exponent and 27 bits of mantissa gives you a full 8 decimal digits of precision, while standard 32-bit floats with an 1 bit sign, 7-bit exponent and a 24 bit mantissa only give you 7 full decimal digits. Double precision floating point came years later; it takes 4x as much hardware.