The VBIOS is around 32-64k. The modesetting path is probably a few k.
And it depends on DOS configuring the memory space to leave an INT 20h call (to terminate the program) at a place that's easy to RET to.
This has always been the case, and actually inherited from CP/M.