The collection act originally was intended to apply to merchandise and merchant ports. The concept was judicially expanded upon in 1925 but wasn't fully ensconced into federal law until 1952.
At actual border crossings, the practice at the time of the framers was that warrantless searches/inspections at border crossings were normal and permissible.