It helps to remember that you are competing with: no event.
If there are other parties happening and you're trying to make a better one, by all means, go all out. But mostly people in their 40s aren't going to many house events, so they're just happy to be somewhere with people. They don't care that you didn't decorate or sweep the floor or prepare an elaborate meal. You made soup and they're thrilled.
I'd also add that first-event nerves (on host and attendee sides) can be an uncertainty problem. No one wants to misunderstand the dress code, social code, etc. Once people have been together, there are now group norms that assuage that (aka "I know what's acceptable to wear and talk about").