[1]: Purely technical. There may be social reasons or legal consequences like "disparate impact" theories coming at you.
But there's plenty of people with high intelligence and problem solving that aren't good programmers, so we probably want to try to hone in on that a little bit instead of picking something with positive correlation.
There are also tall people who are not great basketball players, and vice versa. If you can't do anything but measure height, maybe you should pick for your team that way... but if you aren't so restricted there's surely better ways to go.
Leetcode is popular because:
1. It gauges problem solving ability and pattern recognition, which are essential to any job, and particularly useful for software engineers.
2. It is language agnostic - most jobs would rather have a strong engineer that doesn't know their particular language/tech stack than a weak engineer that is familiar with the stack.
3. It has objective evaluation criteria (correctness + time/space complexity).
It's certainly not perfect, but I have yet to see an alternative that accomplishes all of the above. The one thing I can see potentially complementing it is the new style of AI-assisted assessment where you are dropped into an unknown codebase, and asked to make fixes/features that are assisted by an AI agent. This solves the language-agnostic issue with pre-AI versions of these interviews, where you had to have memorized specific tech stack features or language syntax in order to accomplish the task. Now AI can take care of that stuff, and the interviewee can focus on the problem solving aspect.
Proof of claim required. Leetcode primarily measure's someone's ability to memorize leetcode problems, and secondarily measures their ability to vomit this knowledge up in a stressful situation.
There's an entire test-prep industry surrounding this interview method. Any undergrad will tell you that the way you pass leetcode interviews is to "grind". You'd have to be oblivious to not realize that the system has been gamed to hell and back.
Since you are just asserting things, I will assert a simpler explanation: the primary reason Leetcode is popular is because it is cheap. If you have the problem that you have too many applicants and don't want to spend money interviewing them all, you have to come up with an efficient filter that doesn't violate hiring laws. You don't particularly care if the false negative rate is high, so a test of superficially relevant trivia is what you do. Even better if the trivia can be validated by computer!
Setting aside the fact that almost no companies actually had this problem outside of FAANG, thanks to AI, the approach has gone from "somewhat relevant" to "completely irrelevant" within a couple of years. It's time to take it out back and shoot it.
I’m sure the companies know that Leetcode is not a great indicator for likelihood to succeed in the role. If they had a perfect information about people, then would stop interviewing at all and just take the best people who applied.
They don’t have perfect information. The information is highly asymmetric. The companies are just trying to not be efficient and not get burned.
It sucks for candidates. But it makes sense for companies when they thousands of applicants they know effectively nothing about.
Well, no. It rewards ability (and free time) to repeatedly work through problems and memorize the solutions. And it rewards acting skills so you can perform like you're discovering solutions on the fly, that you've actually memorized.
That's all it does.
What you really want to weed out, at least at top-tier companies who can afford to be more selective, is people who have a resume, but are not that great at fundamentals. Most loud complaints seem to be of this nature: guy had N years of experience at previous company/is author of popular software, but couldn't get through basic linked list/recursion/BFS--as if this is not an acceptable bare minimum expectation of a professional in the field.
Often people are too polite to say it out loud, but realistically, that is exactly when it is working as intended.
It's basically just a grinded leetcode test.
Literal psychologists have done the research on this: A good, predictive interview is a small representative test of what people do in the normal job. Only us techbros seem arrogant enough to want to be the exception to decades of research.
Well then. If the "classic fallacy" is that job interviews should reflect the actual job, then what does it really matter what we do? Let's just have a big pub quiz. That measures "intelligence", too.
> You could test people on random cognitive tasks and predict their job performance.
Show me a single piece of evidence that memorizing code golf answers from a book is a valid proxy for intelligence, and we'll talk.
(Hint: You'll have better luck convincing me that it's a stealth way of discriminating against older, more experienced applicants who are further from undergrad and less willing to re-memorize Cracking the Coding Interview for the 150th time. Because it actually is.)
In any case, I do not understand this...Stockholm syndrome...that some programmers seem to have for the idiotic interview system.
I hadn't actually thought about it this way, but it makes sense. I'm in my early 50s and hella experienced and talented (if i do say so myself), but I refuse to do any interviews that require me to do any more preparation than researching the company I'm applying to.
It's obviously absurd, and while I almost understand it for a company that doesn't know you, you'll get hit with this stuff when you've been referred internally, by people you worked with in the past.
There is a century of data and studies that suggest cognitive ability is positively correlated across tasks. Conducting IQ tests for hiring is practically illegal in the US. Leetcode is the closest you get to an IQ test without tripping some laws.
Please use your intelligence to see the argument I am making, instead of nitpicking the example.
> Conducting IQ tests for hiring is practically illegal in the US. Leetcode is the closest you get to an IQ test without tripping some laws.
No, it isn't. Disparate impact is. Any test that has a disparate impact on a protected class is de facto illegal, but there's case law surrounding IQ tests.
By the way...ironically enough, Griggs v. Duke Power established that any intelligence test must have relevance to the job responsibilities, which sort of blows a hole in your entire theory of the case that this is a "classic fallacy".
I think you have to let this one go. SAT, LSAT, GRE, ASVAB all correlate well with IQ scores. Leetcode should too, after the interviewee does a few weeks of practice (just like all those other tests).
I mean, you're just making stuff up, but ok.
Leetcode is just one step away from that (and much more litigation-proof).
Leetcode is not a general cognitive test, obviously.
I think I'm gonna get a stroke reading this. The classic fallacy is the nonsense you're spouting — that there's anything besides the job itself that is relevant to it. You're hiring people for x, so the only thing you care about is their ability to do x. This is obvious.
The perfect test is just hiring someone on the spot and seeing how they perform. We don't do this because it's expensive but obviously it would be the perfect test, measuring exactly what you're hiring for, by definition.
So then any test that goes further and further away from that ideal becomes worse and worse at predicting job performance. The trick is figuring out a good enough test that's still reasonably quick and cheap to do.
Deliberately going out of your way to test your canditates for something unrelated to the job is insanely stupid.
It's essentially just arrogance. Arrogantly asserting that you know for sure your particular "generic" test actually selects for ~all traits you want while also rejecting ~all traits you don't. I assure you, the chances of you having a good enough model of reality to do this are laughable. Pure arrogance.
Again, easiest to see by thinking of the opposite example to before — the worst test possible would of course be something completely unrelated, so idk, having a chef play ping-pong. I'm sure you could convincingly argue it tests something, but why in the world would you not just have him make a dish, since you know, that's what you're hiring him to do? (it's not actually by the way, now that's a classic fallacy — that the job is the mechanical subset of it and not the job as a whole. Soft skills, anyone? sigh).
I've seen someone suggesting that since a lot of programmers like playing Factorio, why not interview people by having them play Factorio? Ah, because then you're selecting for Factorio players. Do you have a good model of what makes a good Factorio player? Hell, do you have a good model of what makes a good performer at your job? You absolutely do not. Why select for Factorio players when you can select for your-job players?
Another common example of this utter arrogance is thinking that a candidate's performance during a highly stressful, unnatural, and artificial classic interview grilling environment is indicative of their perfomance in the every day feeling-secure-in-their-job state of mind. Oh, look, this fool I'm basically abusing from my position of power is stammering and squirming, must mean he's a good-for-nothing loser! Hah, another fraud found!
So sure, keep putting people in as much stress as possible and asking them how many golf balls fit in an airplane. BTW, I walk out of such interviews immediately. Have fun with your new Leetcode grinder performative actor hires though — you get what you test for.
Multiple trillion dollar companies has not been that bad.