The tech interview prep industry

Before I got into tech, I worked at a tutoring company in the Northern Virginia suburbs of Washington, D.C., doing a mix of schoolwork-related tutoring and standardized test prep.

From a business perspective, it made so much sense - you have kids taking standardized tests and parents who wanted their kids to do well on them. Here we have the SAT, a measurable evaluation that can be an immediate help to your future if you do well on it (though whether or not it’s truly a good evaluation is well beyond the scope of this essay). For better or for worse, it makes up a non-trivial part of the criteria used to evaluate college applications.

With that in mind, let’s play economist: if your SAT score allowed you better future prospects, how much would it be worth it to raise it by 100 points? Sure, we’d have to quantify those future prospects and the probabilities, but it generally seems like it would be a good thing. Furthermore, if you could pay someone to help your kid raise their score by 100 points, how much would you pay?

Even if there isn’t a direct value we can calculate for future benefit on SAT score increase, it seems like one of those things that should be true, at least. Talk about easy marketing! From a business perspective, this seems like the perfect fit; for every graduating student, there’s a new student who will be taking these tests soon enough, frequently with parents that are more than happy to pay for help along the way.

It’s a funny pattern:

1. A standardized test is developed
2. Doing well on that test is beneficial
3. People are willing to pay for help to do better, especially if it’s for their kids
4. Businesses pop up to fill that need

You can see many examples of this pattern in many standardized tests in the US: the SAT, ACT, PSAT, GRE, MCAT, LSAT…

You see this with tech interviews as well.

I don’t know who first came up with this idea of having a candidate work through a tricky algorithm problem as part of their interview, but it has created a beast. Like a butterfly flapping its wings across the world to create a hurricane, we’re now in the depths of the algorithm interviews.¹

(Note: whether or not these kinds of interviews are good is also beyond the scope of this essay)

It’s really an odd game, these interviews. The rules are mostly there for you to see before you play, with books detailing the process, online sites dedicated to the questions asked, and many avenues to practice. Interestingly, some companies also tell you about the rules and what to expect, even going as far as recommending specific subject areas to study (or, helpfully, that they won’t be asking any dynamic programming questions). Some of the companies even recommend specific sites to practice on.

Where did these sites come from? I think it’s the same pattern as before:

1. Companies started doing algorithm interviews
2. Doing well on those interviews can be very beneficial
3. People are willing to pay for ways to do better
4. Businesses pop up to fill that need

The similarities to the test prep industry are very interesting. Is an algorithm interview reflective of how an engineer works in their day-to-day job? Along those lines, what is the SAT really measuring? There’s this odd gap of artificiality between the test and the actual thing it’s purportedly testing for. Much like how bootcamps have popped up to fill in some of the gap between studying CS in college and the skills you need to successfully create websites, these businesses have popped up to help you bridge this gap between what you have to do as an engineer (or student) and what you have to do in the interview (or standardized test).

Look at timing - even a little strategy about how to tackle questions in each SAT section can help students do better. Maybe you aren’t used to writing essays in 50 minutes, but that’s OK! With a little practice and timing (be sure to wear a watch), you can get more used to it. Is that really reflective of life after the test? Maybe you’ll get a multiple choice test in college you have to hurry for or have to write manually during an exam, but the hard part is all of the other learning you’ll have to do.

For interviews, you can practice the timing, too. These businesses can help you get used to solving problems quickly as you identify general patterns. Working as an engineer, will your bugs ever be timeboxed to a time limit? Maybe you have to hurry a little bit every now and then, but I certainly hope it isn’t always the case that you’re working under 45-minute time limits, writing everything from scratch.

The other compelling similarity is that doing well after passing the test is not necessarily related to your performance on the test itself. When my wife was studying for the LSAT (the Law School Admission Test in the US), she found a tutor who had done tremendously well on the LSAT and pivoted that into a job helping other people study for the LSAT. I found this so interesting, because the mismatch between the test and working as a lawyer is just as striking (not to mention the mismatch between law school and working as a lawyer, but that’s also outside the scope here). Here was a test for preparedness to do well in law school (mostly in name, of course), but it clearly doesn’t take a lawyer to teach it.

With the market for helping others pass interviews, suddenly if you’re a few years into your career, there’s another avenue to build a brand doing interview prep. Just look up “coding interview tips” on YouTube - if you’re pretty good at interviews, you can leverage that into building a personal brand, and you get bonus points if you have some of the big companies already on your resume.

As a quick aside, I just wanted to note that there’s no disrespect meant towards those who are making money off of this; I’m more interested in this in a cultural way.

There are also a few interesting differences between this interview/business cycle and the standardized tests above. First, financially, the math is much clearer in favor of spending now to get a positive result later, as doing very well on these interviews can have a direct mapping to future benefits. If you’re able to land a job that increases your salary by $50,000, suddenly the $40 you spend on a book and the $35 you spend on an online subscription pale in comparison to the upside.

Additionally, there’s a strong self-fulfilling cycle here. Let’s say Company A does algorithm interviews and they know that there’s a gap between an engineer’s day-to-day responsibilities (such as writing good code to solve real business needs) and what you have to do in these interviews, so maybe they recommend a few test-prep sites to the candidate so they can get a little practice in. This would be like if signing up for the SAT came with a recommendation for a Kaplan study book - free marketing!²

What are we really measuring here?

If you’re measuring how someone communicates and thinks through a problem, what do you do when there’s a book that walks them through how companies want you to communicate and think through a problem in an interview?

If you’re trying to see how someone thinks about building complicated systems, what do you do when there are YouTube videos that walk them through how companies want them to work through building a complicated system?

If your company has to recommend practicing in advance of the interview, is that indicative that the interview could be better? What about when there’s an entire industry dedicated to being better at technical interviews?

What a weird balance.

It makes me simultaneously want to do all of the following:

1. Start an interview prep startup/personal brand and make money off of it, because shovels sell well in a gold rush and we might as well make a buck. Ideally we get some of these big companies to recommend our product, because that’s great marketing.³
2. Swear off technical interviews, throw away all of my computers, and live in the woods without electricity.
3. Play the dang game and do the stupid practice problems, because the upside is huge and real estate is expensive.
4. Work hard to find better and more equitable ways of interviewing candidates in a kind and human-focused way.

I definitely don’t have all the answers. I think I’ll just focus on #4.