Tackling the system design interview

Jan 26 2018

Disclaimer: These are personal opinions and beliefs, not those of my employer.

There are three broad types of interviews in tech:

• Conversational/behavioral (“soft”)
• Algorithmic/coding (“hard”)
• System design (???)

The Internet is full of advice on conversational interviewing, Cracking the Coding Interview is the classic prep text for algorithms, yet folks are often mystified by system design questions. Though you’re virtually guaranteed to be asked one when interviewing for a software job at a major company, CTCI devotes only six pages to them.

Worse, unlike algorithm questions, there’s no optimal answer, and these questions are impossible to “cram” for. Synthesize a lifetime of systems knowledge into an impromptu design presentation? Who wouldn’t be intimidated?

While it’s hard to practice the concepts, practicing technique can help you approach the design interview with (some) confidence. Here’s what I recommend.

A workable game plan

1) The interview starts with your interviewer reading out the problem. Immediately summarize it on the board.

Interviewer: “Design an appointment-booking system for a doctor’s office.”

You: writes on board “appointment booking for doctor office”

Write down any numbers or constraints your interviewer mentions. 10 doctors in the office? 50 patients per day per doctor? The office is on a cruise ship with unstable access to the internet? Write it down!

2) Start by asking clarifying questions. Write down any revealed constraints in bullet-point form.

You: “I have a few questions…” * What kind of appointments? * Who needs access to this system? Patients? Doctors and nurses? Health aides? Sysadmins? Should access differ? * What operations does the system need to support besides booking? * How many doctors do we need to support? How many offices? * How many patients can a doctor see per day? * Can we assume the office has access to the internet? * Can we assume patients have access to the internet?

(“Can we assume” is a great phrase. Always double-check your assumptions.)

3) Think through what you’ll need for a basic solution. It doesn’t have to be optimal and it doesn’t have to scale, but it should include the major design elements you know you’ll need. You can always tack on additional things later. Don’t spend more than a minute or two, and don’t let the perfect be the enemy of the good.

You: “Let me think for a minute… OK. I’m going to draw out a basic system here and we can improve it as we go.”

4) Draw a system diagram on the board. Draw it large, with lots of room to edit — this is the star of your show. Walk through the elements of your solution as you go, or if you can’t, do so immediately afterward. If your solution involves a data model, draw a diagram of that as well.

You (drawing): “OK, we have some clients over here… who access a frontend site… which talks to a backend… which stores data here… and we have admins with their own site… over here.”

5) The interviewer will likely interrupt you with questions. If they don’t, ask which part you should focus on.

Interviewer: “What about mobile clients?”

You: “OK, we’ll need to make sure this site has responsive design and can deal with intermittent connections…”

Interviewer: “Where does this site live?”

You: “Let’s say it’s in Google Cloud. Should I explain that further?”

Interviewer: “Let’s hear more about that.”

You: “OK, the site is provisioned by the admins through this interface, and we set it up like so…”

6) Drill down into a particular element of your system. Describe how it works, explaining and justifying possible tradeoffs.

You: “The appointment reminder system talks to Twilio to send SMS reminders—”

Interviewer: “Hold on, why use Twilio?”

You: “It’s a pain to set up SMS integration with a bunch of different carriers. On the other hand, we’ll want to make sure we’re either not sending any protected health information, or that our setup is fully compliant with HIPAA.”

7) Either drill down further or move on to other elements, depending on what your interviewer asks.

Interviewer: “How would you scale this up to a hospital with thousands of patients? What about a whole hospital network?”

You: “What kind of scale are we talking about here? Can we say 10,000 people? 100,000?”

Interviewer: “How about a million people? Say it’s a major metro area.”

You: “OK, we’re going to need some kind of sharding solution…”

8) As you’re doing this, ideas, concerns, and so on should pop into your head. Dedicate an area (“Considerations”) of the board to a bulleted list of these. As you address them, cross them out.

You: “Oh, we’re going to need to think about backups. Let’s add that to Considerations here, under ‘HIPAA’ and ‘integration with pagers’.”

9) If appropriate, write down a timeline of the system in use, end-to-end. Use bullet points rather than numbers, and leave lots of space between them (because you’ll want to insert steps later). Walk through it as you’re writing.

You: “Let’s walk through the appointment process, starting from when the user books an appointment to when the appointment is complete. First, the user navigates to the website, or calls the hotline if they don’t have internet…”

10) Ask the interviewer to give you a five-minute warning when you’re nearing the end of your time, and use the final minutes to summarize your solution: the basic outline; elements of the system and why they’re there; optimizations you’ve made to address issues like scale, security, or other constraints; an end-to-end walkthrough of the system in operation; and any outstanding issues and how you’d address them if you had the time.

You: “We’ve presented a solution here that has a highly-available frontend with mobile and desktop sites; phone, SMS, and email integration; a secure backend with health record integration; and special interfaces for doctors and system admins. It can scale to a city-sized population thanks to sharding and multiple datacenters and is protected against denial-of-service attacks. If I had more time, I’d look into applicable health regulations, as well as figuring out what accommodations we’d make for disabled users. I’d also think more about an alerting story: if a user misses their appointment, maybe they’re in trouble. Anything else I should summarize?”

11) Thank the interviewer for their time. Take a deep breath. You’re done.

Specific tips

Have a board organization plan

I like to split up the whiteboard:

• Question statement: upper left
• Systems diagram: top center, should be the largest thing on the board
• Constraints (with room to add): center left, below question
• Considerations and failure modes: bottom left
• Relevant technologies: bottom right
• End-to-end example interaction: bottom center

The value of having a defined board organization is simple: when you suddenly think of something you want to remember, or are interrupted with a question, you know where it goes and can stash it there without interrupting your flow — and at the end of the interview, your board comes out looking like you planned it that way.

No really: Use the board

Here’s a poorly-kept secret: your interviewer is human. They’re focusing on you, yes, and probably taking notes on your solution, but their attention will not be fully on you 100% of the time.

In other words, do not count on them paying full attention to, remembering, and understanding each and every word you say.

Far more lasting and effective is your whiteboard: what you’ve written and how you’ve written and organized it. Hence my number-one tip, and the piece of feedback I’ve given to virtually everyone who’s practiced this type of interview with me:

WRITE.

DOWN.

EVERYTHING.

Seriously. You probably aren’t writing enough. There is no surer sign of a failed interview than a whiteboard with a few lonely lines abandoned in one corner.

I will freely acknowledge: this is hard. Writing and talking at the same time is challenging. Writing, talking, and dynamically generating a scalable, efficient, full-featured, correct system design in your head? Might as well ask you to do calculus while juggling.

If necessary, alternate write-talk-write-talk. It’s OK to pause your monologue for a few seconds while writing. If you’re going to stop talking for more than a few seconds, say “I’m going to write for a bit, then describe what I’ve written.” That’s an example of what’s called a:

Signpost

Signposting is a public-speaking technique you’ve probably heard before. When a speaker says “I have three points: first, foo; second, bar; third, baz;” they’re signposting: helping the audience mentally map their speech.

In short, signposting is when you explicitly state:

• what you’re about to say
• what you’re saying, and/or
• what you just said

You can use this technique to great effect in interviews, and often save yourself from interruptions. When you let your interviewer know what to expect (“I’m going to draw a diagram, then I’m going to discuss the client, frontend, and backend, then we’ll get to scaling”) you’re helping them understand your presentation’s structure. Not only does this make your extemporaneous solution seem more polished, your interviewer might not interrupt you to ask about the backend if they know you’re about to address it. Fewer interruptions means better focus, which means better performance.

Much like clean abstractions in programming, clearly structuring your speech makes you easier to understand — and the easier it is to understand you, the more likely you are to pass.

Examples of good interview signposts:

• “I’m going to discuss three important aspects of this design: sharding, replication, and load balancing.”
• “Moving on to the frontend…”
• “We’ll discuss locking in a bit, but first we’ll address resource management.”
• “So that’s the user flow. Any questions before we talk about the admin view?”

Get ready to change your plan — a lot

On the other hand, don’t try to plan your entire solution as a perfect jewel from the get-go. The structure of the interview will foil that: your interviewer will interrupt you constantly, you’ll think of something you need in step 2 while describing step 6, you’ll suddenly remember an important constraint and need to go back and fix something. It will happen, probably several times.

This is true even if you’ve done the question before. Every interview is different.

Interruptions throw everyone off, and they’ll probably fluster you a bit. Don’t worry about it! You obviously had no time to prepare, and unless you’re psychic (or cheating) you’re not going to know the desired solution ahead of time, so don’t try. Instead, your mission is to:

• stay calm
• leave yourself lots of flexibility, especially in terms of board space
• be OK with jumping back and forth

Again, make sure to signpost what you’re doing. If you’re going to jump back to an earlier point, say something like: “I’ve realized I didn’t address authentication back in step four. Should I talk about it now, or wait until later?”

Speaking of which:

Interact with your interviewer!

Think of yourself as a tour guide walking the interviewer through your solution. You know it better than they do, but you need to make sure they’re happy and aren’t lost. Some interviewers will aggressively interrupt when you’ve lost them, but others might just stay silent. For this reason, make sure they’re with you, and actively solicit their preferences with lines like:

• Does that seem reasonable?
• Should I discuss X or Y next?
• What would you like to see more of?
• Does it seem like I’m missing anything?

Your interviewer may vary, of course, but in my experience asking this kind of question won’t cost you points. On the other hand, if you’ve lost your interviewer and you’re way off in the weeds, occasionally asking “Does that make sense?” and actually waiting for the ACK from your interviewer can save an otherwise-doomed interview. “Actually, I didn’t understand why you added DNS in step 3 — can you explain that?” is infinitely better than walking out of the room with your interviewer still baffled.

Talk to the interviewer, not the board

Ever had a teacher or professor that just talked at the board? Especially if they were writing at the same time? Facing away from your interviewer makes it difficult to understand what you’re saying (especially if they’re deaf or hard-of-hearing) so make sure to always face them when talking.

If you haven’t learned the art of writing on a surface you’re facing away from (a formidable challenge, to be sure) consider the write-talk-write-talk technique mentioned above. Just don’t spend too much time writing without talking or vice versa.

If you know of a relevant technology, mention it (and write it down)

Notice that I said “know of,” not “know.” As long as you’re clear about the limits of your knowledge, mentioning technologies you’re only passingly familiar with, but know are appropriate to the problem, is OK. Don’t expect this to carry your presentation, but saying things like:

• We need to provision many servers on a regular basis, so Chef, Puppet, or Ansible might be appropriate here.
• We’ll need to run jobs on clusters, so we might want a system like Kubernetes.
• We can’t trust the network, so we’ll need to secure the transport with something like TLS or a VPN.

and then adding “I’m not sure of the specific details of those technologies, but if tasked with this in real life I’d investigate them,” casts you as a non-wheel-reinventing professional who does their homework. When writing these on the board, I like to tag them with a (?).

WARNING: Do not bluff. If you claim to know a tech in detail, be prepared to be called on it. Interviewers love nothing more than to catch you in this kind of fakery, an express route to a No Hire. It’s OK to mention a technology you don’t know much about as long as you’re forthright about it.

Conversely, if you know a technology like the back of your hand, make sure it doesn’t take over your presentation unless your interviewer requests it (and maybe not even then). You might ask:

• At this point the server will verify the client’s certificate. Should I describe how that happens?
• We’ll need to build an index across these columns. Should I write out the SQL query?
• Can I assume a reasonable storage backend exists, or should I describe how I’d set up a SAN?

Check whether your interviewer really wants that deep-dive. Maybe they’re happy to handwave it and would prefer to hear detail about something else.

Treat it like a presentation

Notice how I keep using the word “presentation” to describe this interview? That’s intentional. The same skills, techniques, tricks, and traps that apply to presentations (of the boardroom or conference variety) can also apply to systems interviews. Clear verbal communication, effective use of visual media (the board), engaging with the audience, responding effectively to interruptions and questions, incorporating feedback well, and so on, are all important presentation skills that carry over to this form of interview especially.

Takeaways?

• Practicing presentation skills can also help with interviews.
• Nailing this interview may demonstrate proficiency in technical communication, a critical “soft” skill for senior devs. If you’re on the border between two career levels, this can help put you over the top.

Related: Your board is your slide deck

Much like slides, make sure the information on the board is clear, relevant, and a good use of your interviewer’s time and yours.

In particular: Make sure your handwriting is clear. I have terrible handwriting by default and handwriting can be quite painful for me — the best thing to do is slow down. It’s better to have five perfectly clear words than twenty illegible scribbles. This may also force you to be concise (which, if you can’t tell, I also occasionally have trouble with).

Considerations to Consider

There are several frequently-forgotten challenges — “considerations” — that apply to most system designs. In an interview context, some may be optional, but bringing them up (and even adapting your solution to them) can demonstrate experience or care in design, a highly valued commodity in potential hires.

The listings below are brief, but each hides an ocean of potential complexity. Don’t get sucked in unless your interviewer insists. Even a brief mention can show consideration and responsibility — just be sure to follow through when you’re designing for real.

Don’t panic if you can’t remember all of these (I’m likely forgetting some!). If it’s critical, your interviewer will likely prompt you on it.

Security and compliance

If your system has multiple users, is connected to the internet, processes information, or deals with anything of value, you need to think about security. (Yes, this is always.)

You might mention questions like: Who has access? How do you know they are who they say they are? Who might wish to attack the system, and how might they go about it? How is your data protected? Who trusts whom and why?

Tip: Don’t use the abbreviation “auth”. It can mean either authentication (“are you who you claim to be”) or authorization (“are you allowed to do that”) which are separate concerns and are frequently confused.

Compliance

If you’ve never heard this term before, you’re lucky. Compliance refers to ensuring your system adheres to regulations around what you do or the data you deal with, such as (in the US) HIPAA for health data, SOX for financial data, or FedRAMP for products used by the government.

Accessibility

Accessibility (“a11y”) comes down to this: question assumptions about your users. This can mean taking into account physical and mental disabilities (can all your users see? Can all of them read?), culture (see internationalization below) or differing levels of access to technology (can you assume all your users have smartphones? Do they all have phones, period?).

For example, does your system involve a website? You might consult the W3C’s accessibility guidelines, or annotate your web elements with ARIA markup to ensure it works with screen readers. You might also use Chrome’s DevTools to simulate loading your site over slow network connections. Tag your assumptions for further investigation, even if you don’t fully chase them down.

Internationalization and Localization

Another pair of numeronyms: i18n (internationalization) and l10n (localization) — these abbreviations are common enough that you can save yourself a hand cramp by writing them on the board.

Same deal as above — the system’s users might not be located in the same country, speak the same language, use the same alphabet, and so on. This kind of thing is unlikely to cause a deep shift in your design (though consider whether your distributed system might be accessed by people from different geographical regions!) but mention it anyway, as it almost always comes up.

Backups

Does your system deal with data? How will you back it up? Durability of data is often an afterthought, to the bitter (company-ending) regret of more than one startup founder.

You might use a cloud provider’s built-in backup, but is that enough? Will having a datacenter mirror protect you from deliberate file deletion?

Testing, monitoring, and alerting

How will you verify your solution is correct? How will you make sure it’s doing what you think it is? What will let you know when it isn’t? Thinking through a testing and monitoring story, even a brief one, shows your conscientiousness.

If your system goes down, who fixes it? If you’re deploying phone apps, how will you fix security bugs? Will the remote server on that cruise ship be able to notify admins if it loses data?

Conclusion

That’s a lot, isn’t it? If you’ve just read that and are thinking “no way am I going to remember all that,” here’s the TL;DR:

• Write down as much as possible
  • Have a board organization plan
  • Write-talk-write-talk if necessary
  • Concise and neat is better than quick and illegible
• Engage with the interviewer
  • describe what you’re doing
  • ask questions
  • signpost frequently
  • check understanding
  • handle interruptions gracefully
  • don’t talk to the board
• Mention relevant tech
  • but don’t bluff
  • and don’t let it take over the interview
• Track and mention considerations
  • like security, accessibility, and testing

System design interviews aren’t impossible; they’re not even that scary. Done right, they can even be fun. Take a deep breath, follow the plan, and you’ll be OK. Even an unsuccessful interview is a learning experience.

Good luck!