An Elegant Puzzle: Systems of Engineering Management
By Will Larson
I came across this book after facing a challenge in my work where I had to come up with an estimation for the size of the team I wanted to hire in the coming year. I wanted to do this right, so I did a bunch of research for how to best think about what the right size team was. Luckily enough, First Round Review had published an article about An Elegant Puzzle and specifically, about sizing teams.
After reading that article, I bought the book and read it in like 3 days. It was not only helpful for thinking about sizing teams, but it also had a bunch of other management tools, tips, and best practices that I’ve since started to incorporate myself.
Maybe the most underrated part of the book, is the resources section where the author lists out all the resources that he recommends for various topics/subjects. I’ve used this list repeatedly for finding great new books/papers to read.
Summary & Key Takeaways
An Elegant Puzzle is an in-depth look at how to manage engineering teams. It covers everything from sizing teams to how to think about your team’s careers. And while it might be about engineering management, there are a host of great tips and practices for any type of manager. I’ve personally learned and taken away a lot from this book.
Key Takeaways
A systems thinking approach to managing people at large organizations can go a long way.
It’s important to treat people in your team and people looking to join your team well and to view problems as problems with the system, more than with the people.
Career growth is due to luck and learning, so make sure to do what you can to help yourself and others learn faster, since you can’t do anything about luck.
Documenting successful processes and keeping track of key metrics publicly can help create change without needing authority.
Vision is an important tool for getting alignment across many teams and many different people. It also helps you avoid micromanaging when done well.
Consider the people first.
Good books to pair with: Thinking in Systems, High Output Management, Inspired, and Presenting to Win
Notes & Quotes
Organizations
Sizing teams are the fundamental challenge of organizational design
“As I’ve gotten more exposure, I’ve come to believe that the fundamental challenge of organizational design is sizing teams.”
Fewer than four members is not a team
“I’ve sponsored quite a few teams of one or two people, and each time I’ve regretted it. To repeat: I have regretted it every single time...Teams with fewer than four individuals are a sufficiently leaky abstraction that they function indistinguishably from individuals. To reason about a small team’s delivery, you’ll have to know about each on-call shift, vacation, and interruption.”
Prioritize teams based on need
“For each constraint, prioritize one team at a time. If most teams are falling behind, then hire onto one team until it’s staffed enough to tread water, and only then move to the next. While this is true for all constraints, it’s particularly important for hiring.”
Don’t break up high performing teams
“Fundamentally, I believe that sustained productivity comes from high-performing teams, and that disassembling a high-performing team leads to a significant loss of productivity, even if the members are fully retained.”
Better to change scope than people
“Shifting scope works better than moving people because it avoids re-gelling costs, and it preserves system behavior.”
Overwhelmed employees perform worse
“Overwhelmed employees will get slow at finishing work until someone finds a way to help.”
Make sure you finish projects
“My favorite observation from The Phoenix Project by Gene Kim, Kevin Behr, and George Spafford is that you only get value from projects when they finish: to make progress, above all else, you must ensure that some of your projects finish.”
Design your software to be flexible
“In my opinion, probably the most important opportunity is designing your software to be flexible.”
Keep systems and teams decently separated
“When at all possible, build systems with sufficient isolation that you can allow most actions to go forward. And when they do occasionally fail, make sure that they fail with a limited blast radius.”
The idea of organizational debt
“This is the organizational sibling of technical debt, and it represents things like biased interview processes and inequitable compensation mechanisms. These are systemic problems that are preventing your organization from reaching its potential. Like technical debt, these risks linger because they are never the most pressing problem. Until that one fateful moment when they are.”
Tools
Product management is an iterative elimination tournament
“Product management is an iterative elimination tournament, with each round consisting of problem discovery, problem selection, and solution validation.”
Figuring out which problem to solve is a good predictor of long-term success
“Taking the time to evaluate which problem to solve is one of the best predictors I’ve found of a team’s long-term performance.”
Themes for understanding a problem space
Users’ pain
“What are the problems that your users experience?”
Users’ purpose
“What motivates your users to engage with your systems? How can you better enable users to accomplish their goals?”
Cohorts
“What is hiding behind your clean distributions?”
Competitive advantages
“By understanding the areas you’re exceptionally strong in, you can identify opportunities that you’re better positioned to fill than other companies.”
Competitive moats
“Moats represent a sustaining competitive advantage, which makes it possible for you to pursue offerings that others simply cannot.”
Compounding leverage
“What are the composable blocks you could start building today that would compound into major product or technical leverage over time?”
Things to consider for problem selection
Return on investment
“Personally, I think people often under prioritize quick, easy wins.”
Experiments to learn
“What could you learn now that would make problem selection in the future much easier?”
Helpful elements of solution validation
Write a customer letter
“Write the launch announcement that you would send after finishing the solution.”
Prefer experimentation over analysis
“It’s far more reliable to get good at cheap validation than it is to get great at consistently picking the right solution.”
Find the path more quickly traveled
“Try to find the cheapest way to validate”
Justify switching costs
“What will the costs of switching be for users who move to your solution? Even if folks want to use it, high switching costs may mean that they simply won’t be able to.”
Value of having a strategy and vision
“What I needed was a way to coordinate my approach across teams, both in terms of very specific challenges and in terms of our long-term direction. After experimenting with a handful of different approaches, agreeing on strategy and vision has been the most effective approach that I’ve found to alignment at scale.”
Strategy documents
“Strategies are grounded documents which explain the trade-offs and actions that will be taken to address a specific challenge.”
Vision documents
“Visions are aspirational documents that enable individuals who don’t work closely together to make decisions that fit together cleanly.”
“Visions should be detailed, but the details are used to illustrate the dream vividly, not to prescriptively constrain its possibilities.”
Visions help alignment
“If your team is struggling to align with stakeholders, or if you’re struggling to lead a cohesive organization, these documents are exceptionally useful, fairly quick to write as you gain practice, and low risk (at worst, they get ignored).”
Leading change with metrics
“In particular, I’ve found metrics to be an extremely effective way to lead change with little or no organizational authority.”
Provide context to implement change
“The most general and self-managing tool for this is benchmarking. It’s one thing for a team to know that they’re spending $100k a month, and it’s something entirely different for them to know that they’re spending $100k a month and that their team spends the second-highest amount out of 47 teams. Benchmarking is particularly powerful because it automatically adapts to changes in behavior.”
Things should stop working at a new scale
“The fact that something stops working at significantly increased scale is a sign that it was designed appropriately to the previous constraints rather than being over-designed.”
The two managerial skills that have a disproportionate impact on success
“I believe that, at quickly growing companies, there are two managerial skills that have a disproportionate impact on your organization’s success: making technical migrations cheap, and running clean reorganizations.”
How to approach team’s careers
“The options for approaching a career, particularly for those of us privileged to work in technology, are so extraordinarily vast that exploring them effectively requires a different approach. This vastness also means that you, as a manager, can’t simply give folks a career path: you’ll inevitably steer them toward the most obvious avenues and through avoidable competition.”
Opportunity on career ladders
“What I’ve slowly but increasingly come to believe is that there is much more opportunity on career ladders than off of them, and by including those opportunities you’ll make and feel more progress.”
Use a current role to get you to the next level
“A different approach would be to instead work on identifying the gaps that would keep you from being a strong head of engineering, and then start using your current role to help fill those gaps.”
Three rules for speaking with the media
Answer the question you want to be asked
“Don’t Think of An Elephant by George Lakoff is a phenomenal, compact guide to framing issues.”
Stay positive
Speak in threes
“Narrow your message down to three concise points, make them your refrain, and continue to refer back to your three speaking points.”
How to avoid failures
“I try to ensure that there is a manager embedded into every centralized group, and that the manager is responsible for iterating on the format to dodge these pitfalls.”
How to lead without authority — Model, Document, and Share
“Model, Document, and Share assumes: It’s better to adopt a great approach slowly.”
Model
“Frame it as a short experiment with the team, and start trying it.”
Note: something I’ve heard works from others and something I’ve personally done too, just say it’s an experiment. It’s amazing how often it gets adopted for that reason and ends up sticking around for a long time. Of course, there are downsides to this too.
Document
“After you’ve discovered an effective approach, document the problem you set out to solve, the learning process you went through, and the details of how another team would adopt the practice for themselves. Be as detailed as possible: make a canonical document, and even get a few folks on other teams to check that it’s readable from their perspective.”
Share
“You’ll spend the majority of your time refining approaches that work effectively for your team, a bit of your time documenting how you did it, and almost no time trying to convince folks to change their approach. Strangely, in my experience, this has often led to more adoption than top-down mandates have.”
Don’t use the approach to circumvent authority
“It’s particularly important not to try to use this as a strategy to circumvent organizational authority. Operating in direct conflict with authority usually doesn’t end very well.”
Presenting to leadership
Communication is company specific
“Every company has different communication styles and patterns...Each time you watch individuals present to leadership, study their approach.”
Start with the conclusion
“Particularly in written communication, folks skim until they get bored and then stop reading. Accommodate this behavior by starting with what’s important, instead of building toward it gradually.”
Frame why the topic matters
“Start by explaining why your work matters to the company.”
Everyone loves a narrative
“Another aspect of framing the topic is providing a narrative of where things are, how you got here, and where you’re going now.”
Prepare for detours
Answer directly
“Instead of hiding problems, use them as an opportunity to explain your plans to address them.”
Deep in the data
“You should be deep enough in your data that you can use it to answer unexpected questions. This means spending time exploring the data, and the most common way to do that is to run a thorough goal-setting exercise.”
Derive actions from principles
“The other aspect is defining the guiding principles you’re using to approach decisions.”
Discuss the details
“Some executives test presenters by diving into the details, trying to uncover an area the presenters is uncomfortable speaking on. You should be familiar with the details, e.g., project statuses, but I think that it’s usually best to reframe the discussion when you get too far into the details.”
Prepare a lot; practice a little
“If you’re presenting to your entire company, practicing your presentation is time well spent. Leadership presentations tend to quickly detour, so practice isn’t quite as useful...Rather, whether you’re able to present effectively without much preparation will become a signal for whether you’re keeping up with your span of responsibility.”
Make a clear ask
“If you don’t go into a meeting with leadership with a clear goal, your meeting will either go nowhere or go poorly.”
Approach to presenting to leadership
Tie topic to business value
Establish historical narrative
Explicit ask
Data-driven diagnosis
“Explain the current constraints and context, primarily through data. Try to provide enough raw data to allow people to follow your analysis. If you only provide analysis, then you’re asking folks to take you on trust, which can come across as evasive.”
Decision-making principles
What’s next and when it’ll be done
Return to explicit ask
Time Management
An enduring problem of leadership
“Time management is the enduring meta-problem of leadership.”
Get more done by being smarter, not faster
“I’ve gotten much, much better at getting things done, not by getting faster but by getting more logical about solving problems.”
Try quarterly time retrospectives
“Every quarter, I spend a few hours categorizing my calendar from the past three months to figure out how I’ve invested my time.”
Prioritize long-term success over short-term quality
“Ultimately, you have to prioritize long-term success, even if it’s personally unrewarding to do so in the short term. It’s not that I like this approach, it’s that nothing else works.”
Finish small, leveraged things
“If you’re doing leveraged work, then each thing that you finish should create more bandwidth to do more future work.”
Stop doing things
“When you’re quite underwater, a surprisingly underutilized technique is to stop doing things. If you drop things in an unstructured way, this goes very poorly, but done with structure this works every time. Identify some critical work that you won’t do, recategorize that newly unstaffed work as organizational risk, and then alert your team and management chain that you won’t be doing it. This last bit is essential: it’s fine to drop things, but it’s quite bad to silently drop them.”
Size backward, not forward
“Instead, specify the number of hours you’re able to dedicate to the activity, perhaps two per week, and perform as many skip-levels as possible within that amount of time. This method keeps you in control of your time allocation, and it scales as your team grows.”
Trust the system you build
“Handling exceptions can easily consume all of your energy, and either delegating them or designing them out of the system is essential to scaling your time.”
Note: Andy Grove suggested batching these into time slots. Follows similar approach previously mentioned
Decouple participation from productivity
“As you grow more senior, you’ll be invited to more meetings, and many of those meetings will come with significant status. Attending those meetings can make you feel powerful, but you have to keep perspective about whether you’re accomplishing much by attending.”
Hire until you are slightly ahead of growth
“The best gift of time management that you can give yourself is hiring capable folks, and hiring them before you get overwhelmed.”
Calendar blocking
“Creating blocks of time on your calendar is the perennial trick of time management: add three or four two-hour blocks scattered across your week to support more focused work.”
Getting administrative support
“Once you’ve exhausted all the above tools and approaches, the final thing to consider is getting administrative support. I was once quite skeptical of whether admin support is necessary — and, until your organization and commitments reach a certain level of complexity, it isn’t — but at some point having someone else handling the dozens of little interruptions is a remarkable improvement.”
Don’t fall into the trap busy = productive
“If you’re creative and consequent, and if you don’t fall into the trap of believing that being busy is being productive, you’ll find a way to get the workload under control.”
Communities of learning
Be a facilitator, not a lecturer
“Folks want to learn from each other more than they want to learn from a single presenter. Step back and facilitate.”
Brief presentations, long discussions
“Present a few minutes of content, maybe five, and then move into discussion.”
Choose topics that people already know about
“People find it very hard to discuss content that they’ve just learned if it’s too far away from their previous experience.”
Optional pre-reads
“Some folks aren’t comfortable being introduced to a new topic in public, and for those individuals, providing a list of optional pre-reads can help them prepare for the discussion.”
Approaches
Organizations survive by being flexible/adaptable
“Organizations survive by adapting to the dynamic circumstances they live in. An organization stubbornly insisting on its established routines is a company pacing a path whose grooves lead to failure.”
Make new policies worth it
“The fixed cost of creating and maintaining a policy is high enough that I generally don’t recommend writing policies that do little to constrain behavior.”
Be able to justify a “no”
“Folks who communicate a no effectively are not the firmest speakers, nor do they make frequent use of the word itself. They are able to convincingly explain their team’s constraints and articulate why the proposed path is either unattainable or undesirable.”
Communicate that finishing work is important
“When folks want you to commit to more work than you believe you can deliver, your goal is to provide a compelling explanation of how your team finishes work. Finishes is particularly important, as opposed to does, because partial work has no value, and your team’s defining constraints are often in the finishing stages. The most effective approach that I’ve found for explaining your team’s delivery process is to build a kanban board describing the steps that work goes through, and documenting who performs which steps.”
Translate the problem into data
“You have to translate the problem into something resembling data. If you’re following a consistent project methodology, this can be as easy as explaining how you decide the number of story points for each sprint, along with how that number has trended over time.”
Document your work to identify priorities
“Document all your incoming asks, develop guiding principles for how work is selected, and then share subsets of tasks you’ve selected based on those guiding principles.”
Specify quotas for long and immediate-term work
“One technique that I’ve found useful for this particular scenario is specifying quotas for both immediate and long-term work.”
Management is ethical work
“I believe that management, at its core, is an ethical profession. To see ourselves, we don’t look at the mirror, but rather at how we treat a member of the team who is not succeeding. Not at the mirror, but at our compensation philosophy. Not at the mirror, but at how we pitch the roles to candidates. Whom we promote. How we assign raises. Provide growth opportunities. PTO requests. Working hours.”
Strong relationships > any problem
“I believe that almost every internal problem can be traced back to a missing or poor relationship, and that with great relationships it is possible to come together and solve almost anything.”
Pick the right people
“A few years back, one of the leaders I worked with told me, ‘With the right people, any process works, and with the wrong people, no process works.’”
The right process is the one the team enjoys
“Process is a tool to make it easy to collaborate, and the process that the team enjoys is usually the right process. If your process is failing somehow, it’s worth really digging into how it’s failing before you start looking for another process to replace it.”
Spend more time with those you don’t have great relationships with
“If you have a poor relationship with your manager or a member of your team, spend even more time with them. Meet with them every day, or have dinner with them.”
Engage problems head on
“As a leader, you can’t run from problems; engage ‘em head-on.”
Company first, team second, yourself last
“Do the right thing for the company, the right thing for the team, and the right thing for yourself, in that order.”
Think for yourself, don’t just do what others have done
“So much of what we take for granted is cargo-culted instead of done with intention. Early in your management career, you’ll have to figure out how to approach common challenges: interviewing, performance management, promotions, raises. It’s totally all right to start out by following what you see around you — learning from your peers is critical to success. However, it’s also important to be honest about which of your practices are truly best practices and which you’re following on autopilot.”
Note: this is so true of just about everything and everyone. There are so few original thinkers in the world, especially now that there’s social media, podcasts, 24/7 news, etc. This is something I need to be aware of and try to avoid falling into the trap.
There’s a very fine line between learning from others’ experience and just following along what others have already done.
Have a theory of management that’s flexible
“The worst theory of management is to not have one at all, but the second worst is one that doesn’t change.”
Don’t expect to iterate on the status quo for new problems
“These new problems aren’t necessarily novel (most problems are people problems), but they are problems that your company has never prioritized long enough to get a usable solution out the door. This means that you can’t expect to succeed by iterating on the status quo.”
Execution is the primary concern at the growth plates, not new ideas
“What folks in the growth plates need is help reducing and executing the existing backlog of ideas, not adding more ideas that must be evaluated. Teams in these scenarios are missing the concrete resources necessary to execute, and supplying those resources is the only way to help. Giving more ideas feels helpful, but isn’t.”
Manage up by knowing your manager
“A few managers seemingly just don’t care, and I’ve always found that those managers do care, and are too stressed to participate in successful communication. This leads to the other key aspect of managing up: knowing some things about your manager and their needs.”
Grow my pursuing more scope
“As managers looking to grow ourselves, we should really be pursuing scope: not enumerating people but taking responsibility for the success of increasingly important and complex facets of the organization and company.”
There’s a lot less competition for hard work
Seek out cross-cutting projects
“Strong execution in these cross cutting projects will give you the same personal and career growth as managing a larger team. Project-managing an initiative working with 50 engineering managers is a far better learning opportunity than managing an organization of 50, and it builds the same skills.”
You can always find more scope
“You can always find an opportunity to increase your scope and learning, even in a company that doesn’t have room for more directors or vice presidents.”
Culture
Culture stems from the actions an organization takes
“However, in the long run our measure is not in what we say or how skillfully we say it, but in what we do, and the abacus tallying our actions is organizational culture.”
Inclusive culture = access to opportunities and membership
“The basis: an inclusive organization is one in which individuals have access to opportunity and membership. Opportunity is having access to professional success and development. Membership is participating as a version of themselves that they feel comfortable with.”
Access and membership means being challenged and growing
“When I think about having access to opportunity, I think about ensuring that folks can go home most days feeling fulfilled by challenge and growth.”
Good process leads to opportunities to everyone
“The most effective way to provide opportunity to the members of your organization is through the structured application of good process. Good process is as lightweight as possible, while being rigorous enough to consistently work. Structure application allows folks to learn how the processes work, and lets them build trust by watching the consistent, repeated application of those processes.”
Ways to create and distribute opportunity
Rubrics everywhere
“Every important people decision should have a rubric around how folks are evaluated. This is true for promotions, performance designations, hiring, transitions into management, and pretty much everything else!
Selecting project leaders
Explicit budgets
“Instead of saying that you’ll pay for teams to attend a reasonable number of conferences per year, specify a fixed number. Instead of maintaining a general ongoing education budget, make it explicit.”
Nudge involvement
Education programs
People want role models
“People want role models for themselves in senior roles at the company where they work, which is why looking at the representation of underrepresented minorities and women is only a start. You also want to look in each role and level of seniority.”
Membership is the precondition to belonging
Need to have a broad set of options for everyone
“Many activities and events don’t work well for everyone—meals can be difficult for individuals with complex dietary restrictions, physical activities make some uncomfortable, activities after working hours can exclude parents—and success here requires both a broad portfolio of options and a willingness to balance concerns across events and time.”
Have lots of people lead important projects
“This is so important that I’ve come to believe that having a wide cohort of coworkers who lead critical projects is one of the most important signifiers of good organizational health.”
Be aware of what your peer’s teams are working on
“Even with the best intentions, a member cannot optimize for their team if they’re not familiar with other members’ work. The first step to moving someone’s identity to their peers is to ensure that they know about their peers’ work. This will require a significant investment of time, likely in the form of sharing weekly progress, and the occasional opportunity for folks to dive deep into each other’s work.”
You may learn the most from your peers
“The best learning doesn’t always come directly from your manager, and one of the most important things a first team does is provide a community of learning. Your peers can only provide excellent feedback if they’re aware of your work and are thinking about your work similarly to how you are. Likewise, as you’re thinking about your peers’ work, you’ll be able to learn from how they approach it differently than you anticipate. Soon, your team’s rate of learning will be the sum of everyone’s challenges, no longer restricted to just your own experiences.”
Your career will be about luck and learning
“Long term, I believe that your career will be largely defined by getting lucky and the rate at which you learn. I have no advice about luck, but to speed up learning I have two suggestions: join a rapidly expanding company, and make your peers your first team.”
Consider internal candidates fairly for senior positions
“Ensuring that internal candidates take part is essential to an inclusive culture. Fair consideration doesn’t mean that we prefer internal candidates. Rather, it means that there is a structured way for them to apply, and for us to consider them.”
Skills for internal candidates to have when considering for more senior positions
Spoken and written communication. “Can they convey complicated topics in both written and verbal communication? Can they do all this while being engaging and tuning the level of detail to their audience?“
Stakeholder management. “Can they make others, especially executives, feel heard? Can they make these stakeholders feel confident that they’ll address any concerns?”
Get feedback from peers and teams
“Lean into controversial feedback, not away from it. Listen to would-be dissenters, and hear their concerns.”
Have candidates present vision/strategy
“Have the applicant present their vision/strategy document to a group of three to four peers. Have the peers ask questions, and see how the applicant responds to this feedback.”
Internal processes for promotion may take time and have downsides but they’re worth it
“Running the process takes a lot of time, but it’s rewarding time. In fact, this has generated more useful feedback than anything else I’ve done over the past year. It brings an element of intentional practice that’s uncommon in engineering management. Folks get to take risks in their plans. You get to give direct feedback without risk of micromanagement. It’s been useful enough that I’m now figuring out whether we can use a similar format for training managers. Know that internal processes are awkward. You’ll have many internal candidates. They will talk to each other. They will interview external candidates for a role they are applying for themselves. One candidate may end up managing the others. Don’t decide to avoid this morass of awkwardness. You can’t. You’re deferring it until later. It’ll still be awkward, but now awkwardly in the realm of gossip.”
Balance positive and negative freedoms
“I believe that the balancing of positive and negative freedoms is a fundamental task of managers and management.”
Adopt only one new thing for each project
“First, Tom DeMarco’s Slack has an excellent suggestion for a good starting state between positive and negative freedoms for engineering teams: generally follow the standard operating procedure (i.e., keep doing what you’re already doing, the way you’re doing it), but always change exactly one thing for each new project.”
Working harder when a project is behind schedule doesn’t work
“Of course not. Unless your problem is that people aren’t trying hard, the “work harder” mantra only breeds hero programmers whose heroic ways make it difficult for nonheroes to contribute meaningfully. Later, as your new heroes finish martyring themselves on burnout, you’re left with three exceptionally challenging problems: You’ve bred a cadre of dissatisfied and burned-out heroes. You and your heroes have alienated everyone else. Your project is still totally screwed.”
Get rid of hero programmers
“When it comes to solving the problem of the hero programmer, your options are limited: either kill the environment that breeds hero programmers, or kill the hero programmer by burnout.”
Things don’t happen because of just one event or person, they build up over time
“One of the observations from systems thinking16 is that, though humans are prone to interpreting events as causal, often problems are better described in terms of a series of stockpiles that grow and shrink based on incoming and outgoing flows. The Dust Bowl17 wasn’t caused by one farmer or one year of overfarming, but by years of systemic abuse.”
Projects fail slowly
“Projects fail slowly—and fixing them takes time, too. Working at a frenetic pace for a couple of weeks or a month may feel like a major outpouring of effort and energy, but it’s near impossible to quickly counteract a deficit created over months of poor implementation or management choices.”
Careers
Managers can reduce the role of luck in the careers of others
“However, as managers, we have an outsize influence in reducing the role of luck in the careers of others. That potential to influence calls us to hold ourselves accountable for creating fair and effective processes for interviewing, promoting, and growing the folks we work with.”
Growth only comes from change
“don’t treat growth as a foregone conclusion. Growth only comes from change, and that is something you can influence.”
Be kind in interviews
“Being kind comes through in the interview process in a hundred different ways. When an interview runs overtime before you get to the candidate’s questions, the kind thing to do is to allow the candidate a few minutes to ask questions, instead of running on to the next interview to catch up.”
Give time for interviews
“My experience is that you can’t conduct a kind, candidate-centric interview process if your interviewers are tightly time-constrained. Conversely, if an interviewer is unkind to a candidate (and these unkindnesses are typically of the “with a whisper not a bang” variety), I believe it is very often a structural problem with your interviewing process, and not something you can reliably dismiss as an issue with that specific interviewer.”
Make sure everyone agrees on the role they are interviewing for
“The second critical step toward an effective interview loop is ensuring that everyone agrees on the role they are interviewing for, and what extent of which skills that role will require.”
Break the interview down into identifying the key skill sets
“After you’ve broken the role down into a certain set of skills and requirements, the next step is to break your interview loop into a series of interview slots that together cover all of those signals. Typically, each skill is covered by two different interviewers to create some redundancy in signal detection, in case one of the interviews doesn’t go cleanly.”
Effective interview formats
Prepared presentations on a topic. “Instead of asking the candidate to explain some architecture on the spur of the moment, give them a warning before the interview that you’ll ask them to talk about a given topic for 30 minutes, which is a closer approximation of what they’d be doing on the job.”
Debugging or extending an existing codebase on a laptop (ideally on their laptop).
Giving demos of an existing product or feature (ideally the one they’d be working on.)
Roleplay (operating off a script that describes the situation.) This option can be pretty effective if you can get the interviewers to buy into it, allowing you to get the candidate to create more realistic behavior (architecting a system together, giving feedback on poor performance, running a client meeting, and so on).
Always be prepared
“Being unprepared is, in my opinion, the cardinal interview sin, because it shows a disinterest in the candidate’s time, your team’s time, and your own time.”
Have interviewers send notes to the candidate you want to move forward on
“Whenever you extend an offer to a candidate, have every interviewer send a note to them saying that they enjoyed the interview.”
Have feedback loops in the interview process
“The fix for all these issues is to ensure that you build feedback loops into your process, both for the interviewers and for the designer of the interview process.”
Use pair interviews for people new to it
“For pair interviews, have a new interviewer (even if they are experienced somewhere else!) start by observing a more experienced interviewer for a few sessions, and then gradually take on more of the interview until eventually the more senior candidate is doing the observing.”
Each interviewer should write up their initial thoughts and reactions before discussing with the larger group
“To get the full benefit of calibration and feedback, after the interview have each interviewer write up their candidate feedback independently before the two discuss the interview and candidate together.”
Ask candidates for direct feedback on the process
“For direct feedback from candidates, I’ve started to ask every candidate during my “manager interview” sessions how the process has been and what we could do to improve.”
Also have recruiters debrief at the end of the day
“The other, more common, mechanism is to have the recruiters do a casual debrief with each candidate at the end of the day.”
Instrument the process so that you can track it
“Once you have the basics down, the final step of building a process that remains healthy for the long haul is instrumenting the process at each phase (sourcing, phone screens, take-home tests, onsites, offers, and so on) and monitoring those metrics over time.”
But don’t get caught up in instrumenting it
“In the end, I think you’ll find that your process cannot thrive without handling the first six priorities, and that your analytics will direct you toward fixing those issues. Plus, the underlying data is very often poor, and it can be easy to get lost spending your cycles on instrumenting your process instead of improving it.”
Aim for about 600 connections on LinkedIn
“Anecdotally, the number of connections at which things seem to get easier is around 600 or so.”
Personalization of your message doesn’t matter that much
“I’ve personally found that customization matters less than I assumed, because people mostly choose to respond based on their circumstances, not on the quality of your note.”
Be respectful of the candidate’s time
“Being respectful of the candidate’s time is, in my opinion, the most important thing to optimize for.”
Goals of the intro call/coffee
“You have two goals for each of these calls or coffees: figure out if there is a good mutual fit between the candidate and the role, and, if there is a good fit, try to get them to move forward with your process.”
Helpful to use a candidate tracking service
“It’s worth pointing out that this process is much easier to do with an applicant tracking system (ATS) like Lever or Greenhouse, which allow you to have a single place to track whether a candidate has already been contacted by someone else at your company.”
The hiring funnel
Identifying candidates
Motivating them to apply
Evaluating them for your company
Closing them on joining
One of management’s most important tasks is picking role models
“The most sacred responsibilities of management are selecting your company’s role model, identifying who to promote, and deciding who needs to leave.”
You can change your company’s culture by choosing who stays around
“If you want to shape your company’s culture, inclusion, or performance, this is your most valuable entry point.”
Choosing career ladders can identify what’s valuable at a company
“One effective method for reducing the fixed cost of maintaining ladders is to establish a template and shared themes across every ladder. Not only does this reduce fixed maintenance costs, it also focuses the company on a set of common values.”
Performance reviews should not be for poor performance
“Somewhat unexpectedly, performance designations are usually not meant to be the primary mechanism for handling poor performance. Instead, feedback for weak performance should be delivered immediately. Waiting for performance designations to deal with performance issues is typically a sign of managerial avoidance.”
Be careful about sending performance reviews that don’t change to top performers
“But I find that this is unexpectedly somewhat demotivating for high performers, who want consistent, direct feedback on their work so that they can keep improving.”
Give your manager the goals you want to hit
“Propose a set of clear goals to your manager, and iterate together toward an explicit agreement on the expectations to hit the designation you’re aiming for. The goals need to be ambitious enough that your manager can successfully pitch the difficulty to their peers during calibration.”
Moving from general role towards specialists can bring problems
“As you move away from generalized roles and toward specialists, an unexpected consequence is that your organization has far more single points of failure. Where everyone on a team was once able to perform all tasks fairly effectively, now if your project manager leaves, you’ll find that no one is able to fill the role very capably. This brittleness is particularly acute in organizations with frequent structural changes.”
Be careful when hiring for roles you don’t yet have or understand
“This is almost the opposite of the above challenge: sometimes the existing organization has so little experience with the new function they wish to create that they simply don’t have a usable means by which to evaluate candidates. This can lead to evaluations focused on qualities that are largely independent of what the candidates would be doing if they accepted the job.”
All new roles should have career ladders
“In pretty much all cases, new roles should have a career ladder from the beginning. The career ladder is the foundation of a successful performance management system, and it’s not possible for a role to be valued or evaluated coherently without a thoughtful career ladder.”
Don’t try a new interview loop until you’ve instrumented your hiring funnel
“Do not start designing a new interview loop until you’ve instrumented your hiring funnel. It’s pretty hard to evaluate your loop or improve it without this data, so don’t undertake an improvement project without it.”
Understand employee trajectory
“Employee trajectory. For all the candidates you have hired, understand how their work performance relates to their interview performance. What elements correlate heavily with success, and which appear to filter on things that end up not contributing to performance in the job?”
Design tests for demonstrating skills
“For each skill, design a test to evaluate the candidates’ strengths. Whenever possible, prefer tests that have a candidate show their strengths, avoiding formats in which they tell you about it.”
Create rubrics for scoring skill tests
“Once you’ve identified your tests, write a rubric to assess performance on each test. Good rubrics include explicit scores and criteria for reaching each score.”
Avoid design by committee
“Try to avoid design by committee. These almost always lead to incremental change.”
Tools for operating a growing organization
A tension of management
“One tension in management is staying far enough out of the details to let folks innovate, yet staying near enough to keep the work well-aligned with your company’s value structures.”
Adapt process to environment
“Process needs to be adapted to its environment, and success comes from blending it with your particular context.”
Criteria for evaluating if a sprint is working well
Team knows what they should be working on.
Team knows why their work is valuable.
Team can determine if their work is complete.
Team knows how to figure out what to work on next.
Stakeholders can learn what the team is working on.
Stakeholders can learn what the team plans to work on next.
Stakeholders know how to influence the team’s plans.
Importance of your backlog
“Within your sprint process, your backlog is particularly important: it’s the context-rich interface that you’ll use to negotiate changes in direction and priority with your stakeholders. It’s always more interesting to discuss which of two things we should do next, rather than whether something is worth doing.”
Backlog = specialize; roadmap = generalize
“Initially, the distinction between your backlog and your roadmap may be quite small: your backlog a bit more detailed, your roadmap looking a bit further into the future. The value in having both is that this lets you specialize the backlog to be more useful for your team and design the roadmap to be more useful for your stakeholders, rather than relying on one tool to satisfy both sets of constraints.”
Create weekly staff meetings
“As you spend less time with your teams, you’ll want to start a weekly staff meeting with your managers. The best versions I’ve seen start with brief updates from each attendee, at most a couple of minutes per person, and then move into group discussions on shared topics. Topics might include running effective sprints, planning, career development, or whatever else proves useful. Done well, these discussions are the key learning forum for you and the managers you work with.”
Use a vision document
“a vision document: a concise statement of the team’s goals and the strategy for accomplishing those goals.”
Create a directional metric dashboard
“The mechanism I’ve found most helpful in this case is to ensure that every team has a clear set of directional metrics in an easily discoverable dashboard. The metrics should cover both the longer-term goals of the team (user adoption, revenue, return users, etc.) and the operational baselines necessary to know if the team is functioning well (on-call load, incidents, availability, cost, and so on.) For each metric, the dashboard should make three things clear: the current value, the goal value, and the trend between them.”
Systems thinking is an effective universal tool
“For me, systems thinking has been the most effective universal tool for reasoning through complex problems, and this book is a readable, powerful introduction.”