I’m currently an Executive in Residence at Balderton, where I provide support and advice to the firm’s portfolio companies.
Over my 30+ years working with scale ups (as CTO of Criteo and Onfido), large companies (Google, Microsoft, HPE) and startups (as a founder, VP of engineering and technical advisor), I have learned many lessons about what it takes to build great engineering and research teams and increase efficiency.
In this post, I am going to talk about productivity - and in particular, software engineering productivity - by looking at three different aspects:
Let’s assume you are a CTO and your CEO has told you that they want you to track, and increase, the engineering and research velocity. So now productivity is your main goal!
Firstly, it’s worth saying that your CEO is right to ask about this - regardless of whether you are a pre- or post-PMF (Product Market Fit) company. If you are pre-PMF, you need to be able to test and iterate fast. If you are post-PMF, you need to be able to scale the business in a robust way.
But what should you track and optimise for? There are lots of metrics that engineering organisations could monitor - for instance, git pull requests per week, agile story points, frequency of releases, number of models trained per week, outstanding bugs, lines of code written per day… The list goes on and on, and it may not be obvious which of these you should track.
It’s important to know you’re measuring the right thing, while keeping in mind that your team can and will try to game the metrics, either subconsciously or consciously. Goodhart’s Law, named after British economist Charles Goodhart, says - “When a measure becomes a target, it ceases to be a good measure.”
Example: One of the large companies I used to work for experimented with tracking lines of code (LoC)/day for engineers. They found that a fraction of the engineers would only produce a few LoCs per day - surely these must have all been slackers?! Upon looking into it, they realised that these engineers were actually spending most of their time - days and weeks - tracking down some critical and very difficult bugs in the product. Thus LoC/day as a metric was meaningless.
A good place to start is to use your gut feeling, based on your prior experience with similar projects. Does it feel like things are moving slow or fast? Does it feel like you’re delivering value? In my experience, a combination of trusting your experience, and identifying relevant internal and external metrics can produce the best result.
Internal metrics (including some of the ones mentioned above) can be used to measure the sources of friction that are holding back the engineering team. They typically help you identify problems with your engineering processes that you need to address. For instance, if you’re a SaaS business and you only release once a month, you probably need to look into increasing your release velocity quite a bit.
External metrics, however, are key for understanding productivity, as they focus on customer value delivered, i.e. the purpose of your entire company. They are much more relevant than any kind of internal velocity metrics because they are measuring actual outcomes and impact.
Your external metrics will depend on your product and target customers and should be derived from your company’s mission and vision. They need to be owned by all functions, not just engineering.
Once you agree on customer value metrics, the entire company - including the engineering team - will have a meaningful way to measure productivity, by tracking the evolution of and setting quarterly goals for these metrics.
For instance, if you’re a travel site whose mission is to provide your users a way to book amazing vacations in a snap, some of your external metrics could include the percentage of bookings completed, the total time it takes to complete a booking, and customer ratings for the properties they booked.
Example: if you’re a travel site whose mission is to provide your users with a way to book amazing vacations in a snap, some of your external metrics could include the percentage of bookings completed, the total time it takes to complete a booking, and customer ratings for the properties they booked.
First of all, it’s important to understand that you can’t improve productivity from the top down, by dictating how things should be done. Your team needs to be part of the solution, so you should involve them from the beginning.
There are both short and long term fixes to improve your engineering team’s productivity, but unfortunately this is not a Pareto distribution. Short term measures may allow you to reap about 20-30% of the productivity gains, which means most of the gains are only possible over the longer term - months and years.
Looking at the short term, one area to start with is to increase your team’s focus. Anything you can do to reduce interruptions will have an immediate impact - like streamlining the number of meetings and reducing the number of attendees, setting aside focus time, or removing the pressure of immediacy from routine communications (turn off your Slack notifications now!).
Example: One of my reports mentioned that he was quite stressed at work. I noticed that his smartwatch was vibrating more than once a minute, as he had notifications turned on for every single message he was receiving. I suggested that he turn off notifications completely and handle email and Slack messages in batches, at a time of his choosing. When we talked later, he mentioned that this helped relieve his stress significantly.
You can provide relative stability over the quarter by defining - and sticking to - quarterly OKRs (Objectives and Key Results). This helps the engineers know what they are working towards and evaluate on an ongoing basis what their priorities should be.
Within the quarter, focus on small and frequent deliverables. Instead of trying to boil the ocean with projects that don’t produce deliverables for months, have a constant stream of smaller things that can be released and tested more quickly.
It is also important to look carefully at how your engineering team’s bandwidth is being spent. Especially for a product-driven company, if the team is spending more than 10-15% of their time on custom work, that should be a red flag. Ideally, 90% of their time should be focused on the product roadmap and the remaining amount dedicated to specific customer requests that are not included in the roadmap.
Tip: I typically use a timeboxing approach to enforce this split. The 90% of engineering time dedicated to the roadmap should be driven via OKRs that are set jointly by product and engineering. The 10% dedicated to custom work can be managed as a Kanban lane, and prioritised by the sales team. It’s important to enforce this split during the quarter, i.e. not let the 10% “bleed into” the 90%.
But as mentioned above, most of the productivity and efficiency benefits happen over the long term: the journey is the goal because your team can constantly improve and significant benefits can accumulate over time.
Over the long term, there are three key things I would recommend you keep a keen eye on to have a very productive engineering team:
When it comes to building a great team, this may seem obvious but cannot be overstated - great engineers can have significantly more impact than average ones. Hire smart people because they will attract other very smart people, but also hire people that fit well into your culture.
Another aspect to consider when growing your team is the mix of seniority levels. My advice would be to favour more senior talent initially. These engineers will be more difficult to find, take longer to hire, and be more expensive - but it absolutely pays back over time. The risk of having a team that is weighted towards more junior engineers, especially as you’re starting out, is that you may see many avoidable mistakes being made, simply due to their lack of experience.
I have found that it also helps to keep teams light and slightly “hungry”, i.e. just very slightly understaffed. That helps teams focus on what’s essential and doesn’t leave them time to spend on things that don’t really matter.
And finally, pay very close attention to your team’s churn rate. A significant churn rate - anything above 15-20% per year - may create instability and could be devastating to your team’s productivity.
Moving on to the creation of a lean and effective work structure - in my experience, every time something doubles in size (e.g. team, customers, traffic), processes will break. If you're a high-growth startup or scaleup, you need to be extremely agile in terms of adapting to these changes. Team structure, processes, systems need to be constantly reviewed and adapted to these changes.
Tip: 10x reviews are quite useful in a rapidly growing company. Once a quarter, the scalability of the critical parts of the infrastructure can be reviewed by asking the question: “what will break if my (traffic/number of customers/another relevant metric) increases by a factor of 10?” This can help identify and fix scalability issues before they actually happen, at a fraction of the cost.
The last area to focus on, which will increase productivity over the long term, is making sure you have the right tooling and a solid tech foundation in place. This takes more time than hacking things together but pays back significantly.
It should include outlining the best coding practices, maintaining technical documentation, and investing in developer tooling. There are internal metrics that you can track here, for instance, stability and latency of your CI/CD pipeline, and the percentage of releases that need to be rolled back. It is also quite helpful to build a developer sandbox so engineers can try things out very quickly.
Technical debt needs to be paid back over time (preferably before it’s “almost too late” and your product becomes impossible to maintain or evolve), and you need to allocate time for this. Typically, 20% of the engineering bandwidth should be allocated to technical tasks that are prioritised by the engineers.
Last but not least, make sure your engineers use the product regularly so they can relate to your customer’s experience. This is easier for B2C companies of course, but is also possible for a B2B business by creating a separate environment where the team can play around with the product.
To successfully scale a business, everyone in the company needs to understand the customer and how the company delivers value to them.
As mentioned, there should be a set of external metrics agreed across the business that measure customer value. Everyone needs to buy into and track these metrics, this is not the sole responsibility of the engineering product teams.
Engineering is only one critical component that must work together with the rest of the organisation. Engineering can deliver productivity gains, but needs full alignment with, and communication channels into, each part of the business.
The relationship between product and engineering is unique and requires a strong culture of working together. These functions should have a shared understanding of the customer and shared responsibility for driving outcomes.
Product will always push engineering to do more, and it is the engineering team’s role to highlight the trade offs and choices that need to be made. On its own, engineering can produce linear productivity gains, but lack of focus in product can create an exponential increase in the amount of work and lead to failure.
To summarise, here are a few things to remember on your journey to increase your engineering team’s productivity:
Sign up for our newsletter to stay up to date on news from Balderton, and our portfolio.
The Stables
28 Britannia street
London WC1X 9JF UK
+44 (0) 20 7016 6800
+44 (0) 20 7016 6810
