Engineering teams that prioritize risk-taking over stability

We’re not a company of optimization or incrementalism; we consistently support each other in making big bets and finding out how to make the biggest impact we can.

This often means that we fail, so we choose to fail forward. Fail, learn, move on to the next. If you’re a perfectionist or risk-averse, you won’t be very happy at LDT at our current stage. We are currently in a run-around-and-break-shit phase and believe that done is better than perfect. All engineers are on the front lines, operate very close to production, and have complete autonomy to build, release, and revert.

Our technology moves very quickly, too. We don’t mean that in terms of using bleeding edge technology, but that we (probably because of our stage and ongoing search for product market fit) regularly add new tech into the stack, rip out experiments that didn't work, and replace early iterations of ideas as the product evolves.

What might be seen as “cutting corners” (be it some part of QA or rigor in test coverage), we see as necessary in order to meet deadlines. We always try to maximize how much we can fit into a sprint and go as close to committing as much as possible. We also don't worry about overcommitting. If we've estimated poorly and added too much to a sprint or phase, we reassess in the next planning stage and ask ourselves if projects are still a priority or have the goalposts moved (again)? The team members who have come from big and slow movers like Facebook and Google have thrived at how fast we work and get shit done here (after the initial shock that is 😉).

We prioritize experimentation and hypothesis-driven decision making.

When making a decision, we first ask ourselves whether it is a “one-way” or “two-way” door. At our stage, most decisions turn out to be more reversible than we think, and making that explicit helps us contextualize the amount of risk we are actually taking on. For example, we intentionally included features in our alpha product that won’t scale in a cost effective way because we want to learn and prioritize customer feedback early on.

We plan as much as is useful – usually a quarter or two at a time – but we are realistic about our ability to predict the future. We treat our plans and product goals like working hypotheses to be validated through shipping features and actual customer usage and feedback.

We want to build a team that is pragmatic, not dogmatic in our decision making. Sometimes we make decisions based on intuition. Sometimes we have to disagree and commit. And sometimes we turn out to be wrong and need to change course. At the end of the day, our goal is to optimize for the velocity of our learning rather than the percentage of things we get right on the first try.

“Fail fast” is one of our core values.

In order to innovate quickly, we have to be willing to take risks and learn from any missteps. We always ask ourselves, “Are we thinking big enough?” and strive to figure out the best way to work fast and effectively. One way we do this is with our hackathons, which we typically have three times a year. We dedicate a week to coming up with creative, out-of-the-box ideas and the sky’s the limit. For example, during a previous hackathon, the winning team integrated Algolia to improve the search experience in our docs. This change is live today!

While mistakes are inevitable, we don’t dwell on them and always move forward quickly. The platform team migrating from Terraform to Crossplane (to simplify building automation on top of our infrastructure) is a great example. Since Crossplane is a relatively new project, it doesn’t have full feature parity with Terraform, but it offers a better experience in general. We were hoping that most of our resources would easily convert into the Crossplane equivalent. We had success moving some resources over, but for ECR (our Docker image store), Crossplane’s AWS provider didn’t have a feature we needed at the time, but Terraform did. We thought we were stuck, but Crossplane released a new provider that wraps Terraform, and it had the functionality we needed. The caveat was that the provider was still in preview, but it looked like it could be a potential solution. We were able to get it working, but we realized that it wasn’t production ready for a couple of reasons (mainly it made the EKS cluster very slow). We decided to remove that Terraform provider and migrate that code later once the Terraform provider improves or the AWS provider reaches parity.