We deploy many applications on any given day.

We didn’t used to though. In 2015, when Plastiq first moved from Boston to San Francisco, we had a 2-week development cycle followed by a 3- or 4-day feature testing and regression testing cycle that would culminate in a dreaded Thursday night release. It wasn’t a great development environment and we made it a priority to improve.

We automated application builds, enabled working unit tests, and created a simple end-to-end automated regression suite. We next automated our deployments to an integration environment. Today, we’ve completed disabled our QA environment and consolidated all of our non-production environments (our integration environment is our QA and staging environment all in one!).

At Plastiq, we also use the Trunk Based Development model which means that we commit directly to the master branch. Builds and deployment are automatic and then unit, api and end to end tests are run. We are now moving toward continuous deployment so that our commits to master will be automatically deployed to production.

You can learn more about where we started and where we’re headed here.

We believe in delivering updates to our users as soon as they are ready.

Stitch Fix has long practiced continuous deployment. Any build on the master branch that passes its tests (which usually takes <10 minutes) is then followed by a one-step deployment script. This has worked successfully for years because of the quality of our automated tests, which we rely on heavily. Some teams deploy a few times a week while other deploy several times a day. Despite differences between various engineering teams and projects, there is a shared effort to make smaller, more incremental changes at Stitch Fix.

“Software only becomes valuable when you ship it to customers.” - Darragh Curran, VP of Engineering

We ship simple, early and often, and iterate rapidly. It’s not uncommon for us to release our core software 150 times per working day. An Engineer can get their latest PR into the hands of all of our customers in 30 minutes. We recognise that this release process is the pulse of a growing organisation, and realise that software only has value when it’s in the hands of our customers. To support this we have multiple stages of integration tests and also a bunch of tools that allow us to flip features on and off in production in various ways. We’ve also got a bunch of ways of getting in-production feedback and a really tight process for managing and learning from the in-production mistakes we will inevitably make while moving this fast.

We deploy code to production continuously Monday through Thursday, 9:30 am to 4 pm.

Once tests pass (which usually takes about 20 minutes total) on an engineer’s approved pull request, it is deployed to our staging environment, where it runs for 45 minutes before getting deployed to production. We currently use Jenkins to run our deployment scripts for the web application and our Android app, and we use CircleCI to deploy our iOS app. Our iOS and Android apps are released a minimum of once a week, and often more often than that.

We value continuous delivery (and testing our code) because working iteratively gives us the ability to move faster by not bundling too much functionality together at once. It reduces the risk of rolling out many changes, finding an issue, and needing to roll back a lot of work. It is also good for engineers to have the ability to test their changes locally, on staging, and on production quickly, to make decisions for the next iterations. Our ability to ramp code from an admin dashboard also allows us to turn off a feature instantly if there is an issue, without requiring a deploy.

All of these practices (thorough testing, continuous deployment, and utilization of the ramp dashboard) make the basic practice of deploying code less risky, which allows us to make more interesting and experimental choices in how we problem solve in code.

We work using a GitFlow-like process.

Stories are worked on in branches, and then merged to master when they are ready. Our test suite then runs, including mobile tests. If everything is green, we deploy both the server and mobile apps so we can accept and test our work. This all happens automatically.

We use tools like Heroku, CircleCI, and CodePush to manage our operations. We write our tests with tools like rspec, jasmine, and selenium. Being an international company, we’ve also integrated a translation process (both automated and with human review) into our deployment process.

We deploy to production about twice a week, but that’s a choice by our PMs as to when a feature is ready. Just because we work at Airbus, that doesn’t mean that we can’t use modern software practices and tools!

We ship our product multiple times a day.

We are breaking down problems into smaller ones which are easier to solve and test. Not only are these more manageable, but it reduces risk and makes it easier to roll back changes. As a result, we spend less time worrying about deploying and have more confidence each time we push code to production. Typically, features only stay in staging for a couple of hours before going to production. Engineers are responsible for taking their own code to production, so if it breaks, you’re also responsible for fixing it.

We are continuously deploying to production from our master branch.

Code is deployed via pull requests, once another engineer has reviewed the work and builds are passing (build is currently ~8min). We use Github for all issue tracking and code reviews. In the last month, we have deployed to production 150 times (deployment is zero-downtime and currently ~10min).

We like to keep the wheels turning.

We believe in continuous delivery and deployment so that we can address the most pressing user needs and get rapid feedback on our work. We release often (multiple times a day) in incremental changes to our production environments. We also value test-driven development, and this gives us the confidence to release on such a frequent cycle. We use Github to manage our code, Buildkite to deploy our code, and our comprehensive test suite runs in a few minutes.

We like to move fast and innovate.

We have lots of different microservices and applications, and we’re constantly shipping new features. As soon as a PR is merged, tests are run and things are auto deployed to our infrastructure. Our DevOps team has built a great (and open source) layer on top of kubernetes that allows engineers to ship new services quickly and reliably with out-of-the-box logging, metrics, scaling, or load balancing. We like to work with our hands untied so we can make real impact, every day.

We have a really cool deployment process. Hear us out.

We use Jenkins in a way that is deeply integrated with Jira and Bitbucket. We deploy to our non-production environments by simply changing the status of a ticket in Jira. From there, Jenkins gets the branch name from Jira and generates the build on the appropriate server. We do review pull requests before they go in product QA and from there QA can merge to release through a corresponding Jira status. Everyone loves this system!!

Our office is co-located with our fulfillment center, so we’re able to iterate quickly with the teams that use our software every day to get local food to local customers.

We work quickly because the faster we can get a solution in someone’s hands, the faster we can learn how to make it better. You should be able to build a feature, ship it on a Tuesday, and then talk your user to get feedback and make updates before Thursday. We make this possible by having our engineering team work out of the foodhub: the place where farmers show up twice a week, selling locally, organically grown produce and where our operations team sort through produce, pack bags, and deliver food to our waiting customers. While it might not be the quietest work space, it promotes continuous deployment and provides task significance, the perception that one’s job has a positive impact on other people.

We push multiple times a day.

The things we focus on the most is data modeling and our API, so we have almost converged on our coding styles. We work on having a straightforward and repeatable deployment process so that we can write, test, and release code faster and more frequently.

On time. On target. Our word is bond.

A the end of the day it is all about execution. We strive to efficiently prioritize the features in our development pipeline and deploy as quickly as possible. The quicker we have a feature out - the quicker we can learn and improve to provide our users with functionality they really want.

Tests are automatically run against pull requests, and required to pass before merge. They run in ~ 2 minutes.

Once your commit has been approved by another engineer, it is immediately deployed to production when you hit merge. Our services are deployed to ECS and will be gracefully auto-rolled back if they fail to start. We use a variety of tools (Cloudwatch, Sentry, Pingdom) for monitoring and alerting, and Terraform to manage our infrastructure as code. (These are manually applied first to a staging environment, then to our production environment.) We use experiment flags to ship new features to production early, and then develop and deploy incrementally until they’re ready to turn on for all of our customers.

We have bi-weekly sprints.

Our product team researches features and produces a “narrative” which engineering then works with to offer suggestions and modifications to make more achievable. We then translate the feature to a technical specification which is then entered into the sprint for development work. The aim of this is to try and make sure we deliver what Product wants, and that we develop it in the way that will be most future proof.

We use git flow and each PR is reviewed before it’s merged to the develop branch. We use CI/CD, and when a PR is opened, all the unit tests are run automatically. When the PR is merged to develop, all the unit tests are run as well as Rainforest tests. If the tests pass, then develop is automatically merged into master and released into production. We push code to production multiple times a day with high confidence.

We break every feature down into the smallest components and ship code every day.

Right now, we’re not focused on perfecting every feature and we don’t write tests for every line of code. We operate with an “80/20 rule” in mind which means, if it’s 80% of the way to what you’d consider to be perfect, then it’s already ready to roll out. We’re very conscious about how code will look to us a few months from now but we force ourselves to not obsess. It’s the balance we’ve agreed on as a team to iterate quickly. It’s well understood that if it’s 80% of the way there, it’s good enough to push. Iterative deployment is the name of the game. (We should mention though that once every two weeks, we devote an entire day (Squash Days) to fix bugs, re-write tests, or make other minor improvements.)

We are constantly pushing to production.

Whenever a feature is ready and has been tested in our staging environment, we typically push it live asap. We have a modern CI/CD process that integrates CircleCI and AWS to get code out quickly. We make use of feature flags when needed in order to keep our deploys compact and small. New hires can expected to be shipping to production within their first couple weeks.