Impact is Queen. Be Measurably Better.

Two of our core values focus on outcomes, not activities. We use data to ensure we’re always confident in our decision making. Few features make it out to users without first running in an experiment. We have a simple but effective experiment framework. Periscope provides a way to share these metrics with both technical and non-technical team members. Engineers who build features consider how the data is collected, how it is surfaced, and work with the other stakeholders to determine how to measure the success of it. Beyond quantitative data, our design and customer success teams have feedback loops for collecting qualitative responses to new features to surface product challenges.

We collect and use data at every step of our process.

Everything from the profile survey questions a customer initially fills out to the feedback and ratings they provide for each piece we send them is measured and tracked. Data and machine learning is used to solve our cold start problems, improve the items we send out on subsequent boxes (our system gets smarter each time), improve efficiency of our operation, and even to make our own clothes. Almost all of our product roadmap is driven by data analytics. We are constantly running AB tests and evaluating results in order to inform the next step of feature iteration.

Engineering decisions aren’t opinion-based.

If we don’t have a way to measure something, we focus on finding a solution instead of deciding that data collection is too difficult to do. Once we’ve collected data, we discuss (and sometimes debate) its value and then use it to optimize our path forward. For example, if we are evaluating whether or not to lock down temperature ranges, we test those systems in different temperature environments to verify effectiveness before we move forward instead of taking a “this should work” approach.

We have a new “Cannalytics” team at PAX that lead our data-driven approach across all functions. Data is a core part of the feedback loop for both our Hardware and Software Engineering teams. We’re able to look at event data from our devices to understand different use cases and emergent behavior. These insights enable us to improve our existing products and create new products and features that are most valuable to our customers.

Additionally, on the Partner Operations team, we administer a series of surveys for all of our Brand Partners. This allows us to collect data points on pod production and capture the information in Salesforce. As a result, it’s incredibly easy for us to generate reports that help inform data-driven initiatives. We collect data such as units produced per hour, equipment failures, and viscosity readings. This helps inform PAX on how to create solutions, and efficiencies for partners.

We are experienced data engineers and we back up our decisions with available information.

When Przemek (our co-founder) wanted to buy a camper car, he scraped most popular listing sites and checked if the price was seasonal (it was, by the way). We apply data-driven culture in our work and personal lives. Gathering information and presenting it in a clear way is part of our decision making process. We try to share this culture with our clients.

We believe that data and metrics are the foundation of building trust and making impactful decisions.

We strive for integrity and accuracy to ensure that quality data is delivered to the market. We are metrics-driven in our decision-making both internally and externally. We want to make sure the decisions we make - we make with data and metrics backing it. We have Amplitude, Google Analytics, and Custom Grafana dashboards set up so that we can constantly monitor the effects of the changes that we make. We spent countless hours making sure we can trust the internal metrics, and that people have the metrics in place as their north star as they go about their day-to-day.

What is the overall company revenue? A lot of companies actually can’t answer this question. Arriving at a number that can be trusted can be very difficult because of different currencies, custom deal structures, and various payment methods. We constantly monitor and validate our north star metrics (Daily Active Users, Revenue, Press mentions) and discuss our progress during every weekly All Hands meeting.

Data is at the heart of decision making in Digit.

We use Chartio and Redshift extensively to analyze and test all aspects of our product. Our product is designed to be passive in that customers can sign up for Digit and then go on about their lives. Digit will then work in the background to ensure they are financially healthy. Given this passive nature, we rely on data (both qualitative and quantitative) to guide product roadmaps and growth experiments. We currently have 35 features and experiments which are being tested in production right now.

As an engineer at Digit, you’ll be responsible for working with our data analysts and user researchers to define what metrics you care about and which you are trying to improve.

Our mission requires chaining many complex scientific processes together.

When you do this, errors and issues can easily multiply. We use data to tackle this head-on, by systematically tracking the data from all our automated processes, which helps us remove human bias and guesswork from our work. Why did the latest run of a particular protocol fail? We could either try and guess using scientific intuition and previous experience, or look at the data for that run, and compare and contrast it with previous runs to highlight the key differences in proceedings. Did the temperature change? Was a reagent left out for longer than usual?

This approach also helps to optimise our processes. We want to make our system as efficient as possible, and concentrate effort where it will bring most reward. By systematically tracking every process and every parameter, the answers come straight from the data, avoiding a reliance on guesswork to figure out where to target our efforts.

Data is also key for our evolution engine - it parses vast numbers of DNA sequences to infer rules about how the sequences are constructed, in order to shape our experimental testing. As with many ML systems, the more data it receives, the smarter it gets!

Every engineer works with at least one analyst.

We love experimentation and letting the data tell us what our members like best. This allows us to quickly ideate, come up with hypotheses of what we want to test, and see how our users respond. Everything we do is based on analytics derived from these experiments. For example, we use Mixpanel and Optimizely to do things like A/B testing on our site.

As a result, we work very closely with the Analytics department. We currently embed an analyst in every product team, so every engineer works directly with at least one analyst. While each team is responsible for deriving analytics and making decisions based on their data, it is our policy that everyone has access to all analytics at Credit Sesame.

We look at data like click-through rates, how many users are completing a particular funnel, and see what resonates with our members. Before we start any experiment, we have strict guidelines around defining success criteria.

Even internally within our department, we use other data-driven metrics to make decisions. While the product team focuses on the business value, the engineering department considers things like tickets resolved, tickets completed in our sprint backlog, and other work productivity metrics. This helps us improve our turnaround time for tickets, and tells us how many tickets we can take on over the course of a week. We evaluate these metrics during sprint retrospectives.

Note: This is not a stock photo :)

We track everything and make decisions based on numbers, not opinions.

We are an extremely data-driven organization. Our analytics team supports dashboard creation as well as ongoing and ad-hoc analyses. As an organization, we track and report against a plethora of metrics, some of which are related to user events, feature modifications, and revenue changes. The engineering team uses data for suppression, sunsetting, monitoring, and charts creation. Features are rigorously tested using both A/B and bandit algorithms. All teams use robust dashboards for weekly and quarterly strategic planning. Our analytics team uses platforms such as Chartio, Amplitude, and Redash.

We see truth in data.

The evolution of our company, starting as a B2C tech support company, transitioning to B2B tech support, and pivoting into the marketplace we are today, is evidence that we always follow the signal of data. Some of the leadership team have backgrounds in banking, private equity, and financial modeling, so naturally there is a focus on metrics. Every Monday, everyone at the company meets to share their weekly goal. You’ll never hear someone say, “I’m going to do market research this week,” because we only set discrete, quantifiable goals like, “This week, I am going to research 10 companies with a deliverable on Friday.”

We make decisions based on data.

Data informs everything we do at Stitch Fix, from which pants we should order to the images that should appear on the homepage. We believe we can use data to create a better experience for our customers. That's why we employ nearly as many data scientists as we do software engineers. Data informs so many of the decisions that we make here at Stitch Fix, that part of life here is working together with data scientists to craft experiments and solve problems.

One example of how we leverage data is in how our data scientists have improved our demand model. Our buying decisions are much more informed than they would be if they were based on traditional retail models, and give us a better grasp on things like, “How many customers do we think we’ll have?” and “How many customers will be wearing smalls vs. extra smalls?” Where traditional retailers rely on industry standard size breaks, we leverage data to make informed decisions when buying and are able to better utilize that inventory.