Impact is Queen. Be Measurably Better.

Two of our core values focus on outcomes, not activities. We use data to ensure we’re always confident in our decision making. Few features make it out to users without first running in an experiment. We have a simple but effective experiment framework. Periscope provides a way to share these metrics with both technical and non-technical team members. Engineers who build features consider how the data is collected, how it is surfaced, and work with the other stakeholders to determine how to measure the success of it. Beyond quantitative data, our design and customer success teams have feedback loops for collecting qualitative responses to new features to surface product challenges.

To say we’re data-driven is really an understatement. We’re data obsessed.

Enigma’s purpose is to move the world forward by engineering deeper understanding through data, so data is at the core of everything we do.

Having a passion for solving complex problems with data is an absolute must, regardless of whether you’re working directly on a data infrastructure squad, partnering with a data-scientist on a product squad, or building the frontend that powers a best-in-class user experience.

At the core of our offering is a promise to focus on the data and provide transparency into our data-science so customers can focus on getting the intelligence they need in order to make better decisions. Whether we’re working with a large financial institution to help them verify small businesses for loans or screen customers and transactions for sanctioned entities, we’re creating a simple, seamless way for our users to access actionable intelligence from hundreds of authoritative data sources.

We aren’t just data-driven, we are data-obsessed.

When patients sign up, we receive their consent to retrieve and aggregate their medical records. Our records platform is built around human-in-the-loop machine learning to take the hundreds of pages of scans and faxes of medical records per patient, digitize them, then extract and organize key medical concepts. For our patients and research partners, we deliver structured, queryable data to help patients identify key information in their records and help research partners find deeper insights that were previously only unlocked by manual, painstaking analysis of records. In each of these three key pillars, data is a first class product that we are obsessed with. Like most companies, we use data to power our product, technical decisions, strategy, and roadmaps to ensure that we are delivering value to our customer; but where we truly differ is that data is the product that we acquire (with consent), refine, and deliver. Our data is used to accelerate research and treatment programs. We believe in the transformational power of data – for patients, researchers, and our team.

We have access to a vast amount of data that lets us measure the impact of our work on real-world user behavior.

At CircleCI, we are in a unique position to be able to review large amounts of data on how technology delivery teams act in the wild. Our cloud continuous integration and continuous delivery platform processes over 1.6 million job runs per day for more than 40,000 organizations and over 150,000 projects. To that end, we’re constantly looking at data to inform best practices and make improvements.

Every engineering team has a product manager and data scientist/analyst. Engineers work closely with data scientists to determine what data we should be capturing, how easily we can measure the things we’re interested in, and how to measure the impact our work has on user behavior and usage patterns.

We track OKRs at the company, department, and team level. For example, at the team level, we pay attention to delivery data (i.e velocity, cycle time, PR review time), SLOs and SLIs, escalation data, and PR review time. For production incidents, we track mean time to diagnosis and resolution.

We also look at market data, usage data for our product (like adoption rates), and growth metrics using A/B testing. Since insights is part of our product, we’re starting to build out a data intelligence/AI Ops team. Some of our analysis dashboards include Looker, DataDog, Amplitude, Segment, Honeycomb, and more.

We track the percent of time teams spend on different kinds of work: new features, customer support, and technical investment/maintenance. When we see teams that have a consistently high percentage of support and maintenance work, we might do any of the following:

• Review our documentation to ensure customers understand how a capability works
• Identify an opportunity for a new feature to address the need customers are trying to satisfy
• Split the team to reduce its area of responsibility
• Focus on reliability or scalability improvements until support requests and maintenance needs are reduced
• Hire more people to make the team bigger

If you’re a data nerd like we are, New Relic is your paradise.

There are a lot of companies that drive initiatives based on executives’ feelings, but they’re not necessarily grounded in actionable data. At New Relic, simply feeling that something is wrong won’t cut it for us – we aim to make decisions rooted in data. We pride ourselves on collecting data on everything from sprint velocity to data quality in order to make decisions regarding process and system improvements. DOS strives for monitoring-driven development, where monitoring is the first step of a project instead of the last. This is key in establishing a baseline for the current state and for demonstrating the utility of new features and system optimizations. That being said, we try to strike the right balance between making decisions based on math/data and remaining empathetic.

Every decision we make is backed by data.

Unlike other startups where people might take their best guess and just wing it, everything we do at Academia is data-driven. Much like the research scientists whose published papers are on our platform, we use hypothesis-driven decision making to guide product and strategy. Instead of proposing an idea, we present a testable hypothesis based on observation and research. By gathering data that supports the need for a new feature or workflow, we can then experiment to determine whether an expected outcome is achieved. From A/B testing a landing page to restructuring our performance review process, everyone at Academia relies on data to inform decision making.

Our customers are talking to us all the time.

Unlike in many B2C companies where you don’t know exactly what consumers are thinking, our customers are talking to us all the time. When they’re giving us the same feedback and pointing out the same things they’d like to see change, it’s obvious what we need to fix. We take customer data seriously when it comes to product development, because we are creating a tool that people interface with directly. By nature, a lot of those interactions are manual, so we aim to make them as accessible and efficient as possible. To do that, our customer operations team tracks their time. For example, we gather data on how much time is being spent on emails or how much time is spent on personalized payroll plans. We share the time-tracking data results in our monthly all-hands meeting. This allows us to make sure we’re prioritizing major feature developments to be as efficient as possible.

Data is at the heart of decision making in Digit.

We use Chartio and Redshift extensively to analyze and test all aspects of our product. Our product is designed to be passive in that customers can sign up for Digit and then go on about their lives. Digit will then work in the background to ensure they are financially healthy. Given this passive nature, we rely on data (both qualitative and quantitative) to guide product roadmaps and growth experiments. We currently have 35 features and experiments which are being tested in production right now.

As an engineer at Digit, you’ll be responsible for working with our data analysts and user researchers to define what metrics you care about and which you are trying to improve.

Decisions are never made without supporting data.

We are instrumenting all of our company’s processes. In operations, we collect data on customer activity, LTV, threats, and engagement. For engineering, we capture data about performance, failure rates, response times/latency, build times, service throughput, and developer efficiency/happiness/engagement. These are just the tip of the iceberg of what types of data we will collect.

Data informs every decision we make about our product, too. At a very high level, we collect data about the threats our customers see. This data dictates which threat behaviors we model (for machine learning) first. We target the threats that are seen most often by the largest number of customers. In addition to frequency, we track severity of each threat and then model the most severe threats to our customers. Engineers at ActZero always stay close to the data as our team is structured so that engineers and data scientists work in partnership.

As an analytics platform, data drives everything we do!

Our core philosophy is to use data to help us learn where to evolve our product. We use Amplitude to instrument everything (>1 trillion data points per month) we do and all of our teams focus on understanding user behavior and using it to influence our roadmaps. In short, we use our own product in order to build a better product. We rely on Amplitude to research what we should be working on, identify which customers to reach out to, validate hypotheses, and determine leading and lagging indicators as we build new features. When we find shortcomings in our own system, we invest in building it. In fact, we never use other tools to do the job. When we needed a scalable data-ingestion platform, we built it in-house.

Sometimes, what we build for our internal use ends up getting rolled out. For example, Period-over-Period Analysis was a feature we built for ourselves to compare this week’s data to last week’s and give us a look at percentage change over time. We eventually rolled this out as a free feature for all of our customers to use and it continues to be one of the most popular.

Data improves the insurance experience for our brokers and clients.

By running a full-stack brokerage, we sit in a privileged position to analyze data. We collect structured information about risk from clients (e.g. property data, business metrics, etc.) and risk pricing data from insurance markets. This industry is particularly opaque and we have a rare opportunity to provide transparency and build the first ever multi-sided marketplace for risk.

There are lots of thorny technical problems to solve at Newfront, from workflow systems to data modeling to document generation. We always rely on the data we collect from brokers and clients to inform new projects and test initiatives before release.

Several engineers on our team have backgrounds in data analysis and mathematics, and we encourage the entire team to leverage their expertise when shipping products.

We track everything and make decisions based on numbers, not opinions.

We are an extremely data-driven organization. Our analytics team supports dashboard creation as well as ongoing and ad-hoc analyses. As an organization, we track and report against a plethora of metrics, some of which are related to user events, feature modifications, and revenue changes. The engineering team uses data for suppression, sunsetting, monitoring, and chart creation. Features are rigorously tested using both A/B and bandit algorithms. All teams use robust dashboards for weekly and quarterly strategic planning. Our analytics team uses platforms such as Chartio, Amplitude, and Redash.

We make decisions based on data.

Data informs everything we do at Stitch Fix, from which pants we should order to the images that should appear on the homepage. We believe we can use data to create a better experience for our customers. That's why we employ nearly as many data scientists as we do software engineers. Data informs so many of the decisions that we make here at Stitch Fix, that part of life here is working together with data scientists to craft experiments and solve problems.

One example of how we leverage data is in how our data scientists have improved our demand model. Our buying decisions are much more informed than they would be if they were based on traditional retail models, and give us a better grasp on things like, “How many customers do we think we’ll have?” and “How many customers will be wearing smalls vs. extra smalls?” Where traditional retailers rely on industry standard size breaks, we leverage data to make informed decisions when buying and are able to better utilize that inventory.

Instacart is a data-driven organization.

We build and test often, and use as much data as we can gather to inform new features for our customers and shoppers. Our data science team in partnership with our product team develops hundreds of experiments per quarter to improve the customer, shopper, retailer and advertiser experiences across our product portfolio.

Our experiment-driven culture enabled us to ramp up efforts during the busiest period in our eight-year history. In late 2019 we began testing “Leave at My Door Delivery.” Months later, at the onset of the pandemic, we observed more and more customers in test groups opting into Leave at My Door Delivery. Knowing how critical this could be to customer and shopper safety, and having the test data we needed to make a confident decision, we were able to fast-track the experiment window and pushed the feature nationwide in mid-March to meet the moment.