“All happy families are alike; each unhappy family is unhappy in its own way.”, so starts Leo Tolstoy's novel Anna Karenina.
This came to be known as the Anna Karenina Principle - there are many different ways one can fail, but only one way to win: by avoiding each of the routes to failure. The beauty of being a consultant is to experience this first hand. We are able to work with the best data teams in the world, but we're also brought in to try to help companies that are not doing so well.
A data team is any team whose core occupation is dealing with huge amounts of data. Their goals can range wildly: from creating dashboards that help democratize insights and trends, to creating predictive machine learning models that support entire product lines. What they all have in common is that working with data is neither cheap nor trivial. And leaders often get frustrated that their data teams are continuously failing to add value to their business.
What I’ve learned is there is a profound difference between how the very best data teams work and all the rest. By taking a page of the classic Ben Horowitz’s post “Good Product Manager/Bad Product Manager”, I hope to provide a better understanding of what makes a good data team tick.
Good data teams share a compelling vision of how their efforts can change their business. Bad data teams think they are there to produce numbers.
Good data teams know the market they operate in - the product, the product line, and the competition - extremely well. Bad data teams are not interested in learning beyond the technical scope.
Good data teams take their inspiration and ideas from observing their customers’ struggles and analyzing the data they use or generate. Bad data teams gather requirements from the highest-paid person in the room.
Good data teams understand who each of their key stakeholders is and what their constraints are. Bad data teams believe stakeholders are there to issue them requirements.
Good data teams engage directly with the end-users of their models to better understand their needs. Bad data teams think they are the user.
Good data teams understand the importance of having a data model and clear documentation. Bad data teams are fine with having all this knowledge inside one person’s head.
Good data teams love to have brainstorming discussions with smart thought leaders from across and outside the company. Bad data teams get offended when someone outside their team dares to suggest they do something.
Good data teams believe in the importance of having a balanced mix of engineers, data scientists, and analysts. Bad data teams believe they should all have the same expertise and push the other responsibilities to other departments.
Good data teams are obsessed with finding and solving business problems. Bad data teams are obsessed with applying a specific technology they can later show at a conference.
Good data teams can tell you what business metrics were impacted by their models. Bad data teams get confused when asked for numbers that show their models’ impact.
Good data teams investigate the problem before investigating the solution. Bad data teams jump to the solution and take the problem as a given.
Good data teams have a deep understanding of where their data comes from, and will often build data pipelines themselves. Bad data teams think that is someone else’s job.
Good data teams understand the importance of adopting good engineering practices for collaboratively building products. Bad data teams think DevOps/MLOps, code reviews, and testing are a waste of time.
Good data teams start simple, draw experiments, and then iterate. Bad data team starts with the most advanced technique they know.
Good data teams know when to abandon a project. Bad data teams continue to work on something out of stubbornness.
Good teams instrument their models so they can understand how they’re being used and the value they’re generating. Bad teams consider monitoring models and model analytics reporting a nice to have.
Good teams celebrate when their models achieve a significant impact on the business results. Bad teams celebrate when they finally release something.
Realizing you have a bad team is a painful experience. It's a sinkhole of money and talent and, if you don't act, it only grows worse.
We, at DareData, are experienced in finding data scientists/engineers but no talent wants to stick around with a bad team. It's a discussion we don't want to have any more than you do, but we won't shy away from it - and neither should you.