Data Science. Field Cady
id="ulink_2298f80d-af79-56ad-8c43-c441e4b5e683">Two trends have changed that situation. The first is the intrusion of computers into every aspect of life and business. Every phone app, every new feature in a computer program, every device that monitors a factory is a place where computers are making decisions based on algorithmic rules, rather than human judgment. Determining those rules, measuring their effectiveness, and monitoring them over time are inherently analytical. The second trend is the profusion of data and machines that can process it. In the past data was rare, gathered with a specific purpose in mind, and carefully structured so as to support the intended analysis. These days every device is generating a constant stream of data, which is passively gathered and stored in whatever format is most convenient. Eventually it gets used by high‐powered computer clusters to answer a staggering range of questions, many of which it wasn't really designed for.
I don't mean to make it sound like computers are able to take care of everything themselves – quite the opposite. They have no real‐world insights, no creativity, and no common sense. It is the job of humans to make sure that computers' brute computational muscle is channeled toward the right questions, and to know their limitations when interpreting the answers. Humans are not being replaced – they are taking on the job of shepherding machines.
I am constantly concerned when I see smart, ethical business people failing to keep up with these changes. Good managers are at risk of botching major decisions for dumb reasons, or even falling prey to unscrupulous snake oil vendors. Some of these people are my friends and colleagues. It's not a question of intelligence or earnestness – many simply don't have the required conceptual background, which is understandable. I wrote this book for my friends and people like them, so that they can be empowered by the age of data rather than left behind.
1.3 Data‐Driven Development
So where is all of this leading? Cutting out hyperbole and speculation, what does it look like for an organization to make full use of modern data technologies and what are the benefits? The goal that we are pushing toward is what I call “data‐driven development” (DDD). In an organization that uses DDD, all stages in a business process have their data gathered, modeled, and deployed to enable better decision making. Overall business goals and workflows are crafted by human experts, but after that every part of the system can be monitored and optimized, hypotheses can be tested rigorously and retroactively, and large‐scale trends can be identified and capitalized on. Data greases the wheels of all parts of the operation and provides a constant pulse on what's happening on the ground.
I break the benefits of DDD into three major categories:
1 1. Human decisions are better‐informed: Business is filled with decisions about what to prioritize, how to allocate resources, and which direction to take a project. Often the people making these calls have no true confidence in one direction or the other, and the numbers that could help them out are either unavailable or dubious. In DDD the data they need will be available at a moment's notice. More than that though, there will be an understanding of how to access it, pre‐existing models that give interpretations and predictions, and a tribal understanding of how reliable these analyses are.
2 2. Some decisions are made autonomously: If there is a single class of “killer apps” for data science, it is machine learning algorithms that can make decisions without human intervention. In a DDD system large portions of a workflow can be automated, with assurances about performance based on historical data.
3 3. Everything can be measured and monitored: Understanding a large, complex, real‐time operation requires the ability to monitor all aspects of it over time. This ranges from concrete stats – like visitors to a website or yield at a stage of a manufacturing pipeline – to fuzzier concepts like user satisfaction. This makes it possible to constantly optimize a system, diagnose problems quickly, and react more quickly to a changing environment.
It might seem at first blush like these benefit categories apply to unrelated aspects of a business. But in fact they have much in common: they rely on the same datasets and data processing systems, they leverage the same models to make predictions, and they inform each other. If an autonomous decision algorithm suddenly starts performing poorly, it will prompt an investigation and possibly lead to high‐level business choices. Monitoring systems use autonomous decision algorithms to prioritize incidents for human investigation. And any major business decision will be accompanied by a plan to keep track of how well it turns out, so that adjustments can be made as needed.
Data science today is treated as a collection of stand‐alone projects, each with its own models, team, and datasets. But in DDD all of these projects are really just applications of a single unified system. DDD goes so far beyond just giving people access to a common database; it keeps a pulse on all parts of a business operation, it automates large parts of it, and where automation isn't possible it puts all the best analyses at people's fingertips.
It's a waste of effort to sit around and guess things that can be measured, or to cross our fingers about hypotheses that we can go out and test. Ideally we should spend our time coming up with creative new ideas, understanding customer needs, deep troubleshooting, or anticipating “black swan” events that have no historical precedent. DDD pushes as much work as possible onto machines and pre‐existing models, so that humans can focus on the work that only a human can do.
1.4 How to Use this Book
This book was written to bring people who don't necessarily have a technical background up to speed on data science. The goals are twofold: first I want to give a working knowledge of the current state of data science, the tools being used, and where it's going in the foreseeable future. Secondly, I want to give a solid grounding in the core concepts of analytics that will never go out of date. This book may also be of interest to data scientists who have nitty‐gritty technical chops but want to take their career to the next level by focusing on work that moves the business needle.
The first part of this book, The Business Side of Data Science, stands on its own. It explains in non‐technical terms what data science is, how to manage, hire, and work with data scientists, and how you can leverage DDD without getting into the technical weeds.
To really achieve data literacy though requires a certain amount of technical background, at least at a conceptual level. That's where the rest of the book comes in. It gives you the foundation required to formulate clear analytics questions, know what is and isn't possible, understand the tradeoffs between different approaches, and think critically about the usefulness of analytics results. Key jargon is explained in basic terms, the real‐world impact of technical details is shown, unnecessary formalism is avoided, and there is no code. Theory is kept to a minimum, but when it is necessary I illustrate it by example and explain why it is important. I have tried to adhere to Einstein's maxim: “everything should be made as simple as possible… but not simpler.”
Some sections of the book are flagged as “advanced material” in the title. These sections are (by comparison) highly technical in their content. They are necessary for understanding the strengths and weaknesses of specific data science techniques, but are less important for framing analytics problems and managing data science teams.
I have tried to make the chapters as independent as possible, so that the book can be consumed in bite‐sized chunks. In some places the concepts necessarily build off of each other; I have tried to call this out explicitly when it occurs, and to summarize the key background ideas so that the book can be used as a reference.
Конец ознакомительного фрагмента.
Текст предоставлен ООО «ЛитРес».
Прочитайте эту книгу целиком, купив полную легальную версию на ЛитРес.
Безопасно