These steps require more than analytical skill. To bring a solution from idea to implementation usually requires a team with a shared goal, diverse skills and perspectives, and a healthy dose of curiosity. The sometimes-arduous process is not formulaic, but requires depth of insight, creativity, and flexibility.
The skill sets and experience required to perform each of the listed steps are quite varied. They include: business or mission problem solver and/or system engineer, data or software engineer, analytic professional, user experience designer, change management professional, or communications professional.
Where are we headed?
What may have started as a statistician solving analytic problems has become a much more important role that is at the intersection of business/mission, computer science, engineering, and statistics. It is rare indeed for a single person to command mastery of all of these disciplines to the degree required.
I think the business world is maturing in its understanding of the value of data science. Some are seeing beyond the technical hype and realizing that the analytic part of the process is often straightforward compared to the vital surrounding steps of defining the opportunity, collecting data, and successfully delivering the results to a decision maker or automated system.
“…the analytic part of the process is often straightforward compared to the vital surrounding steps…”
Gerhard Pilcher | CEO | Elder Research
A Deeper Look
While the world has begun to realize that a holistic approach requires a multidisciplinary team, it’s been slower to recognize the value of deep experience and advanced quantitative training/degrees in understanding the limitations and assumptions of complex mathematical techniques. This can lead to flawed application of the techniques and thus to misleading results. Some in the software industry have compounded this by promising to create “data scientists” in mere weeks. I liken this to creating “citizen heart surgeons” by using robotic surgery devices. I would rather have a true surgeon at the controls!
As is normal in a healthy growing field, data science is also getting more specialized. We now have experts in individual disciplines of neural networks, graph analysis, natural language processing, image processing, algorithmic compute efficiency, non-linear optimization, simulation, survival analysis, interpretable machine learning, etc. This luxury makes the generalists, who can figure out what specialists to call in for a specific problem, valuable gatekeepers, “triagers,” and interpreters.
Reliability
Data is at the heart of each new discovery in every field of science. Though scientists may be expert in their field they are rarely fluent in data analysis. So data scientists, who are such experts, are useful additions to every possible team. Being well-versed also in every known way that we humans can be led astray by data, data scientists can save many months of project pain and effort by “heading trouble off at the pass” and establishing research and project experimental protocols on a sound foundation from the beginning. They are the best antidote to the current “Crisis in Science” where most work — even published journal articles — cannot be replicated! In the working world, replication-quality work translates into models that reliably predict future events in the real world, and not just on paper in the lab.
Finally, using rational models on carefully curated data is a great way to build ethical models that support decisions free of bias for the betterment of society. In this way, the role of data scientists is steadily changing — to provide careful data curation and problem definition and critical validation of results — to ensure that the value of data continually increases.
