The role of data and analytics in business continues to grow. To make sense of their plethora of data, businesses are lookingÂ to data scientists for help. Job site, indeed.com, shows a continued growth in “data scientist” positions. To better understand the field of data science, we studied hundreds of data professionals.
In that study, we foundÂ that data scientists are not created equal. That is, data professionals differ with respect to the skills they possess. For example, some professionals are proficient inÂ statistical and mathematical skills while othersÂ are proficient inÂ computer scienceÂ skills. Still others have a strong business acumen. In the current analysis, I want to determine the breadth of talent that data professionals possess to better understand the possibility of finding a single data scientist who is skilled in all areas. First, let’s review the study sample and the method of how we measured talent.
Assessing Proficiency in DataÂ Skills
We surveyed hundreds ofÂ data professionals to tell us about their skills in five areas: Business, Technology, Math & Modeling, Programming and Statistics. Each skill area included five specific skills, totaling 25 different data skills in all.
For example, in the Business Skills area, data professionals were asked to rate their proficiency in such specific skillsÂ as “Business development,” and “Governance & Compliance (e.g., security).” In the Technology Skills area, they were asked to rate their proficiency in such skills as “Big and Distributed Data (e.g., Hadoop, Map/Reduce, Spark),” and “Managing unstructured data (e.g., noSQL).” In the Statistics Skills, they were asked to rate their proficiency in such skills as “Statistics and statistical modeling (e.g., general linear model, ANOVA, MANOVA, Spatio-temporal, Geographical Information System (GIS)),” and “Science/Scientific Method (e.g., experimental design, research design).”
For each of the 25 skills, respondents were asked to tell us their level proficiency using the following scale:
- Don’t know (0)
- Fundamental Knowledge (20)
- Novice (40)
- Intermediate (60)
- Advanced (80)
- Expert (100)
This rating scale is based on aÂ proficiency rating scale used by NIH. Definitions for each proficiency level were fully defined in the instructions to the data professionals.
Standard ofÂ Performance
The different levels of proficiency are defined aroundÂ the data scientists ability to give or need to receiveÂ help. In the instructions to the data professionals, the “Intermediate” level of proficiency was defined as the ability “to successfully complete tasks as requested.” We used that proficiency level (i.e., Intermediate) as the minimum acceptable level of proficiency for each data skill. The proficiency levels below the Intermediate level (i.e., Novice, Fundamental Awareness, Don’t Know) were defined by an increasing need for help on the part of the data professional. Proficiency levels above the Intermediate level (i.e., Advanced, Expert) were defined by the data professional’sÂ increasing ability to give help or be knownÂ by others as “a person to ask.”
We looked at the level of proficiency for the 25 different data skills across four different job roles. As is seen in Figure 1, data professionals tend to be skilled in areas that are appropriate for their job role (see green-shaded areas in Figure 1). Specifically, Business Management data professionals show the most proficiency in Business Skills. Researchers, on the other hand, show lowest level of proficiency in Business Skills and the highest in Statistics Skills.
For many of the data skills, the typical data professional does not have the minimum level of proficiency to do be successful at work, no matter their role (see yellow- and red-shaded areas in Figure 1). These data skills include the following: Unstructured data, NLP, Machine Learning, Big and distributed data, Cloud management, Front-end programming, Optimization, Graphic models, Algorithms and Bayesian statistics.
In Search of the Elite Data Scientist
There are a couple of ways an organization can build their data science capability. ItÂ can either hire a single individual who is skilled in all data science areas or it can hire a team of data professionals who have complementary skills. In both cases, the organization has all the skills necessary to use data intelligently. However, the likelihood of finding a data professional who is an expert in all five skill areas is quite low (see Figure 2). In our sample, we looked at three levels of proficiency: Intermediate, Advanced and Expert. We found that only 10% of the data professionals indicated they had, at least, an Intermediate level of proficiency in all five skill areas. The picture looks more bleak you look for data professionals who have advanced or expert proficiencies in data skills. The chance of finding a data professionalÂ with AdvancedÂ skills or better in all five skill areas drops to less thanÂ 1%. There were no data professionals who were considered as Experts in all five skill areas.
We looked atÂ proficiency differences across five industries: Consulting (n = 52), Education / Science (n = 50), Financial, (n = 52), Healthcare (n = 50) and IT (n = 95). We identified dataÂ professionals who had an advanced level of proficiency acrossÂ the differentÂ skills. We found that data professionals in the Education / Science industry have more advanced skills (54% of data professionals have at least an advanced level of proficiency in at least one skill area) compared to data professionals in the Financial (37%) and IT (34%) industries.
The term “data scientist” is ambiguous. There are different types of data scientists, each defined by their level of proficiency in one of fiveÂ skill areas: Business, Technology, Programming, Math & ModelingÂ and Statistics.Â Data scientists can beÂ defined by the skills they possess. So, when somebody tells you they are a data scientist, be sure you know what type they are.
Finding a data professional who is proficientÂ in all data science skill areas is extremely difficult.Â As our studyÂ shows, data professionals rarely possess proficiency in all fiveÂ skill areas at the level needed to be successful at work. The chance of finding a data professional with Expert skills in all five areas (even in 3 or 4 skill areas) is akin to finding a unicorn; they just don’t exist. There were very few data professionals who even had the basic minimum level of proficiency (i.e., Intermediate level of proficiency) in all five skill areas. Additionally, our initial findings on industry differences in skill proficiency suggest that skilled data professionals might be easier to find in specific industries. These industry differences could impact recruitment and management of data professionals. An under-supplyÂ of data science talent in one industry could require companies to use more dramatic recruitment efforts to attract data professionals fromÂ outside theÂ industry. In industries where there are plenty of skilled data professionals, companies can be more selective in their hiring efforts.
Optimizing the value of business data is dependent on the skills of the data professionals who process the data. We took a skills-based approach to understanding how organizations can extractÂ value from their data. Based on our findings, we recommend that organizations avoid trying to find a single data professional who has the skills that span the entire spectrum of data science. Rather, a better approach is to consider building up your data science capability through the formation of teams of data professionals who have complementary skills.