How Many Data Scientists are There and is There a Shortage?

Recently saw this article on KDnuggets (check them out if you aren’t already subscribed) and thought it was worth using to update some of my slides about the current talent shortage for Data Science & Analytics (DSA) skills.

This shortage is definitely acute here in the Philippines.

The 2011 McKinsey report on Big Data said that “The United States alone faces a shortage of 140,000 to 190,000 people with analytical expertise and 1.5 million managers and analysts with the skills to understand and make decisions based on the analysis of Big Data.”


In 2014, KDnuggets examined “How Many Data Scientists are out there?” and came with an estimate of 50-100,000, and did not see much evidence of a massive shortage then. In 2014, we found only about 1,000 job ads for “Data Scientist” on indeed.com. 


Now that we reached 2018, KDnuggets has examined how accurate were those predictions and tried to answer three questions:

1. Is there a shortage of Data Scientists now?
2. How many “Data Scientists” are there , both in name and in function ?
3. What are the future prospects for Data Scientists?

 

The answer to the first question is a resounding YES!
  • LinkedIn Workforce Report for US (August 2018) says “Demand for data scientists is off the charts  … data science skills shortages are present in almost every large U.S. city. Nationally, we have a shortage of 151,717 people with data science skills.
  • Note that LinkedIn reports shortages for people with “Data Science Skills”, not necessarily people with “Data Scientist” title.
  • We can estimate the demand for “Data Scientists” from two popular job search sites – indeed and Glassdoor.
  • Search on indeed.com for “data scientist” (in quotes) in USA finds only about 4,800 jobs. However, in a search for data scientist without quotes, about 30,000 jobs.
US is the largest but not the only market for Data Scientists. We can also see strong demand for Data Scientists elsewhere:
  • UK: 1,100 jobs
  • Germany: 900 jobs
  • France: 718 jobs
  • Philippines: 599 jobs  — You Read That Right! More than India.
  • India: 500 jobs
Glassdoor search for “Data Scientist” finds about 26,000 jobs in USA (same results if quotes are removed).
19989649_1487790424576061_5172679695012103031_n-2
Question 2: How Many “Data Scientists” are There, Both in Name and in Function?

Google search defines a data scientist as “a person employed to analyze and interpret complex digital data, such as the usage statistics of a website, especially in order to assist a business in its decision-making.”

There are many people in the industry and academia who do this work without having the formal title of a data scientist, since Data Science is an interdisciplinary field at the intersection of Statistics, Computer Science, Machine Learning, and Business. We can estimate the current population of Data Scientist by examining popular data science platforms.

Kaggle (now part of Google) is a platform for data science  and analytics competitions. It claims to be the world’s largest community of active data scientists.

While not all Data Scientists take part in Kaggle competitions or have a Kaggle account, and not all Kagglersdo work of data science, it is reasonable to assume a large overlap.

On Sep 19, 2018 Kaggle says they surpassed 2 million members in August 2018.

Since not all Kaggle members are active, Kaggle membership is probably a global upper bound for people engaged in data science.

KDnuggets is now reaching over 500,000 unique visitors per month.

KDnuggets now has about 240,000 subscribers/followers over Twitter, LinkedIn, Facebook, RSS, and email.

On LinkedIn, there are many groups dedicated to data science, and although the engagement in those groups has been falling, we can use their membership as a rough estimate. Here are three of the largest groups

  • Big Data and Analytics  –  339,000
  • Data Science Central – 278,000
  • Data Mining, Statistics, Big Data, Data Visualization, and Data Science – 170,000

Searching LinkedIn for “data scientist”  (quotes are important) we find over 100,000 people with that actual title.  So if globally between 200,000 and 1,000,000 people are doing some Data Science related work, then a majority of them does not have a Data Scientist title.

Screen Shot 2018-09-23 at 10.20.35 AM.png

We can also estimate the by looking at activities related to languages and platforms most connected to Data Science: R, Python, Machine Learning libraries, Spark, and Jupyter.

  • Apache Spark Meetups had 225K members recently and growing every month.
  • Intel Capital estimated that there 1 million R programmers worldwide.
  • Based on the public data on python.orgwebsite, there have been around 2.75 million downloads.
  • Jupyterproject has around 3 million users at present.

These numbers can give us a rough upper limit on the number of data analysts/data scientists around the world.

So yeah, to answer the question, there are at least 200,000

Screen Shot 2018-09-23 at 10.25.53 AM

Question 3: What are the Future Prospects for Data Scientists?

The near-term future for Data Scientists looks bright.

LinkedIn 2017 emerging jobs report claims that machine learning engineers working today has increased by 9.8 times as compared to 5 years ago.

Machine Learning Engineers, Data Scientists, and Big Data Engineers rank among the top emerging jobs on LinkedIn. Data scientist roles have grown over 650% since 2012.

Job growth in the next decade is expected to outstrip growth during the previous decade, creating 11.5M jobs in the Data Science/Analytics area by 2026, according to the U.S. Bureau of Labor Statistics.

IBM recently claimed that by 2020 the number of Data Science and Analytics job listings is projected to grow by nearly 364,000 listings to approximately 2,720,000. No matter what the true number of data professionals out there currently, their number is likely to grow in the near future.

So What are the Future Prospects for Data Scientists in the Philippines?

APEC_500K

Based on Data from APEC (Asia Pacific Economic Cooperation), there is both a huge demand here in the Philippines as well as in the jobs where the Philippines already has an outsourcing pipeline too.

SO what does that mean for you?

You Need to Know Exactly What You Need to Hire/Learn how to have/be a Data Scientist?

And it’s not easy.

Eco_Picture1

To assemble a team of DSA Practitioners, you need to make sure you have the right combination of talent.

Here is how I would start.

Eco_Picture2

Make sure you have people who can do these functions.

And if you want to learn how to be one of these key players, I’m betting you need to know where to start.

So wether you want to be a DSA enabled professional or you want to assemble a DSA team, here is a better understanding of how that looks.

Eco_Picture3

Yeah I know. It is a lot!

So, what now?

Connect with DMAIPH and we will get you started!

42309040_572124856542481_6576071598531411968_n

DMAIPH – Decision-making, Analytics & Intelligence Philppines

Over the past few years businesses in the Philippines have invested heavily in big data, analytics and data science, but still have not achieved the expected outcomes of data-driven companies.

Based on our learnings from the 100’s of Filipino businesses and 10,000s of Filipinos who have taken part in DMAIPH Analytics trainings all across the country, we have crafted a proven,  non-technical approach to upskilling your team in analytics.

In 2019, we will be launching two new training programs: (1) Our DMAIPH Applied Analytics Master Class series for executives, leaders and decision-makers and our (2) DMAIPH Applied Analytics Boot Camp series for practicing analysts.

We will feature case studies of real Filipino run business, exercises based on actual analytics challenges being solved by Filipino analysts, and provide you with a copy of my book, Putting Your Data to Work, an analytics guidebook for the Filipino professional

Connect with us via our marketing partner, http://www.sonicanalytics.com to learn about upcoming analytics trainings and events. 

Advertisements

Quick Data Science Survey

If you are in the Philippines and work with data everyday in your job, I’d like to invite you to take my survey.

Next week I will be speaking about data science in the Philippines, specifically trying to answer the question, “Just How Many Data Scientists Are There In The Philippines Anyway?”

It’s a short 7 question survey that will help me validate some of my research.

Here’s the link:

https://www.surveymonkey.com/r/WKG9VJ5

Thanks for taking a few minutes to help address on of the biggest questions facing the Philippines today.

Analytics Survey – DMAIPH conducts quarterly analytics surveys to collect data on current trends in analytics. We specialize in surveys that assess analytics culture and measuring how aligned an organization is to using data and analytics  in its decision-making.

Contact DMAIPH now at analytics@dmaiph.com or connect with me directly to find out more about how DMAIPH can conduct surveys to help you assess the analytics culture in your business.

So Just How Many Filipino Data Scientists Are There Anyway?

One Question I have been asking myself for awhile is just how many Filipino Data Scientists are there anyway?

I ask this  question because based on my observations of job postings and job titles and industry buzz, there are way more data scientist jobs open here in the Philippines then there are actual data scientists to fill them.

And I’m not even trying to separate the “true” data scientists with the “quasi” data scientists that are doing data science but without all the pre-requisites required.

Lumping everyone together into one number, I wonder what it is. An educated guess would be a few hundred at best. Yet, there are literally 1000 job postings out there right now for data scientist.

This speaks to a fundamental lack of awareness across the country as to what truly a data scientist is and what they do. But that’s for another blogpost.

Right now Im simply going to do some research, survey my network and try and come up with a best guess on exactly how many data scientists there currently are here in the Philippines.

Please feel free to contribute to the conversation… its not going to be easy to get a universally acceptable answer without a lot of discourse first.

Let the hunt for a unicorn head count begin!

Screen Shot 2016-06-28 at 9.02.36 AM

Analytics in the Philippines – The Philippines is at the center of the action when it comes to solutions to the global need for analytics. Blessed with a solid foundation of young, educated and English speaking workforce, companies around the world are look for Filipino analytics talent to fill analytics positions. DMAIPH was set up to facilitate these solutions and bring the talent and the business together. Contact DMAIPH now at analytics@dmaiph.com or connect with me directly so we can help you take advantage of this unique global opportunity.

What % of Companies Can Both Afford Data Science Teams and Understand How to Use them?

Woke up this morning thinking about the future of analytics and data science in the Philippines.

I created this image to help visualize one of the biggest challenges. From my perspective the majority of companies out there don’t really understand data science and probably cant afford a traditional data science team.

who-can-afford-it

Best guess is ever 50% of companies operating in the Philippines aren’t yet at a place where data science is practical.

The smallest group are ones who both understand data science and know how to use it. These are the ones who are players in the data science consulting, training and outsourcing business.

Best guess its about 10% of the companies operating here and just about all of them are big corporations with lots of resources.

The next segment is the ones who do understand the value of data science, but don’t have the resources to compete for a the talent it takes to build a top data science team.

I’ll peg this at about 15%. But its  growing quickly. Awareness and adoption are actually ahead of the talent, as you see so many “data scientists” in name only out there filling open jobs right now.

That leaves the 25% of companies operating in the Philippines who have money and resources but just don’t know how to get started using data science and building a team. This is the target market of companies like mine.

DMAIPH has the capability to educate your managers and decision-makers on how to use data science to add value to the business. We can also train your team in the basics of data science to allow you to cultivate a data driven culture and promote from within to build a data science team. And we can also take some of the load off your shoulders in the terms doing some of the data science for you.

Analytics Consulting – DMAIPH specializes in a variety of analytics consulting solutions designed to empower analysts, managers and leaders with the tools needed for more data-driven decision-making. We have helped dozens of companies get more analytics in their business. Contact DMAIPH now at analytics@dmaiph.com or connect with me directly so we can tailor an analytics solution made just for your unique requirements.

Q7: What exactly is data science and why the rapid rise of data scientists?

A year ago I might have found it challenging to really answer this question. The first time I had heard of the term data science and a data scientist wasn’t that long ago. And I have been doing some pretty advanced analytics for close to 20 years now.  I know the term has been around in academic and research circles awhile longer, but 2014 is the first time I ever saw a job posting for data scientist in big business.

So what is data science? Besides simply being the study of data, it generally refers to using complex models, machine learning, predictive and prescriptive analytics and powerful technology to analyze business data in much greater volume, velocity and variety then possible a few years ago.

And of course the ones charged with doing the data science are data scientists. They understand math, statistics, and theories that can be applied to business data using new technologies and methodologies.

The biggest challenge to being a true data scientist is that you have to be adapt at both technology and working with people. Being a business data expert, knowing how to code and doing higher math are only half the job. You have to also share your data, communicate it in ways that drive action, share and engage with non-data centric people. It’s hard to find people who are good at both.

ByugG_cIEAAL6wM

Image from Forbes Magazine. 

In addition, whole some data scientists are educated to be data scientists, very, very few actually have any kind of degree in data science. That kind of degree really didn’t exist until very recently. Instead most data scientists have advanced degrees is related subjects and have migrated into the business world do to market demand.

That demand has been growing at a staggering rate the past few years as every day we generate more and more data across the planet. President Obama first employed a data scientist for his campaign in 2012. The White House now has a chief data scientist position.

If you were to compare results from job board searches form 2012, you’d see maybe 100 data scientist job postings. Now its easily in the 1000’s.  So that’s why the job market for data scientist is one of the hottest around.  Lack of training programs, having both tech and people skills, and the booming demand due to unending new data to being analyzed.

Some people ask me if I’m a data scientist I am careful with my answer. True data science is not something I am academically prepared for nor I have never published anything in a scholarly journal. But my real world experience working with data has made me an expert on many aspects of data science.

I guess I feel more like an analyst, but a freakin awesome analyst who can do a lot of things using data that are super important to a business.

img_8168

Analytics Education – Facilitating a mastery of the fundamentals of analytics is what DMAIPH does best. As a key parnter of the Data Science Philippines Meetup Group, DMAIPH champions the use of using data. All across the world, companies are scrambling to hire analytics talent to optimize the big data they have in their businesses.

We can empower students and their instructors with the knowledge they need to prepare for careers in analytics. Contact DMAIPH now at analytics@dmaiph.com or connect with me directly so we can set a guest lecturer date, On-the-Job Training experience or other analytics education solution specifically tailored to your needs.

I Always Say… Data Science Is Just As Much Art As It Is Science!

“A data scientist is that unique blend of skills that can both unlock the insights of data and tell a fantastic story via the data.” … that’s a quote by DJ Patil, who is generally considered one of the world’s top data scientists.

The term data scientist has really taken off in the analytics world the past few years and the number of job postings, college courses and certifying bodies continues to rocket upwards. Here in the Philippines I am starting to see a number of analytics gurus using that title and have even started seeing some of the bigger BPOs look for skills sets inline with this type of title,

However, even the most veteran data geeks, have a hard time really determining what role data scientist can play in the BPO industry and to what level you can set up outsourcing team to do data science… aka analytics… aka data-driven decision-making. Its really, quite a wide open topic without a lot of clear guidance trickling down into the industry here.

That will change. And it will change quickly. The number of analytics trainings and data-centric conferences and college course geared towards data science careers is exploding. In fact i will be speaking at two of events of this kind later this month. One geared to college students looking into analytics careers and one for a cross section of practitioners looking for deeper understanding of the analytics environment.

To that end, I wanted to share a few visuals that I will be using in my presentations… both with a slant towards guiding students towards the right analytics career for them.

The first is a great visual showing the diverse skills a true data scientist needs to be awesome. Source: http://www.datasciencecentral.com/profiles/blogs/the-22-skills-of-a-data-scientist

ByugG_cIEAAL6wM

The second is a a great way to show what data geeks really need to become data rock stars. Source: http://www.datasciencecentral.com/profiles/blogs/are-you-a-data-scientist

MDL_P1_DS_1