How Many Data Scientists are There and is There a Shortage?

Recently saw this article on KDnuggets (check them out if you aren’t already subscribed) and thought it was worth using to update some of my slides about the current talent shortage for Data Science & Analytics (DSA) skills.

This shortage is definitely acute here in the Philippines.

The 2011 McKinsey report on Big Data said that “The United States alone faces a shortage of 140,000 to 190,000 people with analytical expertise and 1.5 million managers and analysts with the skills to understand and make decisions based on the analysis of Big Data.”

In 2014, KDnuggets examined “How Many Data Scientists are out there?” and came with an estimate of 50-100,000, and did not see much evidence of a massive shortage then. In 2014, we found only about 1,000 job ads for “Data Scientist” on indeed.com.

Now that we reached 2018, KDnuggets has examined how accurate were those predictions and tried to answer three questions:

1. Is there a shortage of Data Scientists now?

2. How many “Data Scientists” are there , both in name and in function ?

3. What are the future prospects for Data Scientists?

The answer to the first question is a resounding YES!

LinkedIn Workforce Report for US (August 2018) says “Demand for data scientists is off the charts … data science skills shortages are present in almost every large U.S. city. Nationally, we have a shortage of 151,717 people with data science skills.
Note that LinkedIn reports shortages for people with “Data Science Skills”, not necessarily people with “Data Scientist” title.
We can estimate the demand for “Data Scientists” from two popular job search sites – indeed and Glassdoor.
Search on indeed.com for “data scientist” (in quotes) in USA finds only about 4,800 jobs. However, in a search for data scientist without quotes, about 30,000 jobs.

US is the largest but not the only market for Data Scientists. We can also see strong demand for Data Scientists elsewhere:

UK: 1,100 jobs
Germany: 900 jobs
France: 718 jobs
Philippines: 599 jobs — You Read That Right! More than India.
India: 500 jobs

Glassdoor search for “Data Scientist” finds about 26,000 jobs in USA (same results if quotes are removed).

19989649_1487790424576061_5172679695012103031_n-2

Question 2: How Many “Data Scientists” are There, Both in Name and in Function?

Google search defines a data scientist as “a person employed to analyze and interpret complex digital data, such as the usage statistics of a website, especially in order to assist a business in its decision-making.”

There are many people in the industry and academia who do this work without having the formal title of a data scientist, since Data Science is an interdisciplinary field at the intersection of Statistics, Computer Science, Machine Learning, and Business. We can estimate the current population of Data Scientist by examining popular data science platforms.

Kaggle (now part of Google) is a platform for data science and analytics competitions. It claims to be the world’s largest community of active data scientists.

While not all Data Scientists take part in Kaggle competitions or have a Kaggle account, and not all Kagglersdo work of data science, it is reasonable to assume a large overlap.

On Sep 19, 2018 Kaggle says they surpassed 2 million members in August 2018.

Since not all Kaggle members are active, Kaggle membership is probably a global upper bound for people engaged in data science.

KDnuggets is now reaching over 500,000 unique visitors per month.

KDnuggets now has about 240,000 subscribers/followers over Twitter, LinkedIn, Facebook, RSS, and email.

On LinkedIn, there are many groups dedicated to data science, and although the engagement in those groups has been falling, we can use their membership as a rough estimate. Here are three of the largest groups

Big Data and Analytics – 339,000
Data Science Central – 278,000
Data Mining, Statistics, Big Data, Data Visualization, and Data Science – 170,000

Searching LinkedIn for “data scientist” (quotes are important) we find over 100,000 people with that actual title. So if globally between 200,000 and 1,000,000 people are doing some Data Science related work, then a majority of them does not have a Data Scientist title.

Screen Shot 2018-09-23 at 10.20.35 AM.png

We can also estimate the by looking at activities related to languages and platforms most connected to Data Science: R, Python, Machine Learning libraries, Spark, and Jupyter.

Apache Spark Meetups had 225K members recently and growing every month.
Intel Capital estimated that there 1 million R programmers worldwide.
Based on the public data on python.orgwebsite, there have been around 2.75 million downloads.
Jupyterproject has around 3 million users at present.

These numbers can give us a rough upper limit on the number of data analysts/data scientists around the world.

So yeah, to answer the question, there are at least 200,000

Screen Shot 2018-09-23 at 10.25.53 AM

Question 3: What are the Future Prospects for Data Scientists?

The near-term future for Data Scientists looks bright.

LinkedIn 2017 emerging jobs report claims that machine learning engineers working today has increased by 9.8 times as compared to 5 years ago.

Machine Learning Engineers, Data Scientists, and Big Data Engineers rank among the top emerging jobs on LinkedIn. Data scientist roles have grown over 650% since 2012.

Job growth in the next decade is expected to outstrip growth during the previous decade, creating 11.5M jobs in the Data Science/Analytics area by 2026, according to the U.S. Bureau of Labor Statistics.

IBM recently claimed that by 2020 the number of Data Science and Analytics job listings is projected to grow by nearly 364,000 listings to approximately 2,720,000. No matter what the true number of data professionals out there currently, their number is likely to grow in the near future.

So What are the Future Prospects for Data Scientists in the Philippines?

APEC_500K

Based on Data from APEC (Asia Pacific Economic Cooperation), there is both a huge demand here in the Philippines as well as in the jobs where the Philippines already has an outsourcing pipeline too.

SO what does that mean for you?

You Need to Know Exactly What You Need to Hire/Learn how to have/be a Data Scientist?

And it’s not easy.

Eco_Picture1

To assemble a team of DSA Practitioners, you need to make sure you have the right combination of talent.

Here is how I would start.

Eco_Picture2

Make sure you have people who can do these functions.

And if you want to learn how to be one of these key players, I’m betting you need to know where to start.

So wether you want to be a DSA enabled professional or you want to assemble a DSA team, here is a better understanding of how that looks.

Eco_Picture3

Yeah I know. It is a lot!

So, what now?

Connect with DMAIPH and we will get you started!

42309040_572124856542481_6576071598531411968_n

DMAIPH – Decision-making, Analytics & Intelligence Philppines

Over the past few years businesses in the Philippines have invested heavily in big data, analytics and data science, but still have not achieved the expected outcomes of data-driven companies.

Based on our learnings from the 100’s of Filipino businesses and 10,000s of Filipinos who have taken part in DMAIPH Analytics trainings all across the country, we have crafted a proven, non-technical approach to upskilling your team in analytics.

In 2019, we will be launching two new training programs: (1) Our DMAIPH Applied Analytics Master Class series for executives, leaders and decision-makers and our (2) DMAIPH Applied Analytics Boot Camp series for practicing analysts.

We will feature case studies of real Filipino run business, exercises based on actual analytics challenges being solved by Filipino analysts, and provide you with a copy of my book, Putting Your Data to Work, an analytics guidebook for the Filipino professional

Connect with us via our marketing partner, http://www.sonicanalytics.com to learn about upcoming analytics trainings and events.

Working on an Analytics Internship/OJT program…

400 Hour DMAIPH Data Science & Analytics OJT/Internship Program

The end goal is to develop a DSA strategy presentation for manager. Start out by getting to know the physical data environment, the tools being used and the main players in the business. Move on to assessing the maturity of the analytics culture and it’s use of DSA talent, techniques and technology. Design a business dashboard prototype and deliver a compelling data story to improve management reporting.

Three tracts for interns… HR Analyst, Business Analyst and Data Analyst.

Interns will spend 60% of the internship at the place of business and 40% of the internship in a classroom. This will facilitate the application of theory to real business data in order to help managers get a better idea of the what’s working and what’s nor when it comes to the data in their business.

Based on the APEC DSA Competencies which is close to being adopted by 20+ countries across Asia and the Pacific as a guide for current and future DSA training efforts.

Week 1 – Fundamentals of DSA

APEC DSA Competencies
Company Background
How This Internship Works

Exercise: LinkedIn Profile

Company Deliverable: Company/Organization DSA Profiles

Week 2 – DSA in the Philippines

Putting Data into Context
Emerging Trends
Cultures of Innovation

Exercise: Glossary of Data

Company Deliverable: Defining Where the Cutting Edge Is

Week 3 – Data Management & Governance

Data Management Macro View
Data Governance
Information Security

Exercise: Data Survey

Company Deliverable: Info Security Risk Assessment

Week 4 – Data Analytics Methods & Algorithms

Data Management Micro View
The Right Data
Machine Learning

Exercise: Who’s Who of Data in the Business

Company Deliverable: Data MVPs

Week 5 – Data Science Engineering Principles

Data Map
Identify Right App
Feedback Loop

Exercise: A Visio Data Map

Company Deliverable: Map of Business Data Lake

Week 6 – Computing and Computational Thinking

MS Excel
Query Data
Programming Languages

Exercise: Top 10 Excel Tips Video

Company Deliverable: Top Ten Data Tips

Week 7 – Statistical Techniques

Getting IT
Analytics Maturity Model
Predictive Analytics Model

Exercise: Flight Risk Model

Company Deliverable: Results of Maturity Assessment

Week 8 – Operational Analytics

Management Reporting
Public Big Data
Business Dashboards

Exercise: Tableau Public Mock Up

Company Deliverable: Business Dashboard Prototype

Week 9: Data Visualization & Presentation

Data Visualization
Enchantment
Data Storytelling
Exercise: D.R.A.P.S
Company Deliverable: A Business Data Story

Week 10 – Final Project/DSA Strategy Presentation

My goal is to create and promote a hybrid approach that offers both supplemental education and hands on experience. We need to get past the days of having OJT do data encoding or simple research projects… they need skills that they can apply day one.

They need it, we need it, the country needs it.

Any ideas or suggestions? This is just the first draft.

Hoping to roll this out in the next month or so.

Analytics Education – Facilitating a mastery of the fundamentals of analytics is what DMAIPH does best. As a key parnter of the Asia Pacific Economic Cooperation’s Project DARE initative, DMAIPH champions the use of using data. All across the world, companies are scrambling to hire analytics talent to optimize the big data they have in their businesses. We can empower students and their instructors with the knowledge they need to prepare for careers in data science and analytics. Contact DMAIPH now at analytics@dmaiph.com or connect with me directly so we can set a guest lecturer date, On-the-Job Training experience or other analytics education solution specifically tailored to your needs.

The Analytics Puzzle for Higher Education in the Philippines

When you look at the picture on the box of puzzle pieces, you generally think it won’t be so hard to fit all the pieces together. But then when you lay out all the pieces and connect them one by one it can often feel like a sense of this is a lot harder then I thought.

In many way, that’s how I feel about efforts to date regarding the teaching of Data Science and Analytics in the Philippines. The end product is clear, just about all the 2,000+ HIEs across the Philippines offering some level of DSA education to a wide range of students.

Everyone agrees that we need more education to meet both the high current demand and the expected huge future demand for DSA talent for both domestic and global consumption. We have seen a lot of awesome initiatives popping up trying to train educators to teach DSA subjects and have seen a number of industry-academe partnerships. CHED has even set aside significant resources to promote the training of faculty and the incentive to offer DSA programs.

So things are going well, but when you look at the simple math of how many educators need to be training in the very near future, some like me get a little concerned. Current programs train a few dozen here and maybe a few hundred there, bit by bit. But if you need thousands then current efforts are just going to come up short.

What we need is a unified front. Bringing together all the interested parties, many of whom are already working on this issue, is the only way to get to critical mass. By my estimation we should be looking at training 5,000 educators in the next 3 years. And a one week overview is just the start. To really become adept at teaching DSA, educators need an apprenticeship that lasts months to really learn the tools of the trade like data storytelling, business intelligence and predictive analytics.

And that is just the faculty… when you think about the 100,000s of students who need to taught DSA, you start to see that this puzzle is gonna take a lot more effort to complete then it may have looked like at first.

So thats where I am at now… both evangelizing and empowering. Raising awareness of what the puzzle looks like when solved and why we need to solve. And empowering to build collaborations to connect the pieces faster then each puzzle expert can work on their own.

And that is exactly why I started Augment BPO.

Augment BPO. The Augment BPO Data Science and Analytics Advocacy Project (Augment BPO) is empowering BPO Companies, Executives, and Workers in the Philippines to prepare for and address the clear and present danger posed by Artificial Intelligence Chatbots (AI Chatbots) to BPO revenue growth and jobs through Data Science and Analytics strategy planning, awareness building and upskill training.

DMAIPH Analytics Education – Facilitating a mastery of the fundamentals of analytics is what DMAIPH does best. As a key parnter of the Asia Pacific Economic Cooperation’s Project DARE initative, DMAIPH champions the use of using data. All across the world, companies are scrambling to hire analytics talent to optimize the big data they have in their businesses. We can empower students and their instructors with the knowledge they need to prepare for careers in data science and analytics. Contact DMAIPH now at analytics@dmaiph.com or connect with me directly so we can set a guest lecturer date, On-the-Job Training experience or other analytics education solution specifically tailored to your needs.

Teaching Analytics: APEC List of Competencies

Teaching Analytics: An Instructor’s Guide to implementing the recommendations of Project DARE (Data Analytics Raising Employment).

APEC (Asia Pacific Economic Cooperation) hosted an event on May 4-5 bringing together over 50 analytics experts and visionaries from over 14 counties across the Asia-Pacific region to form an advisory group.

The APEC Project DARE (Data Analytics Raising Awareness) Advisory Group started with an agenda set to develop recommended “APEC Data Analytics Competencies.

There are still being finalized, but as of last week here is a high level of the competencies that we came up with:

Business and Organizational Skills

Operational Analytics: Use data analytics and specialized business analytics (i.e. business intelligence) techniques for the investigation of all relevant data to derive insight in support of decision-making.
Data Visualization and Presentation: Ability to create and communicate compelling and actionable insights from data using visualization and presentation tools and technologies.
Data Management and Governance: Develop and implement data management strategies and governance, incorporating privacy, data security, polices and regulations, and ethical considerations.
Domain Knowledge and Application: Apply domain-related knowledge and insights to effectively contextualize data, achieved by practical experience (e.g. apprenticeships) and exposure to emerging innovations.

Technical Skills

Statistical Techniques: Apply statistical concepts and methodologies for data analysis.
Computing: Apply information technology, computational thinking and utilize programming languages and software and hardware solutions design and development for data analysis.
Data Analytics Methods and Algorithms: Capture, clean and inspect data. Evaluate and implement data analytics and machine learning methods and algorithms on the data to derive insights for decision making.
Research Methods: Utilize the scientific and engineering methods to discover and create new knowledge and insights.
Data Science Engineering Principles: Use software and system engineering principles and modern computer technologies, incorporating a data feedback loop, to research, design, and prototype data analytics applications. Develop structures, instruments, machines, experiments, processes, and systems to support the data lifecycle.

Workplace Skills

21^st Century Skills: Exhibit crosscutting skills essential for DSA at all levels including, but not limited too; collaboration, customer focus, communication and storytelling, organizational awareness, critical thinking, planning and organizing, problem solving, decision making, business fundamentals, awareness of social and societal awareness, intelligibility, cross cultural awareness, dynamic (self) re-skilling, professional networking, ethical mindset and entrepreneurship.

Once this list is finalized I will update you all.

My end goal is two fold, (1) to help craft a version of the competencies into an academic discipline that the Analytics Council can present to CHED for adoption and (2) to design a vocational education track to address the basic skills needed for entry level DSA work here in the Philippines.

Stay Tuned!

Analytics Training – DMAIPH offers a wide range of analytics centric training solutions for professionals and students via public, in-house, on-site, and academic settings. We tailor each training event to meet the unique needs of the audience. If you need empowerment and skills enhancement to optimize the use of analytics in your organization, we are here to help. Contact DMAIPH now at analytics@dmaiph.com or connect with me directly to set up a free consultation to learn which of our DMAIPH analytics training solutions is best for you.

Become a Data Science & Analytics Pro! Apply for the DMAIPH Apprenticeship Program

A standard definition of Apprenticeship is a kind of job training that involves following and studying a master of the trade on the job instead of in school.

Learning to become an analyst for the most part has been something done on the job. after working in a company and gaining subject matter expertise, those who had good analytics skills often found themselves going down the analyst career path.

Big Data and technological advancements in analytics processing and data science have changed all that.

Nowadays, there are so many analyst jobs available that the natural order of learning to become an analyst isn’t working fast enough.

Higher education is trying to come up with solutions to offer analytics themed course, a few are already in place. But that’s only training 100’s when industry needs 1,000’s.

SO, to help fill the skills gap between the very finite supply of Data Science and Analytics (DSA) talent and the huge demand in the form of open jobs, we have to get outside the box.

You will see a lot more ideas like the DMAIPH Data Science & Analytics Apprenticeship program coming in the near term.

But don’t wait for the future, get ahead of the game.

Learn the DSA skills you need for a long and profitable career as an analyst.

E-mail me your resume today if you would like to learn more. danmeyer@dmaiph.com

I will be taking on a few more apprentices in the coming months as we grow the program to implement APEC’s Project DARE recommended approach to gaining a basic understanding of what it means to be a DSA professional.

Hope to hear from you soon!

Dan

jobspicture2

If you need empowerment and skills enhancement to optimize the use of analytics in your organization, we are here to help. Contact DMAIPH now at analytics@dmaiph.com or connect with me directly to set up a free consultation to learn which of our DMAIPH analytics training solutions is best for you.

Why Focus on Data Analytics Competencies?

Later this week I will be attending the APEC Project DARE (Data Analytics Raising Awareness) Advisory Council in Singapore to discuss Recommended APEC Data Analytics Competencies.

Why Focus on Data Analytics Competencies?

Jobs requiring a familiarity with data analysis are forecasted to dramatically rise, resulting in a massive shortage of qualified employees. According to reports, some economies face a shortage of up to 1.5 million data analytics-enabled managers and analysts, costing billions of dollars in lost revenue annually. There is an urgent need to ensure that the future workforce is equipped with data analytics competencies to secure the jobs of tomorrow and move with ease in the labor market.

This is where Project DARE comes in. Project DARE aims to facilitate development of a data analytics-enabled workforce across the APEC region to effectively support sustainable economic growth and prosperity in the Asia-Pacific region. To do so, Project DARE developed a set of Recommended APEC Data Analytics Competencies which will be a resource to academic institutions and governments to align curricula, courses and programs so APEC economies are equipped to educate its workforce with the data analytics skills needed by employers in a data-driven future.

How was the Recommended Data Analytics Competencies Developed?

The Recommended APEC Data Analytics Competencies was developed through a public-private partnership with input from over 40 Advisory Group members (see Acknowledgements) comprised of distinguished business and higher education leaders who oversee data science and analytics needs for their organization and data science inter-disciplinary initiatives and curriculum. The Advisory Group was led by the private sector partner co- chairs, global skills and knowledge company Wiley and the Business Higher Education Forum (BHEF), with technical support by the EDISON (Education for Data Intensive Science to Open New Science Frontiers) Project.

17426122_1095825400562902_5936762267508985241_n

DMAIPH and Analytics Education

Facilitating a mastery of the fundamentals of analytics is what DMAIPH does best. As a key parnter of the Data Science Philippines Meetup Group, DMAIPH champions the use of using data. All across the world, companies are scrambling to hire analytics talent to optimize the big data they have in their businesses. We can empower students and their instructors with the knowledge they need to prepare for careers in analytics. Contact DMAIPH now at analytics@dmaiph.com or connect with me directly so we can set a guest lecturer date, On-the-Job Training experience or other analytics education solution specifically tailored to your needs.

About Project DARE

Project DARE is an initiative of the Asia-Pacific Economic Cooperation led by the United States (U.S. Department of Labor) with co-sponsorship from the governments of Australia, Japan, Malaysia, Peru, Chinese Taipei, and Viet Nam. As a project of APEC’s Human Resources and Development Working Group (HRDWG), Project DARE seeks to enable APEC workforce with the data analytics competencies demanded by employers today and to secure the jobs of tomorrow.