It’s No Longer Just Enough To Know

In a recent conversation about using more analytics in the measurement and evaluation of public policies and programs, one of my colleagues said that in today’s world, “It’s no longer just enough to know.”

The point being if you aren’t using data and analysis to enhance your efforts and empower decision-makers with actionable insights, then you are not serving the public to the best of your ability.

A lot of government programs, non-profits and philanthropic organizations are what he called, “Information Rich, but Data Poor.”

Check out my upcoming webinar on Feb 15, 2017! https://dmaiph.com/2017/01/14/analytics-and-data-driven-decision-making-webinar-on-feb-15/

Just because you gather massive amounts of information in the form of data points, does not mean the data is adding value. In fact one of the biggest challenges the corporate world has been dealing with the past few years is how to optimize Big Data.

We live in a world where so much data is produced and captured, then analyzed and published in reports and article, yet the data and analysis alone is often not having the impact our policies and projects were intended to have.

In effect, we might know things, but we aren’t able to influence decisions because our data is not compelling enough.

To this end, I have advocated importing some analytics themed best practices from the corporate world to educate more on what to do with the data and how to put the data to use. To in short, be Information Rich, Data Rich to move towards more Data-Driven Decision-Making.

Starting backwards, I will first focus my training on the How. How do we make more data-driven decisions?

The I will focus on the Why. Why do we need to make more data-driven decisions?

From there we will go into several business analytics concepts like Data Visualizations, Public Data Mining, Data Lakes, Demographic Profiling using Big Data, and Data Blending.

amupicture1

A final topic of interest that I will bring to the discussion is the Plus Minus Implications for Unstructured and Qualitative Data. Things that at first can be hard to assign a number too, but are just important as any piece of traditional data used in decision-making.

At the conclusion of my work, public policy and project reporting will be much more data rich, influence will improve and decision-making enhanced.

Now we won’t just know, we will be able to champion what we know in ways that will make a difference.

Analytics Education – Facilitating a mastery of the fundamentals of analytics is what DMAIPH does best. All across the world, companies are scrambling to hire analytics talent to optimize the big data they have in their businesses. We can empower students and their instructors with the knowledge they need to prepare for careers in analytics. Contact DMAIPH now at analytics@dmaiph.com or connect with me directly so we can set a guest lecturer date, On-the-Job Training experience or other analytics education solution specifically tailored to your needs.

Advertisements

Survey Results: Devote More Time For Data Analysis

Most Analysts Spend 50% of Their Time Finding Data

% Finding Analyzing Reporting
10 12% 6% 33%
20 14% 10% 39%
30 20% 31% 24%
40 6% 14% 2%
50 31% 16% 2%
60 14% 18% 0
70 0% 0% 0
80 0% 2% 0
90 0% 0 0
100 0% 0 0
       

Most analysts spend most of their time finding data.

Among other thing this can mean they are setting up data mining or data gathering process to look for the data or it can mean they reviewing their data for relevancy.

My experience is that when you spending this much time on the finding the right data phase it reflects a poorly structured data environment or a unfamiliarity with the data needed.

Dirty data is also a big time waste.

Experience is the best solution for challenges with finding data. The fact the finding phase % is so high speaks to both the explosion in the 3 V’s of Big Data (Velocity, Volume and Variety)  as well as the number of analytics newbies.

To me this should be no more than 20% of your time.

I expected finding data would be the biggest chunk, but was surprised that over 50% of my analyst connections using at least 40% of their time finding data.

If you have one day to answer a key business question, this means you are using your entire morning just finding the data.

When you get back from lunch you haven’t even started the actual analysis yet and the clock is ticking.

Data is based on a survey I sent to 3,000 of my LinkedIn connections who are either analysts or work closely with data and analysis.

Analytics Survey – DMAIPH conducts quarterly analytics surveys to collect data on current trends in analytics. We specialize in surveys that assess analytics culture and measuring how aligned an organization is to using data and analytics  in its decision-making. Contact DMAIPH now at analytics@dmaiph.com or connect with me directly to find out more about how DMAIPH can conduct surveys to help you assess the analytics culture in your business.

DMAIPH Quick Data Survey

A few months back I sent a quick survey to 3,000 of my LinkedIn connections who are either analysts or work closely with data and analysis.

Here is the question I asked.

Greetings!  I’m hoping you can help me gather some data for a book I’m working on. If you had to breakdown the work you do into 3 buckets; finding data, analyzing data and reporting data, what would the % of each be? A quick reply with your breakdown would be hugely helpful in my research. Thanks!   Dan Meyer, Analytics Champion, www.dmaiph.com

I got back over 400 replies.

Here is how they broke down.

 

% Finding Analyzing Reporting
10 12% 6% 33%
20 14% 10% 39%
30 20% 31% 24%
40 6% 14% 2%
50 31% 16% 2%
60 14% 18% 0
70 0% 0% 0
80 0% 2% 0
90 0% 0 0
100 0% 0 0
       

The higher the %, the more each analyst spent time doing that particular phase of analytics.

Here are some of my takeaways from this simple (and very nonscientific survey)

  • I was surprised to see 45% spend half their time or more on finding data. To me this is one of the telling signs that Big Data has led to a shortage of top analytics talent.
  • Only 1 out of 4 analysts are spending 20% of less of their time finding data. These are generally senior analysts, well established in their company.
  • Only half of my analyst connections are spending 40% of more of their time on conducting analysis. With significant time spent on finding and/or reporting data you can imagine a lot of important discoveries are being missed and opportunities lost.
  • Only 1 out of 3 analysts are getting spend my recommended 50% or more of their time actually doing analysis work.
  • Based on my survey, reporting gets shortchanged a lot. All in, 96% of respondents spend 30% of their time of less on reporting.
  • My recommendation is that you spend about 30-40% of your time on the reporting aspect, and sadly only 4% of my analytics connections are able to do that.

In an ideal world, I would expect an analyst to spend no more the 30% of their time on finding data, and at least 30% on reporting their findings, leaving more or less 40% to do the actual analysis.

IMG_1310

This breakdown is based on my own experience as an analyst as well as seeing how analyst working for data-driven companies work.

Only about 30% of my 400+ analytics focused LinkedIn connections come close to meeting my recommended breakdowns.

Which means I have a lot of work to do.

Analytics Survey – DMAIPH conducts quarterly analytics surveys to collect data on current trends in analytics. We specialize in surveys that assess analytics culture and measuring how aligned an organization is to using data and analytics  in its decision-making. Contact DMAIPH now at analytics@dmaiph.com or connect with me directly to find out more about how DMAIPH can conduct surveys to help you assess the analytics culture in your business.

 

Why Analytics Projects Fail – #9: Bad Data

In my experience, most of the time analytics projects fail its generally traceable back to a purely human problem. However, sometimes you see things fall apart because of technology, the misuse of technology and/or just bad technology. This is the case when projects fail because of bad data.

There are a lot of ways bad data can happen.

One common way you end up with bad data, is the data was not captured correctly. Perhaps the data was manually input with lots of error. Or maybe your data is not consistently collected so it has gaps. Knowing what exactly goes into capturing your data and being able to understand how it is collected is extremely important.

zurich_image_april_10-323x353

Another cause of bad data is that you are not getting all the data or you are getting data that has been altered. A lot of times when data passes from the collection point to you, it might be being truncated, or blended, filtered or converted. Lots of databases are structured for optimal data storage, not usage. A lot of database admins who don’t really know the data will add data flow shortcuts. Or maybe the fall under the datakeepers category and partition or cut out some of the data you need.

Bad data also comes in the form of old and out of date data. When you are making decision on data that just not recent enough, it can lead to a lot of problems. Keeping data fresh is something some companies just don’t value. If that’s the case, you will likely see your analytics initiatives come up with analysis that points you in the wrong direction.

In all three of these examples, one solution I suggest to mitigate the chance you have bad data is to build a data map. Learn about every point in a data flow that touches your data. Talk to the ones in charge of each touch point to make sure your data is not being impacted in any way that can result in bad data. Even if you cannot fix the problem, understanding it can help you set more realistic expectations of what your analytics project can achieve.

I have found using Visio to build data flow visuals is the best way to explore, document, and report how the data being used in my projects is being impacted by the environment it lives in. Knowing Visio is a valuable skill for an analyst. If you don’t use it, I promise you that once you do you’ll be sending me a thank you.

may18

Analytics Culture – The key to using analytics in a business is like a secret sauce that fuels Data-Driven Decison-Making. It is a unique combination of analytics talent, technology and technique that are brought together to enrich and empower an organization.

A successful analytics culture is not easy to create, but DMAIPH can show you how. Contact DMAIPH now at analytics@dmaiph.com or connect with me directly so we can build a strategic plan to turn your company into analytics driven success story.

Q3: What are some of the current trends in analytics?

Every few months I devote a day to discover what are the current trends in analytics. I do this both to refresh the slides in my presentation and to refresh my mind to see what I may have missed.

The amount of literature out there on analytics continues to blossom at an amazing rate, making it a true challenge to stay well versed on what’s hot and what’s not. I read a new analytics themed book about once a month and I have well over 200 blogs, web sites and social media groups cataloged. So I like to think I’m pretty well versed on what is current.

Every time I go to list the top 5 analytics trends, I find that some things change and some stay the same. Ever since I have been doing this, data visualization is near the top. Business dashboards continue to be a big need. Business intelligence tools evolve and new ones’ pop up, but Tableau continues to be a market leader. 90% of us still use Excel for 90% of our analytics work.

275

Still a lot has changed. When I started this just 5 years ago no one was really talking about Big Data or Data Science. People just stared discussing using predictive analytics and now its all about prescriptive, even though most of us are still just doing descriptive analytics. For the newbie, descriptive = historical, predictive = forecast models, and prescriptive = really complicated models with a lot of variables to not just predict the future but to show a lot of alternatives as well.

Now if you talk to experts they make think nothing I have mentioned so far is new. But to the novice analyst or to the manager who really doesn’t care what’s it called, she just want’s results… its all new to them.

So I try each time to really find something really new not just to me but truly new to analytics. Six months ago that was the idea of using a data lake instead of a data warehouse. For those still unsure what a data warehouse is, it’s a collection of databases stored and/or connected centrally. Data lakes are used to describe the reality that more and more data is now unstructured data.

The discussion on what is unstructured data and how best to mine it and integrate it with structured data has really been at the forefront for a while now. Going from 80% structured to 90% unstructured in in just a few short years as mankind generates unprecedented amounts of data not easily captured in a database every day.

As of today, if I had to pick 5 topics to talk about it would be (1) Hiring Data Science and Analytics Talent, (2) Big Data Analytics, (3) Data Warehousing and Data Lakes, (4) Data Blending and (5) Mining Public Unstructured Data

Check back with me in a few weeks and this list will change.

The Fundamental of Business Analytics – Business Analytics is the application of talent, technology and technique on business data for the purpose of extracting insights and discovering opportunities. DMAIPH specializes in empowering organizations, schools, and businesses with a mastery of the fundamentals of business analytics. Contact DMAIPH now at analytics@dmaiph.com or connect with me directly to find out how you can strengthen your business analytics fundamentals.

All The Tools And All of The Talent but none of the Technique… Where Good Analytics Intentions Go Bad

I have seen so many examples of this. A majority of companies throw money at analytics in the form of buying new technology, but don’t spend a fraction as much on the people who need to make the technology work.

A good analyst using Excel is much more powerful then a mediocre analyst using a cutting edge BI tool. Without the innate curiosity, knowledge of the business and ability to communicate discoveries that come with a good analyst, your analytics plans will fall short no matter what the sales reps from the analytics companies promise you.

Now we have the 2016 Presidential Election results to analyze. Most predictive models had Clinton winning. Most of the polls had Clinton winning.

So where did the analytics go wrong? Well, its definitely not the technology. And I don’t think it was the talent.

In the coming days, I am pretty sure we will find it was the technique.

It was not getting deep enough data.

It was looking at the data and seeing what you expected to see.

Curiosity was lost.

Finding new perspectives to make sure we have the right data next time.

Hillary Clinton’s campaign will be a case study in where good analytics where not good enough.

Analytics Culture – The key to using analytics in a business is like a secret sauce. It is a unique combination of analytics talent, technology and technique that are brought together to enrich and empower an organization. A successful analytics culture is not easy to create, but DMAIPH can show you how. Contact DMAIPH now at analytics@dmaiph.com or connect with me directly so we can build a strategic plan to turn your company into analytics driven success story.

 

Internet Research Tips

About to do a training for some of my team on how to master internet research. Here are some of the excellent tips on how to optimize Google searches that I will be sharing:

1. Use unique, specific terms – It is simply amazing how many Web pages are returned when performing a search. You might guess that the terms blue dolphin are relatively specialized. A Google search of those terms returned 2,440,000 results! To reduce the number of pages returned, use unique terms that are specific to the subject you are researching.

2. Use browser history – Many times, I will be researching an item and scanning through dozens of pages when I suddenly remember something I had originally dismissed as being irrelevant. If you can remember the general date and time of the search you can look through the browser history to find the Web page.

3. Don’t use common words and punctuation – Common terms like a and the are called stop words and are usually ignored. There are cases when common words like the are significant. For instance, Raven and The Raven return entirely different results.

4. Set a time limit — then change tactics
Sometimes, you never can find what you are looking for. Start an internal clock, and when a certain amount of time has elapsed without results, stop beating your head against the wall. It’s time to try something else:
> Use a different search engine, like Yahoo! Bing, Startpage, or Lycos.
> Ask a peer.
> Call support.
> Ask a question in the appropriate forum.
> Use search experts who can find the answer for you.

540

The Internet is the great equalizer for those who know how to use it efficiently. Anyone can now easily find facts using a search engine — assuming they know a few basic tricks.

Never underestimate the power of a skilled search expert.