The Analytics of Measurement and Evaluation

By taking inspiration from the way corporations use business analytics to optimize their Big Data, our Program Measurement and Evaluation processes can be greatly enhanced.

To understand the connection, let’s start with the mission of the Measurement & Evaluation program.

“The ability to effectively evaluate projects, programs and processes is becoming increasingly essential to organizational success today. American University’s online Master of Science (MS) in Measurement & Evaluation provides you with the knowledge to lead these evaluation efforts and the technical skills needed for analytically demanding roles in upper management.” 1

A good analytics solution constructs a universal framework for collecting, analyzing and utilizing data to determine project effectiveness and efficiency.

Likewise, an efficient measurement and evaluation of projects, programs and policies using analytics should ensure success. An analytics centered approach will likely work with corporate, non-profit and governmental organizations across various sectors and industries.

IMG_6912

We can look specifically to two key business analytics concepts I have used in my twenty plus years of analysis work; Key Performance Indicators (KPIs) and Data Visualization. The key to my success was my ability to answer important business questions using analytics.

Analytics is generally defined as the discovery of patterns in data that provides insight and identifies opportunities. As Carly Fiorina, former CEO of HP said about analytics, “The goal is to turn data into information, and information into insight.” 2

Organizations that invest in analytics generally make much better business decisions then one’s that don’t. In fact, IBM found that organizations who use analytics are up to 12x more efficient and 33% more profitable. 3

In the corporate world, business analytics is widely use to track, analyze and report Key Performance Indicators (KPIs).

KPIs are rolled up to senior leadership to drive business strategy, identify and mitigate risk and to optimize operational productivity.

This approach is very similar to the way projects in the Measurement and Evaluation are tracked, analyzed and reported.

So we need to ask ourselves, what are the KPIs for the project, program or process we are measuring? What points of data need to be captured, analyzed and reported to determine success?

A successful analyst is able to remove the noise when analyzing data and isolate what matters most to his or her organization. That is what is at the heart of measurement, knowing what data is important and what is not.

Once we have the right data, we can measure what the data tells us to determine success, causality, impact… whatever the outcome may be.

A quote often attributed to management guru Peter Drucker perfectly sums up why big corporations rely so heavily on analytics when he said “What gets measured, gets managed.”

Similarly, policy decisions can be made based on what is measured. Project funding can be impacted by what is measured. Process optimization can be directed by what is measured.

Once we are able to measure what is truly important to policy-makers, managers and decision-makers, we need to make sure we present the data in a compelling way.

This is where data visualization comes in.

I often make the analogy that if a picture is worth a thousand words, then a good pie chart is worth a thousand rows of data.

We all know that most people learn more by seeing something then by reading or hearing it. Data visualization takes that a step further.

Data visualization is not only important to presenting our insights but also for exploring the data for insights. Most people find it easier to process information when it is in the form of a picture then a collection of data.

Chip & Dan Heath, Authors of Made to Stick, found that, “Data are just summaries of thousands of stories – tell a few of those stories to help make the data meaningful.”

The ability to take all of the data gathered in the measurement phase and use it in the evaluation phase will make a significant difference in the success of the project, program or process you are working on.

According to the Office of Planning, Research and Evaluation, “Program evaluation is a systematic method for collecting, analyzing, and using information to answer questions about projects, policies and programs, particularly about their effectiveness and efficiency”. 5

Data Visualization can be used to paint a picture of a program, project or policy that influences outcomes based on the KPIs. And by appealing to the basic human fascination with stories, a persuasive graph, chart or infographic can make all the difference in the world.

By adopting the business analytics concepts of KPIs and Data Visualization, and applying them to the world of programs, policies and projects, you can find the same level of success I found in the corporate world.

  1. American University, “Certificate in Measurement & Evaluation” http://programs.online.american.edu/online-graduate-certificates/project-monitorin Accessed October 20, 2016
  2. Carly Fiorina Speech from December 6, 2004 http://www.hp.com/hpinfo/execteam/speeches/fiorina/04openworld.html . Accessed October 20, 2016
  3. Simon Thomas, Senior Analytics Consultant for IBM https://youtu.be/Zi8jTbXnamY . Viewed October 20, 2016
  4. Chip & Dan Heath, Authors of Made to Stick, http://heathbrothers.com. Accessed October 20, 2016
  5. OPRE, http://www.acf.hhs.gov/opre/resource/the-program-managers-guide-to-evaluation-second-edition. Accessed October 20, 2016

Analytics Education – Facilitating a mastery of the fundamentals of analytics is what DMAIPH does best. All across the world, companies are scrambling to hire analytics talent to optimize the big data they have in their businesses. We can empower students and their instructors with the knowledge they need to prepare for careers in analytics. Contact DMAIPH now at analytics@dmaiph.com or connect with me directly so we can set a guest lecturer date, On-the-Job Training experience or other analytics education solution specifically tailored to your needs.

IBM’s SMART Approach to Analytics

Came across this 5 point methodology for applying analytics in a business… very similar to the 3 I’s that I use in my training. I’ll start using both going forward.

S=Start with Strategy
What problems do you need big data to help you solve? If you’re running a business you might think it’s as simple as “How do I increase my profits?” But a question like that is inevitably going to lead you to more questions.
How do you generate more sales? How do you increase visitors to your site or store? How do you make your customers happier?

In this first step you need to be clear about your strategic objectives as well as the key strategic questions you want to have an answer to. You need to have this nailed down before you worry about collecting your first kilobyte of data.

M = Measure metrics and data
Once you know what data you need to answer your most strategic business questions, you can work out how you are going to capture it. Everything we do, online and, increasingly, in the real world, is capable of being recorded and stored. If we visit a website, records are kept of how long we browse for and where we head off to next. GPS systems in our phones as well as CCTV surveillance keep track of our physical movements.

Of course much of it is (hopefully) anonymized. Big data collection isn’t about tracking individuals, it’s about tracking the masses, so patterns can be spotted giving clues to overall trends. This part of the process involves designing the actual systems that will collect what your strategy tells you is needed.

A = Apply analytics
Increasingly, we are finding that the sort of data which contains really valuable insights is very messy. The slightly more technical term we use for this is that it is unstructured data. The sort of neat and tidy data you get when, for example, you ask someone to fill in a form giving you their age, height, weight and data of birth, is structured. The sort of messy, disjoined data you get when you analyze the contents of an email exchange or CCTV recording is unstructured.

The hidden value in this unstructured data is where most big data divers are finding the real sunken treasures. If you’re a business, being able to spot trends affecting your industry before your competitors is what will give you your edge. In order to implement this part of the process you will need to get to grips with the ever-growing range of tools and methods becoming available for making sense of messy, complex data sets.

R = Report results
The most insightful insight ever is useless if you can’t explain what it means to the key decision-makers in your business. Presenting the information necessary to drive change in a clear and digestible format is as vital as any other step of the operation. This part of the process has analogies to storytelling. There will be a beginning, a middle and an end, detailing why you need the insights, what you did to find them, and how they will result in everyone living happily ever after.

If you use data visualization and narratives to tell that story in a focused and interesting way, it’s far more likely people will understand what you are trying to do, and be as motivated as you are yourself about implementing data-driven change.

T = Transform your business
Change—specifically positive change—is the ultimate aim. Transformations you make to your products, service, marketing strategies or internal processes, guided by insights from your Smart Big Data analysis, is the catalyst which will drive that change.

cropped-dmai-fb-cover-photo1.jpg