Tag: KPIs

The Analytics of Measurement and Evaluation

By taking inspiration from the way corporations use business analytics to optimize their Big Data, our Program Measurement and Evaluation processes can be greatly enhanced.

To understand the connection, let’s start with the mission of the Measurement & Evaluation program.

“The ability to effectively evaluate projects, programs and processes is becoming increasingly essential to organizational success today. American University’s online Master of Science (MS) in Measurement & Evaluation provides you with the knowledge to lead these evaluation efforts and the technical skills needed for analytically demanding roles in upper management.” 1

A good analytics solution constructs a universal framework for collecting, analyzing and utilizing data to determine project effectiveness and efficiency.

Likewise, an efficient measurement and evaluation of projects, programs and policies using analytics should ensure success. An analytics centered approach will likely work with corporate, non-profit and governmental organizations across various sectors and industries.

IMG_6912

We can look specifically to two key business analytics concepts I have used in my twenty plus years of analysis work; Key Performance Indicators (KPIs) and Data Visualization. The key to my success was my ability to answer important business questions using analytics.

Analytics is generally defined as the discovery of patterns in data that provides insight and identifies opportunities. As Carly Fiorina, former CEO of HP said about analytics, “The goal is to turn data into information, and information into insight.” 2

Organizations that invest in analytics generally make much better business decisions then one’s that don’t. In fact, IBM found that organizations who use analytics are up to 12x more efficient and 33% more profitable. 3

In the corporate world, business analytics is widely use to track, analyze and report Key Performance Indicators (KPIs).

KPIs are rolled up to senior leadership to drive business strategy, identify and mitigate risk and to optimize operational productivity.

This approach is very similar to the way projects in the Measurement and Evaluation are tracked, analyzed and reported.

So we need to ask ourselves, what are the KPIs for the project, program or process we are measuring? What points of data need to be captured, analyzed and reported to determine success?

A successful analyst is able to remove the noise when analyzing data and isolate what matters most to his or her organization. That is what is at the heart of measurement, knowing what data is important and what is not.

Once we have the right data, we can measure what the data tells us to determine success, causality, impact… whatever the outcome may be.

A quote often attributed to management guru Peter Drucker perfectly sums up why big corporations rely so heavily on analytics when he said “What gets measured, gets managed.”

Similarly, policy decisions can be made based on what is measured. Project funding can be impacted by what is measured. Process optimization can be directed by what is measured.

Once we are able to measure what is truly important to policy-makers, managers and decision-makers, we need to make sure we present the data in a compelling way.

This is where data visualization comes in.

I often make the analogy that if a picture is worth a thousand words, then a good pie chart is worth a thousand rows of data.

We all know that most people learn more by seeing something then by reading or hearing it. Data visualization takes that a step further.

Data visualization is not only important to presenting our insights but also for exploring the data for insights. Most people find it easier to process information when it is in the form of a picture then a collection of data.

Chip & Dan Heath, Authors of Made to Stick, found that, “Data are just summaries of thousands of stories – tell a few of those stories to help make the data meaningful.”

The ability to take all of the data gathered in the measurement phase and use it in the evaluation phase will make a significant difference in the success of the project, program or process you are working on.

According to the Office of Planning, Research and Evaluation, “Program evaluation is a systematic method for collecting, analyzing, and using information to answer questions about projects, policies and programs, particularly about their effectiveness and efficiency”. 5

Data Visualization can be used to paint a picture of a program, project or policy that influences outcomes based on the KPIs. And by appealing to the basic human fascination with stories, a persuasive graph, chart or infographic can make all the difference in the world.

By adopting the business analytics concepts of KPIs and Data Visualization, and applying them to the world of programs, policies and projects, you can find the same level of success I found in the corporate world.

  1. American University, “Certificate in Measurement & Evaluation” http://programs.online.american.edu/online-graduate-certificates/project-monitorin Accessed October 20, 2016
  2. Carly Fiorina Speech from December 6, 2004 http://www.hp.com/hpinfo/execteam/speeches/fiorina/04openworld.html . Accessed October 20, 2016
  3. Simon Thomas, Senior Analytics Consultant for IBM https://youtu.be/Zi8jTbXnamY . Viewed October 20, 2016
  4. Chip & Dan Heath, Authors of Made to Stick, http://heathbrothers.com. Accessed October 20, 2016
  5. OPRE, http://www.acf.hhs.gov/opre/resource/the-program-managers-guide-to-evaluation-second-edition. Accessed October 20, 2016

Analytics Education – Facilitating a mastery of the fundamentals of analytics is what DMAIPH does best. All across the world, companies are scrambling to hire analytics talent to optimize the big data they have in their businesses. We can empower students and their instructors with the knowledge they need to prepare for careers in analytics. Contact DMAIPH now at analytics@dmaiph.com or connect with me directly so we can set a guest lecturer date, On-the-Job Training experience or other analytics education solution specifically tailored to your needs.

Advertisement

Finding The Time To Talk About Analytics Strategies

One of the biggest frustrations I hear from people when we are talking about how empower more data-driven decision-making in an organization is that no one has enough free time to get serious about analytics.

When facing this kind of challenge, a short presentation full of useful data can be used to start the ball rolling. When people say they are too busy then they are often actually saying I have too many things to keep track of and I am not as organized as I should/could be.

There are indeed times when there is more work than can be done in 40 hours a week, but that is actually not the case in most situations. Most corporate cultures reward those with the “I’m so busy I don’t have time to waste on reports” badge of honor.

To counter this you need to do some research into what are the potential time and cost savings that come from fixing the reports and getting better data to decision-makers. It wont take long to find a lot of numbers to use in your business case after a quick Google search. LinkedIn is also a good place to find lots of supporting documentation.

Once you have that info, you can add it to your own assumptions about what KPIs are not being used or what KPIs are missing from current reporting or which KPIs are miss reporting. This can all be turned into a short, compelling case for others taking the time to talk about broader reporting across the organization.

11756770_934770113231654_1677809504_o

To really add some punch to your argument, ask one of the too busy people to give  you a few minutes of time to ask them what more they want/need out of their reports. Hardly anyone is truly happy with the reports they get. Gain a few allies in your crusade.

Most people think that using analytics effectively is all about the technology, but the real secret to success is getting the people on board and on the same page. True analytics centric cultures are based on engagement and that engagement needs a facilitator.

Once you have your story and a few key advocates it will be a lot easier to get closer to the data-driven decision-making you are looking for.  If you need help polishing your story or advice on how to get the engagement of a really tough “I’m too busy” co-worker, let me know. I’m sure I can lend a hand.

Analytics Culture – The key to using analytics in a business is like a secret sauce. It is a unique combination of analytics talent, technology and technique that are brought together to enrich and empower an organization. A successful analytics culture is not easy to create, but DMAIPH can show you how. Contact DMAIPH now at analytics@dmaiph.com or connect with me directly so we can build a strategic plan to turn your company into analytics driven success story.

When Your KPIs Aren’t Really KPIs

One question I get asked a lot is what should someone do when they know the data they are reporting and/or using in their analysis is not the best data to available?

No matter what part of the business you work in, the first thing to do is to define the current Key Performance Indicators (KPIs) being used in decision-making.  Often right off the bat, some of the KPIs being reported aren’t even being used.

You can do a simple survey, asking end users to rank in order of importance the KPIs they get. Also ask if the ones at the bottom are even useful or should they be eliminated if no one is using them.

Growthink_Dashboard_Hero_w_Background_PSD_0

At the same time you should be working on understanding what computations go into each KPI. Often we just do simple counts, total and averages that mask more important data. On the flip side, we tend to over complicate things with extravagant weighing and scoring. Either way, we need to make sure we know exactly what is being reported and how does the final data point come to its end state.

The next step is to look at the data architecture to make sure there is nothing happening upstream that might impact the data we are using in the KPIs. Before making changes to the KPIs we want to have the full view of what happens before the data gets to the end user.

Now we are at the point where we can start experimenting. What happens when we swap out data points? Or if we change a variable in a calculation? Or we pull the data from a different source? The questions are endless. Pick a few, make some changes in a test environment and start sharing the updated KPI data. See if it has more value with the end users.

Again, this shouldn’t be hard. But of course in many organizations a lot of consequences can result from a simple change to just one KPI. Spreadsheets may have to be reformatted, review processes may have to be updated, dashboards may have to be redesigned. But in the end, what is more important… making decisions with crappy data or setting a standard to let the reporting process evolve as the business evolves?

This come back to my point earlier, changing KPIs is as much sales as it is analysis… that you have to be ready to share a story, back it up with data, and really influence the minds of senior management that updating the KPIs makes good business sense.

If you are at a point where you are trying to figure out what KPIs aren’t working anymore or you need help is building a business case to change some KPIs, let me know. I’m here to help.

jobspicture2

General Analytics – Analytics is the application of using data and analysis to discover patterns in data. DMAIPH specializes in empowering and enabling leaders, managers, professionals and students with a mastery of analytics fundamentals. To this end we have parterned with Ariva Events Management/Ariva Academy to offer a wide range of analytics themed trainings across the Philippines. Contact DMAIPH now at analytics@dmaiph.com or connect with me directly to find out what we can do to help you acquire the analytics mastery you and your organization need to be successful in today’s data-driven global marketplace.