Finding the Right Data at the Right Time

Sir Conan Doyle’s famous fictional detective, Sherlock Holmes, couldn’t form any theories or draw any conclusions until he had sufficient data. Data is the basic building block of everything we do in analytics: the reports we build, the analysis we perform, the decisions we influence, and the optimizations we derive.

resized_99261-basil_85-16636_t300 — Basil Rathbone as Sherlock Holmes

Several years ago I came across a book called the Accidental Analyst (*www.accidentalanalyst.com). The book opens with the questions, “Are you drowning in a sea of data? Would you like to take control of your data and analysis to quickly answer your business questions and make critical decisions? Do you want to confidentially present results and solutions to your managers, colleagues, clients and the public?”

Written by two Stanford professors, the book explores how and why people become good analysts and goes into detail about how to approach analytics successfully. After reading the book I was inspired to come up with a way to teach analytics to college students and fresh graduates.

The core of both the book and my program hinges on the ability of an analyst to find the right data at the right time. The authors suggested that identifying your data is where it all starts. Identifying exactly what you need to address whatever it is that you need to report.

Back at Wells Fargo, the single greatest attribute that I had that made me successful was my ability to size up how long it would take to deliver something. Knowing what data I would need, where I would find it and how long it would take to analyze it to come up with something useful made me somewhat of a wizard in the minds of the team.

Finding the right data at the right time requires one to first know ends and outs of their data. You have to know how the data is captured, where it is stored and how it makes its way to you. Knowing the data architecture in your business is the key.

So you have to get to know the people who know where your data comes from and how it gets there. Learn from them. Partner with them. Buy them doughnuts.

A couple of years ago I came across an analogy being used to describe data in a business. That of a data lake. A data lake is the living, breathing, evolving pool of all the data in a business. If you have a good data architecture, and you can navigate it fairly easily, then you have a data lake. Ideally, your business has data structured in such a way you can live off it. Data to a business is like water to living things… it sustains life

So once you have the lake mapped out, then you have to learn how to fish it. Knowing where the fish are biting is another key. Once you know what data you need, you have to know how to get to it quickly.

Business Intelligence tools help us here. As does coding languages to extract data from a database. These are your fishing tools. You have to practice using them to be good at getting the right data at the right time.

Another way to optimize your data search is to save your work. Of as I call it leave yourself breadcrumbs. Save the query. Cut and paste the code into a document and save it. Write down the steps. Whatever you need to do to replicate what you just did so you can do it again in the future without starting over from scratch.

So to recap, if you know data structure, you understand how data is stored and you leave yourself clues to do things faster next time.
Now the other part of the equation is knowing if the data you are using is the right data. Finding data quickly doesn’t do you any good if you bring back the wrong data.

So, how do you know if the data you are using is the right data to be using?
I can’t count the number of times I asked myself that question. In general, just about every new analysis or project or research or whatever it is you are using data for, you have to ask that question at some point.

Even data you have used a hundred times and comes from a highly trusted source needs to be scrutinized.

Now if you work with data every day in a familiar format, from the same source and with no changes to the data gathering and storage process you don’t have to spend much time validating it. Usually you will see problems when something just doesn’t look right when you are doing the analysis.

On the other hand, things get a whole lot trickier when you are using data from a source you don’t use often, or something has changed in the way the data is populated or if it’s the first time you are using the data.

When this happens, I have a few suggestions on how to validate the data.

First off, pull the data, do your analysis and draw some conclusions. If it passed the eye test and it feels ok to you, then your job is just to validate it.
One simple way to do this is pull the data again the exact same way to make sure you get the exact same data. Or change one parameter like the dates used in the query. See if that significantly alters the way the data looks and feels.
Another option is to have someone else do the same thing independently. See if they get the same results you do. You can also find someone who knows the data to look over your work to see if it makes sense to them.
Whatever you do, the best way to prevent publishing or using bad data is to involve someone else. Not always possible, I know, but it’s the best way to go.

Another suggestion is to (1) get the data, (2) do some analysis, and then (3) step away for a while. Come back to it with fresh eyes. Don’t let our minds play tricks on us by making us see what we want to see and not what is really there.

I have seen several articles showing research that most time doing data analysis is actually spent cleaning data. In a lot of businesses, the data lake has become a data swamp, clogged with bad or unusable data. As the % of unstructured data increases daily, it’s easy to see how data swamps have become the norm. Even the most robust data collection and mining can run afoul if the data is not trustworthy.

I can’t stress this enough. No matter how good you are at analysis, or what tool you are using to do the analysis, if you don’t have an understanding of what happens to the data before it gets to you then you are probably not drinking from a clean lake.

DMAIPH – Decision-making, Analytics & Intelligence Philppines

Over the past few years businesses in the Philippines have invested heavily in big data, analytics and data science, but still have not achieved the expected outcomes of data-driven companies.

Based on our learnings from the 100’s of Filipino businesses and 10,000s of Filipinos who have taken part in DMAIPH Analytics trainings all across the country, we have crafted a proven, non-technical approach to upskilling your team in analytics.

In 2019, we will be launching two new training programs: (1) Our DMAIPH Applied Analytics Master Class series for executives, leaders and decision-makers and our (2) DMAIPH Applied Analytics Boot Camp series for practicing analysts.

Connect with us via our marketing partner, http://www.sonicanalytics.com to learn about upcoming analytics trainings and events.

Staying Current with Analytics

Every few months I devote a day to discover what the current trends in analytics are. I do this both to refresh the slides in my presentation and to refresh my mind to see what I may have missed.

The amount of literature out there on analytics continues to blossom at an amazing rate, making it a true challenge to stay well versed on what’s hot and what’s not. I read a new analytics themed book at least once a month and I follow dozens of blogs, web sites and social media groups. Being well versed on what is current in analytics is a key to success.

Every time I go to list the top 5 analytics trends, I find that some things change and some stay the same. Ever since I have been writing about analytics, data visualization is near the top. Business dashboards continue to be a big need. Business Intelligence (BI) tools evolve and new ones’ pop up, but Tableau continues to be a market leader.

That said, we are still squarely in an MS Excel dominated world. Upwards of 80% of Filipino professionals I recently surveyed still use Excel as their primary tool for data analysis. And even the ones who have dedicated BI tools, still use Excel for 75% of their analytics work. The adoption of BI tools is trending upward, but the curve is still very step.

Another trend that has been on the upswing is how more and more data is now unstructured data. The discussion on what is unstructured data and how best to mine it and integrate it with structured data has really been at the forefront for a while now. Going from 80% structured to 90% unstructured in just a few short years as mankind generates unprecedented amounts of data not easily captured in a database every day.

As October 2018, if I had to pick 5 current trends in analytics to talk about it would be:

(1) How to Conduct Impactful Data Storytelling,

(2) The Analytics and Data Science Talent Shortage,

(3) Using Big Data Analytics for Digital Transformation,

(4) Optimizing Data Warehousing and Data Lakes,

(5) Which Tool Is Best; Tableau or Power BI, R vs Python, etc

And thats is not even touching topics that are on the cutting edge like machine learning, artificial intelligence and augmented analyst. Although those are super important to an overall understanding of how we can optimize data, these topics generally are several steps down the road from where my audience sits. They are still trying to master the fundamentals of business analytics and introductory data science.

So I spend a fair amount of time looking for YouTube videos or TED Talks on these topics to add to what i read.

The amount of information available to consume if immense. I guess as we have more and more data and more and more tools to analyze data, we will have more and more people writing about how to use data.

Its a fun time to be the Data Guy.

42650914_1540261689452602_6471348166867288064_n

DMAIPH – Decision-making, Analytics & Intelligence Philppines

Over the past few years businesses in the Philippines have invested heavily in big data, analytics and data science, but still have not achieved the expected outcomes of data-driven companies.

Connect with us via our marketing partner, http://www.sonicanalytics.com to learn about upcoming analytics trainings and events.

From Putting Your Data to Work… A Basic Overview of Analytics

Analytics is about looking for patterns in data to help answer questions. Most businesses use analytics to help ensure more data-driven decision-making.

The primary people responsible for conducting analytics on the massive amounts of data we have today are analysts. Analysts are skilled in using various technologies and methodologies to identify, inventory and integrate large amounts of data quickly.

Many Analysts today feel like they are drowning in a sea of data. They need to know how to take control of their data and analysis to quickly answer business questions and make critical decisions. They want to confidently present results and solutions to their managers, colleagues and clients.

You should get started by building a baseline understanding of analytics. The term analytics can often be used interchangeably with statistics and data science. What separates analytics from disciplines like statistics and data science is generally the speed of the analysis and the focus on solving business problems.

The most common form of analytics is business analytics, which is usually used by senior leaders and decision-makers to investigate problems, validate assumptions and to guide strategic planning.

Business analysts are therefore the most common type of analyst. If you do a job search on the title analyst, as many as half the posting will likely be business analysts. However, analytics can be used in an almost limitless number of business functions in specific areas like HR, recruitment, marketing, finance, and so on. Each one can have its very own analyst.

13920727_10154471484642425_3148729421440288448_n

Analysts have been around a long time, but recent technological advances have both allowed us to produce and capture more data as well as give us the ability to analyze immense data sets quickly. Thus we are amidst a huge boom in the applications of analytics and the need for analytics talent across the globe. Analytics is something just about every business leader is trying to figure out how to use more effectively in their business. To add to our challenge, the demand for good analysts is booming just as fast as the explosion in big data.

As a result, there is a huge shortage of people who are skilled in working with data to answer questions and solve problems. This is why you have seen the number of analyst job postings increasing at an amazing rate.

If you are a business leader, manager, owner, and/or executive are not actively trying to surround yourself with analysts and if you are not infusing an analytics centric culture in your business, you will most likely soon see your business fail.

A business needs analysts to make sense of big data, manage the storage of the data, and know when to use which of the 4 types of analytics (descriptive, diagnostic, predictive, and prescriptive). To be effective, analysts need to have business intelligence tools to create data visualizations, build business dashboards and tell stories with data.

So whether you are an analyst or someone who oversees analysts, Putting Your Data to Work is designed as guidebook to help you get the most out of your business data.

1ee69-1o4ohlnayqgythuxcsj-nca

DMAIPH – Decision-making, Analytics & Intelligence Philppines

Over the past few years businesses in the Philippines have invested heavily in big data, analytics and data science, but still have not achieved the expected outcomes of data-driven companies.

TRAINING IN DATA ANALYTICS

By Maureen Andrei Lepatan

INTRODUCTION

We create data everyday. How? We, especially in this generation spend many hours in accessing our social media accounts, doing online shopping, playing games, watching movies online. Part of our daily routine includes internet and technology. By doing so, all of our hobbies generate data that are captured in various places and in different ways.

Every time we post pictures on Instagram, rant something on Twitter and post our status and photos on Facebook, we create a lot of data. There is a corresponding data point every time we comment or like something online. Imagine how many data we can generate everyday if every person of this planet accesses online.The data become closer and closer to infinity. That is why the term “big data” was created.

With that being said, data analytics is key to handle pool of data. Analytics is about searching for clues that will enable us to find answers to our problems. We find, we analyze and we present our data.

Primary people for conducting analytics are called analysts. The problem would be that they are overwhelmed by massive amount of data and have trouble to handle them properly.

In order to be effective, analysts should master effective and current business intelligence (BI) tools that could help them to interpret the data properly and guide the companies and businesses regarding their strategies and decision making processes.

I started having interest in dealing with data when I was 3rd year in college. Before, I was a Math person. I am the kind of person who likes challenging activities and work on complex subjects. In the pursuit of my Economics degree, I used a lot of data and created graphical representations in order to survive essay crises and loads of research papers.

Somehow, economics has the same idea as data analytics which is to tell a story out of the representations. The difference lies upon the frequency of the usage of business intelligence tools in data analytics.

Why did I dive into data analytics? It fits my personality, hobbies and skill sets. I am curious in nature and love to learn new things. I love editing videos, photos and creating infographics and graphical representations. And data analytics made me combine all of these hobbies in data analytics. It enables me to be creative, analytical and communicative all at once. There is no wrong and right approach. I can be my own self. As long as I get the right data, visualize and verbalize them well, I’m good to go.

Data analytics gave me a sense of purpose. I think in this generation, being an effective analytics talent is what the world needs. I do not mean to disregard other jobs. I just want to be realistic about the present and the future. More and more businesses will build their companies using online platforms requiring more data analytics talents. If businesses do not adapt to the demands of the society, they will most likely fail. As a student and future professional, I need to prepare for these changes. Although I have a background in dealing with data, I need to learn timely business intelligence tools and to train myself to be a better data storyteller.

DMAIPH can help analysts and aspiring individuals who want to learn data analytics. The company conducts trainings to help increase effective and efficient analysts in the Philippines and meet the demands of the society when it comes to data enthusiasts.

Screen Shot 2018-10-08 at 1.04.02 PM

EXPERIENCE

Last September 25 and 26, I attended the training of Sir Dan Meyer regarding Data Management and Data Visualization. In a span of two days, I was able to have an overview of how data analytics works and how to use business intelligence tools to tell a story. Moreover, I was also assigned to tasks like to welcome the guests and to assist Sir Dan in helping the participants to use Tableau since I also need to fulfill my duties as a business analytics trainee.

At first, I was really intimidated with the participants when they introduced themselves. I never thought that the people whom I say “Good Morning/ Hello” to are CEOs and various kinds of analysts in their respective companies. This really reflects that the demand for analytics talents in the Philippines is greater than the supply. When I talked to some of them, they said that companies have sent them to have trainings with Sir Dan and some of them personally wanted to learn to help their companies.

Training people is really a must to adjust in this day and age. As time goes by, more and more data are generated and unstructured data gradually increase. If data continue to produce increments, the world needs more and more analysts to handle them. In the case of the Philippines, Excel still dominates the analytics industry and is used primarily by professionals to conduct data analysis despite the evolution of business intelligence (BI) tools. On the second day of the training, the practical application of the concepts taught in Day One were applied. Sir Dan tackled about business intelligence tools, data visualization, business dashboards and data storytelling.

I have 5 major takeaways that I want to share with you:

Data Visualization is just half the job. We need to interpret the data correctly and relay the information such that a grade school student can understand the story behind the data. This is in order to create an impact to various kinds of people and encourage decision-makers to make relevant changes in their businesses. Just be simple and precise!
Learning data science and analytics is all about experimentation. We shall be ready for mistakes along the way. We must continuously attend trainings in order to guide us and persistently practice on our own to obtain mastery.
Companies are enchanting because people like them and trust them. As part of a company, we want to reflect the enchantment our companies have to give to the customers. Without the right strategies to be enchanting, people will not believe us leading to a low profitability and a bad reputation. We can be enchanting as analysts if we can deliver the data persuasively and we can work well with other people.
Being an effective data scientist is a combination of being mobile when it comes to changes in technology and being adaptable in dealing with people.
There are three types of analytics which include descriptive, predictive and prescriptive. How do we use them properly? Descriptive analytics can be effectively utilized if we want to know what happened to have insights in present trends. For example, we want to know about the profits in each month from 2015-2017. Secondly, predictive analytics is used to develop projections and provide information what might happen in the future. Expected sales can be best represented by predictive analytics. Lastly, prescriptive analytics is used to know what to do. We can use this especially if we want to build a model out of multiple sources and include many variables.

DMAIPH: FIRST TRAINING, FIRST INTERNSHIP

DMAIPH really provided me a brand new experience. Although I love dealing with data and graphical representations before I become an intern, I felt more impactful when I started my training. I got to help the participants how to navigate Tableau and had to work with wonderful people. I was able to apply what I learned in the past and at the same time acquire new skills that will be beneficial for me in the future. I look forward to the trainings and more involvement that I can get from the company.

So far, so good.

Screen Shot 2018-10-08 at 1.04.21 PM

ABOUT THE AUTHOR:

Maureen Lepatan is an Economics student in De La Salle University and currently a business analytics intern in DMAIPH. She has a passion in data analytics especially using business intelligence tools such as Tableau and Excel. She has an eagerness to learn data structures such as SQL.

Revisiting “My Analytics Story” for the upcoming 3rd Edition of Putting Your Data to Work

My Analytics Story

Analytics is in many ways a new profession and up until very recently few people have seen being an analyst as career path. In fact, the majority of analysts became so by accident. To understand analytics, the first thing you should know is that there is no one, right way to analyze things.

As with my case, most analysts are drawn to analytics because they like to solve problems, have an affinity for working with data, are tech savvy and above all else… insatiably curious. By the time I first had analyst in my title, I had already been doing analytics for several years.

Before I was even out of college I became the “Data Guy. I found my novice skills with Excel, my interest in sharing knowledge and my ability to solve problems made me highly employable. Still, there is nothing specific in my background that would suggest I’d become an analytics expert someday.

I majored in History with a plan to be a teacher and even obtained my Master’s Degree in Education. After college I started to teach, but the school I was working at went bankrupt. So I took a job with Wells Fargo Bank just to pay the bills and 15 years later I had amassed a wide range of analytics skills.

If you ask anyone with analyst in their job title, most of them have similar stories. Until recently you could not even get a degree in analytics as schools are just now offering analytics focused courses and degrees.

In 1998, I had the good fortune of being hired by Wells Fargo. The factors that contributed most to my success with the bank were two things inherit in the culture; the progressive use of data in decision-making and the accepted practice of moving up the corporate ladder by moving between departments.

If I had to pick one thing above all others that had made me a good analyst, it is my ability to quickly assess a problem and then identify the data needed to solve the problem. For me, finding the right data is the most important trait to have and also the hardest to teach. It comes out of being curious and letting that curiosity drive you to find answers.

For 15 years that drive lead me to add new skills, learn new technologies, and develop new methods to become a proverbial jack of all trades when it comes to analytics. I often describe myself as a super hero, curiosity being my super power and the wide range of skills I’ve picked up being items on my analytics utility belt.

I am far from an expert on most of the ever increasing number of analytics tools out there, but I know what they can do and what they are good at. There are definitely a lot of people who are better at different aspects of analytics and no one can know it all. But in the end, I have become in many ways a guru of analytics.

I love talking about the fundamentals of analytics, explaining it in layman’s terms, empowering people new to the concept. I also have a passion for sharing my experience with predictive analytics models, using SQL code to write a complex series of table joins between data sources or figuring out what tool would be best use to build a business dashboard.

For the past 7 years now I have been exploring data sets, answering questions, and providing solutions here in the Philippines and loving every minute of it. Analytics… it’s more fun in the Philippines! 🙂

42196437_1884637428252670_3179760826932264960_n

DMAIPH – Decision-making, Analytics & Intelligence Philppines

Over the past few years businesses in the Philippines have invested heavily in big data, analytics and data science, but still have not achieved the expected outcomes of data-driven companies.

Connect with us via our marketing partner, http://www.sonicanalytics.com to learn about upcoming analytics trainings and events.

More and More Frequently I Find What I Do Being a Form of Data Evangelist

The concept of a key influencer in a field being called ad Evangelist has been around business for awhile. I remember reading how early on, called their marketing people evangelists. One the comes to mind is Guy Kawasaki, who wrote about this in his book Enchantment.

When you have something that is either new or still young in its overall adoption cycle, you need evangelists to raise awareness and being the process of acceptance.

When it comes to data evangelism, there has definitely been a number of key leaders pushing for more and more adoption of analytics across various organizations. Bernard Marr is one I have followed quite extensively.

Based on the importance so many companies have placed on analytics in recent years, you would expect to find that just about every business leader buys into the concept of using data to drive decision-making.

To be sure, the tech giants and the banks have been on board for a long time and you have seen the adoption of large-scale analytics really start to have a lasting impact among major players in fields like professional sports, the entertainment industry and politics.

And nowadays, most social media platforms have lots of built in analytics that provide instant insight into what’s hot and what’s not.

However, both factual evidence and general observation are showing this is not necessary the case across the board. We still see headlines saying things like “62% of businesses have no data analytics strategy”.

In fact, many small and medium sized companies are still not where they could be when it comes to optimizing their business data.

Three of the biggest challenges they face are not really knowing what question to ask, how to manage the data so that it is well governed and getting the data to decision-makers seamlessly.

Terms like data interpretation, data collection, data governance, and data automation are not concepts easily articulated by many business leaders.

Having a data intelligent business culture is a lot more than just buying a business intelligence tool and putting it on top of Excel.

There needs to be solid foundation in all aspects of the data life cycle, a clean and well governed data lake to house all business data, and the ability to present data impactfully.

I have found that this is often the missing element when organizations are trying to craft an analytics strategy. They focus on technology or they go out and acquire high priced talent, but in the end they struggle because not enough of their decision-makers are on the same page.

As I train groups of professionals, both in public and in in-house trainings, I find most of the attendees do not have a solid foundation in how data is collected, stored and managed. They just know how to run reports, build simple dashboards and share data in ways that do not often influence the audience as intended.

So that is what I have found myself doing hundreds of times the past several years. Helping build that foundation. Connecting the dots between the various phases of the data life cycle and helping define the data value chain. Once an organization has that down, then going out and getting a fancy new tool or bringing on data science makes sense.

Being a data evangelist is all about getting not just 1-2 executives to buy into the power of analytics, but making sure the whole organization is in a place where they can truly optimize their data, become more business intelligent and compete on the same level as the big boys when it comes to making data-driven decisions.

42250800_1884641561585590_7664423913429925888_n

DMAIPH – Decision-making, Analytics & Intelligence Philippines

Over the past few years businesses in the Philippines have invested heavily in big data, analytics and data science, but still have not achieved the expected outcomes of data-driven companies.

We will feature case studies of real Filipino run business, exercises based on actual analytics challenges being solved by Filipino analysts, and provide you with a copy of my book, Putting Your Data to Work, an analytics guidebook for the Filipino professional
Connect with us via our marketing partner, http://www.sonicanalytics.com to learn about upcoming analytics trainings and events.