Let’s embark on an searching quest, akin to finding ‘Dollar Waldo’ in the expansive world of data. Just as Waldo blends into the crowd, waiting to be discovered, valuable actionable insights often hide within vast datasets, requiring a keen eye and systematic approach to be unearthed. In this post, we will embark on a journey through the Data Analytics Lifecycle (1), using the engaging allegory of searching for Waldo to explain each step of the process. We will see how the Data Analyst (2) kick-starts this Dollar Waldo investigation sequence by performing raw data exploration and then dives into decribing its features and explaining the patterns causality. The sequence continues with the Business Intelligence Analyst (3) who will keep his eye open on the field and regularly monitor the previously identified KPIs. Finally the sequence culminates with the Data Scientist (4) who will forecast Dollar Waldo’s location, predict the chances he could be anyone and then pair with the Business Analyst to refine the searching strategy.
1. The Sequence: Analytics Lifecycle
Let’s start with an overview of each step of the data analytics lifecycle, from the initial exploratory analysis to the sophisticated realm of prescriptive analysis. For each step of out Dollar Waldo’s quest, what ouputs are concerned, the mission behind them, and the person responsible for delivering them.
| Step | Where’s Dollar Waldo | Ouput | Mission | Role |
|---|---|---|---|---|
| 1 | Embarking on the quest | Exploratory data analysis | Identify the dataset features: discover patterns, relationships, or anomalies in the data without having a prior hypothesis or expectation. | Data Analyst |
| 2 | Mapping out the scene | Descriptive data analysis | Understand and describe the dataset features: quantify and summarize the discovered patterns. | Data Analyst |
| 3 | Solving the puzzle | Explanatory data analysis | Identify and explain the factors driving the dataset features: hypothesize and prove the causes of the described patterns and the implications of the previous successes and failures. | Data Analyst |
| 4 | Keeping watch | Monitoring Dashboard | Ensure KPIs and conditions are performing as expected by monitoring and reviewing them regularly. | Business Intelligence Analyst |
| 5 | Forecasting the next hideout | Forecasting variables | Forecast future events based on trends estimated from the past and present data of a single variable. | Data Analyst, Data Scientist |
| 6 | Identifying Dollar Waldo | Predictive modelling | Predict the likelihood of an outcome. | Data Scientist, AI Engineer |
| 7 | Strategizing the search | Prescriptive analysis | Prescribe the best course of action to achieve desired outcomes or mitigate risks. | AI Engineer, Business Analyst |
Now that we are familiar with the sequence, we can dive into more details and clarify for each step of our Dollar Waldo’s quest, which question and answer are usually expected, which methodology to follow, and the finally the data to be mined and when.
2. Initiating the Sequence with Data Analytics
The Data Analyst is reponsible for three types of analysis: exploratory, descriptive and explanatory. An analysis is a detailed examination of a topic. It involves performing research and separating results into smaller, logical topics to form reasonable conclusions. It presents a specific argument about the topic and supports that argument with evidence. These reasonable conclusions’s purpose is to help the organizations make informed decisions. Let’s dive into these 3 types of analysis.
Embarking on the quest. Imagine opening a ‘Where’s Waldo?’ book. Your first task is to understand the landscape – the crowded scenes filled with red herrings. Similarly, exploratory analysis involves diving into raw, unstructured data. You don’t know exactly what you’re looking for yet, but you’re scanning for patterns, anomalies, or any clues that stand out.
| Step 1 | Exploratory Data Analysis |
|---|---|
| Question | What can the data tell us? |
| Mission | Identify the dataset features: discover patterns, relationships, or anomalies in the data without having a prior hypothesis or expectation. |
| Answer | Did pattern “a” occurred? The outcome is typically more questions or hypotheses that need further testing, exploration and explanation. |
| Methodology | Use visual methods like graphs, plots, and summary statistics to get a feel for the data. Asking questions and being open to any direction the data might lead. |
| Timing | Post-mortem |
| Mining | Historical data |
| Example | Let’s explore the effectiveness of a marketing campaign. We scrutinize click-through rates and conversion data across various digital platforms and we uncover that social media channels have a markedly higher engagement compared to traditional online ads. We then segment the audience data, and it becomes evident that younger demographics respond more positively to the campaign, particularly in mobile app environments. Our analysis also reveals that the time of ad display significantly impacts customer interaction, with late evenings yielding the highest engagement rates, suggesting an optimal time frame for future ad placements. |
Mapping out the scene. Now that you’ve got a lay of the land, it’s time to start mapping out where Dollar Waldo isn’t. In data terms, descriptive analysis helps chart out the ‘knowns,’ providing statistics and visualizations that summarize and quantify your initial findings.
| Step 2 | Descriptive Data Analysis |
|---|---|
| Question | What happened and How it happened? |
| Mission | Understand and describe the dataset features: quantify and summarize the discovered patterns. Making no conclusions about what the data might imply beyond the sample. |
| Answer | Pattern “A” occurred. |
| Methodology | Use descriptive statistics like the central tendency (mean, median, mode) and dispersion (range, variance, standard deviation). You can extend the exercise to include classifying, clustering, and reducing dataset features for the Machine Learning (ML) models you plan to build later. |
| Timing | Post-mortem |
| Mining | Historical data |
| Example | Let’s describe the marketing campaign’s data. We observe that the overall click-through rate averages around 2.5% across all digital platforms, with social media accounting for 60% of the total clicks. Age-wise, the majority of our audience (about 70%) falls in the 18-35 age bracket, showing a younger demographic preference. Additionally, the data indicates that the peak time for ad engagement is consistently between 7 PM and 9 PM, suggesting this as the most effective time slot for ad placements. |
Solving the puzzle. You’ve spotted a few individuals who look like Dollar Waldo – but are they? This stage is about verifying your hunches, testing hypotheses, and understanding why certain patterns exist in your data, much like narrowing down potential Waldos based on distinctive traits.
| Step 3 | Explanatory Data Analysis |
|---|---|
| Question | Why it happened? |
| Mission | Identify and explain the factors driving the dataset features: hypothesize and prove the causes of the described patterns and the implications of the previous successes and failures. |
| Answer | Pattern “A” occurred because Pattern “B” occurred. |
| Methodology | Use inferential statistics to test hypotheses or theories and uncover relationships (correlation, causes, consequences) and patterns (trends) between variables. Using more complex modelling methods to validate or refute these hypotheses (regression, classification). Conduct interviews, focus groups and use in-depth observation to work on a more qualitative approach. |
| Timing | Post-mortem |
| Mining | Historical performance |
| Example | Let’s explain the effectiveness of a marketing campaign. We suggest that the higher engagement on social media ads, particularly among younger users, is likely due to the visually appealing content and the platform’s alignment with their daily social browsing habits. The increased interaction with ads during late evenings can be explained by user behavior, as people are more likely to browse social media and shop online after work or school hours. Furthermore, the data indicates that the format and messaging of the ads resonate more effectively with this demographic, explaining why mobile app environments yield better responses compared to other digital platforms. |
To learn more, check this article: How To Perform An Explanatory Data Analysis?
3. Pursuing the Sequence with Business Intelligence Analytics
Once the Data Analyst is able to communicate his insights, the Business Intelligence Analyst will leverage them to build Business Intelligence solutions like real-time monitoring dashboards and bechmarking reports. Let’s focus on these tasks below.
Keeping watch. Imagine having a magic magnifying glass that updates you whenever Dollar Waldo moves. In the data world, this is your monitoring dashboard – a dynamic tool that keeps you updated with real-time data, helping you track Dollar Waldo’s (i.e. your key metrics’) latest movements.
| Step 4 | Monitoring Dashboard |
|---|---|
| Question | Are our KPIs performing as expected? |
| Mission | Ensure KPIs and conditions are performing as expected by monitoring and reviewing them regularly. |
| Answer | Patterns “A” and “B” are occurring now. |
| Methodology | Use visual display of data to check performance against expected standards or objectives. Identify any deviations or issues, and potentially trigger alerts or actions if certain thresholds or conditions are met. |
| Timing | Current/ transactional data. |
| Mining | Key Performance Indicators |
| Example | Let’s build a monitoring dashboard to track the performance of our marketing campaign. We start with the Engagement Metrics Panel where we display real-time updates of click-through rates and conversion rates. It highlights top-performing platforms, with a special focus on social media channels and mobile app environments. Then we build a Demographic Insights Section that continuously tracks the age and device usage of the audience interacting with the campaign. It provides a breakdown of engagement by age groups, with a specific emphasis on the 18-35 demographic. Finally we add the Time Analysis Graph that offers an interactive timeline showing peak engagement hours. It allows us to dynamically observe and adjust to the most effective times for ad placements, particularly focusing on the 7 PM to 9 PM window. |
4. Finalising the Sequence with Data Science
For the last step of the lifecyle, the Data Scientist and AI Engineers will try to forecast future events and predict a desired outcome. The cherry on top of the cake will be the contribution of the Business Analyst in order to prescribe the best course of action for the organisation to take in order to achieve their desired outcome. Let’s learn more about it below.
Forecasting Dollar Waldo’s next hideout. You’ve been keeping track of the patterns of Dollar Waldo’s appearances and you’ve noticed that Waldo tends to appear more frequently near peculiar landmarks or crowded events. Based on his past hideouts, can you guess where he might be next? Forecasting a future event based on historical trends or patterns is a crucial step for strategic planning and staying one step ahead.
| Step 5 | Forecasting Variables |
|---|---|
| Question | How this variable might evolve in the future? |
| Mission | Forecast future events based on trends estimated from the past and present data of a single variable. |
| Answer | Because pattern “A” occurred, we predict that pattern “A” will reoccur in the future. |
| Methodology | Take a variable and project its future value looking at its data trends. Account for unique trends, seasonal patterns, and cyclic behavior. Build a Machine Learning (ML) model that makes time series and sequential predictions (ARIMA, RNN, Markov). |
| Timing | Post-mortem |
| Mining | Historical time series data, where the temporal dimension is crucial. No separate input or output variable. |
| Example | Based on the insights from our exploratory analysis, let’s forecast the future performance of our marketing campaigns. We predict a continued increase in engagement rates on social media platforms, especially among users aged 18-35, suggesting a potential 15% rise in click-through rates over the next quarter. We also anticipates a gradual shift in the audience demographic, with an increasing proportion of the 35-45 age group engaging with mobile app ads, likely leading to a 10% increase in interactions from this demographic. Finally, we foresee the late evening engagement peak (7 PM to 9 PM) to remain consistent, recommending this time slot for future high-impact ad placements, with a potential increase in user interaction by 20% during these hours. |
Identifying Dollar Waldo in a crowded scene. You use a model that incorporates historical characteristics (Waldo’s distinctive clothing and typical location tendencies) to analyze existing characters in the current crowded scene and predict the likelihood of each being Waldo based on these features. In predictive modeling, you’re analyzing current and historical data to predict an outcome, which could be classifying a current condition or predicting a future outcome.
| Step 6 | Predictive Modelling |
|---|---|
| Question | What might happen in the future? |
| Mission | Predict the likelihood of an outcome. Not necessarily focused solely on forecasting future events. |
| Answer | Because patterns “A” occurred and “C” is occurring, we predict that pattern “D” will occur in the future. |
| Methodology | Indicate a desired outcome regardless of the temporal sequence; you can predict current, future, or past (in case of missing data) events. Work backwards to identify traits in data that have previously indicated the outcome was ready to be reached. Identify the risks and opportunities to train the model with. Perform simple Hypothesis Testing or build a Machine Learning (ML) model that makes predictions (leverage statistics, probabilities, regression, decision trees, random forests, support vector machines, neural networks, etc.). Run the data into the model and perform rigorous randomized testing to find the purest predictions. |
| Timing | Post-mortem training and live deployment. |
| Mining | Use several input variables (not just one) to arrive at a single output variable. Building the model on historical data. Training the model on current/ transactional data. |
| Example | Based on our forecast, let’s predict the future performance of our marketing campaigns. Our Sales Projection Model estimates a 25% increase in overall sales over the next quarter, particularly influenced by the heightened engagement of the 18-35 age group on social media platforms during the late evening hours. Then, based on user interaction trends, our model also predicts a significant rise in sales within specific product categories popular among mobile app users, suggesting a potential 30% increase in these categories. |
Strategizing the search. Finally, with all the knowledge and tools at your disposal, you formulate a strategy to find Dollar Waldo. Prescriptive analysis takes all your insights and recommends the best course of action to not only find Dollar Waldo but also to make the most of the data you’ve gathered.
| Step 7 | Prescriptive Analysis |
|---|---|
| Question | What should we do to influence or benefit from future outcomes? |
| Mission | Prescribe the best course of action to achieve desired outcomes or mitigate risks. Provide strategies consisting of actionable recommendations. |
| Answer | To achieve goal “X”, we must take action “Y”. |
| Methodology | Combine the predictive models with rules, algorithms, and business intelligence extracted from the decision-making processes (supply chain optimization, resource allocation, operational efficiency improvement) to suggest specific actions. It often involves simulation and optimization techniques to explore different scenarios and outcomes. |
| Timing | Forward-looking and action-oriented. Timely implementation of suggested actions is crucial for the effectiveness of prescriptive analysis. |
| Mining | Various factors that could influence the outcomes of different recommended actions. |
| Example | Let’s prescribe an approach to boost by 30% the overall sales over the next quarter. Our model prescribes 3 main actions. First, to launch targeted social media ad campaigns at 7:45 PM on weekdays, focusing on visually engaging content that resonates with the 18-35 age group’s interests until increasing the click-through rate by at least 35%. Second, to reallocate an additional 20% of the marketing budget within the mobile app environment to the high-demand eco-friendly product category. Third and last, our model suggests implementing a loyalty program with tailored rewards and exclusive early-access deals to the same 18-35 age group to enhance their purchase frequency by at least 20% and their average transaction value by at least 15%. |
Explore more
To learn more about the working methodologies from the different stakeholders contributing to Data Analytics, check this article: Guess Who? Data Science Edition. And to deep dive even more into thow to brief, ticket and action each of these analytics investigations, check this one: Why Should we Ticket our Jobs?