Last updated on May 5, 2020
This is an unpublished exploratory study we wrote with Professor Jim Jansen for Machine Learning and Data Analytics Symposium (MLDAS2018), held in Doha, Qatar.
Change of landscape: For a long time, automation has been invading the field of marketing. Examples of marketing automation include the various scripts, rules, and software solutions that optimize pay-per-click spending, machine learning techniques utilized in targeting of display advertising (Google, 2017), automated tools that generate ad copy variations, and Web analytics platforms that automatically monitor the health of marketing performance, alerting the end users automatically in case of anomalies.
In particular, several steps of progress towards automating analytics insights are currently being made in the industry and research fronts of data analytics.
For example, there are several tools providing automated reporting functions (e.g., Google Analytics, TenScores, Quill Engage, etc.). While some of these tools require pre-configuration such as creating report templates, it is becoming more common that the tool itself chooses the relevant insights it wants to portray, and then delivers these insights to the decision makers, typically pinging via email. An example of such an approach is provided in Figure 1 that shows Quill Engage, a tool that automatically creates fluent text reports from Google Analytics data.
Figure 1: Quill Engage. The Tool Automatically Generates Fluent Reports From Google Analytics Data, And Provides Numerical Comparisons Based on Outliers And Trends.
As can be seen from Figure 1, the automatic analytics tool quickly displays key information and then aims to provide context to explain the trends in the key performance indicators.
Benefits: The benefits of automatic analytics are obvious. First of all, automation spares decision makers’ time, as they are not forced to log into systems, but receive the insights conveniently to their email inboxes and can rapidly take action. Since cognitive limitations (Tversky & Kahneman, 1974) are imposing serious constraints for decision makers dealing with ever-increasing amounts of “big data,” the need for smart tools that pre-process and mine the data at the user’s convenience are highly beneficial.
The core issue that automatic analytics is solving is complexity.
As a marketing manager, one has many platforms to manage and many campaigns to run within each platform. Multiple data sources, platforms, and metrics quickly introduce a degree of complexity that hinders effective processing of information by human beings, constrained by limitations of cognitive capacity.
In general, there are two primary use cases for business analytics: (1) deep analyses that provide strategic insights, and (2) day-to-day analyses that provide operational or tactical support. While one periodically needs to perform deep analyses on strategic matters, such as updating online marketing strategy, creating a new website structure, etc., the daily decisions cannot afford a thorough use of tens of reports and hundreds of potential metrics. That is why many reports and metrics are not used by decision makers in the industry at all.
The solution to this condition has to be automation: the systems have to direct human users’ attention toward noteworthy things. This means detecting anomalies on marketing performance, predicting their impact and presenting them in actionable format to decision makers, preferably by pinging them via email or other channels, such as SMS. The systems could even directly create tasks and push them to project management applications like Trello. A requisite to automatic analytics should therefore be the well-known SMART formula, meaning that Specific, Measurable, Appropriate, Realistic and Timely goals (Shahin & Mahbod, 2007). Through this principle, decision makers are able to rapidly turn insights into action.
Interfaces for automatic analytics: To accomplish the goal of automatic analytics, one trending area of is natural language systems, where users find the information by asking the system questions in free format. For example, previously, Google Analytics had a feature called Intelligence Events, which detected anomalies. Currently, Google provides automatic insights via a mobile app, in which the user can ask the system in natural language to provide information. An example of this is provided in Figure 2.
Figure 2: Screenshot from Google Analytics Android App, Showing the Functionality of Asking Questions From the Analytics System.
However, even asking the system requires effort and prior knowledge. For example, what if the question is not relevant or misses an important trend in the data? For such cases, the system must anticipate, and in fact analyze the data beforehand. This form of “intelligent anticipation” is a central feature in automatic analytics systems.
Examples: In the following, we provide some examples of current state-of-the-art tools of automatic analytics. We then generalize some principles and guidelines based on an overview of these tools.
First, in Figure 3, we present a screenshot from email sent by TenScores, a tool that automatically scans Quality Scores for Google AdWords campaigns.
Figure 3: TenScores, the Automatic Quality Score Monitoring Tool.
In search-engine advertising, Quality Scores are important because they influence the click prices paid by the advertisers (Jansen & Schuster, 2011; Salminen, 2009). In this particular case, the tool informs when there is a change in the average Quality Score of the account.
From a user experience perspective, the threshold to alerting the user is set to very low change, resulting in many emails sent to the users. This highlights the risk of automation becoming “spammy,” leading into losing user interest. The correct threshold should be set experimentally, e.g., according to open rates by experimenting with different increments of messaging frequency and impact thresholds.
In Figure 4, we can see a popular Finnish online marketplace, Tori.fi. Tori sends automatic emails to its corporate clients, showing how their listings have performed compared to previous period, and enabling the corporate clients to take direct action from within the email.
From example, one can click the blue button and the particularly listing which is not performing well, is boosted. In addition, there is a separate section (not visible from the screenshot) showing the best performing listings.
Figure 4: Tori’s Marketplace Insights Automatically Delivered to Inbox.
Risks: There are also risks associated with automatic analytics. For example, In search-engine advertising, brands are bidding against one another (Jansen & Schuster, 2011). Thus, an obvious step to further optimize their revenue by providing transparent auction information is Google sending automatic emails when the relative position (i.e., competitiveness) of a brand decreases, prompting advertisers to take action.
This potential scenario also raises questions about morality and ethics of automated analytics, especially in click auctions where the platform owners have an incentive to recommend actions that inflate click prices (Salminen, 2009). For example, in another online advertising platform, Bing Ads, the “Opportunities” feature gives suggestions marketers can implement in a click of a button. However, many of these suggestions relate to increasing the bid prices (see Figure 5).
Figure 5: An example of Bing Ads Recommending to Increase Keyword Bid.
If the default recommendation is always to raise bids, the feature does not add value to end user but might in fact destroy it. From an end user point of view, therefore, managers are encouraged to take recommendations with a grain of salt in such cases. From a research point of view, it is an interesting question to find out how much the automatic recommendations drive user actions.
Discussion: The current situation is that marketing optimization consist of various micro-tasks that are inter-connected and require analytics skills and creativity to be solved in an optimal way. The role of automated analytics, at least with the current maturity of technology, is pre-filtering this space of potential tasks into a number that is manageable to human decision makers, and, potentially, assigning the tasks priority according to their predicted performance impact.
In this scenario, humans are still needed to make the final decisions. The human decides which suggestions or insights to act upon. Nevertheless, the prospect of automatic filtering and sorting is highly beneficial in maneuvering the fragmented channel and campaign landscape taking place in practical online marketing work.
Practical guidelines:
- As each vertical has its own KPIs, metrics and questions, there is a requirement of using many tools. For example, search-engine optimizers require drastically different information than display advertisers, and therefore it makes no sense to create a single solution. Instead, an organization should derive the tools from its business objectives and based on the specific information needed to achieve them.
- An example of fine-grained automatic analytics is TenScores that only specializes on monitoring one metric in one channel (Quality Score in Google AdWords). Their approach makes sense because Quality Score is such an important metric for keyword advertisers and its optimization involves a complexity, enabling TenScores to provide in-depth recommendations that are valuable to end users.
However, even though the tools may be channel-specific, their operating principles can be similar. For example, stream filtering and anomaly detection algorithms are generalizable to many types of data, and thus have wide applicability. Moreover,
- setting the frequency threshold to pinging decision makers is a key issue that should be experimented with when designing automatic analytics systems.
Even if there is automation, it is too early to speak of real artificial intelligence. The current systems always have manually set parameters and thresholds, and miss important things that are clear for individuals. For example, the previously shown Quill Engage cannot provide an explanation why the sales dropped when going from December to January — yet, this is apparent to any individual working in the gift business: Christmas season was the reason.
Implications for developers of automatic analytics systems: Developers of various analytics systems should no longer expect that their users log in to the system to browse reports. Instead, the critical information needs to be automatically mined and sent to decision makers in an actionable format (cf. SMART principle). There is already a considerable shift in the industry to this direction which will only be emphasized as customers realize the benefits of automatic analytics. Thus, we believe the future of analytics is more about detecting anomalies and opportunities, and giving decision makers easy choices to act upon. Of course, there are also new concerns in this environment, such as biased recommendations by online ad platforms – is the system advising you to increase bids because it maximizes your profit or because it increases the owner’s revenue?
Conclusion: Analytics software providers are planning to move toward the direction of providing automated insights, and researchers should follow suite. Open questions are many, especially relating to users’ interaction with automatic analytics insights: how responsive are users to the provided recommendation? What information do the users require? What actions do users take based on the information? We expect interesting studies in this field in the near future.
References
Google. (2017). Introducing Smart display campaigns. Retrieved February 12, 2018, from https://adwords.googleblog.com/2017/04/introducing-smart-display-campaigns.html
Jansen, B. J., & Schuster, S. (2011). Bidding on the buying funnel for sponsored search and keyword advertising. Journal of Electronic Commerce Research, 12(1), 1–18.
Lee, K.-C., Jalali, A., & Dasdan, A. (2013). Real Time Bid Optimization with Smooth Budget Delivery in Online Advertising. In Proceedings of the Seventh International Workshop on Data Mining for Online Advertising (p. 1:1–1:9). New York, NY, USA: ACM. https://doi.org/10.1145/2501040.2501979
Salminen, J. (2009). Power of Google: A study on online advertising exchange. Turku: Master’s thesis. Turku School of Economics.
Shahin, A., & Mahbod, M. A. (2007). Prioritization of key performance indicators: An integration of analytical hierarchy process and goal setting. International Journal of Productivity and Performance Management, 56(3), 226–240.
Tversky, A., & Kahneman, D. (1974). Judgment under uncertainty: Heuristics and biases. Science, 185(4157), 1124–1131.