Skip to content

Tag: metrics

Platform metrics: Some ideas

I was chatting with Lauri [1] about platform research. I claimed that the research has not that many implications for real-world companies apart from the basic constructs of network effects, two-sidedness, tipping, marquee users, strategies such as envelopment, and of course many challenges, including chicken-and-egg problems, monetization dilemma, and remora’s curse (see my dissertation on startup dilemmas for more insights on those…).

But then I got to think that metrics are kind of overlooked. Most of the platform literature comes from economics and is very theoretical and math-oriented. Yet, it’s somehow apart from practical Web analytics and digital metrics. Those kind of metrics, however, are very important for platform entrepreneurs and startup founders.

On the face of it, it seems that the only difference that platforms have compared to “normal” businesses is their two-sidedness. If we have supply side (Side A) and demand side (Side B), then the metrics could be just identical for each and the main thing is to keep track of the metrics for both sides.

However, there are some dynamics at play. The company has one goal, one budget, one strategy, at least typically. That means those metrics, even though can be computed separately, are interconnected.

Here are some examples of platform metrics:

  • Number of Users/Customers (Side A, Side B)
  • Revenue (Side A, Side B)
  • Growth of Revenue (Side A, Side B)
  • Cross-correlation of Number of Users and Revenue (e.g., Side A users => Side B revenue)
  • Cost per User/Customer Acquisition (Side A, Side B)
  • Support cost (Side A, Side B)
  • Average User/Customer Lifetime (Side A, Side B)
  • Average Transaction Value (Side A, Side B)
  • Engagement Volume (Side A, Side B)
  • Profitability distribution (Side A, Side B)

Note the cross-correlation example. Basically, all the metrics can be cross-correlated to analyze how different metrics of each side affect each other. Moreover, this can be done in different time periods to increase robustness of the findings. Such correlations can reveal important information about the dynamics of network effects and tell, for example, whether to focus on adding Side A or Side B at a given point in time. A typical example is solving the cold start problem by hitting critical mass, i.e., the minimum number of users required for network effects to take place (essentially, the number of users needed for the platform to be useful). Before this point is reached, all other metrics can look grim; however, after reaching that point, the line charts of other metrics should turn from flat line to linear or exponential growth, and the platform should ideally become self-sustainable.

Basic metrics can also be used to calculate profitability, e.g.

Average Transaction Value x Average Customer Lifetime > Cost per Customer Acquisition

Business models of many startup platforms are geared towards “nickel economics,” meaning that the average transaction values are very low. In such situations, the customer acquisition cost has to be low as well, or the frequency of transactions extremely high. When these rules are violated, the whole business model does not make sense. This is partly because of competitive nature of the market, requiring sizable budgets for actual user/customer acquisition. For platforms, the situation is even more serious than for other businesses because network effects require the existence of critical mass that costs money to achieve.

In real world, customer acquisition cost (CPA) cannot usually be ignored, apart from few outliers. The CPA structure might also differ between the platform sides, and it is not self-evident what type of customer acquisition strategies yield the lowest CPAs. In fact, it is an empirical question. A highly skilled sales force can bring in new suppliers at a lower CPA than a digital marketer that lacks the skills or organic demand. Then again, under lucrative conditions, the CPA of digital advertising can be minuscule compared to sales people due to its property of scaling.

However, as apparent from the previous list, relationship durations also matter. For example, many consumers can be fickle but supplier relationships can last for years. This means that suppliers can generate revenue over a longer period of time than consumers, possibly turning a higher acquisition cost into more a more profitable investment. Therefore, churn of each side should be considered. Moreover, there are costs associated with providing customer support for each side. As Lauri noted based on his first-hand experience working for a platform company, the frequency and cost per customer encounter differ vastly by side, and require different kind of expertise from the company.

In cases where the platform has an indirect business model, also called subvention because one side is subventing the cost of the other, the set of metrics should be different. For example, if only Side B (paid users) is paying the platform but is doing so because there is Side A (free users), Side B could be monitored with financial metrics and Side A with engagement metrics.

Finally, profitability distribution refers to uneven distribution of profitable players in each market side. This structure is important to be aware of. For example, in e-commerce it is typical that there are a few “killer products” that account for a relatively large share of sales value, but the majority of the total sales value is generated by hundreds or thousands of products with small individual sales. Understanding this dynamics of adding both killer products and average products (or complements, in using platform terms) is crucial for managing the platform growth.

Footnotes:

[1] Lauri Pitkänen, my best friend.

Basic formulas for digital media planning

Planning makes happy people.

Introduction

Media planning, or campaign planning in general, requires you to set goal metrics, so that you are able to communicate the expected results to a client. In digital marketing, these are metrics like clicks, impressions, costs, etc. The actual planning process usually involves using estimates — that is, sophisticated guesses of some sorts. These estimates may be based on your previous experience, planned goal targets (when for example given a specific business goal, like sales increase), or industry averages (if those are known).

Calculating online media plan metrics

By knowing or estimating some goal metrics, you are able to calculate others. But sometimes it’s hard to remember the formulas. This is a handy list to remind you of the key formulas.

  • ctr = clicks / imp
  • clicks = imp * ctr
  • imp = clicks / ctr
  • cpm = cost / (imp / 1000)
  • cost = cpm * (imp / 1000)
  • cpa = cpc / cvr
  • cpa = cost / conversions
  • cost = cpa * conversions
  • conversions = cost / cpa

In general, metrics relating to impressions are used as proxies for awareness and brand related goals. Metrics relating to clicks reflect engagement, while conversions indicate behavior. Oftentimes, I estimate CTR, CVR and CPC because 1) it’s good to set a starting goal for these metrics, and 2) they exhibit some regularity (e.g., ecommerce conversion rate tends to fall between 1-2%).

Conclusion

You don’t have to know everything to devise a sound digital media plan. A few goal metrics are enough to calculate all the necessary metrics. The more realistic your estimates are, the better. Worry not, accuracy will get better in time. In the beginning, it is best to start with moderate estimates you feel comfortable in achieving, or even outperforming. It’s always better to under-promise than under-perform. Finally, the achieved metric values differ by channel — sometimes a lot — so take that into consideration when crafting your media plan.

On digital marketing ROI

Introduction

There are many sub-types of ROI calculations in digital marketing. This post aims at making an argument that digital marketers should measure digital marketing returns as a sum of sub-returns from different channels/actions. Through that, they are able to capture the ROI impact on a wider scale than just looking at overall sales. Some metrics which inevitably have (some, albeit often hard to quantify) effects on dollar-returns, can only be accessed via a sub-type examination.

Profit, not revenue

Before going into the ROI types, I have to mention one important caveat in ROI  calculation — whenever possible, use profit as the upside, not revenue. This is simply because you want to measure the real profitability of your marketing efforts, which you cannot determine without including production costs into the equation. Don’t only measure marketing cost, measure the cost of being in business (because that’s what your bottom line consists of).

Digital marketing ROI

Figure 1  Digital marketing ROIs

So, here are different ROI sub-types in digital marketing:

  • dmROI = digital marketing ROI
  • oROI = organic digital marketing ROI
  • pROI = paid digital marketing ROI
  • osmROI = organic social media ROI
  • seoROI = search-engine optimization ROI
  • cmROI = content marketing ROI
  • psmROI = paid social media ROI
  • seaROI = search-engine advertising ROI
  • dROI = display ROI

And they can be divided like this:

dmROI = digital marketing ROI

consists of

oROI = organic digital marketing ROI

consists of

osmROI = organic social media ROI
seoROI = search-engine optimization ROI
cmROI = content marketing ROI

and

pROI = paid digital marketing ROI

consists of

psmROI = paid social media ROI
seaROI = search-engine advertising ROI
dROI = display ROI

Different returns

Now, the ROI equation has two sides: the cost and the return. As said, the return side measures the profit. But what happens when the profit is not directly computable? Such can be the case in deferred conversions, multi-channel effects and word-of-mouth, for example.

In this case we need to substitute profit with some other quantifiable measure. If one is not available, we have to calculate it.

The returns can be something like this:

  • Value of sales — this is simply euros
  • Value of customer lifetime — this is average order value times average frequency of repurchases during average customer lifetime (a lot of averages here…)
  • Value of impressions — for example, the increase of brand searches and their association with sales
  • Value of social shares — for example, the increase of organic reach leading to likes and associated returns
  • Value of likes — for example, the amount of sales from a social media channel divided by the number of followers in the channel in a given period
  • Value of email subscribers — the amount of sales from email channel divided by number of subscribers in a given period
  • Value of leads — the closing rate times average deal size gives the value of a lead
  • Value of organic traffic increase — the sales uplift from SEO activities vis-à-vis normal development of organic search traffic

We should aim at isolating the marketing effects to the best of our ability, i.e. determine what the baseline metric would have been without the marketing intervention and what it was; the difference between the two is our return. In a similar vein, we should seek to attribute not only direct but also indirect (assisting) interaction effects in the return side of a given marketing channel/effort. Not everything that should be observed can be observed (cf. Einstein), so we have to use arbitrary mechanisms such as attribution modeling.

Different costs

In turn, how should we define the costs?

  • In paid channels, they include media + labor costs
  • In organic channels, they include labor costs

There is a good rule of thumb: to achieve a certain reach, you need either high labor cost (and low media cost) or a high media cost (and low labor cost). Of course, the practical implementation decides the outcome, but this is the ceteris paribus scenario. The labor cost can be determined by internal accounting, e.g. activity-based costing (ABC). This cost calculation you can also use to determine “make or buy” decision – i.e., whether outsourcing digital marketing is feasible or not.

Conclusion

ROI is a fascinating question of which there is not certainty or absolute truth. Bringing in the sub-type examinations widens the scope of ROI and makes its constituency more accurate, yet leads into some sort of relativism, manifested e.g. in the choice of attribution models.

The correct way to calculate ROI for online marketing

Introduction

This is a short post explaining the correct way to calculate ROI for online marketing. I got the idea earlier today while renewing my Google AdWords certificate and seeing this question in the exam:

Now, here’s the trap – I’m arguing most advertisers would choose the option C, although the correct one is option A. Let me elaborate on this.

The problem?

As everybody knows, ROI is calculated with this formula:

ROI = (returns-cost)/cost*100%

The problem is that the cost side is oftentimes seen too narrowly when reporting the performance of online advertising.

ROI is the ‘return on investment’, but the investment should not only be seen to include advertising cost but the cost of the product as well.

Let me give you an example. Here’s the basic information we have of our campaign performance:

  • cost of campaign A: 100€
  • sales from campaign A: 500€

So, applying the formula the ROI is (500-100)/100*100% = 400%

However, in reality we should consider the margin since that’s highly relevant for the overall profitability of our online marketing. In other words, the cost includes the products sold. Considering that our margin would be 15% in this example, we would get

  • cost of products sold: 500€*(1-0.25) =425€

Reapplying the ROI calculation:

(500-(100+425)) / (100+425) * 100% = -4.7%

So, as we can see, the profitability went from +400% to -4.7%.

The implications

The main implication: always consider the margin in your ROI calculation, otherwise you’re not measuring true profitability.

The more accurate formula, therefore, is:

ROI = (returns-(cost of advertising + cost of products sold)) / (cost of advertising + cost of products sold)

Another implication is that since the ROI depends on margins, products with the same price have different CPA goals. This kind of adjustment is typically ignored in bid-setting, also by more advanced system such as AdWords Conversion Optimizer which assumes a uniform CPA goal.

Limitations

Obviously, while the abuse of the ‘basic ROI’ calculation ignores the product in the cost side, it also ignores customer lifetime value from the return-side of the equation.

Dr. Joni Salminen holds a PhD in marketing from the Turku School of Economics. His research interests relate to startups, platforms, and digital marketing.

Contact email: [email protected]

Chasing the “true” CPA in digital marketing (for Pro’s only!)

This is a follow-up post on my earlier post about “fake” conversions — the post is in Finnish but, briefly, it’s about the problem of irreversibility of conversions in the ad platforms’ reporting. In reality, some conversions are cancelled (e.g., product returns), but the current platforms don’t track that.

So, my point was to include a ‘churn coefficient’ which would correct for the CPA calculation. In other words, it adjusts the CPA reported by the ad platform (e.g., AdWords) in regards to churn from “conversion” to conversion (as per the previous explanation).

The churn coefficient can be calculated like this:

1/(1-churn),

in which churn is the churn from the reported conversion to the lasting, real conversion.

However, I got to think about this and concluded this — since we consider the churn taking place due to real world circumstances as a lift to the reported CPA, we should also consider the mitigating factor of customer-to-customer references (i.e., word-of-mouth).

Consider it like this – on average, converted customers recommend your company to their friends, out of which some convert. that effect would not be correctly attributed to the referring customers under normal circumstances, but by attributing it uniformly to the average CPAs we can at least consider it in aggregate.

So, hence the ‘wom coefficient’:

1-(Cn / Cm), in which

Cn: conversions from new customers non-affiliated with any marketing channel
Cm: conversions from all marketing channels

The idea is that the new visitors who convert can be attributed to wom while conversions from marketing channels create the base of customers who are producing the recommendations. Both pieces of information can be retrieved in GA (for Cn, use an advanced segment).

So, the more accurate formula for “true” CPA calculation would be:

1-(Cn / Cm) * 1/(1-churn) * CPA

In reality, you could of course track at least a part of the recommendations through referral codes (cf. Dropbox). In this case you could have a more accurate wom coefficient.

Limitations:

Consider that in period t, not all Cn are created by Cm. Hence, it would be more realistic to assume a delay, e.g. compare to period t-1 (reference effect does not show instantly).

The formula does not consider cases where the referred customers come through existing marketing channels (this effect could be eased by not including branded search campaigns in Cm which is a good idea anyway if you want to find out the true performance of the channel in new customer acquisition).

Finally, not all customers from non-marketing channels may not originate from wom (especially if the company is using a lot of non-traceable offline marketing). Thus, the wom efficient could have a parameter that would consider this effect.

Dr. Joni Salminen holds a PhD in marketing from the Turku School of Economics. His research interests relate to startups, platforms, and digital marketing.

Contact email: [email protected]