June 26, 2017
Introduction. In this post, I’m exploring the usefulness of personas in digital analytics. At Qatar Computing Research Institute (QCRI), we have developed a system for automatic persona generation (APG) – see the demo. Under the leadership of Professor Jim Jansen, we’re constantly working to position this system toward the intersection of customer profiles, personas, and analytics.
Three levels of data. Imagine three levels of data:
Which one is the best? The answer: it depends.
Case of advertising. For advertising, usually the more individual the data, the better. The worst case is the mass advertising, where there is one message for everyone: it fails to capture the variation of preferences and tastes of the underlying audience, and is therefore inefficient and expensive. Group-based targeting, i.e. market segmentation (“women 25-34”) performs better because it is aligning the product features with the audience features. Here, the communalities of the target group allow marketers to create more tailored and effective messages, which results in less wasted ad impressions.
Case of design and development. In a similar vein, design is moving towards experimentation. You have certain conventions, first of all, that are adopted industry-wide in the long run (e.g., Amazon adopts a practice and small e-commerce sites follow suit). Many prefer being followers, and it works for the most part. But multivariate testing etc. can reveal optimal designs better than “imagining a user” or simply following conventions. Of course, personas, just like any other immersive technique, can be used as a source of inspiration and ideas. But they are just one technique, not the technique.
For example, in the case of mobile startups I would recommend experimentation over personas. A classic example is Instagram that found from data that filters were a killer feature. For such applications, it makes sense to define an experimental feature set, and adjust if based on behavioral feedback from the users.
Unfortunately, startup founders often ignore systematic testing because they have a pre-defined idea of the user (à la persona) and are not ready to get their ideas challenged. The more work is done to satisfy the imaginary user, the more harder it becomes to make out-of-the-box design choices. Yet, those kind of changes are required to improve not by small margins but by orders of magnitude. Eric Ries call this ‘sunk code fallacy’.
In my opinion, two symptoms predating such a condition can be seen when the features are not
In contrast, iterative (=repetitive) analysis of performance of each feature is a modern way to design mobile apps and websites. Avoiding the two symptoms is required for systematic optimization. Moreover, testing the features does not need to take place in parallel, but it can be one by one as sequential testing. This can in fact be preferable to avoid ‘feature creep’ (clutter) that hinders the user experience. However, for sequential testing it is preferable to create a testing roadmap with a clear schedule – otherwise, it is too easy to forget about testing.
Strategic use cases show promise. So, what is left for personas? In the end, I would say strategic decision making is very promising. Tactical and operational tasks are often better achieved by using either completely individual or completely aggregated data. But individual data is practically useless at strategic decision making. Aggregated data is useful, e.g. sales by region or customer segment, and it is hard to see anything replace that. However, personas are in between the two – they can provide more understanding on the needs and wants of the market, and act as anchor points for decision making.
Strategic decision aid is also a lucrative space; companies care less about the cost, because the decisions they make are of great importance. To correctly steer the ship, executives need need accurate information about customer preferences and have clear anchor points to align their strategic decision with (see the HubSpot case study).
In addition, aggregated analytics systems have one key weakness. They cannot describe the users very well. Numbers do not include information such as psychographics or needs, because they need to be interpreted from the data. Customer profiles are a different thing — in CRM systems, enrichment might be available but again the number of individual profiles is prohibitive for efficient decision making.
Conclusion. The more we are moving towards real-time optimization, the less useful a priori conceptualizations like target groups and personas become for marketing and design. However, they are likely to remain useful for strategic decision making and as “aggregated people analytics” that combine the coverage of numbers and the details of customer profiles. The question is: can we build personas that include the information of customer profiles, while retaining the efficiency of using large numbers? At QCRI, we’re working everyday toward that goal.
June 10, 2017
Googlen algoritmin tarkka toiminta on liikesalaisuus. Siihen, mitä tiedetään, paras lähde on Googlen oma sivusto.
Tässä keskeisimpiä linkkejä:
Hakukoneoptimoinnista löytyy paljon spekulointia – parasta on nojautua Googlen viralliseen tietoon. Monet ns. tutkimukset, joita näkee alan blogeissa, eivät kestä akateemista tarkastelua, eikä niitä siksi voi käyttää lähteinä. Aina aika ajoin ne esimerkiksi julistavat, että “SEO on kuollut”, tai että “linkit eivät enää merkitse”, tms. juttuja, joilla ei ole todellisuuspohjaa. Virallisen sivun lisäksi Googlen blogi ja Matt Cutts ovat hyviä lähteitä – Matt ei enää töissä Googlella, mutta Twitter-historiasta löytyy paljon vastauksia (vastasi hakutulosten laadusta).
Google ei näytä kaikille käyttäjille samoja tuloksia: https://googleblog.blogspot.qa/2009/12/personalized-search-for-everyone.html
Toiseksi Google käyttää satoja tai tuhansia signaaleja, ja jokainen hakutulossivu on niiden päätösten uniikki tulos. Googlen algoritmi saattaa sisältää neuroverkkotoimintoja (syväoppimista), joiden tuloksena on hankala jäljittää tietyn hakutuloksen näyttämisen syytä. Näiden syiden vuoksi on parasta välttää yleistyksiä “parhaista käytännöistä”. Paras sääntö kokemuksen mukaan on: tee käyttäjäystävällisiä ratkaisuja, niin teet parasta hakukoneoptimointia.
May 28, 2017
I was reading through old emails while backing up my Gmail inbox with Gmvault, an open source command-line tool. Among other interesting trips to memory lane, one message was about the first startup user study I was involved with. It was for a life-coaching startup that eventually failed. In retrospect, it’s interesting to reflect on what went wrong and how we could have improved. In that spirit, I’m sharing these notes on the user study.
Okay, here’s my analysis based on user feedback.
We have three big issues to tackle at the moment:
Value refers to a) providing such benefits customers want to pay for and b) setting the right price point.
Mobilizing refers to practical issue of getting customers to use the service on daily basis – i.e. facilitating the data input process as much as possible.
Clarity refers to layout and functionality of the site.
Are users willing to pay for the service?
“Sivusto ei tuonut mitään uutta tässä vaiheessa muihin vastaavanlaisiin sivustoihin verrattuna, paitsi jos sivustonne pysyy ilmaisena.” (“The site doesn’t bring anything new compared to other similar sites, except if it stays free.”)
-> There’s a need to introduce features people are willing to pay for.
I’ll doubt they’d pay at the moment, although we should have asked that in the feedback form.
-> Creating a mobile version for paying customers (cf. Spotify)
“Ehdottomasti mobiiliversiota tarvitaan.” (“A mobile version is absolutely needed.”)
I agree that an optimized version for mobile phones is needed. Technically this would require mobile device detection and loading appropriate html+css files to fit the smaller resolution. Later on, if the service succeeds, an iPhone/Android app would be great 🙂
-> Other additional features for paying users such as meal (1) and training recommendations (2) (integrated in the app)
(1) “You could include an suggestion for the diet. You now have something like 50% proteins, 30% Carbo., 20% fat, but would be great to see already a diet suggestion based on that.
“It could include in the categories some traditional dishes like: Fish and potatos, Pasta and tomato, Kebak, etc…”
This was a constant concern by many respondents. As Valtteri suggested earlier, building meals is a great way of providing value. We could for example divide them to three categories based on nutritional values: snacks (välipalat), light meal (kevyt ateria) and heavy meal (raskas ateria). By combining these three, the service could propose a personalised diet based on consumption and need of calories calculated from person’s weight and other factors.
Also users could be given the possibility to add own meals and make them public for other users, as suggested in this comment:
“Yksittäisten ruokalajien sijaan voisi lisätä malliannoksia ja tarvittaessa muokata niitä samalla tavalla, miten annos pitää nyt koota täysin itse.”
(2) Another idea is creating a mentoring/peer supporting system that enables users to get support
b) Problem: “How much you’re thinking to charge the subscription?”
Free of charge -> revenue coming from advertisements/affiliate marketing
So, I think we’ll need to make a decision now of whether to offer the service for free and acquire sponsors, or make it paid and introduce such features that would increase likelihood of paying.
Also, we can think of providing a free version for free and premium with extra features for a small fee.
2. MOBILIZING USERS
Problem: Users are too lazy/busy to add daily information, ergo the app is not used actively.
“In general I think that it is really great, but the problem is to add the information on a daily basis…. :(“
“Jos palvelua alkaa käyttää niin on riskinä, että sitä vain kokeilee, eikä käytä jatkuvasti”
-> Creating a mobile version that allows on-the-go editing of profile
-> Creating a scheduled email system that allows easy updating (“Did you do the assigned training? Answer ‘yes’ to this message and information is updated automatically to your Muscler profile”)
-> Making it possible to update via sms? (requires an sms gateway + might be complex to use)
-> Creating community pressure for using the service (assigning personal mentors who have access to a person’s progress data — while conserving anonymity)
-> Offering rewards after completing milestones (e.g. reductions of sponsor products such as proteins)
3. CLARITY AND PURPOSE
“On vähän epäselvä. Ohjeet voisi olla vaikka tyyliä ‘ranskalaiset viivat’ settiä” (“A bit unclear. Bulleted instructions needed”)
“Parantakaa/tiivistäkää putkea, joka alkaa tarvoitteiden asetannasta ja etenee seurannan aloittamiseen ja raportteihin, tällä hetkellä minun tulee itse välillä muistaa edetä seuraavaan kohtaan ilman kehhoitusta/vinkkiä” (“Improve the process starting from setting goals to tracking and reports – atm, i have to remember to move myself”)
“Tutoriaali tai step-by-step-ohjeet olisivat hyvät :)” (“tutorial or step-by-step instructions would be great”)
-> Inserting tooltips
-> Creating “Proceed to [next]” buttons (Proceed to Tracking/Reports)
-> Creating a “How to use?” page
Besides the video, a separate “How to use?” page is needed. a bullet-type list would do ok, like suggested by the user. it could contain steps of using the service along with links, being short and simple.
“Mitä tarkoittaa tarkalleen Vahvistus tai ‘Lisää kestävyyttä’. Näistä olisi hyvä olla tietoa, sillä eri lajien ihmiset saattavat nähdä nuo eri tavoin.”
(“What does strenghten or more endurance mean?”)
-> Maybe these could be removed and focus on mass increase?
4. Bugs & Language
“Suosittelisitko palvelua ystävillesi?” kohdassa sanasta “Kyllä” puuttuu toinen l-kirjain (“Kylä”). :)”
-> Kylä should be Kyllä on feedback page
“Tavoite laatikon perässä saisi olla mitä yksikköä siinä halutaan”
-> Add units after goal form field to clarify what is asked.
“Kun lisäsin ruokia omalle “tililleni”, ei missään tuntunut olevan hiilihydraatteja. Se valikko, mistä ruuat valitaan, voisi olla selkeämpi (esim. isompi ja ADD-nappula aina samassa kohtaa). Haku-toiminto ruoka-aineille olisi myös kätevä.”
-> search function for nutrition, bigger add button with fixed position
“Itse tykkään etsiä aina lisätietoja varsinkin tällaisista urheiluun liittyvistä aiheista, joista löytyy aina monia mielipiteitä. Siksi olisin kaivannut esim. linkkejä lisätietoon “The Harris Benedict Equation” -menetelmästä”
-> Insert a link to “The Harris Benedict Equation”
“-Report is showing for today: 364% ACHIEVED (!!??)”
This is an issue I spotted myself, too. There must be something bizarre in the calculation formula, or can you give me example of a proper diet of 100% per day for let’s say a person weighing 60kg?
“- Many times a orange button option is dimmed, and little confusing (do I need to enter more infp? Does it work?)”
-> The button should be yellow in normal state and changed in mouseover, not other way around.
In reports/diet, “viikottain” should be “viikoittain”
Traning advice -> Training advice
Also, server location still shows Ukraine. There’s still some timelag which can be annoying.
There’s a logic problem -> when i insert a weightlifting training of 5 x 35kg in goals and then go into tracking, i shouldn’t reinsert the same stuff. i should be able to select the activity and then check that it’s done. if the weight was different, it should be changed in goals and not here.
May 21, 2017
Remora’s curse takes place when startup attaches itself to a large platform in the attempt to solve the chicken-and-egg problem of getting users. The large platform then exercises its greater power to void the investments made by the startup into the platform, essentially causing more or less deadly delays and needs for re-design. The idea originates from Don Dodge who wrote about the Remora Business Model.
The popular platform is not your friend. If their interests collide with yours, they will walk over you. Period.
Purposefully limit the role of platform to user acquisition as opposed to being core value prop. Platforms, seen this way, are just like other marketing channels – if they work, scale. if not, kill. The benefit of platform integration is that it may partially solve the cold-start problem: get faster traction and accelerate user growth.
Read more about Remora’s curse in my dissertation: Startup dilemmas – Strategic problems of early-stage platforms on the internet
May 17, 2017
One important thing in machine learning is feature engineering (selection & extraction). This means choosing the right variables that improve the model’s performance, while discarding those reducing it. The more impact your variables have on the performance metric, the better. Because the real world is complex, you may start with dozens or even hundreds of variables (=features), but in the end, you only want to keep the ones that improve the model’s performance.
While there are algorithms, such as information gain, to help, expert judgment can be of help as well. That’s because experts may have prior information on the important inputs. Therefore, one could interview industry insiders prior to creating a machine-learning model. Basically, the expert opinion narrows down the feature space. While this approach has risks, primarily foregoing hidden or non-obvious features, as well as potential expert biases, it also has obvious advantages in terms of distinguishing signal from noise.
So, the premise of narrowing down search space is the motivation for this article. I got to think, and do some rapid research, on what features matter for performance of Facebook advertising. These could be used as a basis for machine learning model e.g. to predict performance of a given ad.
A simple model could only account for C (=independent and dependent variables) and D (independent variables), while more complex models would run a more complex analysis of text and images using linear or non-linear optimization, such as neural networks (shallow or deep learning). Also, some of these features could be retrieved by using commercial or public APIs. For example,
Ideally, each advertiser has his own model, because they may not generalize well (e.g., different advertisers have different target groups). However, feature selection may benefit from learning from earlier experiences. Also, given that there is enough data, it may be possible that the model learns which features apply across different advertisers, achieving a greater degree of generalizability.
April 30, 2017
More than truthfulness of the numbers, investors evaluate the assumptions underneath a pitch. They are not asking “Are these numbers real?” but “Could they be real?”.
The assumptions reveal the logic of thinking by the founders. When examining them in detail, one should get logical answers to questions like:
(Assuming an enterprise sales case; the questions in a B2C market would be different, so you need to consider the circumstances.)
The investors are looking for “intellectual rigor” and “completeness of thought” from the founders. Therefore, the pitch needs to show that you understand how to run the business, and how those actions are linked with growth within a defined timeframe. Like one investor said, it is better to be roughly right than exactly wrong.
April 30, 2017
We had an interesting week with the Qatar Science and Technology Park (QSTP) that had invited several high-profile entrepreneurs from the US to evaluate the technologies of Qatar Computing Research Institute (QCRI).
Unfortunately, I wasn’t able to attend all the sessions, but from what I saw I picked up a few pointers for due diligence work done by investors when evaluating the startups. Here they are:
It’s worthwhile to mention that the formal due diligence process is something different from an informal one – the latter takes place when the investor does some initial inquiries about the team and the tech, and then decides whether he wants to pursue further discussions. After reaching an adequate level of confidence, formal and detailed due diligence procedures conducted with the help of experts (e.g., tech, legal, science) ensue.
April 16, 2017
However, after the meeting I realized this might not be enough, and in fact be naïve thinking. It may not matter that algorithms and social media platforms show people ‘this is false information’. People might choose to believe in the conspiracy theory anyway, for various reasons. In those cases, the problem is not the lack of information, it is something else.
And the real question is: Can technology fix that something else? Or at least be part of the solution?
Because, technically, the algorithm is simple:
=> results in a balanced and informed citizen!
But, as said, if the opposing content is against what you want to believe in, well, then the problem is not “seeing” enough that content.
These are tough questions and reside in the interface of sociology and algorithms. On one hand, some of the solutions may approach manipulation but, as propagandists could tell, manipulation has to be subtle to be effective.
The major risk is that people might rebel against a balanced worldview. It is good to remember that ‘what you need to see’ is not the same as ‘what you want to see’. There is little that algorithms can do if people want to live in a bubble.
Originally published at https://algoritmitutkimus.fi/2017/04/16/the-balanced-view-algorithm/