Last updated on November 18, 2024
Observed myself citing various “superpowers” to my PhD students in occasional emails.
So, thought of writing these down (the list might get updated).
Currently, I can think of eleven research superpowers [updated: November 11, 2024]. Here they are, in no particular order.
1. SPECIFICITY
Be specific. Applies pretty much to any form of communication:
- email messages to colleagues
- writing sentences in a research paper
- creating marketing copy
- giving instructions to others.
Being specific is one #researchsuperpower (but it applies well beyond research).
2. BOLDNESS
Be bold in your writing. I mean, literally, bold your contributions. Just to clarify: contribution is something general and helpful for other researchers. (It is NOT every specific finding – contribution is a broader thing, but well argued.)
Claiming contributions is one of the research superpowers!
3. WRITING LIKE YOU MEAN IT
So, your writing is pretty good and mature.
However, since you’re always interested in leveling up (as you should be!), here’s a tip: write like you mean it.
Look for every opportunity possible to claim a contribution, whether empirical, conceptual, theoretical, or methodological —- I’ll bold examples of this tactic in the edited version (we can keep these bolded parts; reviewers may complain about highlightings but in fact they are more likely to get the points with them! — because the reviewers are busy and you need to clearly communicate the contributions), and you can also check out my post about this: https://jonisalminen.com/about-forms-of-contribution-in-research/
The writing is still too bland — remember that there will be thousands of submissions to CHI. You have to be bold and ambitious — bland papers get lost in the haystack. So, level up your ambition in your writing: write it like you mean it, be very hungry to claim a contribution. Try to turn concepts, ideas, methods as your contributions that you argue will benefit others, too. This is much stronger than expecting that the reviewer will magically see the value in the paper – they won’t.
In the CHI review form, there are typically two sections: (1) is this study rigorous? (methodologically sound and present valid findings) and (2) does it contribute to HCI?
You need to tick both boxes —- write it so that the reviewer can remember: “oh yeah, they had 54 participants, real on-site study, very convincing and principled analysis” (=you need to use convincing arguments to make them think this); “oh yeah, the bolded parts detailed a hell of a lot of contributions and most of them made sense”.
Instead, the quick-reading reviewer will often think, “oh yeah, the method seemed standard. The findings were self-evident or boring. There was no clear contributions argued.”
Read this a couple of times. It’s important. I want you to internalize this idea and start writing in a hungrier and bolder way.
4. ABILITY TO *THINK*
Basically, you need to level up the thinking behind the paper. At this point of your journey, you’re expected to produce conceptually mature writing — this means that you’re no longer describing self-evident thinks but can move higher up in the abstraction. Practice that. Once you get the hang of it (and I’m not sure I’m explaining it well — what I mean by this is that you want to probe into things that matter for the research community. They need to be excited about it.), it’s unlocking a new superpower in research skills.
5. TURNING YOUR PROBLEMS INTO CONTRIBUTIONS
So, when we do empirical research, we face problems. We run algorithms, we do statistical analyses, we visualize the data, and so on.
We observe things that don’t make sense or we struggle to evaluate the results.
Sometimes we feel we get stuck because of these challenges.
What to do in those cases?
Write it up. Write your problems up. What is the problem you’re having?
By articulating the very practical problem at hand, you’re simultaneously creating a path for solution.
A well-defined problem is easier to solve.
So, you can start thinking about possible solutions – and write them down.
You’re writing to other experts who are likely to face the same challenges if they work on the same problem. So when you articulate the problem you’re facing and how you think about them, you are now producing text that is interesting to other experts in the field.
Compare this to a case where you try to “hide” the problem, or you’re just not making it explicit – it will be very hard to solve. And once you solve it, it might be hard to defend your solution, because you haven’t written up the journey of coming to the solution. So now the reader (reviewer) doesn’t know why you made the choices you did – why you picked certain variables, why you did certain preprocessing, why you chose a specific library, how you decided about threshold or hyperparameter values, and so on. Whereas, if you write all this down – for yourself! – then the logic is understandable to anyone.
Make your thinking transparent. Don’t hide it. Don’t expect people to understand automatically.
I used an analogy of writing a diary in an earlier call — if people want to know you, they could learn a lot about you by reading your diary. Think of your research paper as a diary – you want to expose your thinking to others, so they feel there’s human intelligence behind it; there’s an expert (you) that has given this problem a great deal of thinking.
Once they see that, they will relate to it, and they’re more appreciative, even when the results/methods have flaws.
I guarantee if you learn to write like this, it will be easier for both you to write the papers and for the people reading your papers.
Write. It. Up.
And yes — we all are struggling!
…I much less, after discovering this style 🙂 But I can vividly remember how it felt facing a problem before I had realized all of this. I went into a lock and couldn’t proceed.
It opened my eyes when I realized I can turn that problem into something worthy to share with other researchers, so instead of an obstacle it becomes a “finding” of sorts. (But it still needs to be tackled, of course!)
6. SEEING EVERYTHING AS EDITABLE (ACCEPT NOTHING WITHOUT EDITING IT!)
I realized I didn’t instruct you properly on how to “edit and comment” papers. I want to fix my mistake.
When we talk about “edit and comment”, we don’t mean proof-reading. Proof-reading is the least important part. I generally proof-read my own work and there are few mistakes remaining once the paper reaches you. So, you can’t add much value by proof-reading.
Jim has a principle: “edit the paper for grammar and content”. Let’s see what this means:
First, grammar means not only spelling errors but making the language overall more fluent. Can some sentences be said with fewer words while retaining the meaning? Are some sentences useless (contain no valuable information)? Are some words redundant? Can a whole paragraph be removed? Should a paragraph be placed in a different section? Should a sentence be in some other section? Is there a disconnection in the flow of the sentences (i.e., does each sentence logically follow from the previous one)? These are some questions for “grammar”.
Second, content. This is the more important one. Essentially, you should switch your mindset from a collaborator to critic while reading the paper. You should assume the role and perspective of a reviewer who wants to reject this paper. You try to find things that are unclear, details that are missing, information that is conflicting, results that are not convincing, arguments that are too general or not really backed by the study. You want to think, “Is this paper interesting? Is it clearly expressed? Is it actually worth publishing?” And all this reflection you will add as comments.
Notice, though, that there’s one important principle, too: whatever YOU can fix regarding these issues, you will. Because you still are a co-author. There are some co-authors that just mention problems, like “Research gap is not clearly defined.” My reaction to them is, “Fix it, then!”. Only the issues that you notice that you cannot fix should be shouldered back to the paper’s lead, e.g., when it requires reanalyzing the data. So, generally, you want to be very critical in finding flaws but also very helpful in fixing those flaws that you identify. This is what it means to be a good co-author.
I realize you’re still learning research and may not actually know what issues to pay attention to. So, I’m adding below a list of questions that you can use next time. You can think of these questions while editing and commenting the paper. Try to challenge the paper and make a real contribution in improving it. Next time, I really want to see all of you try.
- Does the work identify a plausible research gap?
- Are the RQs clearly understandable? Are they relevant? Can they be answered via a literature review?
- Does the introduction define the key concepts of the study? Are those definitions clear and appropriate?
- Does the title match the content? Does it contain all the key concepts?
- Is the method description clear enough to understand what choices the author made and why? (if a method section was included)
- Does the method section contain specific details like search terms used, number of articles found and screened, databases used, screening criteria and so on? (if a method section was included)
- Are the references from reputable sources? (Impact Factor above 1, JUFO level 2 or 3)
- Do each of the references clearly address one or more of the RQs?
- Are there enough references to build a solid understanding of the literature?
- In its analysis, does the work compare the findings of different studies? (A practical way to see this is to look if the paragraphs contain references to multiple studies with comparative arguments — in contrast, if a paragraph only references one work repeatedly, it’s not a good sign)
- Does the author exhibit original thinking? (subjective assessment, but you’ll notice if the work leaves you thinking, “wow, the author really presented a novel angle to this topic”)
- Are there tables and figures that summarize information? Are these the original work of the author?
- Does each result section end with a short synthesis of the findings?
- Does the discussion section briefly summarize the answers to the RQs?
- Does the discussion section list practical implications for relevant stakeholders?
- Does the discussion section mention key limitations?
- How well does the discussion section formulate future research directions, and do those directions form a logical continuum from the current work?
- Does the conclusion clearly summarize the findings?
ps. I’m also attaching an example; a US7 paper that Jim edited. Look at his edits: they are thorough and thoughtful — while I’m not expecting the same kind of depth from you (yet :), you can learn more and strive toward this example. It’s a superpower that will be useful for you 🙂
pps. anything in the paper is editable! Including the title (and often, the title CAN be improved)!
so, don’t be afraid to suggest changes.
Ilkka wanted to add one point: because you’re using track changes, you “can do whatever” (his words :). Meaning that the lead author can then use his or her own judgment to accept or reject the changes.
7. THINKING BACKWARDS FROM DONE
ALL: There’s no replacement for Jim’s drive to get these books DONE. He’s counting backwards from a deadline and keeps hammering reminders until it’s done 🙂
I don’t think I’ll ever learn that skill but it would be one superpower I wish I’d have.
8. NOT AIMING FOR PERFECTION
Another tip: don’t aim for perfection.
Try to say what you want to say, as quickly as possible.
Try to have the mindset, “I want this paper out of my hands as quickly as possible”.
…that’s because you’ll get it back with a lot of comments and edits from the co-authors (=when co-authors are Jim and me).
So, there will be a lot to fix. You don’t know what perfect is (nobody does), so don’t try to go for it. Just express yourself. When you feel like you’ve now said what you need to say, then you pass the paper forward.
So, focus more on freedom of expression than perfection. Freedom of expression doesn’t mean you shouldn’t be critical when writing — be “free and critical in turns” 🙂 Hard to describe but it’s like you need to have two personas taking turns. Persona 1 is positive and creative – she writes without fear. Persona 2 then looks at what Persona 1 wrote and tries to poke holes at it.
This is the best way I can explain it. In my own writing, I literally do that —- allow myself the freedom to think. But then immediately criticize that thinking. It’s odd but it works!
9. ONLY DOING *FOCUSED* READING
When you review articles for your PhD (or in fact for any purpose), try to *immediately make use of your observations* of said work in your own research—use it to contrast your findings, support your findings, and/or contextualize your research purpose.
You shouldn’t read papers with “oh, this was nice to know in general” attitude—-from each paper, you want specific nuggets of information that support specific claims you are making in specific papers.
(A good way to start this mindset is to consider your research plan as the “anchor point”, around which everything else is built.)
(ps. this method also helps avoid unnecessary reading, because you can skip whole articles or specific content if it’s not relevant to your research. IMMEDIATELY relevant, not “everything is related to everything” kind of way.)
10. WRITING UP AS YOU GO
Hi all,
many people do two mistakes:
(1) They read papers without immediately integrating them into the paper(s) they are working on.
(2) They do an analysis and think they will write it up later.
Both practices result in slow and inefficient work.
If you don’t immediately integrate what you’re writing, you’re “reading for fun”. This is fine if reading is your hobby. It’s not fine when reading is your profession. Write up your thoughts immediately. If you’re not getting any thoughts from what you’re reading, then you’re not reading deeply enough (again, you’re reading for fun, not seriously).
If you’ll not write down the analysis results immediately, then you’ll forget, first of all, and you won’t be making a follow-up analysis as easily.
The best method for me has been this: I write the analysis up immediately in a *self-contained way* (self-contained == it includes a brief explanation of why this analysis is done, how it is done, then immediately it reports the main finding and follows up with elaboration in the form of textual clarification and pictures/tables where needed). Already while I’m writing this, I’m getting ideas for a follow-up analysis! So, after I complete my short write-up, I can immediately proceed to the follow-up analysis.
…naturally, if you *didn’t find* anything significant or interesting, you DO NOT write it up. There’s no point. You move onto the next analysis, until you find something significant or interesting to report. You’re on a “scavenger hunt” for a story to tell.
11. CLARITY
Hi all,
that clarity is a beautiful thing.
When you have it, it feels nice, manageable, purposeful.
Here are some “cues” to identifying if you have clarity—-if you can identify with these, you probably have clarity!
“Today, I’m doing this.”
“I’m working on this one thing.”
“I want to get this specific thing done.”
If you’re uncertain or doing things that really don’t take you toward a clear, tangible outcome (in research, this is the “paper”), you likely lack clarity.
For PhD students, my purpose is to get you into a state of clarity. Ideally, I’d like you to be at the clarity mode 90% of the time!
[some examples cut out]
So yeah, clarity is a beautiful thing. Making my job also easier because if I see you have clarity, I’m confident to let you run. And when I see you don’t have clarity, I know I have to jump in and help you find it.
Best,
Joni