The Process

Understanding the final goal of a Ph.D. is the first step towards understanding the process and, in my experience, it was important to scope the problem as a whole. To achieve this, there is one thing which every student should get clear in his or her mind: your Ph.D. does not have to be big.

This post from prof. Matt Might conveys the point. In visual terms, you need to push the boundaries of the red circle—in a very focused manner—until the boundary gives up and you expand the existing knowledge of just a little bit.

Setting the bar too high—i.e., thinking to make a big bump in the circle—can be detrimental for (at least) two reasons:

you will have problem understanding where your thesis is done

you will feel you not going to make it.

On the other hand, this is not an excuse for slacking off and work on a small (even worse irrelevant) problem. I believe that one skill to develop as a scientist is not only the capacity to identify a problem but also understanding when to stop working on it and communicating the results. Lionel also covered the issue of selecting a worthy problem for a Ph.D. which generated some interesting discussion, see his slides here.

The next logical step in the process is to understand where the boundary lies. My suggestion here is to invest time in studying the literature. My process is to dedicate a single day of the week to deliberately read the key papers in the field of your research. Those are not only the must-know-by-heart, high-cited papers but also the ones published in areas close to your sub-field which are recently published. For example, the area of research for my Ph.D. was unit-testing and test-driven development, but I would also read papers about Agile software development, in general, to be up-to-date with the larger scope of the topic.

To get a hold of the literature, my approach is to set up one or more Google Scholar Alerts to get a digest of newly published papers that are matching a Scholar search string (or a specific author). What I mean by deliberate reading is the process of going through the paper, annotate it and collate the notes about:

Motivation—Does the same motivation apply to my research? If yes, what is my specific perspective?

Relevant literature—Identify the key citations in the paper (usually the one that the authors build on, contrasts, or compares) and snowball them to identify other relevant papers.

Methodology—Can the methodology be applied to your problem? If that is the case, make sure you understand it and try to replicate it.

Limitations—Do the same limitations apply to your study? Can your study be motivated by addressing one (or more) of such limitations? If that’s the case be sure to understand them properly.

Discussion—A good paper is characterised by a thorough discussion section (which is also the most difficult to write). Pay extra attention when reading this section to learn how to write your discussion section (not regarding contents, of course, but regarding form).

While you are busy figuring out where the boundary is, you should also stock up the tools needed to push it. Usually, these are obtained by attending courses (for those who require clearing some credits as part of their doctoral training) which I divide into two categories:

Hands-on—these are the practical skills, specific to your niche of research, which you usually learn in engineering school or advanced courses.

Heads-on—these courses teach less concrete concepts (like philosophy of science or research methods) but are very beneficial for writing the dissertation. Having a good understanding of research methods (in general, not only the one used in your research) and their philosophical assumptions helps to justify your choices of methods, sampling, addressing threats to validity, etc. Remember that you are a scientist first and an engineer later.

Writing is a big part of being a scientist, so much that you should consider yourself as a writer (or so I have been told 🙂). One of skill a scientific writer should definitively possess is cogency. From the New Oxford American dictionary:

cogency | ˈkəʊdʒ(ə)nsi | noun [mass noun] the quality of being clear, logical, and convincing; lucidity.

One of the most powerful advice I came across is to start writing your papers early, earlier than you think. I got this idea from a presentation by Simon Peyton Jones of Microsoft Research Cambridge. I would start writing down a paper before doing the study itself. For example, I would start by writing the introduction, jot down the background part (which forces to understand the key references), the methodology and design I envision for the study, and the possible limitations. This approach drammatically helps to crystallize the idea of the paper.

Many times you want to cram as much stuff as possible in a single paper. In my experience, this is detrimental to you as a writer since you need to juggle different lines of thoughts (and eventually trying to connect them), and for the readers (e.g., your reviewers) who can then get lost in the manuscript without understanding what is the point you are trying to make.

Deciding on the main idea early on, helps you to more clearly communicate it, let alone giving you the possibility to focus and improve on it while doing the study. However, as I pointed out in my presentation, this is no excuse for least publishable unit—i.e., trying to publish thin slices of findings to increase one’s publication counts (sadly, a large part of career advancement in academia relies on bean counting).

In the quest for properly communicating the results of my work, there were two clear fallacies I fell for:

Believing that I could formally communicate the rationale for my proposed approach.

Believing that since I spent most of the time on a particular aspect of the work, it should be the focus of the paper.

I guess that the above originates from a naïve view of science in which everyone is supposed to understand everything given the least amount of words. According to the above definition of cogency, (scientific) writing can be recognized as an act of persuasion—an aspect easy to overlook. As Jones says in his presentation, you should spoon-feed the intuition of your ideas to the readers before formalizing it. Moreover, the best way to achieve the latter is through examples which gets the reader to understand the idea “as if you were jotting it on the whiteboard the first time it came to your mind.”

Regarding the second point, the temptation is to go over the whole journey that took you to write the paper thinking that the time invested in a specific activity (e.g., deciding on the experiment design) should be proportional to the number of pages dedicated to reporting it. Do not believe that the readers want to know every detail but instead choose the most direct path to the idea.

As a fresh Ph.D. student, I found myself naïvely believing that I (i.e., my work) was right while others were wrong. I soon discovered that scientific studies are like a blanket too short to cover all the parts of the bed and that there is no such a thing a the perfect scientific study, but rather a study which took a good trade-off. I, nowadays, put much effort in identify the shortcomings of my work and communicate them upfront to the readers. (It turns out that reviewers appreciate such effort and in several of instance they praised my “threat to validity” section.)

Alongside, I figured that giving praise to other researchers is important too. Here, I follow another advice from Jones and put an effort in acknowledging others work not only in the paper but also on a personal level. I would write to the authors of a paper asking for feedback on a paragraph where I cite them.

Related to this, I have also figured out the importance of reviews. Initially, you might be upset when getting a rejection and discard the comments as the work of an imbecile who didn’t understand your work. The important realization here is that if someone could not understand you, 80% of the time you are the culprit as you should have better explained your ideas in the first place (I leave the remaining 20% to an incompetent reviewer, not by his or her choice but due to how the peer-reviewing system currently works, at least in software engineering research). Note that your gratefulness for a good review (even if for a rejection) will increase with your understanding of how much effort goes into (properly) reviewing a paper.

It is finally important to understand that your writings (and presentation) will always have several audiences. These can be organized as in the picture below.

The first readers of your paper (after you and your inner circle of colleagues and hopefully supervisors) will be the reviewers. Mainly, they will be interested in the Form of the paper and check whether it fulfills the requirements for publications—is the paper coherent? is it properly written? etc. The reviewers will also check the Information contained in the paper. These are the bits that make your work worthwhile to be communicated to the scientific community (e.g., novelty, validated evidence). Do not forget that reviewers are researchers too, and one of the reasons why they agreed to review your paper (without getting paid for it) is because they found it interesting (in the best case scenario by reading title or abstract, but hopefully not at random) and want to get early access to the Content.

On the bottom part, your audience will focus on specific content. Other researchers are unlikely to read the whole paper but mostly will look for easy access to content interesting for their research. They will hunt for graphics, tables, algorithm and so on that quickly communicates Content. Similarly, practitioners are interested in another kind of Content, the one they could apply in their work (e.g., a new development process, or a piece of software). The important takeaway is that you should make clear where each of audiences can get what they’re looking for in the paper. This could be as easy as explicitly stating where the information and using an appropriate typeface.