Obstacle: How to evaluate my work’s originality?
“Most of the times when I believe I have found an original idea to develop, I find that it has already been developed. How can I evaluate if my research is novel enough or not? Are slight changes enough?”
How many ‘Eureka!’s do you need for a PhD.? None. Most research is not revolutionary, nor Nobel Prize winning. Most is ‘bread and butter’ research: plodding, incremental, ‘normal’ research. This is not to denigrate such work: it sustains and enriches the discourse.
What you need is insights or increments, which are much, much smaller than Eurekas. Insights can be things like: showing that a technique works in a new (relevant, possibly surprising) context – or showing that it doesn’t and why; drawing together two existing ideas and showing that the combination reveals something new and useful; providing evidence that received wisdom is incorrect; making concrete someone else’s idea, and hence showing how it works in practice and what its limitations are; conceiving and justifying a new explanation for a problematic phenomenon; showing that the application of a theory to a new situation (e.g., importing and applying theory from another domain, such as decision theory, economics, motivation) is revelatory, explanatory, or predictive; showing that a ‘tweak’ to an algorithm or technique is more effective; showing what’s missing or overlooked in existing work and beginning to fill the gap; correlating the results of a number of existing studies to show patterns, omissions, or biases; etc.
Students often worry about being ‘gazzumped’, about someone else publishing the same insights on the eve of their submission. This is a false concern: even if someone else is working on the same problem and has come to similar conclusions, there will be something distinctive about what you have done. Having said that, if the competing publication precedes your submission by months instead of days, then it is appropriate to revise your dissertation to accommodate the competing work – this is usually a minor issue, rather than a major one. Remember that well-founded corroboration is a contribution, too.
Tips
1. Understand the difference between asking: “How good is this?” and “Is this good enough?”
A Ph.D. is pass/fail. We may all want ‘grade As’, but the truth is that we all just need to pass. Aim for an ‘easy pass’ – for work that is clearly good enough. Remind yourself of the difference between ‘good enough’ and ‘as good as it could possibly be’ much less ‘perfect’.
2. Make (and implement) a publication plan.
A decent Ph.D. should yield a sound journal paper. A good Ph.D. yields more than one.
Securing yourself some refereed publications en route to the Ph.D. is a good way of reassuring yourself (and your supervisors and examiners) in advance that the work meets the criteria of originality, contribution, and publishability. Even if you don’t publish during your studies, it’s a good idea to have thought about how the work might be parcelled up for particular audiences, and which audiences those should be.
3. Analyse examples.
What contributions do ‘passed’ PhDs make? How are those contributions identified? How are they substantiated?
4. Make your contributions explicit.
There are some nice rhetorical devices you might consider for your dissertation, for example mirroring the ‘research objectives’ you list in your introduction with a list of how you met those objectives in your conclusion. Another example is making good use of the conclusion or summary sections of each chapter, ensuring that you state in each case not just what the chapter accomplished, but what the thinking contained in it contributed to the discourse.
5. Expose your work.
Giving external seminars, presenting conference papers, corresponding with researchers in the field, and having conversations with other researchers at meetings are all ways of exposing your research to external scrutiny, which gives you a chance both to identify weaknesses and to confirm insight.
Obstacle: Completeness of review
“I keep reading and reading; how do I know when I’m done?”
One of the things a good doctoral student will accomplish – one of the key aspects of education at this level – is to amass a first ‘core’ literature. Most good researchers carry around in their memories a good, accessible database of relevant papers. That means, out of all the reading they do, they maintain a working knowledge of about 100-150 papers. As they continue to read, the core adjusts, shifting to follow developments in the discipline or to follow their changing interests. But some of that core will persist for years. Interestingly, the lower limit for a good bibliography in a dissertation tends to be around 100 citations. Of course, there’s huge variation, but the numbers don’t really matter – the idea of keeping a selection of pertinent literature accessible in memory does.
One of the ways to gain perspective on ‘how much is enough’ is to understand that you’re trying to master a literature – to understand its scope, shape, and structure – rather than to beat it to death. ‘Enough’ then is enough to be confident of your ‘core’ literature, and hence to be able to cope readily with new material, including to recognise where it fits into to the literature you’ve mastered.
Tips
1. You’re never done reading. But there are signs that your coverage is probably sufficient for your PhD. For example, when you check the citations of the latest papers you’re reading, you’ve read most of them, and all of the important ones. For example, when people in your network send you pointers to reading, you’ve already read the papers.
2. Good ‘reading hygiene’ is good protection.
Keep visiting the library throughout your studies, and check out the contents of the incoming journals and books in your area.
3. Your best protection is your network.
People in your network should be sending you pointers to material they think will be relevant to you (as you should be doing for them). People in different environments, with different networks of their own, are likely to come across different material, which will help to keep your search broad.
Obstacle: How can I make sense of so much information?
Students begin their Ph.D. studies with high ambition and optimistic ignorance. Rarely do students come equipped with conceptual structures into which to organise the material they’re reading, and so they must induce structures as their reading progresses. Similarly, they typically have yet to form a basis for critical assessment of what they read, and so all the early material is of the same status. Students read frantically for months, before they begin to identify patterns of ideas and thinking, and before they begin to identify landmark papers of high quality against which to judge the rest. The time before the patterns emerge and the critical faculties are developed can be a time of overload and panic.
The big transition comes when students stop reading to survey, and start reading with a focus, for a purpose. Students’ use of the literature usually matures and focuses during the course of their research, in a way that corresponds to the development of their research question. The development goes through some phases, for example:
| Entering student | Completing student | ||
| knows what research area | knows what research topic | knows what research question | knows what research evidence |
| reads to find what’s there, what’s known | reads to find what isn’t there, what isn’t known | ||
| surveys, collects, reports | organises | selects (in terms of research question) | judges (critique and gap-finding) |
| here are the sources: how can I organise them? | here is the problem: what do people have to say about it? | here is the problem: what else should people have to say about it? |
So, the difference between a literature survey and a literature review is the difference between report and critique. Ideally, the completing student should have developed a ‘critical voice’. The literature review in the dissertation should ‘make sense’ of the literature in terms of the thesis. If the literature review is well-structured and appropriately critical, then, ultimately, the research question ‘emerges’ as an ‘inevitable’ conclusion of the literature review.
The problem of organising and making sense of a new literature is amplified for inter-disciplinary Ph.D.s, when there are multiple literatures to encompass, both in their own terms and in relation to each other. The problem is exacerbated by differences of disciplinary perspective.
Tips
1. Keep an annotated bibliography.
This is the single most powerful research tool you can give yourself. It should be a personal tool, including all the usual bibliographic information, and also the date when you read the paper, and notes on what you found interesting / seminal / infuriating / etc. about it.
A key thing to remember is that your perception of a given paper is likely to change over time, as your knowledge of the domain and your own research develop. So the annotated bibliography should include enough information to allow you to audit the progress of your own thinking about the paper, to understand the context in which you made the notes, and to understand why you valued it in the way you did at the time you made the notes.
2. Learn to mind map.
Mind mapping is a handy mechanism for drawing structure out of a collection of things. It’s easy to learn and to do. There are many other organising practices, too, like keeping index cards which can be sorted and re-sorted.
3. Stand on other people’s shoulders.
There’s nought as useful as a ‘good organic database’ – that is, an experienced, well-read researcher who has already made sense of the literature and can steer you to the key papers. People are incredibly good information filters. Other people’ literature reviews, review articles, and personal bibliographies can provide good insight into how others make sense of a literature and what structure they identify in it. If you start from someone else’s conceptual structure, be prepared to adapt it for your own purposes.
Obstacle: Deadlines (I’m always too late)
Writing is hard, and it takes a long time. Research doesn’t always go as planned. Life happens. There are always mitigating circumstances. People miss deadlines for a variety of reasons, such as unrealistic expectations, self-sabotage, lack of commitment. If you miss them regularly, then you need some life coaching from your supervisor or someone else, in order to diagnose and address the cause.
Tips
1. Always allow at least twice as much time as you think you might need. Marian’s algorithm for time estimation: think of how long you think the job will take; double the number, increase the units, and add one. So, if I first think a job will take 5 minutes, Marian’s algorithm tells me to allow 11 hours.
2. Reassess and re-adjust your plan regularly.
3. Grow up. Deadlines don’t go away. As your career progresses, there will be more deadlines, and more responsibilities competing for your time. Face your responsibilities and learn some discipline.
4. Carrot and stick. What motivates you? Set up a system of rewards (for meeting deadlines) and penalties (for missing them) – choose things that are significant to you. For some people, this means holding themselves accountable to someone they respect. For others, it means holding out a treat, like a weekend rock-climbing, subject to meeting a deadline. For others, it means holding out a threat, like having to sit in your supervisor’s office until you meet the deadline. For others, it means paying forfeits (i.e., public admissions of failure).
5. Figure out what you do while you’re not meeting that deadline. Learn to recognise the symptoms of self-sabotage and ‘wheel-spinning’ in order to combat them early.
Obstacle: Thought delay
“I never think of the right thing to say until after the supervision meeting.”
Lots of students experience this. Sometimes their supervisors are intellectually dazzling, sometimes, their supervisors just talk too fast, but usually it’s just that their supervisors are more experienced than they are. Supervisors are used to this sort of critical discussion, they’ve spent years learning which sorts of questions to ask, they’ve seen more research, and they’re familiar with the sorts of questions and guidance students need – and they’re used to being pressed for time. That tends to make them faster in a meeting. Some students find this intimidating. Smart students, however, assume that their supervisors are acting in good faith.
Tips
- Assume that your supervisor actually wants to know what you have to say.
Students keep forgetting this. It’s hard to supervise a brick wall. It’s far more interesting and stimulating to supervise a responsive student who thinks independently and communicates well. Don’t give up on your supervisor because the meetings move too fast. Work at communicating your ideas, even if it takes a few tries. If necessary, ask for a pause, so that you can collect your ideas – it’s OK to say ‘I need to slow down a bit’.
2. Follow up in writing.
If you get your ideas straight afterward, then write them down promptly, and send them to your supervisor, relating your ideas to what was said in the meeting. You can even explain that this is what you’d liked to have said during the meeting.