Students ask: What does it mean to keep good records? How can I maintain traceability between my writing and my notes?
Research hygiene is about keeping all the materials of your research in order, so you have space to work (mentally and practically) and you can find things when you need them – even years later. As part of research hygiene, the audit trail is about keeping a record of what you did, and why. Ideally, it also includes a record of what you decided not to do, and why.
We’ve all had the experience of looking back through our notes and reacting: “What was I thinking!” Frustrating as it can be, sometimes the process of trying to make sense of the notes provides new insight. Sometimes re-acquainting ourselves with our earlier thinking reminds us about something that got lost in the process, or an insight that was overlooked. For example, one student was struggling with the discussion chapter of her dissertation. So she went back through her notes, which included a repository of draft and discarded material, and found something she had written for her supervisors three years earlier. That ‘naïve’ document turned out to have captured key elements of her thinking, anticipated key findings in her research, and provided a narrative for her discussion chapter. (Dan Berry has an interesting line on the importance of ‘ignorance’ in research (See “The Importance of Ignorance in Requirements Engineering” by Daniel Berry (JSS 28:2, pp, 179–184 (1995)).
Research hygiene is a bit like dental hygiene: done well, it becomes a regular habit that doesn’t take much time or attention; but, if neglected, problems can accumulate and escalate into a crisis. Cleaning one’s teeth daily is easier, cheaper, and less stressful than coping with oral infections and root canal work.
There are lots of practices that help us keep track of what we’ve done/read/learned in the course of our research, for example:
- Annotated bibliographies (including importantly when we read something and what we made of it) – along with the annotated publications [see ‘Keeping an annotated bibliography’ in Resources].
- Study protocols (I often ask students to write up the methodology for a study as though for submission even before they’ve completed the data collection, just as a ‘forcing function’ to get them to record each step. Protocols need to include sample selection, data collection, data processing, analysis – the whole process, not just the beginning. And they’re often hard to re-construct six months later.
- Supervision meeting notes (what was discussed, what was decided, what actions were agreed…).
- Networking notes (whom you met, when, where, what you discussed, why they interest you…).
- Research journals or private blogs (where students write about what they’re thinking, or where they ‘file’ ideas they don’t have time to pursue; I often refer to ‘the shoebox under the bed’ where we can drop cool ideas for which we don’t have time – and which can fuel future work and grant proposals).
- Draft writing (with version control). If appropriate, you can leave comments in your writing that point to any notes that make a significant contribution (just as you’ll cite relevant literature – but for your consumption, rather than the reader’s), so that you can re-trace the important steps.
And, of course, research hygiene includes regular backups. I’ve met more than one student who had to write a dissertation from scratch after a laptop theft or computer failure.
Given the scope and complexity of the material you’re likely to amass over the course of doctoral study, you do need to take care about naming conventions and about how you structure your repository – so that, even if your priorities and thinking change, you can still find things. And you need to revisit your conventions periodically, which also provides an opportunity to remember what’s there.
Ultimately, the thesis evolves and matures – it is often a post-hoc rationalisation of what was done, with things like uninformative digressions edited out, or expressed in terms of a conceptual structure (rather than a historical structure). Which is why I break it down to those basic questions: what do I know now (that I didn’t know previously – or didn’t know in the same way); how do I know it; why does it matter.
So, with respect to the thesis (or to any publications along the way), think of the audit trail this way: Can the examiners ‘walk in your shoes’? Do they have enough information to repeat your studies and understand your argument – both structure and evidence? Do they have enough information to understand why you made those design choices rather than others? Is the empirical basis for any conclusions clear and explicit – and are the conclusions supported by the evidence? Is the relationship of your work to existing knowledge explicit? No examiner wants to repeat your whole PhD journey, but they do want insight into your thinking, so you need to give them a clear enough map to ‘walk with you’.