It is common at British universities to ask undergraduate students to write a critical evaluation of a research report from a journal in your subject area. This type of assignment asks you to read critically.
If this is one of your assignments, then here's a list of questions you need to ask yourself while reading and to address in your written evaluation. Of course, these questions apply equally to your own work; in other words, use these to evaluate critically any write-up of your own primary research (e.g. your dissertation) before submitting it to your tutor.
- Is it peer-reviewed? In other words, are articles read/reviewed/commented on by academic peers and revised before accepted for publication?
- Is it clear and concise? Do you know what this report is going to be about?
- What do you know about the authors? Did they receive any funding for the project? If so, from whom/what organization? Is there any potential conflict of interest or bias?
- Does it summarize the main points of the study adequately and accurately?
a) The problem:
- Is the problem or purpose of the study clearly stated?
- Is the problem and/or key terminology associated with it defined?
b) The question:
- Is it/are they stated clearly and concisely?
- Does it/do they follow logically from the problem?
- Are they worth answering? Better yet, are they answerable?
c) The need:
- Is the significance of the research stated, i.e. does the author carve a niche for his/her own study? Is it making a contribution to our current knowledge?
d) The aims:
- Are they stated clearly and concisely? Are they logically related to the original question?
The literature review:
- Is the background information sufficient? (Does it leave you asking more questions?)
- Does the author appear to know his/her subject?
- Is the author critical (positive & negative) of related research? Or has he/she patched together quotes/paraphrases which support her/his own position without considering the counter-arguments?
- Are specific theories/frameworks used in order to put the study/findings into context?
- Does the theory seem relevant?
- Is it described in detail?
- Does it follow logically from the original problem?
- Is it narrative (words), numerical (numbers) or both (mixed-methods)?
- Was a pilot study completed, e.g. were the methods tried out first on someone/non-participant equivalents? Were changes made? What and why?
b) Ethical considerations
- Does the author adhere to specific ethical guidelines? If yes, which one/s?
- How does the author make his/her research ethical, e.g. was consent granted? Are pseudonyms used? Were participants informed that they could: ask questions; withdraw from the study; gain access to the findings, etc…?
c) Samples and participants
- Who was chosen? How were people selected?
- How many people were selected? Why this many?
- Was a specific size of sample chosen for a reason (e.g. statistical purposes)?
- Is the sample representative? Does it need to be, e.g. does the author intend to make generalisations?
d) Data collection
- What methods are used to collect data (e.g. questionnaires, interviews, focus groups)?
- Are the methods described adequately?
- Could you replicate this piece of research from the description?
- Are the questionnaires, interview schedules etc. included in the text or appendix?
- Is the method of analysis understandable?
- Are reasons given for the type of analysis? Do these adequately justify the chosen method?
- If numerical analysis, what kind of statistical tests were used? Does the author explain these well enough?
- If narrative analysis, was there evidence of reading/re-reading/coding/categorizing the data?
- Was it done by one researcher or more (e.g. inter-rater reliability)?
- Are results presented in a clear and coherent way so that you can interpret them and come to your own conclusions?
- Are raw data supplied or only proportions, percentages, etc. after manipulation? Are actual quotes given? Are numbers given before turned into proportions/percentages?
- Are charts, tables and other graphic representations labelled and explained?
- If results are based on responses to a questionnaire what is the response rate?
- Are statistical results included? Are they meaningful?
- Are the results interpreted in relation to the original question?
- Are the original questions answered?
- Does the author discuss any weaknesses in the methods and factors which may have affected validity (i.e. accuracy of assessment - the study measures what it sets out to measure) and reliability (consistency of assessment - the study can be replicated by someone else)?
- Do findings link back to/make comparisons with previous studies reported in the literature review?
- Are implications discussed?
- Do you reach the same conclusions based on your interpretation?
- Do they relate logically to the results?
- Are there recommendations for practice or for further research?
- Could you attempt to implement them (should you)?
- Is the length of the list more impressive than its quality?
- Some work is seminal and important to the field and may have older (old) publication dates. This is fine, but there should also be up-to-date sources. So, look for a mix.
- Are there any references not included which you think should be?