Are you interested in reading about research in the field of online teaching and learning? Could you use some help in reading and digesting the results of various research reports in the field? Would you like to be able to identify the strengths and weakness of the study reports that you read? If you answered “yes” to one or more of these questions then you might be interested in the Ecampus Research Unit’s new resource: the Report Reader Checklist.
The Report Reader Checklist includes a comprehensive set of criteria that offers you a guide to evaluate the quality and rigor of study reports. The checklist is intended to provide an overview of the foundational elements that should be included when reporting on the results of a study. You can apply each checklist criterion to a report to see whether that element has been included or not.
Here is an overview of the six areas of the checklist and the criterion in each area:
- Context: Does the report describe the larger purpose of the study? Does it explain the history or theoretical framework? Does the report include research goals and suggestions for further research?
- Methodology: Does the report have a methodology section? Is it clear how data were collected and analyzed? If the study used statistics, were they named? If coding was used, was the procedure described?
- Sample: Are the study participants described in detail? Is it clear how participants were recruited? Does the sample represent an appropriate level of diversity? Are subgroups appropriately identified?
- Reporting Results: Are all numbers in the report easy to comprehend? Is the “N” provided? Does the report identify missing data? Is it clear where study findings fit with the study’s purpose? Do data visualizations enhance your understanding of the results?
- Transparency: Are raw data included in the report? Are instruments or study protocols provided in the report? Are the authors clear about any conflicts of interest? Is the discussion rooted in data results?
- Reader Experience: Does the report use language that is easy to understand? Is the report ADA accessible? Does it include a summary or abstract? Is the study an appropriate length?
There are no “points” or “weighting” within the checklist, but if you find one area (e.g., “Context” or “Methodology”) that is missing several criteria within a report, that would indicate that a report is weaker in that particular area.
You can download a one-page PDF of the checklist or visit our supplementary website that provides more details on each of the criterion. Further, the site includes sample reports for each criterion so that you can learn more about areas that you are unfamiliar with.
We hope you find this resource useful for reading and evaluating reports in the field. We also hope it helps you make data-driven decisions for your work.
About the Oregon State University Ecampus Research Unit: The Oregon State University Ecampus Research Unit makes research actionable through the creation of evidence-based resources related to effective online teaching, learning and program administration. The OSU Ecampus Research Unit is part of Oregon State Ecampus, the university’s top-ranked online education provider. Learn more at ecampus.oregonstate.edu/research.
