Frontiers | Formalized Journal-Style Review Process: Improving the Quality of Students’ Work (2024)

Introduction

Over the last few years, I have experimented with a student feedback assessment method that follows a formalized scholarly journal-style review process. Although the peer review system has numerous drawbacks, it is still considered the gold standard (Bloom, 1999; Enslin and Hedge, 2018). Like us, students often research an area searching for the truth by writing term papers. They can be creative, show novelty, significance, and produce sound scholarship. However, students are also not experts as they are learning simultaneously and require assistance from experts. The instructor is the expert who takes on the role of being the reviewer of students’ papers. Our role is to assess the submission and provide constructive feedback to improve the quality of the students’ discoveries. The instructor is a reviewer helping students articulate their results and interpretations better. The instructor can help make the paper easier to understand and enhance the communication of the paper’s thesis. The objective is to make the student’s manuscript sound, impactful, rigorous, and communicative. For example, the peer review feedback I received on this perspective piece was very insightful and led to significant changes in macrostructure, improving the message I wanted to deliver. I am thankful for the long narrative form type of feedback. Students can incorporate the feedback and resubmit their work for re-evaluation, addressing our comments in a rebuttal similar to the journal-style review process. Hence, why not use the gold standard for student term papers?

The Formalized Journal-Style Review Process

Students submit their papers before the end of a semester’s deadline for review. The instructor acts as a reviewer who can reject the paper in the case of plagiarism detection or accept it subject to either minor, moderate, or significant changes. Minor revisions or “surface” changes include typographic and grammatical errors, formatting, and small clarifications. Moderate changes add to the minor revisions. Examples include more elaboration and clarifications in certain sections and citations to support statements. Major revisions are “macrostructure” changes requiring additional research, organizational changes, and other substantive revisions. Examples of significant issues are inaccuracies and unsupported conclusions, erroneous data analysis, significant flaws in syntax, and extensive rewritten sections to fortify the paper. These five decision outcomes are hierarchical, from accepted outright to minor to moderate to significant and to outright rejection.

The review is an open peer review by construction since the instructor is the expert reviewer and the student is the manuscript’s author, openly known to each other. This open peer review process aligns with recent suggestions to make the scholarly journal-style peer review process less secret and more open (Enslin and Hedge, 2018). Nevertheless, it could also be single-blinded (i.e., anonymous marking), whereby the instructor does not see the student names, removing any potential human biases. However, recent findings indicate that anonymous marking intervention has done little to reduce ethnic, gender, and socio-economic mean performance differences (Hinton and Higson, 2017; Pitt and Winstone, 2018).

The instructor prepares a report starting with a summary of the paper in a narrative form and then lists the recommended revisions with major issues first, followed by moderate and then minor comments. The student then makes the revisions and prepares a written rebuttal indicating how they dealt with the concerns and submits the revised paper and the rebuttal for review by the instructor. The rebuttal could be in the form of replies to each critique or a long-form narrative. Although a rebuttal is currently not requested, it is an essential document as it provides metacognitive benefits for reflecting and thinking about the work and research findings (McGuire, 2015).

The instructor can assign a tentative grade based on the original submission with or without informing the student. For example, if revisions are significant, the instructor can assign a tentative low grade depending on the extent of modifications requested. On the other hand, if they are considered minor, a letter grade such as B or higher may be appropriate. It is rare for the instructor to accept a paper without any comments for improvement. Similarly, academic papers are rarely accepted outright for publication in a peer-review journal.

Outright rejection without the possibility of re-submission, although it commonly happens with journals, never happens with student manuscripts under this journal-style review process. There are two cases to consider. First, never flatly reject a student’s submission, even if atrocious, because these are the students who will significantly improve their manuscript by incorporating the feedback. They cannot do any worse but only improve their manuscript and grade in the second round of revisions. Second, if the student plagiarised, they are brought to the office given a stern warning of the consequences, receive a zero grade, and get a second chance to resubmit a new non-plagiarized manuscript (Devlin, 2006). The idea is to correct behavior instead of punishment by reporting the incidence to the University (Kara and MacAlister, 2010). Thus, treating this issue in a formative manner may result in student remorse and behavior change. Unfortunately, plagiarism is also rising in academic papers in journals due to the publish or perish paradigm (Butler, 2010; Awasthi, 2019).

Not assigning any grade is an option if the rebuttal and revisions are to the instructor’s satisfaction. Then the students receive full marks. When we publish our research, we are not assigned a letter grade. The quality of our published papers arises indirectly, initially by the journal impact factor and later by citation counts, which are also subject to manipulation (West and Bergstrom, 2021). Unfortunately, students do not usually publish their work in journals. Still, why do we letter grade their papers when ours are not assigned a letter grade? A gradeless learning environment has pitfalls but also offers a positive learning environment (McMorran and Ragupathi, 2020) and is something we instructors should consider. See Figure 1 for a summary of the Journal-style review process of students’ work.

FIGURE 1

Discussion

What if feedback is provided only after grading the paper? Unfortunately, this type of feedback does not allow the students to improve the manuscript since they cannot incorporate this into their paper and resubmit. The feedback provided to students is usually to defend the grade they received. However, academic manuscripts have a preferential treatment than students’ manuscripts that do not have a journal-style review intervention, as discussed previously. Scholars can resubmit, sometimes multiple times, for publication by incorporating some recommendations and rebutting other suggestions.

Is the journal-style review process more time-consuming activity for the instructor? Anecdotal evidence indicates that this is not the case. Why? The instructor will have a copy of the student’s initial manuscript and the report. Hence, reviewing it a second or third time does not take long, mainly if a rebuttal indicates how students dealt, or not, with the reviewers’ recommendations. The time-consuming task of grading is spread across time, with more time spent assessing earlier in the semester and less later than the grading lumped together at the end of the semester.

Why not have students do the peer review instead of the instructor? First, some literature advocates for this approach, but training students to be successful peer reviewers is difficult (Min, 2005). Second, students may not be independent reviewers (Bejdová et al., 2014). Third, students are not experts. Fourth, the instructor also knows how to peer review, although there is some evidence that this may not be the case at the graduate level (Navalta and Lyons, 2010). Hence, the students will get better feedback from the instructor than from their classmates to incorporate into their paper.

What is the evidence on feedback? Providing feedback to students is commonly done at all levels of education and is a thoroughly examined research area (Orrell, 2006; Hattie and Timperley, 2007; Winstone et al., 2019). Literature shows written comments provided by teachers improve students’ work only if they incorporate the feedback into their work (Dohrer, 1991; Slade, 2017; Slade and Miller, 2017; Bowden, 2018). Slade (2017) found that improvement requires multiple submissions of mini post-lab assignments whereby students pay attention to feedback and review their work until they “Do it Right!”. Slade and Miller (2017) showed that requiring four drafts of an introduction to mimic journal-style chemistry writing resulted in significant improvements as students re-assess their work by responding to constructive feedback. Contrary to the claim that students do not read or use such comments, recent evidence indicates that many collect and use such feedback (Slade, 2017; Zimbardi et al., 2017). Hence, the proposed formalized journal-style review process has similarities to the existing literature and should improve the quality of the students’ papers.

Should the submission for review be mandatory or voluntary? Making early submission of a manuscript for review mandatory is vital because the weaker students will benefit the most but are the ones who are the least likely to submit if the process is voluntary (Dohrer, 1991). Moreover, to detect noticeable improvement in the quality, students need to submit multiple paper drafts for review (Dohrer, 1991; Slade and Miller, 2017).

Until now, students submitted their work for feedback voluntarily. The thinking was that students would take this great opportunity to get feedback to improve their work and get a better grade. Unfortunately, many of the undergraduate students submitted only a final draft. Why undergraduate students did not take this opportunity remains unknown. One potential reason is procrastination since it would require students to have the draft at least 2weeks before the deadline, the last day of classes. Student procrastination occurs because of task aversion, risk-taking, and time delay of future incentives (Zhang et al., 2019; Svartdal et al., 2020).

The subsequent two case studies illustrate differences between undergraduate and graduate students opting in to get feedback when it is voluntary. In a recent third-year undergraduate course entitled: “The Economics of Climate Change” during Winter 2021, out of the 40 students, only six students (i.e., 15%) submitted their papers for feedback, and a couple submitted multiple times until they felt they “did it right!”. They wrote a commentary piece like those published in a scholarly journal with a word limit of 600 words, one table or figure, and 3–5 scholarly references. Students followed the style of an article published in Nature. The six students that submitted their work for review and revised their paper received an A grade worth 20% of their overall grade. Some of the commentaries were even publishable. Some students found it challenging to keep the manuscript to 600 words but found it a great skill to acquire as every word counts. To give one example of many, a student submitted for review a short commentary entitled “What has happened to CO₂ emissions after Kyoto Protocol relative to before for China?”. The student used data from the World Bank and the scientific methodology of Waggoner and Ausubel (2002) and reported the results in a commentary style.

In a graduate course entitled: “Environmental and Natural Resource Economics,” all 14 students submitted their work for review multiple times. The students wrote a research paper on the value of ecosystem services provided by local urban parks using the holistic approach by Sutton and Anderson (2016). After a few revisions during the semester, students presented their work at an end-of-term symposium. An invitation was sent out to the whole school to attend. Furthermore, one of the papers is under review with a journal. Not all graduate students ended up with an A grade for their work, but every paper improved from the initial draft submission.

Author Contributions

PT conceptualized the opinion piece.

Conflict of Interest

The author declares that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Publisher’s Note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

Acknowledgments

The author would like to thank Lian Dumouchel, Laura Lamb, Mike Henry, Bruce Martin from Thompson Rivers University and Jaime Teixeira da Silva an independent scientist from Japan for the numerous discussions and valuable feedback about the proposed journal-style review process for students’ term papers.

References

McGuire, S. Y. (2015). Teach Students How to Learn: Strategies You Can Incorporate into Any Course to Improve Student Metacognition, Study Skills, and Motivation. Sterling, VA: Stylus Publishing, LLC.