enter search term and/or author name
This guide is for Associate Editors of the Transactions on Computing Education of the ACM (TOCE). It explains about the editorial cycle, the AE’s role and responsibilities, finding reviewers, recommendation categories for manuscripts, the review criteria used for all manuscripts, expectations concerning manuscript recommendations by reviewers and the AE, and how revisions are handled. Each of these items is elaborated in its own section below.
ACM TOCE operates on a monthly review cycle. At the beginning of each month, the Editor-in-Chief (EiC) performs a preliminary review of all papers received by 8:00 am. Pacific Time on the first day of the month. The EiC's review of each manuscript ensures that (a) it is within the scope of the journal, and (b) it contains no major or obvious issues that would prevent it from being accepted. If it passes the EiC's preliminary screening, the EiC assigns the paper to an Associate Editor (AE).
The role of the Associate Editor in overview
When an Associate Editor (AE) is assigned a manuscript, the AE first takes a quick read of the manuscript, and finds at least three reviewers with relevant expertise. The target is to complete this assignment within one week of being assigned as the AE of the paper. As reviewers perform their reviews of the paper, the AE ensures that the reviews are received in a timely manner, sending reminders as needed. We specify 30 days as a default time for reviewers to complete their reviews, but indicate flexibility if more time is needed. In some cases, an uncooperative reviewer may need to be replaced by another reviewer.
Once the reviews are received, the AE rereads the paper and the reviews, and writes a thorough meta-review, along with a recommendation to the EiC for publication. Possible recommendations include Accept, Minor Revision, Major Revision, and Reject. The meta-review should be completed within one week of receipt of all of the reviews. If a substantial amount of time has passed and an AE has received two thoughtful reviews, it is acceptable to make a recommendation based on only those tworeviews, provided that the two reviews are in good agreement.
The EiC makes a final decision on the paper, based on the reviews and the meta-review, generally within one week of receipt of the AE’s recommendation. The EiC communicates this decision to the authors, along with the reviews and AE meta-review. If the authors decide to make revisions, they submit a revised manuscript along with a detailed cover letter that summarizes changes and responds to the points raised in the reviews and meta-review. The cycle continues, defaulting to the original AE and reviewers, until either the author does not return with a satisfactory revision or the paper is accepted. The reviewers and AE are anonymous to the author; only the EiC is identified by name.
Consider the following sources of reviewers for papers. First, rely on your professional networks, soliciting colleagues working in relevant areas. Second, request previous JERIC/TOCE authors of similar papers to undertake reviews. Third, look for authors of papers on the same topic (often referenced in paper that is being reviewed), as well as known experts in the field. It is good to get a mix of expert disciplinary researchers and educators in the area. For many papers, especially those with a strong research or evaluation component, consider obtaining at least one reviewer who is a Computing Education researcher familiar with evaluation and/or appropriate research methodology. The EiC can provide help with finding reviewers. The quality of the journal will largely be determined by the quality of the reviews; choosing reviewers carefully is thus an important part of the TOCE enterprise.
Here is how to interpret the different recommendations:
It is important that reviewers use the latest review form, last updated in January of 2016. As of July, 2016, this form is directly integrated into the Manuscript Central System. This means that reviewers will have to use the form to complete their reviews.
There are three primary reasons reviewers need to use our form. One is because the form tends to prompt more helpful and comprehensive reviews. Second, the review form is linked to the journal's review criteria, so it helps authors to see that they need to write their papers to the published criteria. Third, the form makes it is easier for the AE and the EiC to understand the reviewer's rationale.
To provide some history, here is what Josh Tenenberg and Robert McCartney wrote about the review criteria in their first editorial for JERIC as EiCs.
Our review criteria were developed from those in the Journal of Engineering Education, Computer Science Education, and the International Computing Education Research Conference. Common among these is that scholarship must address a question or issue of significance, that it is linked to previous relevant research, that the methods of investigation are appropriate to the problem studied, that the results are important, and that the reporting of the research is honest and careful. Additional and distinct scholarly characteristics of a journal on computing education are that the articles have a broad appeal to computing instructors and curriculum designers, provide sufficient details for these educational practitioners to replicate the approaches taken, and establish a clear connection to student learning that occurs in the use of whatever educational approach is described.
These criteria reflect a scholarly approach to teaching and learning within the disciplines. Adhering to this standard (or one like it) is one of the key things that distinguishes TOCE papers from those of other conferences and journals.
It is not necessary for every paper to satisfy all criteria. However, there should be a good reason if some criteria is weak or overlooked; there should be significant strengths elsewhere.
Although TOCE's review criteria are explicit, everyone will interpret them through their existing conceptual framework. Because we do not have the kind of common ground in our knowledge, experience, and practice in computing education as we do in disciplinary aspects of computing (such as theory, databases, and programming languages), all of us will need to make much more of our reasoning explicit than we might if serving similar roles for one of the mainstream 3 disciplinary journals. Reviewers thus need to make comments that are thorough (they adequately cover all aspects of the paper), specific (they provide concrete, actionable recommendations that authors can carry out and reviewers can check), and justified (there is a reasoned rationale). They also need to provide concrete suggestions for improvement. These reviews form the basis for the AE’s meta-review, and are also seen by the EiC and the authors.
You may sometimes receive reviews that are perfunctory and provide little basis for making a recommendation. In these cases, consider one or both of the following. First, ask the reviewers for a more thorough review, explaining to them the importance of making their reasoning explicit. If they have not used the review form, we ask that they do so, since it often allows them to see the range of criteria on which we would like them to comment.
If the reviewer is unwilling to do this, or if you do not believe that this will yield a better review, then it is important to solicit an additional review.
AEs provide a recommendation (into one of the four categories mentioned above) to the EiC along with a detailed rationale. This is in the form of a draft letter to the authors. The AEs do not sign this letter with their names, and do not send this letter to the author. Rather, the EIC makes the final decision as to whether to accept this recommendation. If so (the most common case), the EiC adds an introduction to the letter, communicates the decision to the author, and signs this letter as EiC. If not, the EiC and AE will discuss the paper, and hopefully reach an agreement concerning publication. As indicated above, it is important for the AE to provide a detailed rationale for their recommendation. In the case of major revisions, it is important for the AE to clearly state what changes need to be made before the paper can be considered for re-review. In the case of minor revisions, it is important for the AE to clearly state what changes need to be made in order for the paper to be published. Please use the existing templates of these recommendation letters provide by Manuscript Central.
The default assignment is for the original AE to continue as AE through the first and subsequent revisions. Please let us know immediately if you unwilling or unable to carry out one or more revision cycles.
We require that authors provide a cover letter with their revision that summarizes the changes that they made as well as provides a response to each of the actionable items by the AE and each reviewer. If they have not provided this, or if this letter is inadequate, please contact the reviewer and indicate that you will not re-review the paper until they provide this detailed cover.