If you’re like most editors, you’re always looking for new ways to optimize your journal’s peer review process. Of course, in order to know why bottlenecks are occurring in your workflow and come up with solutions to stop them you have to figure out when and where they’re happening first.
Many journals have begun to focus on tracking journal metrics to get a granular view of their peer review processes - from simple stats like average annual submission rate, to the average number of days it takes individual editors to send authors manuscript decisions. Tracking metrics can help journals stay abreast of how they are performing externally in terms of volume, quality, and scope of submissions, and internally in terms of the speed of journal-wide and editor and reviewer specific performance.
“The big saying out there right now is data is king,” said Christine Dymek, senior managing editor at leading journal management consultancy Kaufman Wills Fusting & Company.
According to Dymek, who consults journal editors on peer review best practices, all journals should produce analytics reports to go over during regular team meetings.
“I think that it’s absolutely something journals need to do and audit annually, if not bi-annually,” said Dymek. “It’s a good way to check progress to see where you stand and to set goals.”
In Academic Journal Management Best Practices: Tales from the Trenches, a Scholastica eBook, Dymek overviews the core metrics she encourages journals to track. Below we roundup those main metrics in greater detail. For editors using journal management software with built-in analytics, you will likely be able to see all of this information from your account. If you are not using journal management software, Dymek said not to be overwhelmed by having to manually track multiple stats. She encourages journals to start out small choosing one or two metrics that matter most to them.
Time to Manuscript Decision
According to Christine Dymek, one metric that all journals should track and look to improve is the average amount of time it takes them to make decisions on submissions, from the time a manuscript is first received. It’s important for journals to ensure that their peer review process is moving forward and that they aren’t accruing backlogs of submissions that need to be assigned to editors or that editors need to assign to reviewers. Dymek encourages the journals she works with to agree on a benchmark for the number of days in which decisions should be made on new submissions, so that editors have a shared goal to work towards. Once that benchmark is in place, journals can determine if they’re on or missing their mark by tracking their average time to decision on a bi-annual or annual basis.
In addition to tracking journal-wide time to decision, Dymek said journals could also benefit from tracking how long it is taking their individual editors to move manuscripts through peer review, to determine if everyone is working at the same pace or if one editor is struggling to keep up with the manuscripts assigned to him.
“There’s a lot of benefit to that,” said Dymek. “You can take a look at who your high and low performing editors are and ask - why are the low performing editors moving at a slower rate?” Dymek said editor-specific time to decision metrics can quickly reveal gaps in editor support that need to be addressed, such as insufficient software training, or simply whether or not journals need to modify their manuscript assignment process to account for editors with too much on their plate.
Acceptance and Rejection Rate
Another core metric journals can track is the average number of submissions they accept and reject on a bi-annual or annual basis and where rejection decisions are being made in their peer review process. In her experience, Dymek said she’s seen that many journals can benefit from assessing their average number of desk rejections in particular and what those numbers say about their process.
“Desk rejects is a really important topic now,” she said. “The big talk in academic publishing has been the increased strain on reviewers. You want to make sure you’re only sending reviewers submissions that have potential.”
Dymek said, on one hand, journals should make sure they aren’t making too many desk rejections. If you notice a rise in desk rejections over time, you may want to meet with your editors to discuss the change and whether or not you’re putting enough manuscripts through peer review. However, on the other hand, if you find that a high number of manuscripts are being rejected during first-round peer review you may need to consider whether your editors are screening manuscripts well enough before sending them out to reviewers.
Your journal’s acceptance rate can also reveal a lot about your peer review process and the quality of your submissions. If you find that you have a very high acceptance rate or that your acceptance rate is growing, it’s a good idea to compare acceptance rate by submissions rate every six months or year to see if a decline in submissions has caused your editors to begin accepting more manuscripts than usual. If that’s the case, you should look to acquire more quality submissions to ensure that your journal remains selective.
Manuscripts Per Reviewer and Average Time to Review
One of the hardest parts of peer review for all journal editors is ensuring that they have enough peer reviewers to reach out to and that their reviewers are completing assignments in a timely manner.
“It’s very important to make sure that the reviewer pool you have is accurate and up-to-date, and that you only have people in it who actually want to review,” said Dymek.
To gauge the quality of their reviewer database, she encourages journals to track how often reviewers decline review requests.
“If you have someone that’s been in a reviewer database for three years and is yet to accept an assignment, you need to step back and ask if it is really necessary to have that person in the database,” she said.
Dymek said tracking the amount of time it takes each of your reviewers to complete manuscript assignments is another indicator of whether or not you are reaching out to the right people. You may find that your reviewers continue to accept assignments, but that their average time to decision is growing. If that’s the case, it may be a sign that it’s time to find new reviewers and to give your go-to referees a needed break.
To proactively ensure that your journal is not burning out reviewers, Dymek said it’s also important to keep track of how often you are assigning manuscripts to specific reviewers. She advises journals to continually seek new reviewers so that they can alternate the people they reach out to often.
