As far as we know, there are no secret “official rules for successfully running scholarly journals” (though we’re sure many editors would love to find them!).
Aside from following available disciplinary norms, journal teams generally have to devise their own submission and peer review processes based on their unique needs. And a big problem for many is sticking with somewhat arbitrary workflows indefinitely, regardless of potential areas of inefficiency, because it’s all they know, which can stagnate publication development.
To avoid this situation, it’s vital for editors and publishers to periodically evaluate their journal performance with a critical eye and look for areas to improve. One of the best approaches is to perform operational journal audits to get a pulse on submission and peer review processes and ensure everything is flowing as efficiently as possible.
Kathey Alexander, a freelance consultant in professional and academic publishing who has worked in various editorial roles, regularly performs operational audits of scholarly society journal programs and helps editors optimize their workflows. In this blog post, we’re rounding up some of the primary areas Alexander says operational audits should focus on and steps journal teams can take to address commonly uncovered challenges.
Alexander suggests having an outsider run journal audits to ensure objectivity, but publishers and editorial teams can also perform audits in-house.
Start with submissions data
Of all the possible metrics to look at when performing an operational audit, Alexander says statistics on submission should be your main concern. She recommends beginning journal audits by checking to see if submissions are going up or down over time. Diminishing submissions can be indicative of a problem, which may be related to poor author experience. When assessing submissions volume, Alexander stresses that journals look at submissions by type, rather than just aggregate data, as some submissions matter more than others.
“It’s the old devil’s in the details,” she explained. “Once you get in and really start looking at all the pieces, that’s where you see if there’s a problem that hasn’t been explored.”
Alexander gave an example of a journal she worked with that showed increasing submissions in all of its reports, but when the team dug deeper realized they were actually losing submissions in key areas.
“The journal had run a five-year history of submissions by type of article and were looking at only the aggregate data. But, the most important type of article was certainly the research article and those submission numbers were going down. Other material that I would call fluff was going up. If you only looked at the total, there was no cause for concern, but if you looked at the detailed report you could see exactly where they were losing ground.”
If your journal is seeing a drop in submissions, you’ll want to look for problem areas in your peer review or publishing process that may be deterring authors from working with you. In her experience, Alexander said journals drawing out the peer review process can often be the culprit. Lengthy times to decision can occur for various reasons such as delays in manuscripts being assigned to editors or reviewers, journals overusing the same reviewers resulting in longer times to review, or journals building extra steps into the peer review process.
One mistake Alexander said she’s seen journals make is bringing in new reviewers after a manuscript has already gone through one or more rounds of revision.
“Some journals will throw in a third reviewer and suddenly the paper is going back to the author again with a different set of revisions being requested. That’s really annoying. If it goes two or three times around, that’s when the author says ‘no, I’m not going to do this anymore. I’m taking my paper elsewhere,’” she said.
Another issue that can deter authors from submitting to a journal is if it has delayed publication of accepted articles in either or both digital or print format. For journals that publish online and in print, Alexander said editors should not assume that getting out digital articles quickly nullifies the need for rapid print publication.
“I work primarily in medicine and most of the medical journals are still published in print as well as online. If a journal is publishing an issue a month that’s great. But, if I look at the date a published article was submitted and see that it was two years ago and then I go to the online journal and I find 500 articles in the publish-ahead-of-print queue I think that’s a problem.”
While some may say it doesn’t make a difference whether an article is in print as long as it’s online, Alexander said she disagrees. “I think a backlog is a backlog whether it’s online or in print, and I realize once you’re online you’re fully findable, you’re fully citable, and all those things, but particularly if you’re a society-owned journal where they are delivering members a print issue, if members are not getting the most up-to-date information that’s a problem.”
Query authors and reviewers to see what they think of working with your journal
In order to avoid peer review and publishing faux pas that could lead to alienating authors, Alexander recommends journals or their publishers query authors about their submission experience on a regular basis and use feedback collected in future audits. Queries should not only be sent to published authors but also authors who submitted desk reject articles and articles that were rejected after one or more rounds of revision. She also recommends that journals solicit feedback from referees to hear their thoughts on the peer reviewing experience.
“I think part of your responsibility is to make sure you know what your stakeholders think of the job you’re doing,” she said.
Once author and reviewer feedback starts coming in, Alexander said it’s paramount that journal teams take it seriously.
“I was doing a review of a big journal that had been consistently querying its authors, so as part of the review I read all of those surveys. I saw that year after year there were the same complaints about two particular items. No one ever did anything about it because the editors would say ‘that’s not true’ even though they were hearing it repeatedly,” she said.
To avoid situations like this, be sure to seriously consider all feedback your journal is receiving and to raise a red flag among your editorial board when the same concerns keep coming up over time.
Don’t get bogged down with bloated processes
In situations where journals aren’t making timely decisions or getting articles to publication fast enough, Alexander said identifying bloated editorial processes is the first step to solving the problem. She frequently finds that teams have processes in place that aren’t really necessary and can result in lost time.
“One of my prime examples is when I started working with one organization and discovered that the staff was spending an entire day sitting in a room, reading page proofs out loud to each other to make sure there was absolutely no error in any article. They felt that was crucial to delivering the highest quality content. I have to question something like that.”
For teams with extensive proof review processes in place, Alexander said it may be time to reconsider what they’re spending extra time on and to find ways to be more selective in choosing what really matters.
“We’re dealing today with a world in which the attention span of most of us is a nanosecond. If you want to pick something to emphasize re-checking, what is the one thing that everybody reads? It’s the abstract. If you want to put a lot of time and attention into something, I’d put it into the abstract and perhaps the conclusions. But to sit there and wordsmith materials and methods, no, I don’t think that’s time well-spent. It would be nice to think that way, but I don’t know if it’s very realistic.”
Make sure journal editors and the publisher are fulfilling their designated obligations
Sometimes issues with publishing that editorial teams are facing aren’t actually their problem. Alexander said she’s come across situations where a society journal that’s working with a publisher is trying to figure out processes they aren’t supposed to worry about in the first place. This can either occur because the editors don’t like how their publisher is handling something and choose to take it into their own hands, or because the journal team simply doesn’t realize something is a publisher duty.
“I remember once I was working with a journal where the editor in chief was creating his own special processes for things he shouldn’t have been involved in,” Alexander recalled. “He was completely out of sync with the production process, so I set up a meeting with the publisher. We sat through a day of meetings and the editor was sitting next to me and he kept turning to me and saying ‘do you mean I don’t need to do that?’ I’d go ‘no that’s their job.’ He was shocked, he would go ‘wow, I just got two hours of my life back a day.’”
Alexander said in this situation and others that she’s seen like it, a lack of communication between journals and publishers is often the problem.
“Often nobody ever sits down and says ‘here’s the journal’s process, here’s the duties of everyone involved, and this is why.’ Really, regardless of whether your journal is published by a commercial house or self-published, you’ve got to have that kind of meeting so everybody’s on the same page.”
Have an onboarding plan for new editors
In order for journals to have process meetings and ensure everyone knows their particular duties, it’s imperative that they have a clearly documented onboarding plan for new editors. Alexander said a problem she often comes across at journals is that they are not adequately prepared for editorial board transitions and information gets lost as members come and go.
“I am constantly discovering journals that treat editorial transitions as casual process where the new editors are told ‘here’s everything, now it’s yours,’ instead of sitting with the publisher, all the incoming staff, and all outgoing staff, to review everything and make sure no balls get dropped. Casual turnovers normally cause a great deal of trouble.”
As part of outlining a journal’s publishing process, Alexander said it’s important that lead editors also take their role as managers seriously. “You can definitely see the difference when journals don’t have active management of peer review. It’s important that editors are aware of and aren’t afraid to acknowledge when their team’s performance isn’t up to par. Otherwise a journal can provide authors a poor experience and become less competitive as a result.”
Be prepared to be critical of your own ideas
Throughout any journal audit process it’s paramount that all parties involved be ready and willing to acknowledge areas where improvements can be made. Alexander said one of the biggest barriers journals face to speeding up peer review and publication is failing to admit that there might be faults in their current workflow.
“I can go back to one situation I ran into a few years ago where a journal had gotten a new editor in chief, a very creative guy, and he was questioning everything they did. The old editor had been slow, so it was easy for the staff to keep up with peer review because it was extraordinarily lengthy. So the new editor walks in and he wants everything done yesterday. The call I got from the society was: ‘we’ve got a real problem with this editor.’ But what I found was it was the staff that was the problem, because they didn’t want to work the way he worked even though it was more efficient.”
The takeaway - be prepared to question everything when you initiate a journal publishing audit! It may be that your existing processes are as efficient as possible, but it’s also likely that your journal has room for improvement. Don’t miss those opportunities by failing to look at the full picture.
