Today’s post is the second in a series of three parts that comprise Laura’s story. The first part, Just trying to get them to think about the nitty gritty of the process, unpacked her approach to teaching peer review in the classroom. The portion of our conversation shared in this post discusses Laura’s experiences as a referee, and her reflections on how she learned to do refereeing work.
Laura Saunders
Pronouns: She/her/hers
Position: Associate Professor, Simmons University School of Library and Information Science
Fun Fact: Laura just got elected Secretary of the Board of Trustees for the Somerville Public Library.
Emily: So how did you learn to referee? Was it just practice over time? Did anyone mentor you?
I think it was a little bit of a few things. I wouldn’t say I was really mentored per se, except that I did work with two faculty members who were editors of a journal—even just by observing some of the things that they did. Both of them gave me feedback on some of my early drafts of articles. So kind of seeing how they did it I think actually was helpful. I guess there was some mentoring. I think the more that I was teaching—I don’t want to say that peer review is like grading papers because it’s not exactly—but there was some sort of give and take there where I also learned over time how to give better feedback on papers, just again by practicing and doing more of it. And so I feel like over time I just learned how to give better feedback in general. Things that were more constructive, more specific, etc. And then I think also some of it came from being reviewed myself and recognizing the things that were helpful and the things that were less helpful.
Reviewing your peer student on a paper they’re writing for class is not exactly the same as peer reviewing a journal article, but at least you learn a little about giving feedback.
Emily: It sounds like you were self-directed in learning about it. It was just experience.
Yes, I think so. I mean I would say there wasn’t really anything in any of my formal training as a master’s student or PhD student where we were actually really taught how to do that. I mean there may be some places where this is happening, where it’s being done better now because I know a lot of my colleagues, for instance, are having their students peer review each other more, and they’re doing it with more guidance. Again, reviewing your peer student on a paper they’re writing for class is not exactly the same as peer reviewing a journal article, but at least you learn a little about giving feedback. I had been an English major and so how you write papers as an English major is pretty different than how you’re supposed to write research articles. And once I took the research methods class as a PhD student, essentially where they really broke down a research article, a few things clicked for me. It was like okay, now that I understand the structure of this article, I also know what I’m looking for. It was helpful for me in writing articles, but then over time as I reflected on it, it became helpful in terms of how do I analyze an article.
Emily: Do you have any experiences or anecdotes as a referee that are sticking out in your mind?
There have been one or two times where I’ve gotten some articles that started off strong, so they posed a question that I thought was really interesting, but then it just really seemed to fall apart. And those are hard because I want to be supportive in my feedback—and I’m always thinking about the fact that these are my colleagues even if they’re not people that I know directly. These are my colleagues who, like me, are trying to just do their work and do it well. So I think what’s sticking out is in some of these cases where I feel like there’s really some major issues, but trying to think about: do I just say reject this, forget about it? Or do I say, “here are all the things that you could do to try and fix this article.” Like I said I’ve been leaning more towards just giving the more detailed feedback and trying to encourage people. Again, we’re also working in a field where in most cases the issues that we’re looking at are either not executing the methodology as well as they should, or not doing a good job of analysis. It’s not like clinical trials or medical research or anything like that where anyone is going to be harmed necessarily by this. I feel like there’s more opportunity to say “look, just keep working on it. I think there’s something here that you really need to tease it out, give it more support.”
I do remember one time, and this was quite a while ago, I did suggest rejecting a particular article and it was mainly because it was incredibly, it seemed to me to be a very narrowly researched question with an incredible small sample size. And what’s interesting is that article did get published [and I don’t know that it went through a second round of review] or if it did it didn’t come back to me.
I rejected something but it got published anyway, and it really got me thinking about whether I was not understanding what the perspective of the journal was.
…. it really kind of made me think of was I using really looking at it from the perspective of this particular journal…. So basically I rejected something but it got published by that journal anyway, and it really got me thinking about whether I was not understanding what the perspective of the journal was. And it’s not as though I review always for the same journal either and so trying to remember and think about who is the readership of this journal and what is really appropriate for them I guess.
Emily: How do you think we could solve that problem? What would you as a reviewer need to be able to do that work better?
I think part of me worries that the information is all there and maybe I’m not just taking enough time to think about it. Like I said a lot of times they will send you some guidelines and things like that about the journal. In some cases I’m just thinking like “oh, I know this journal. I’ve written for this journal.” To an extent I think that the information in a lot of cases is there and we just need to take the time. I also think we probably could do a better job of packaging and sharing that information. And I think one of the problems – like I said we’re already volunteering our time to do the peer review. It takes a substantial amount of time to do it well, and if I have to add another – even if it’s another 20 minutes or half an hour to read through all of these guidelines and get this particular perspective—it is a challenge. It makes it harder for me to agree to do it, so I wonder if there’s some way without losing the quality and all the nuance. If there’s a way to just really, in a streamlined way, just say, “here are the things we want you to look at when you’re looking at this article. Here are the things you really need to know about the readership of our particular journal.”
This is a little bit tangential, but the other thing I have found really frustrating is that some journals give you a template. Often it’s set up like a survey where there’s a few closed-ended questions that you have to fill in the dots. Sometimes the questions do not line up with the article that you have just read. That’s really challenging and I think that’s something they need to think about. If you publish a range of types of articles, maybe research articles and more philosophical kinds of things, you might have to have a couple of different templates for people to respond to.
Stories of Open 