This story comes from Laura Saunders, a professor at the Simmons University School of Library and Information Science. We spoke in late July of 2020. This story will be published in three parts. This first chapter discusses Laura’s approach to teaching peer review in the classroom.
As you read through Laura’s story, you may find tension in her telling of how she teaches peer review in the classroom and how she herself approaches peer review when she practices it. (I know I noticed it.) We got to think about that tension together. Maybe her story will help you uncover the tensions in your own teaching practices as they relate to the practice of the thing itself.
Laura Saunders
Pronouns: she/her/hers
Position: Associate Professor, Simmons University School of Library and Information Science
Fun Fact: Laura is currently studying Italian
In addition to her own research and publishing, Laura works to elucidate the peer-review process for her students. Among the classes that she regularly teaches, she teaches information Sources and Services, a user instruction and information literacy class, and an academic libraries class. Prior to becoming a professor, Laura was managing a small career resource library. It was affiliated with an academic library that also housed career counselors. Half of Laura’s time was spent providing reference and instruction services as well. She then pursued a PhD as a part-time student while working, and landed at Simmons. In our discussion Laura talked about learning to referee and refereeing, teaching peer review to students, and we also ruminated together on editorial power and open review.
Emily: Can you tell me a little bit about why you wanted to participate in this interview?
I think it’s a few reasons. One reason of course is just because I have been a peer reviewer and been the subject of peer review. So the process definitely interests me, but it also interests me in a little bit more of an academic sense for a couple of different reasons. One being because I teach librarians, emerging professional librarians and so part of what we talk about is peer review in the sense that some of them, especially the ones who want to be academic librarians, are going to have work with patrons who will have to look at peer-reviewed papers for their research, and so part of what they need to be able to do is not just locate those papers, but really understand what peer review is and what the process is so they can help their patrons understand that. Some of them are going to go on to publish themselves, and so they might be involved in that.
One of my PhD students who recently graduated, did her dissertation on financial conflicts of interest in scientific publications. And part of what she looked at—even though it was a little bit outside of the scope—she ended up including in her review some information about scientific things that had been retracted, and so failures of peer review. In that regard it’s kind of an interesting question.
Emily: How do you teach peer review in the classroom?
I actually it’s a little bit of a mix of trying to just give as good an overview as I can of what the process is and how it’s supposed to work. So just kind of explaining to my students why do we do peer review and what is it that we’re hoping to accomplish by this and then how does it work. It’s this idea that I focus a lot on peer-reviewed journal articles, so I’ll talk about that process of somebody writing an article, sending it out and that it will get reviewed by two experts in the field who will give feedback, and then you’ve got this range of possibilities from accept to reject and everything in between, with major revisions and minor revisions and so on. I’ll kind of go through that on a really basic level, and then from there I’ll usually just share with them some of my personal experiences because I feel like it helps them to understand, especially those who haven’t ever been through it.
I’ll talk to them a little bit about when peer review fails, and at times they’ll get really, I think rightfully, kind of annoyed with the idea that people aren’t doing their jobs. But then I’ll often ask them, “Well how much do you think peer reviewers get paid to do this?” And there’s this look. And people are looking around. And when I tell them, “Nothing. We do this out of professional courtesy,” it gets them thinking more about what is really involved, what the labor is that’s involved, and why it may not always be done as thoroughly as we would hope and expect. So just trying to get them to think about the nitty gritty of the process. Because on the surface level it sounds like something that should work really well, and it’s a nice setup and whatever, but thinking about all the little pieces and how they fit together and the fact that most of the time reviewers are doing this work without getting paid—and you do get some professional return on it in the sense that you can say you did this service to the profession, but since most of it is blind peer-reviewed no one is ever really going to know what you did and how well you did it. So just talking through all of that with them and sharing some of my experiences.
What were they looking for that was different from what I was actually giving them?
Emily: Do you have particular experiences that you find yourself sharing over and over with them?
I suppose so, yeah. There’s one, and it’s funny because I don’t really remember what the review itself was, but it was one of those things where the advice really stuck with me. It was while I was working on my PhD I was collaborating with my advisor on a paper. We sent it out and it came back with a request for revisions. I was reading through it and talking through it with him and he said something along the lines of, “you have to understand, sometimes people are reviewing the paper they wish you wrote instead of the one you actually wrote.” And it made something click with me that I have found to be helpful going forward. Sometimes when I get reviews now, instead of reading them as being overly critical or they just didn’t understand, I’ll start thinking about, “okay, what might have been in their mind? What were they looking for that was different from what I was actually giving them?” I share that story quite a bit.
I was recently collaborating with a master’s student who ran a survey for an independent study that I oversaw. I thought it was a really interesting research question, and I encouraged her to publish it, and I worked with her and we ended up going through two full rounds of revisions and at one point she asked me “is it normal for this to happen?” I could tell that she was feeling maybe a little bit like they were saying it wasn’t good work or something like that. As we talked through it and then as we got to the final round, she said to me “wow, I really see how much better this paper is now and how it was really worth it.” The paper is going to come out pretty soon.
The person who gave me the lowest score gave me two words of feedback.
I’ve seen that full range of the feedback that really isn’t particularly helpful. I submitted a grant proposal two years ago and it was reviewed by five people and they sent all five reviewers scores. It was interesting because the person who gave me the lowest score gave me two words of feedback. I have shared that story with my classes since then because I said, “what am I supposed to do with this?” I mean they told me nothing. If you’re going to volunteer to review you should at least put some effort into it.
Emily: At a journal you might have an editor that will tell you where to focus, what to work on. Does a grant officer play that same role?
Yes, at least for the one that I applied for they do. You’re able to meet with them and they can give you some high-level feedback. With regard to that particular grant proposal that did not get funded, she couldn’t go into a whole lot of detail, but she was able to give me a sense of what was the general consensus. What are some of the areas that I might work on to strengthen it the next time around and stuff like that.
Emily: You mentioned something about this whole idea of reviewers reviewing the article that they thought that they wanted to read versus actually reading the article that was submitted. Can you talk a little bit more about that?
I can remember in particular one study that I did where in two rounds of feedback one of the reviewers both times said something like, “well, you really should have asked this question as part of the survey.” And it was kind of like well, sure. To be honest I’m not sure I even agreed that that question necessarily should have been there, but whether it should have been or not, it wasn’t, so there was nothing we could do about it at this point. I think the first time around I kind of glossed over it, but then the second time around in my response to the reviewers I tried to say something like I appreciate this point and I mentioned in the area for further research that that could be something that someone could look at in the future. I responded, in reality it wasn’t there and here are some of the reasons why it may have been just as well not to have it there. I tried to give a response that I hoped would help them to understand what my thinking was.
Emily: Do you think the reviewer even got to see that comment?
I am not sure. It’s a really good question because I think it depends on the journal. When I have been a reviewer and I’ve gotten a second round, it has worked both ways. In some cases I just get a new revised article and in some cases I get the revised article along with my original comments and any response that the author wrote so I think it really depends on the journal.
There was a part of me that thought this could have actually been an interesting conversation and who knows, maybe even one that might have been of interest to the readers of that publication.
Emily: Are you screaming into an abyss where you’re offering this response to feedback and the only person reading it is an editor? Where does that conversation go?
It’s a great question and, in fact, that made me think, too, just recently I submitted an article. I’ve been trying to look at how faculty members in LIS teach soft skills in their classrooms, so things like interpersonal skills, and I’m working with a collaborator, and we ended up deciding to survey people outside of LIS, too, so we could do some cross-disciplinary comparisons. So we sent the article out, the one that looks at these several fields that we surveyed. I sent it to one journal in higher education that I thought looked like a good match. We made a few points about active learning, that in order to learn these kinds of personal and interpersonal skills you really do need to practice, because interestingly and probably not surprisingly, a lot of the top methods the people said they used were readings and lectures. How well is someone going to learn interpersonal communication if all they’re doing is sitting and listening to a lecture? So it seemed like a good match, and the editors sent it back and said, “I’m sorry but I’m not going to forward this to the reviewers because your suggestions” – she said something along the lines of “you’re making suggestions about how to change classes that instructors could not use in their individual classes. Those kinds of changes would have to be made at a program level.” I was thinking, “no, they wouldn’t actually.” But again, it was kind of this awkward position where I definitely disagreed with what the editor was saying, and she wrote me this really long thoughtful response but I kind of felt like, “no, I really don’t think this is the case,” but I also felt like she had made her decision and the way it was written it just kind of seemed like it was pretty clearly a no and it didn’t seem like it was worth it to write back and argue. But there was a part of me that thought this could have actually been an interesting conversation and who knows, maybe even one that might have been of interest to the readers of that publication.
Emily: I was thinking about the power that this editor had in terms of selection bias of what’s getting published in that journal and what’s actually getting out in the literature and how it’s approached. So I feel like peer review is one of those where, without practicing it, how do you learn it?
Well I think you raise a really good point about the power that the editor wields. Definitely this was interesting so it was one of the very few times that I had a general editor reject something out of hand. And I did think that was interesting because like you said it does mean that there’s definitely a lot of power there in terms of just deciding what gets through the – it’s a gate-keeping function, right? And so what actually gets through the gate and when one person is doing all of that or most of that, there’s probably going to be some skew towards whatever their particular areas of interest might be, even unconsciously. I think that’s interesting and I think that could have been a case where maybe if it had gone out to a couple of different people, their responses or reactions may have been slightly different. Who knows? It is something that I’ve thought about a lot and again something that I’ve brought back to the classroom.
You do wonder what are all those things that are out there that would be valuable knowledge that aren’t getting published.
I remember quite a few years ago when I was first starting to teach in LIS, hearing a story on NPR that they had done a study of articles in health and medical journals. What they found is that, and I’m not sure if I’m remembering all the statistics correctly, but in clinical trials when the results were positive, the articles got published something like 60 percent of the time. When the results were negative, they only got published 12 percent of the time. I remember that really striking me because those negative results could be just as important to know about. But, of course, for lots of different reasons they’re not getting published and there’s a reason to look at those kinds of publishing trends in medicine, but we’ve got to figure it’s happening in other fields, too. I’m sure in LIS there aren’t a lot of people writing articles about how we failed or how it didn’t work for us. Again, you can think of all the reasons why people may not want to write those things, but you do wonder what are all those things that are out there that would be valuable knowledge that aren’t getting published. And then it raises questions, too, about not just the content necessarily, but the voices that are being stifled and the idea that often there’s going to be some bias towards different questions, different kind of methodology, different content, different authors, all of that kind of stuff. I think this idea of this objective blind peer review suggests that all of those biases get pulled out. But, of course, we know that’s not the case.
So as you said I think that does come back to what I try to teach in the classroom, and just really getting folks to think about, because so often and it’s been a long time since I’ve been in the library—teaching as a librarian as opposed to a library faculty member. I remember back when I was in classes where I would see a librarian say to the class, “okay, I know you need to look for peer-reviewed articles for this class, so in this database you can check this little box and get peer-reviewed articles.” And the implication seemed to be, or I was at least afraid that the message the students were getting in these cases were, “once I click that box, I can trust anything that comes back.” I felt like rarely did we have a discussion about how what you get might still be problematic. I feel like those discussions might be happening more now than they were 15 years ago or so. But trying to get my students to think about that and then finally I think it’s absolutely true: we’re not really taught how to peer review. Sometimes the journal will send you some guidelines when they ask you to peer review and they’ll go throw a set of guidelines at you or something. It is absolutely something that I had to learn over time and I think I probably do a much better job of it now. Do you know what I mean? I think about some of those people who got my first reviews who probably weren’t feeling too much like they were more useful than some of the ones that I’ve been complaining about because I didn’t really know what I was doing.
Stories of Open 
2 thoughts on “Just trying to get them to think about the nitty gritty of the process”