This is the third and final installment of my conversation with Sarah, following up on “…this was a thing that really meant a lot to me and so it really hurt a lot when I got these comments that were just basically that it was terrible” and “…you have to just figure out the rules and no one is going to tell you what the rules are until you screw up…”
In this installment we talk more explicitly about what peer review should be for. Why do we do this thing? What can we do to improve it? Sarah shares her thoughts on what a hybrid open and opaque process could be and do.
Sarah Ward
Pronouns: She/her/hers
Position: Associate Professor, Outreach Librarian
Institution: Hunter College – City University of New York (CUNY)
Fun Fact: Sarah’s favorite way to get around the city, aside from the subway, is by bicycle (especially during a pandemic). She’s never owned a car.
Emily: You mentioned the word gate keeping in terms of peer review and I’m wondering if that’s really what peer review should be doing. Do you have thoughts on why we do this process with publishing and what it should be? What should peer review do, ideally?
That’s a really good question and I don’t know that I have a good answer for it. I’ve experienced traditional peer review that I’ve talked about, I’ve also experienced open peer review, which is a different thing entirely.
I published something two years ago In the Library with the Lead Pipe, which is open peer reviewed. I knew who the reviewers were, they knew who we were, and that was a totally different experience. And I liked it because I felt like more care was taken in the exchanges because we knew who each other were. Now, I understand that one of the reasons for traditional peer review is allegedly to eliminate or reduce bias in the review process—you don’t know who the people are who wrote or whatever—but I feel like in certain fields like you can still guess, so it’s sort of artificial. I don’t have a fully formed thought on this, but I feel like academia in general is about gate keeping: who is in and who is out, whose voices are represented, whose are not? So peer review traditionally has obviously favored the white western male cis-gender heterosexual perspective; that’s who academia publishes. But that’s opening up and more and more voices are being heard.
I feel like the reviewer comments just go into the void and you never get to personally respond to them…
I feel like at its best, peer review should be an honest review by colleagues or peers of the value of your work. I don’t necessarily feel like that should be shrouded in mystery. I don’t feel like there should be unwritten rules that you have to follow to get through that process. I feel like it can be more transparent. And I don’t know if totally open is the way to go, maybe a hybrid version of some anonymous review and some open review so that you have an open exchange with somebody and so that you can have a dialogue. I feel like the reviewer comments just go into the void and you never get to personally respond to them to say, “okay can you tell me more about why you said this,” or have a conversation so that there’s more of a human understanding between people about the reasons for doing things. I feel like there’s a value in being able to talk it out to say, “oh I misinterpreted that; you could make it clearer by doing these things.” That would be really valuable for somebody for revisions, to be able to say “I did it this way on purpose” or “thank you for that, that really helps me think about another way to write my work.” I’m not a scholarly communications person so I don’t have a lot of really well-founded arguments about all of this. I really liked that open peer review process, but I also see the problems that could come along with it if you’re reviewing people you know and you don’t really like those people, or you do like those people, as if you’re going to overlook things if they’re your friends or whatever. But I feel like there’s a place for a hybrid model that would still maintain the integrity of the process but also enable a little bit more of a dialogue between reviewers and researchers.
Emily: It sounds like it was a good experience and you liked it, at Lead Pipe, but you said that it could be problematic. Did you feel like you experienced some of those problems with that process?
I don’t know. I had met one of the reviewers at a conference once, so I knew who she was and I knew the names of the other people and so there was maybe a little bit of intimidation on my part. I really respected their work, but that could be said for any reviewer even if you don’t know who they are they should be someone whose work is valid in the field and they need due respect or whatever. I didn’t run into any of those problems really with that process. And like I said, it was a co-authored piece so we had the ability to have a conversation with each other about the process. It was interesting and it was totally different than anything we had done before.
Emily: Why did you and your collaborator decide to submit to Lead Pipe?
This article that was in Lead Pipe was another bit of my research that I care very much about actually, because this work was about this ongoing program that we had developed with a summer bridge program at our school. We developed a curriculum for students to be delivered over the summer while they were between high school and college. We had written and presented about developing the curriculum and about all this other stuff up until this point and then we felt like we had one more thing to say about this process. We interviewed a number of former students and asked them about their experiences with this curriculum. We wanted it to be a back and forth, if not fully co-authored, but certainly collaborative process between the students and us. And so we thought Lead Pipe was a good place for this because, stylistically, they publish lots of different types of articles. They publish qualitative research studies, they publish quantitative research, they publish theoretical stuff, they publish all different types of things. We decided to submit to them because we had submitted to a couple of other journals and had been rejected for lack of formality again, so this comes back to this voice that we wanted to tell it in. We wanted to tell this story and we wanted to use the student’s voices and we wanted to incorporate our own voices. So pitched it to them and they said, “yeah we’ll work with you on this” and we shaped it for their publication. I felt pretty good about how that ended up. We arrived at that because we knew that what we had written was not another traditionally academic style article so that was why we submitted there.
Emily: Thank you for sharing that. I don’t know if you know this but I was one of the co-founders of In the Library with the Library Pipe. It’s a place where we can have our own voice shine through and they don’t have to look traditionally academic but they still can be academic, there’s that flexibility. The open peer review process that we tried to create is still working. So, I just want to reflect that my heart is just full to hear that.
I was really pleased and I agree with you like it’s one of the few bits of professional reading that I keep up with because I think like the content is so interesting and so varied and I feel like a lot of it is still relevant to my work as well. So, thank you for founding the journal. I appreciate it.
Emily: Thank you for sharing that. I want to unpack this idea of a hybrid open since you mentioned it. I know you say you haven’t really codified for yourself what that hybrid would look like, but what should peer review look like with a degree of more transparency?
…I feel like it’s that humanity piece that’s lacking
I mean I don’t have fully formed thoughts about this, but as I was talking it out with you, the idea of an anonymous reviewer has power because then you don’t have that sort of personal, “oh this person knows who I am therefore they can trash me if I make bad comments about them.” So I feel that there’s power in that, but I feel like it’s that humanity piece that’s lacking. And I know that’s partially the job of the editor, to communicate with writers and between reviewers and writers, but I feel like there’s an interim step. If you had an article that was reviewed by an anonymous reviewer and an open reviewer, then an editor or a co-editor could facilitate a dialogue between the reviewers and the writer to say, “so I’ve spoken with the anonymous reviewer and here are the things that they have commented on and I see that the other reviewer has commented on these things, now let’s talk about what all of this means for you.” If the editor could really make that a conversation rather than a one way: I’m going to give you comments and you have to do it or don’t do it. Let’s make it an exchange. There’s always value in talking about your research and writing rather than just writing and submitting, but to actually have a conversation with somebody about it and about your process and to get feedback to clarify your thinking. When I’m with students that’s what I say, too. “Talk with somebody about it, you should be able to talk about your research in a way that makes sense to somebody else, and if it doesn’t, then you have some work to do.” To be able to build that into the process—the reflective discursive process between an anonymous reviewer, an open reviewer, an editor, and an author. It’s a lot more work for an editor and I think it’s a heavier lift, but I think systems need to change. I think that there’s a place for that now, given the move toward open review and open scholarship.
Emily: I like the collaborative process. The approach that frustrates me – the purely gate keeping approach – makes it feel almost robotic. I feel like research is a human endeavor and what we’re doing as librarians is a human endeavor and I feel like peer review practices should reflect the humanity, just like you said, it should be a human interaction, it should be approached with care and love and collaboration and support.
…the process by which this stuff gets published should also reflect the humanity of the profession.
Particularly in this field. I think maybe in the hard sciences it’s a little different when everyone’s trying to submit to Nature and get published in these high impact journals, but I feel like in librarianship our humanity is part of what makes us good librarians. That, I think, should be reflected in our literature and the process by which this stuff gets published should also reflect the humanity of the profession. Same thing with social work other of these professions that are caring professions or human centered professions—I think there’s space for that. I don’t think we need to mimic the hard sciences to be valid in the research, I think we can carve our own path.
Emily: But why do we mimic the hard sciences or why have we in the past?
I think that some of that has to do with the inferiority complex in academia that librarians have with “we’re not really faculty but we are called faculty but we don’t have PhD’s all the time.” We walk that line between service providers and researchers and I think that there’s always a bit of a struggle there. Some folks purely identify as academics, where some people say we really are in a service profession. I think that there is space for both of those things to be true, but I think that there’s that conflict. And I think that the people—particularly who I know in CUNY—who fought for faculty status for librarians are not willing to give that up and are not willing to bend that to mean anything other than, “I am a faculty member, I am a researcher, therefore I will mimic the processes that have been in place for however many years even if I don’t think that they’re appropriate for our field.” I respect that and I respect the folks who came before who fought those fights. I think it was really important for people to do that and to speak up about the fact that we are researchers and academics as well, but I also think that times have changed and the profession is necessarily changing and evolving and academia also is necessarily changing and evolving and that there is room for more conversation about what is the purpose of this now. Do we need to mimic these systems of oppression? (For a dramatic turn of phrase.) Do we need to continue in this vein or can we say, “okay so we did that for a while and now we see a different way for ourselves forward?”
Emily: I’ve been calling it our insecurity problem and actually it comes through in my book quite a bit, but I’ve been pretty influenced by the introduction to this book The Self as Subject. Ann-Marie’s introduction, she posits that if we as librarians – she’s quoting someone or paraphrasing someone – if we don’t really have a certain guiding theory so where we have this insecurity – are we faculty? Are we not faculty? Are we service providers? Are we not? How do we exist in this world? When that happens our research will be guided not by what we want, but by what our institutions want. And so if our institutions want us to be doing quantitative research with methodologies that are “tried and true” or positivist or whatever, then that’s the direction that we’re going to go because we’re uncertain of who we are. And so, we’re letting the institution dictate our research, our peer-review processes, et cetera.
We’ve been having those conversations on our teams. We are revising our tenure guidelines now and I’m on the committee that’s doing that. We’re talking about the need to push for giving weight and value to the things that we value in this profession rather than mimicking what other, differently focused departments have done. I think those are really important conversations to have. Then also engaging the administrators who are reviewing files in order to be able to make a case that these things are important. I remember a while ago somebody at the college saying, “oh you need to have solo authored blah, blah, blah X number of solo authored peer-reviewed articles,” and I pushed back. I said, “we don’t do anything solo. I have had one solo article. Everything I’ve been writing, all of my work, is collaborative. I have to work with other people.” That’s not the point or the realm in our profession; we don’t have to write a book, we don’t have to do a solo research study, we worked together. I publish with the same people a lot. You’ll see on my CV that early on I had these two colleagues I worked with them two or three different studies, and then later on it was two different colleagues that I worked with on a bunch of other things; that’s how we work. We work in teams and we have to reflect that in our literature. It’s disingenuous to do it otherwise, I think.
Stories of Open 