Just trying to get them to think about the nitty gritty of the process

This story comes from Laura Saunders, a professor at the Simmons University School of Library and Information Science. We spoke in late July of 2020. This story will be published in three parts. This first chapter discusses Laura’s approach to teaching peer review in the classroom.

As you read through Laura’s story, you may find tension in her telling of how she teaches peer review in the classroom and how she herself approaches peer review when she practices it. (I know I noticed it.) We got to think about that tension together. Maybe her story will help you uncover the tensions in your own teaching practices as they relate to the practice of the thing itself.

Laura Saunders

Photograph of Laura Saunders
Laura Saunders

Pronouns: she/her/hers

Position: Associate Professor, Simmons University School of Library and Information Science

Fun Fact: Laura is currently studying Italian

In addition to her own research and publishing, Laura works to elucidate the peer-review process for her students. Among the classes that she regularly teaches, she teaches information Sources and Services, a user instruction and information literacy class, and an academic libraries class. Prior to becoming a professor, Laura was managing a small career resource library. It was affiliated with an academic library that also housed career counselors. Half of Laura’s time was spent providing reference and instruction services as well. She then pursued a PhD as a part-time student while working, and landed at Simmons. In our discussion Laura talked about learning to referee and refereeing, teaching peer review to students, and we also ruminated together on editorial power and open review.

Emily: Can you tell me a little bit about why you wanted to participate in this interview?

I think it’s a few reasons. One reason of course is just because I have been a peer reviewer and been the subject of peer review. So the process definitely interests me, but it also interests me in a little bit more of an academic sense for a couple of different reasons. One being because I teach librarians, emerging professional librarians and so part of what we talk about is peer review in the sense that some of them, especially the ones who want to be academic librarians, are going to have work with patrons who will have to look at peer-reviewed papers for their research, and so part of what they need to be able to do is not just locate those papers, but really understand what peer review is and what the process is so they can help their patrons understand that. Some of them are going to go on to publish themselves, and so they might be involved in that.

One of my PhD students who recently graduated, did her dissertation on financial conflicts of interest in scientific publications. And part of what she looked at—even though it was a little bit outside of the scope—she ended up including in her review some information about scientific things that had been retracted, and so failures of peer review. In that regard it’s kind of an interesting question.

Emily: How do you teach peer review in the classroom?

I actually it’s a little bit of a mix of trying to just give as good an overview as I can of what the process is and how it’s supposed to work. So just kind of explaining to my students why do we do peer review and what is it that we’re hoping to accomplish by this and then how does it work. It’s this idea that I focus a lot on peer-reviewed journal articles, so I’ll talk about that process of somebody writing an article, sending it out and that it will get reviewed by two experts in the field who will give feedback, and then you’ve got this range of possibilities from accept to reject and everything in between, with major revisions and minor revisions and so on. I’ll kind of go through that on a really basic level, and then from there I’ll usually just share with them some of my personal experiences because I feel like it helps them to understand, especially those who haven’t ever been through it.

I’ll talk to them a little bit about when peer review fails, and at times they’ll get really, I think rightfully, kind of annoyed with the idea that people aren’t doing their jobs. But then I’ll often ask them, “Well how much do you think peer reviewers get paid to do this?” And there’s this look. And people are looking around. And when I tell them, “Nothing. We do this out of professional courtesy,” it gets them thinking more about what is really involved, what the labor is that’s involved, and why it may not always be done as thoroughly as we would hope and expect. So just trying to get them to think about the nitty gritty of the process. Because on the surface level it sounds like something that should work really well, and it’s a nice setup and whatever, but thinking about all the little pieces and how they fit together and the fact that most of the time reviewers are doing this work without getting paid—and you do get some professional return on it in the sense that you can say you did this service to the profession, but since most of it is blind peer-reviewed no one is ever really going to know what you did and how well you did it. So just talking through all of that with them and sharing some of my experiences.

What were they looking for that was different from what I was actually giving them?

Emily: Do you have particular experiences that you find yourself sharing over and over with them?

I suppose so, yeah. There’s one, and it’s funny because I don’t really remember what the review itself was, but it was one of those things where the advice really stuck with me. It was while I was working on my PhD I was collaborating with my advisor on a paper. We sent it out and it came back with a request for revisions. I was reading through it and talking through it with him and he said something along the lines of, “you have to understand, sometimes people are reviewing the paper they wish you wrote instead of the one you actually wrote.” And it made something click with me that I have found to be helpful going forward. Sometimes when I get reviews now, instead of reading them as being overly critical or they just didn’t understand, I’ll start thinking about, “okay, what might have been in their mind? What were they looking for that was different from what I was actually giving them?” I share that story quite a bit.

I was recently collaborating with a master’s student who ran a survey for an independent study that I oversaw. I thought it was a really interesting research question, and I encouraged her to publish it, and I worked with her and we ended up going through two full rounds of revisions and at one point she asked me “is it normal for this to happen?” I could tell that she was feeling maybe a little bit like they were saying it wasn’t good work or something like that. As we talked through it and then as we got to the final round, she said to me “wow, I really see how much better this paper is now and how it was really worth it.” The paper is going to come out pretty soon.

The person who gave me the lowest score gave me two words of feedback.

I’ve seen that full range of the feedback that really isn’t particularly helpful. I submitted a grant proposal two years ago and it was reviewed by five people and they sent all five reviewers scores. It was interesting because the person who gave me the lowest score gave me two words of feedback. I have shared that story with my classes since then because I said, “what am I supposed to do with this?” I mean they told me nothing. If you’re going to volunteer to review you should at least put some effort into it.

Emily: At a journal you might have an editor that will tell you where to focus, what to work on. Does a grant officer play that same role?

Yes, at least for the one that I applied for they do. You’re able to meet with them and they can give you some high-level feedback. With regard to that particular grant proposal that did not get funded, she couldn’t go into a whole lot of detail, but she was able to give me a sense of what was the general consensus. What are some of the areas that I might work on to strengthen it the next time around and stuff like that.

Emily: You mentioned something about this whole idea of reviewers reviewing the article that they thought that they wanted to read versus actually reading the article that was submitted. Can you talk a little bit more about that?

I can remember in particular one study that I did where in two rounds of feedback one of the reviewers both times said something like, “well, you really should have asked this question as part of the survey.” And it was kind of like well, sure. To be honest I’m not sure I even agreed that that question necessarily should have been there, but whether it should have been or not, it wasn’t, so there was nothing we could do about it at this point. I think the first time around I kind of glossed over it, but then the second time around in my response to the reviewers I tried to say something like I appreciate this point and I mentioned in the area for further research that that could be something that someone could look at in the future. I responded, in reality it wasn’t there and here are some of the reasons why it may have been just as well not to have it there. I tried to give a response that I hoped would help them to understand what my thinking was.

Emily: Do you think the reviewer even got to see that comment?

I am not sure. It’s a really good question because I think it depends on the journal. When I have been a reviewer and I’ve gotten a second round, it has worked both ways. In some cases I just get a new revised article and in some cases I get the revised article along with my original comments and any response that the author wrote so I think it really depends on the journal.

There was a part of me that thought this could have actually been an interesting conversation and who knows, maybe even one that might have been of interest to the readers of that publication.

Emily: Are you screaming into an abyss where you’re offering this response to feedback and the only person reading it is an editor? Where does that conversation go?

It’s a great question and, in fact, that made me think, too, just recently I submitted an article. I’ve been trying to look at how faculty members in LIS teach soft skills in their classrooms, so things like interpersonal skills, and I’m working with a collaborator, and we ended up deciding to survey people outside of LIS, too, so we could do some cross-disciplinary comparisons. So we sent the article out, the one that looks at these several fields that we surveyed. I sent it to one journal in higher education that I thought looked like a good match. We made a few points about active learning, that in order to learn these kinds of personal and interpersonal skills you really do need to practice, because interestingly and probably not surprisingly, a lot of the top methods the people said they used were readings and lectures. How well is someone going to learn interpersonal communication if all they’re doing is sitting and listening to a lecture? So it seemed like a good match, and the editors sent it back and said, “I’m sorry but I’m not going to forward this to the reviewers because your suggestions” – she said something along the lines of “you’re making suggestions about how to change classes that instructors could not use in their individual classes. Those kinds of changes would have to be made at a program level.” I was thinking, “no, they wouldn’t actually.” But again, it was kind of this awkward position where I definitely disagreed with what the editor was saying, and she wrote me this really long thoughtful response but I kind of felt like, “no, I really don’t think this is the case,” but I also felt like she had made her decision and the way it was written it just kind of seemed like it was pretty clearly a no and it didn’t seem like it was worth it to write back and argue. But there was a part of me that thought this could have actually been an interesting conversation and who knows, maybe even one that might have been of interest to the readers of that publication.

Emily: I was thinking about the power that this editor had in terms of selection bias of what’s getting published in that journal and what’s actually getting out in the literature and how it’s approached. So I feel like peer review is one of those where, without practicing it, how do you learn it?

Well I think you raise a really good point about the power that the editor wields. Definitely this was interesting so it was one of the very few times that I had a general editor reject something out of hand. And I did think that was interesting because like you said it does mean that there’s definitely a lot of power there in terms of just deciding what gets through the – it’s a gate-keeping function, right? And so what actually gets through the gate and when one person is doing all of that or most of that, there’s probably going to be some skew towards whatever their particular areas of interest might be, even unconsciously. I think that’s interesting and I think that could have been a case where maybe if it had gone out to a couple of different people, their responses or reactions may have been slightly different. Who knows? It is something that I’ve thought about a lot and again something that I’ve brought back to the classroom.

You do wonder what are all those things that are out there that would be valuable knowledge that aren’t getting published.

I remember quite a few years ago when I was first starting to teach in LIS, hearing a story on NPR that they had done a study of articles in health and medical journals. What they found is that, and I’m not sure if I’m remembering all the statistics correctly, but in clinical trials when the results were positive, the articles got published something like 60 percent of the time. When the results were negative, they only got published 12 percent of the time. I remember that really striking me because those negative results could be just as important to know about. But, of course, for lots of different reasons they’re not getting published and there’s a reason to look at those kinds of publishing trends in medicine, but we’ve got to figure it’s happening in other fields, too. I’m sure in LIS there aren’t a lot of people writing articles about how we failed or how it didn’t work for us. Again, you can think of all the reasons why people may not want to write those things, but you do wonder what are all those things that are out there that would be valuable knowledge that aren’t getting published. And then it raises questions, too, about not just the content necessarily, but the voices that are being stifled and the idea that often there’s going to be some bias towards different questions, different kind of methodology, different content, different authors, all of that kind of stuff. I think this idea of this objective blind peer review suggests that all of those biases get pulled out. But, of course, we know that’s not the case.

So as you said I think that does come back to what I try to teach in the classroom, and just really getting folks to think about, because so often and it’s been a long time since I’ve been in the library—teaching as a librarian as opposed to a library faculty member. I remember back when I was in classes where I would see a librarian say to the class, “okay, I know you need to look for peer-reviewed articles for this class, so in this database you can check this little box and get peer-reviewed articles.” And the implication seemed to be, or I was at least afraid that the message the students were getting in these cases were, “once I click that box, I can trust anything that comes back.” I felt like rarely did we have a discussion about how what you get might still be problematic. I feel like those discussions might be happening more now than they were 15 years ago or so. But trying to get my students to think about that and then finally I think it’s absolutely true: we’re not really taught how to peer review. Sometimes the journal will send you some guidelines when they ask you to peer review and they’ll go throw a set of guidelines at you or something. It is absolutely something that I had to learn over time and I think I probably do a much better job of it now. Do you know what I mean? I think about some of those people who got my first reviews who probably weren’t feeling too much like they were more useful than some of the ones that I’ve been complaining about because I didn’t really know what I was doing.

“I feel like at its best, peer review should be an honest review by colleagues or peers of the value of your work.”

This is the third and final installment of my conversation with Sarah, following up on “…this was a thing that really meant a lot to me and so it really hurt a lot when I got these comments that were just basically that it was terrible” and “…you have to just figure out the rules and no one is going to tell you what the rules are until you screw up…”

In this installment we talk more explicitly about what peer review should be for. Why do we do this thing? What can we do to improve it? Sarah shares her thoughts on what a hybrid open and opaque process could be and do.

Sarah Ward

Sarah Ward

Pronouns: She/her/hers

Position: Associate Professor, Outreach Librarian

Institution: Hunter College – City University of New York (CUNY)

Fun Fact: Sarah’s favorite way to get around the city, aside from the subway, is by bicycle (especially during a pandemic). She’s never owned a car.

Emily: You mentioned the word gate keeping in terms of peer review and I’m wondering if that’s really what peer review should be doing. Do you have thoughts on why we do this process with publishing and what it should be? What should peer review do, ideally?

That’s a really good question and I don’t know that I have a good answer for it. I’ve experienced traditional peer review that I’ve talked about, I’ve also experienced open peer review, which is a different thing entirely.

I published something two years ago In the Library with the Lead Pipe, which is open peer reviewed. I knew who the reviewers were, they knew who we were, and that was a totally different experience. And I liked it because I felt like more care was taken in the exchanges because we knew who each other were. Now, I understand that one of the reasons for traditional peer review is allegedly to eliminate or reduce bias in the review process—you don’t know who the people are who wrote or whatever—but I feel like in certain fields like you can still guess, so it’s sort of artificial. I don’t have a fully formed thought on this, but I feel like academia in general is about gate keeping: who is in and who is out, whose voices are represented, whose are not? So peer review traditionally has obviously favored the white western male cis-gender heterosexual perspective; that’s who academia publishes. But that’s opening up and more and more voices are being heard.

I feel like the reviewer comments just go into the void and you never get to personally respond to them…

I feel like at its best, peer review should be an honest review by colleagues or peers of the value of your work.  I don’t necessarily feel like that should be shrouded in mystery. I don’t feel like there should be unwritten rules that you have to follow to get through that process. I feel like it can be more transparent. And I don’t know if totally open is the way to go, maybe a hybrid version of some anonymous review and some open review so that you have an open exchange with somebody and so that you can have a dialogue. I feel like the reviewer comments just go into the void and you never get to personally respond to them to say, “okay can you tell me more about why you said this,” or have a conversation so that there’s more of a human understanding between people about the reasons for doing things. I feel like there’s a value in being able to talk it out to say, “oh I misinterpreted that; you could make it clearer by doing these things.” That would be really valuable for somebody for revisions, to be able to say “I did it this way on purpose” or “thank you for that, that really helps me think about another way to write my work.” I’m not a scholarly communications person so I don’t have a lot of really well-founded arguments about all of this. I really liked that open peer review process, but I also see the problems that could come along with it if you’re reviewing people you know and you don’t really like those people, or you do like those people, as if you’re going to overlook things if they’re your friends or whatever. But I feel like there’s a place for a hybrid model that would still maintain the integrity of the process but also enable a little bit more of a dialogue between reviewers and researchers.

Emily: It sounds like it was a good experience and you liked it, at Lead Pipe, but you said that it could be problematic. Did you feel like you experienced some of those problems with that process?

I don’t know. I had met one of the reviewers at a conference once, so I knew who she was and I knew the names of the other people and so there was maybe a little bit of intimidation on my part. I really respected their work, but that could be said for any reviewer even if you don’t know who they are they should be someone whose work is valid in the field and they need due respect or whatever. I didn’t run into any of those problems really with that process. And like I said, it was a co-authored piece so we had the ability to have a conversation with each other about the process. It was interesting and it was totally different than anything we had done before.

Emily: Why did you and your collaborator decide to submit to Lead Pipe?

This article that was in Lead Pipe was another bit of my research that I care very much about actually, because this work was about this ongoing program that we had developed with a summer bridge program at our school. We developed a curriculum for students to be delivered over the summer while they were between high school and college. We had written and presented about developing the curriculum and about all this other stuff up until this point and then we felt like we had one more thing to say about this process. We interviewed a number of former students and asked them about their experiences with this curriculum. We wanted it to be a back and forth, if not fully co-authored, but certainly collaborative process between the students and us. And so we thought Lead Pipe was a good place for this because, stylistically, they publish lots of different types of articles. They publish qualitative research studies, they publish quantitative research, they publish theoretical stuff, they publish all different types of things. We decided to submit to them because we had submitted to a couple of other journals and had been rejected for lack of formality again, so this comes back to this voice that we wanted to tell it in. We wanted to tell this story and we wanted to use the student’s voices and we wanted to incorporate our own voices. So pitched it to them and they said, “yeah we’ll work with you on this” and we shaped it for their publication. I felt pretty good about how that ended up. We arrived at that because we knew that what we had written was not another traditionally academic style article so that was why we submitted there.

Emily: Thank you for sharing that. I don’t know if you know this but I was one of the co-founders of In the Library with the Library Pipe. It’s a place where we can have our own voice shine through and they don’t have to look traditionally academic but they still can be academic, there’s that flexibility. The open peer review process that we tried to create is still working. So, I just want to reflect that my heart is just full to hear that.

I was really pleased and I agree with you like it’s one of the few bits of professional reading that I keep up with because I think like the content is so interesting and so varied and I feel like a lot of it is still relevant to my work as well. So, thank you for founding the journal. I appreciate it.

Emily: Thank you for sharing that. I want to unpack this idea of a hybrid open since you mentioned it. I know you say you haven’t really codified for yourself what that hybrid would look like, but what should peer review look like with a degree of more transparency?

…I feel like it’s that humanity piece that’s lacking

I mean I don’t have fully formed thoughts about this, but as I was talking it out with you, the idea of an anonymous reviewer has power because then you don’t have that sort of personal, “oh this person knows who I am therefore they can trash me if I make bad comments about them.” So I feel that there’s power in that, but I feel like it’s that humanity piece that’s lacking. And I know that’s partially the job of the editor, to communicate with writers and between reviewers and writers, but I feel like there’s an interim step. If you had an article that was reviewed by an anonymous reviewer and an open reviewer, then an editor or a co-editor could facilitate a dialogue between the reviewers and the writer to say, “so I’ve spoken with the anonymous reviewer and here are the things that they have commented on and I see that the other reviewer has commented on these things, now let’s talk about what all of this means for you.” If the editor could really make that a conversation rather than a one way: I’m going to give you comments and you have to do it or don’t do it. Let’s make it an exchange. There’s always value in talking about your research and writing rather than just writing and submitting, but to actually have a conversation with somebody about it and about your process and to get feedback to clarify your thinking. When I’m with students that’s what I say, too. “Talk with somebody about it, you should be able to talk about your research in a way that makes sense to somebody else, and if it doesn’t, then you have some work to do.” To be able to build that into the process—the reflective discursive process between an anonymous reviewer, an open reviewer, an editor, and an author. It’s a lot more work for an editor and I think it’s a heavier lift, but I think systems need to change. I think that there’s a place for that now, given the move toward open review and open scholarship.

Emily: I like the collaborative process. The approach that frustrates me – the purely gate keeping approach – makes it feel almost robotic. I feel like research is a human endeavor and what we’re doing as librarians is a human endeavor and I feel like peer review practices should reflect the humanity, just like you said, it should be a human interaction, it should be approached with care and love and collaboration and support.

…the process by which this stuff gets published should also reflect the humanity of the profession.

Particularly in this field. I think maybe in the hard sciences it’s a little different when everyone’s trying to submit to Nature and get published in these high impact journals, but I feel like in librarianship our humanity is part of what makes us good librarians. That, I think, should be reflected in our literature and the process by which this stuff gets published should also reflect the humanity of the profession. Same thing with social work other of these professions that are caring professions or human centered professions—I think there’s space for that. I don’t think we need to mimic the hard sciences to be valid in the research, I think we can carve our own path.

Emily: But why do we mimic the hard sciences or why have we in the past?

I think that some of that has to do with the inferiority complex in academia that librarians have with “we’re not really faculty but we are called faculty but we don’t have PhD’s all the time.” We walk that line between service providers and researchers and I think that there’s always a bit of a struggle there. Some folks purely identify as academics, where some people say we really are in a service profession. I think that there is space for both of those things to be true, but I think that there’s that conflict. And I think that the people—particularly who I know in CUNY—who fought for faculty status for librarians are not willing to give that up and are not willing to bend that to mean anything other than, “I am a faculty member, I am a researcher, therefore I will mimic the processes that have been in place for however many years even if I don’t think that they’re appropriate for our field.” I respect that and I respect the folks who came before who fought those fights. I think it was really important for people to do that and to speak up about the fact that we are researchers and academics as well, but I also think that times have changed and the profession is necessarily changing and evolving and academia also is necessarily changing and evolving and that there is room for more conversation about what is the purpose of this now. Do we need to mimic these systems of oppression? (For a dramatic turn of phrase.) Do we need to continue in this vein or can we say, “okay so we did that for a while and now we see a different way for ourselves forward?”

Emily: I’ve been calling it our insecurity problem and actually it comes through in my book quite a bit, but I’ve been pretty influenced by the introduction to this book The Self as Subject. Ann-Marie’s introduction, she posits that if we as librarians – she’s quoting someone or paraphrasing someone – if we don’t really have a certain guiding theory so where we have this insecurity – are we faculty? Are we not faculty? Are we service providers? Are we not? How do we exist in this world? When that happens our research will be guided not by what we want, but by what our institutions want. And so if our institutions want us to be doing quantitative research with methodologies that are “tried and true” or positivist or whatever, then that’s the direction that we’re going to go because we’re uncertain of who we are. And so, we’re letting the institution dictate our research, our peer-review processes, et cetera.

We’ve been having those conversations on our teams. We are revising our tenure guidelines now and I’m on the committee that’s doing that. We’re talking about the need to push for giving weight and value to the things that we value in this profession rather than mimicking what other, differently focused departments have done. I think those are really important conversations to have. Then also engaging the administrators who are reviewing files in order to be able to make a case that these things are important. I remember a while ago somebody at the college saying, “oh you need to have solo authored blah, blah, blah X number of solo authored peer-reviewed articles,” and I pushed back. I said, “we don’t do anything solo. I have had one solo article. Everything I’ve been writing, all of my work, is collaborative. I have to work with other people.” That’s not the point or the realm in our profession; we don’t have to write a book, we don’t have to do a solo research study, we worked together. I publish with the same people a lot. You’ll see on my CV that early on I had these two colleagues I worked with them two or three different studies, and then later on it was two different colleagues that I worked with on a bunch of other things; that’s how we work. We work in teams and we have to reflect that in our literature. It’s disingenuous to do it otherwise, I think.

“…you have to just figure out the rules and no one is going to tell you what the rules are until you screw up…”

This is the second post in a series of three that tells Sarah’s story. See the first installment, “…this was a thing that really meant a lot to me and so it really hurt a lot when I got these comments that were just basically that it was terrible.” This first post gives an overview of her in-depth research project with an art collection at Hunter College, and the resulting article documenting her efforts to understand the collection, and return it to the public. When it came to publishing an article about the project, she ran into many barriers, including intense feelings of rejection.

Sarah Ward

Sarah Ward

Pronouns: She/her/hers

Position: Associate Professor, Outreach Librarian

Institution: Hunter College – City University of New York (CUNY)

Fun Fact: Sarah is from Illinois but New York is her adopted home. She says, “I can’t imagine living anywhere else, and I’m so glad to be raising my kids here (even during a pandemic).”

Let’s pick up where we left off, with feelings.

Emily: How did you find the emotional fortitude to move forward after feeling so – I mean you didn’t use this word but it sounds to me like you felt really devastated by that.

It did. I think that’s an adequate description. It took a moment. I read the comments, I got really upset about it and then I thought, “well if this is important enough to me to work on then I’m going to take the comments in the spirit of constructive criticism.” And I also talked to colleagues. I said, “look I’ve got these conflicting comments from three different reviewers, some of them were helpful, some of them are harsh, what do you do with this information?” And a couple of my colleagues who were tenured who had gone through this process before told me “you have to figure out what’s important to you to stick to your guns on and you have to figure out what you’re willing to bend on and how you can still maintain your integrity and make the revisions that they’re asking for.” They said when you send the revisions in you can go reviewer by reviewer or comment by comment and say here’s how I modified it according to his comment. I didn’t agree with this one so I didn’t make any changes based on the this one. You can go and you can justify that it’s not just like someone is grading your paper that you have to do the things they ask you to do, you can actually push back and say, “no actually it’s written in this journalistic style because that’s the way that I wanted to present the work and I maintain that this is an appropriate way to tell this story.” So I think talking to other people and understanding that it was okay for me to do that was really important. I was able to, after my initial emotional reaction, go back and say okay now that my feelings are in check, I can see the value of these comments and I will then address the ones that I feel are worth addressing and I will tell them why I didn’t address the other ones. And that’s exactly what I did.

Emily: Was that the first time you had ever learned that you could do that? You had no idea what to do and that kind of negotiation of the process was totally new for you?

…that was the first time that anybody had told me that it’s okay for you to push back on this and say I’m not doing this because it doesn’t maintain the integrity of my research.

Yup. It was totally new for me. I had no idea what to do. I thanked the reviewer and said they gave me a lot to think about. To address the comments I outlined the changes I made. And to the reviewer number two I said, “I was unsure how to respond to the comments from reviewer number two, as it seemed to me like he or she wanted me to write an entirely different article than the one I submitted.” I said, “I do intend to continue my research in the broader area of these areas, but this is not the focus of this particular manuscript.” So, that’s how I addressed it. But yeah that was the first time that anybody had told me that it’s okay for you to push back on this and say I’m not doing this because it doesn’t maintain the integrity of my research. That was powerful to learn and I think and that’s something that I try to pass on when I’m asked to informally review others’ writings. If you get comments back that you disagree with, you are allowed to say that you disagree with those comments.

Emily: It’s such a black box. We’re thrown in as librarians into these academic positions, and we have these promotion and tenure requirements, and we don’t know what we’re doing when it comes to this stuff. Wow do we learn?   

There was basically no mentorship in my institution. CUNY is a huge place, there’s a lot of colleges; Hunter has a pretty big library. When I first started I asked if there were mentorship opportunities for people who are new in the tenure track. And there wasn’t really so I started a committee of junior faculty who get together and talk about their research, but it’s similar to the peer-review process. The tenure process is so wonky and so ill-defined at most campuses; each one has different requirements and each chief librarian has a little bit different advice. Nobody was willing to state “here’s what you need to do” because they don’t want the liability. They want to have flexibility and so nobody is really willing to mentor in a formal way because they don’t also want to be in trouble over telling people the wrong thing. So, there’s this real culture of fear around it, which I think sucks, but at least I had some colleagues who were willing to talk to me about it and have these informal opportunities to talk about my research.

Looking outside the library I had other faculty colleagues that I spoke with. The chief librarian at the time gave me some good advice. He was not an academic, he came from the corporate world and so it was interesting to hear his perspective on it. I think so much in academia, there’s so much gate keeping involved, you have to know the rules before you even begin, you have to just figure out the rules and no one is going to tell you what the rules are until you screw up and then they’re like, “oh by the way you could have done it this way.” Well, you could’ve told me that in the beginning and set me up to succeed rather than setting me up to fail. I just think it’s unfair. So, I’ve tried to pay forward the lessons that I’ve learned when other people come and ask me about things.

Emily: So, how did this experience compare with other experiences you’ve had in peer review? Has it informed how you serve as a referee? Has it in formed how you submit or how you write?

But for me there’s value in the fact that somebody put in the work to submit this thing and so I need to take that in the spirit in which it was submitted…

I’ve only refereed a handful of times but it’s definitely had an impact. I feel as a reviewer you get this feeling of, “I have the power to make or break this thing,” and it can be a power trip. Knowing what I know about it now, my approach was to be very careful with the work because it was somebody’s very hard work. Just think about the way you word things. It’s like when you’re talking to people who you care about, you don’t speak harshly to them, you try to gently encourage them in a particular direction. It’s really informed that for me that I don’t make mean comments. I feel like some reviewers take it as an opportunity to be mean. I don’t know why, I guess it’s a power trip or human nature—sometimes people need somebody to take stuff out on. But for me there’s value in the fact that somebody put in the work to submit this thing and so I need to take that in the spirit in which it was submitted and give thoughtful feedback that would help them rather than tear them down. I teach a one-credit class for the library and it’s helped me giving student feedback as well. Students are trying to do their best. So don’t be mean, just tell them what you want them to do, and tell them when they did it right, and tell them when they didn’t, and offer them an opportunity to revise their work. I think it’s pretty simple.

In terms of submitting my work, everything else I’ve authored has been collaborative and so that’s been a totally different process. I think the reason this article about the Puerto Rican prints was so personal was because it was my only solo-authored piece. When you get rejected for a group effort there’s other people to experience it with. “Okay well clearly that wasn’t a good fit, so let’s figure out how to move on.” It doesn’t feel as personal to me; I don’t care about that research as much. I haven’t solo authored anything since this, if that’s any indication of how rough it was. I would love to, but I haven’t had the time. I’m working on another big archival research project now, sort of slowly, obviously during COVID I can’t get into the archives, but I’m a little nervous about that one, too, because it feels awfully personal because I care about this research as well.

Emily: You said that you didn’t care as much about your other research projects, but I’m going to interpret that a little bit differently so please tell me if I’m wrong. I’m going to posit that maybe when you’ve had rejection of works that were collaboratively authored it’s that there was a sense of camaraderie in the rejection and that there was a space for processing that.

I think that’s a fair way to say it and that definitely was the feeling. Saying that I didn’t care about it as much feels flippant, but quite frankly it’s true to a point; I’m not as invested in academic library literature as I was in this one particular research study. I have to publish for tenure, and I got tenure, and I’m really pleased with that, but I know that the stuff that I have written—aside from this one bit of research that I did—is not going to be as impactful on the world in any meaningful way as. That’s why I say I didn’t care about it as much, because I wasn’t as personally invested in it. There are things, obviously, that I have cared about since then and that I do still care about. Actually, now that I think about it there was one chunk of my research I do care very deeply about as well, but that was also collaborative and so when that stuff would get rejected, we would come back to the team and say, “okay so how can we make this better because we do think this needs to be told.” But yeah, I guess that’s the danger of writing about something that you really care about. It’s not that I think the other work is bad work, it’s just that I don’t think it’s necessarily as impactful or important.

“…this was a thing that really meant a lot to me and so it really hurt a lot when I got these comments that were just basically that it was terrible.”

This is the first in a series of three sharing Sarah Ward’s story. Sarah and I spoke in July 2020. The air was hot, the pandemic was raging, and we both agreed we were happy to be discussing something other than COVID-19 or strategies for remote teaching and learning.

Sarah Ward

Sarah Ward

Pronouns: She/her/hers

Position: Associate Professor, Outreach Librarian

Institution: Hunter College – City University of New York (CUNY)

Fun Fact: Sarah’s undergraduate degree is in theatre production and design with a minor in dance. Her first career was as a costume technician and wardrobe person in various theatre, dance, and opera companies in Chicago. Librarianship is her second career.

Sarah Ward is a librarian at Hunter College in New York City, where she works as Outreach Reference and Instruction Librarian. Prior to coming to Hunter, Sarah worked as an art librarian first at the Met, and then at the Parsons School for Design. When she responded to my call for interviewees, Sarah let me know she had a very particular story in mind that she wanted to tell. The story she shared is connected to publishing and research in art librarianship. It is a story about a project Sarah says is “…the work that I’m actually most proud of and had the most lasting impact.”

When I started working at my current job at Hunter College the library had their artwork hanging in the wall of the library, which I thought was really interesting. And nobody really knew much about it. We had folks who worked at the library had been there since the 80s and they were like, “they’ve been around since then.” So, I started poking around to find out more about this collection and it turned out that it was a fairly significant collection of prints from the 50s through the 80s by Puerto Rican artists. This was really a hot time of printmaking in Puerto Rico—part of it was Puerto Rican nationalists and rallying around the people who were fighting for Puerto Rican independence. The collection had different types of prints—linocuts and silk screens and all different kinds of stuff. They ended up digging through the archives at the college and people kept just finding things in storage rooms and under broken glass and so they would send me a picture and say, “is this part of the collection too?” And I’m like, “yes, it is,” and so I started gathering all this stuff. The research for this took about five years and I actually had two interns helping me at one point and inventorying the collection, learning about the collection. So I pieced together the whole story about this collection of art, which was not insignificant; the history of Puerto Ricans in New York is an important part of the city’s history and continues to be so. At Hunter we have the Center for Puerto Rican Studies and they have an archive in the library, so I started looking around at their collections and discovered they had some of these artists’ works. I ended up drawing enough attention to the collection that we ended up transferring the it to their library and archives for preservation and digitization and they have been keeping them ever since. They have used them in exhibitions about these different art makers and for lots of different things. So the works were supposed to be public art, and they are now being returned to the public because they’re digitizing them and making the finding aids available, which is really cool and that’s what I ultimately hoped would happen.

The way that the art got into the college was that the entire undergraduate student government, in about 1983, started using their money for things that were unsanctioned. They were going out to restaurants and renting cars and all kinds of stuff. And one of the officers went to Puerto Rico and one of the artist Lorenzo Homar’s prints was autographed and signed over to the students of Hunter College. The student officer was Puerto Rican and he went to the island, but he used this money to pay for his trip to Puerto Rico to stay at a hotel and purchased a bunch of artwork with undergraduate student government funds. I started off thinking misappropriation of funds, but then the more I looked into it and the more I thought about the time in history that it happened, I feel like this was an act of defiance and an act of representation. He would’ve been in the era of the children of the young lords and the people who were pushing for healthcare reform and the students who were occupying city college and shutting down CUNY and saying CUNY needs to be free, we need to open admissions. There was a big push in the 60s and 70s for open admissions and for Black and Puerto Rican Studies departments. I’m making wild assumptions about his family, but I assume that in his community he would have had been around other people who were fighting for these things. And so, when he got to Hunter and saw – I need to see my people represented at the college in some way – and so he purchased this artwork as a way to get it there.

So, that’s my speculation. And I know it’s like a big overreach but I do feel like it’s appropriately timed and that perhaps there was a bit of student activism there, rather than just – ‘I’m going to spend money on something that I like.’ It was a fairly well-curated collection for an undergraduate student to be able to pull together this collection of prints by major Puerto Rican artists, but also by people who trained at the workshops of those artists. So it was this nice story through history and art making. Anyway, I thought this was really interesting and also sort of embarrassing for the college, because it betrayed the fact that the college was still not adequately representing students of color who comprise the majority of the student body. They also totally overlooked this misappropriation of funding. Then there was this big crack down, and a big investigation, and legal got involved, and people in the budget office got fired, and so it was this embarrassing thing for the college, too. Anyway, I digress.

Anyway, that’s the research that happened and I thought it was significant and interesting. So I reached out to the editor of Art Documentation, and she’s like, “this sounds like a great story. Please submit your article.” So I submitted it and the first reviewer comments were brutal – so painful. This had been five years of my life and I was feeling very invested in it. It was also my first solo-authored article—everything else I had done had been collaborative—so this was a thing that really meant a lot to me and so it really hurt a lot when I got these comments that were just basically that it was terrible. And the editor was so kind because she said the reviewers agree that it needs significant revisions, so I have to revise and resubmit, which was fine. I took the comments for what they were. Of course they were contradictory just like reviewer comments often are. Some people loved certain parts of it, and some people hated the same parts of it, and I didn’t know what to make of it.

It was a valuable learning experience for me of trying to draw meaning from contradictory statements and figuring out what was important to me to keep in the article and maintaining my integrity as a researcher.

It was a valuable learning experience for me of trying to draw meaning from contradictory statements and figuring out what was important to me to keep in the article and maintaining my integrity as a researcher. I think that especially as a relatively new researcher I was still thinking, “oh everybody else knows better than me so I should take what they say,” and change the things that I had done. And then I started thinking more. I was like, “no this is really important.” This is an important thing for me to include. So I went back and revised and I resubmitted and I got another round of reviewer comments, which were kinder and more constructive, but they still rejected the paper. I was like, “well what do I do now?” I didn’t give up, but I stopped and thought, “I don’t know where else to submit this because it’s about an art collection in a library, it’s part librarianship, it’s got all the things.” And so I paused for a little bit and just regrouped and started thinking. I ended up presenting at the REFORMA National Conference in 2015, which actually was a really good fit for this work because it was still not published and it was still a work in progress. I got some really great feedback from that presentation about the collection and about the work and I thought it was a really excellent place to have debuted the work.

And then out of the blue the editor emailed me and she said, “I haven’t stopped thinking about your article and I really want to publish it and I’m willing to work with you on it.” It was as if she was like, ‘I am the editor; I’m going to decide that this is a thing that I want to publish even if my reviewers disagree.’ I found her email the other day and here is what she said, “I was going through some art doc files this evening and came across your article. I feel like you were so close with it and I really would like to see it published. How would you feel about doing one more round of revisions based on the second group of reviewer’s comments and then submitting?” Because I made the choice not to revise again and resubmit. She was so kind and she went through the reviewer comments with me and said, “here are the things that I think you needed to change or to improve on or modify on this article in order for it to be publishable by Art Documentation.” I felt like that was such a valuable thing for someone who is new to all of this; to have an editor say, “I’m going to take a chance on this even though my expert reviewers have said no,” that she saw something in the work. And it probably me being so immersed in the archival research that I over told the story; there was too much information and so the importance of what happened got lost.

And it read like a narrative, it wasn’t formal, like what you would think of as a typical scholarly article, it read like a story because I was telling about the journey of this collection and my journey with this collection. So she helped me shape it in a way that it would be appropriate. And ultimately it got published and I was super proud of the way that it turned out. It was the research I was most proud of because it allowed a little bit of light to be shed on a piece of history that would have probably gotten lost or destroyed as our library was being renovated because they were just removing things from the walls and putting them in the closets, so the fact that I spoke up and said, “hey this is important” and I got other people involved who were experts and said, “hey this is important and we need to do something with this.”

Emily: One of the things that you started talking about early on as you were relaying this experience is how emotional it was for you because you had spent so much time working on this project. I would like to unpack that a little bit. Were the comments really personal and harsh?

Knowing what I know now, and in retrospect, of course harsh comments are going to feel personal if it’s something that you really care for.

Actually revisiting this after a while is really good because at the time it felt personal and I don’t think it was. Do you know what I mean? It felt personal because of my investment in the research, but I don’t think that they were. I think that that’s why I had such a reaction to it is that I was so invested in it I wanted everyone to love it. Knowing what I know now, and in retrospect, of course harsh comments are going to feel personal if it’s something that you really care for. It’s so funny to look back at these emails about “I found another print.” [laughs]

Emily: You got everyone onboard. It was like a treasure hunt.

Yeah. It was like a treasure hunt. That’s actually funny that you say that; that was one of the comments that somebody made in a derisive manner, “this reads like a mystery story,” and I was like, “it kind of was, so it’s okay that it read like that.” It was just funny. One of the reviewers said that it wasn’t appropriate for Art Documentation, which I disagree with. They wanted more of a connection to art libraries rather than just art and libraries. “This paper does tend to read like a newspaper article.” That’s what they said. So, it was journalistic rather than academic, which I took very harshly, but now I’m just like, yeah it really is actually and that’s fine. You know what’s funny, is I’m reading these now and they’re not that harsh but they felt really harsh at the time.

A lot of it was stylistic that they didn’t like, which I think felt more personal because it was my voice and so they were criticizing my voice essentially.

They didn’t like the informality with which I wrote it. Because it was a personal journey story, I wrote it from the first-person perspective. It felt very silted for me to say, “the researcher blah, blah, blah” because it’s me; I was doing the work. The reviewer’s said it needed to be formalized and to remove my presence from the article, which I utterly disagreed with. I was like, “I can’t remove myself from this because I am involved in it.” They didn’t like the personal narrative style. They all thought that it filled a gap in the literature, which was nice to hear. A lot of it was stylistic that they didn’t like, which I think felt more personal because it was my voice and so they were criticizing my voice essentially. Here’s one: “this article isn’t engaging enough. It’s not an engaging story and reads like a mystery.” [laughs]

Emily: When I hear people say that their work is rejected or made to be revised based on voice and style, especially if it’s not performing academic tone, that to me reeks of elitism and privilege. That is my personal reaction to those comments, and especially since you say it’s about your personal experience. It’s a phenomenological approach. It sounds like you were able to get it published, but that was only because you had a champion in the editor. Do you feel like you were still able to have your voice and your tone?

Yes. Because as I’m looking through these notes the first round of reviews was three reviewers, and that reviewer number two was the one who, of course, you focus on the ones that hurt your feelings the most, so that one is the one I focused on. And I’m looking at reviewer three’s comments and theirs were super constructive and helpful, and I actually incorporated a lot of their comments into the article eventually because they said “the style is not what we usually see in this journal but it’s appropriate for the unfolding of the story” and I was like, “okay validation.” Somebody saw the reason why I wrote this thing. And then they also gave me suggestions like, “can you connect it to this historical context to me?” Or “can you connect it to these other types of things? This is a singular event but it’s not unique; there’s other things that have happened that you can connect to this.” They gave me some very good suggestions for other sources to look into, which I ultimately did. So it’s funny to look at this now and think how I only focused on that one really negative review and this one was really helpful. It really was that the editor ended up seeing promise and taking a chance on me. And again, the second round of reviewer’s comments were better, more helpful, less harsh but still wanted substantial revisions, that I didn’t think would take the article in a direction that I wanted it to go, and so I started looking for somewhere else to publish it. So I don’t know that they were as personal as I took them, but that second reviewer really hurt my feelings. And that’s silly now that I think about it.

And the funny thing is I’ve written my share of these academically-toned articles; I’ve written plenty of research studies with other people in this academic voice and I felt like an art library journal would be a place for it, there would be a little bit more freedom to have a narrative. And I had read Art Documentation–not everything is hard-core social science-y research-y, they include other kinds of stuff as well.

Emily: Do you think that if you hadn’t heard from the editor you would have continued to pursue this?

I don’t know. That’s a very good question because I see now that I tried another journal and they also said out of scope for the journal, like they were interested in the story but they said it’s not really in their scope. I just didn’t know where else I could publish it. I feel like in library science literature research librarians who are publishing things like that, we often think, “well wouldn’t it be great to publish outside the discipline?” But then you get into –I also am not a historian and so I would feel really outside my realm in a history journal or in an art history journal because it’s not my field. I know enough to guide students in research in those areas, but I’m not an art historian and I would feel like it would get torn to pieces if I tried to publish it somewhere else. And so I felt stuck because I didn’t know where else it would fit because it’s at that intersection of art and history and libraries so where else do you put it?

I was so disappointed because I had worked for so long on this and it was so important to me I think I ultimately – presenting at REFORMA really helped me see the value in the research for the impact that it had rather than for my tenure portfolio– and that was eye-opening to me. When I got into academia I was so idealistic at the beginning that I thought I don’t want to just publish for other librarians, I want to have my work have an impact and most of what I published isn’t read by anybody but librarians. Fine. Whatever. But this I felt like it was important enough culturally and historically that I was glad, even if it couldn’t be published, that it had made a difference in preserving a bit of cultural heritage that was important to New York. I feel like at least at the very least the collection and the representation of these artists was not taken care of. And like it’s in the hands of people who will care for it such their library archives are something that they do so much great work with Puerto Rican history and featuring like Puerto Rican lives in New York and stuff like that. Anyway, it’s a good thing that it happened even if it never got published.

“I am sure that there are many voices that are being excluded because they can’t surmount some invisible hurdle that we don’t necessarily have justification for.” On power and privilege

This post is the fourth in a series of four sharing Amanda Nichols Hess’s story with publishing, peer review, and thoughts on open peer review. (Her first chapter, “Did I just get a dissertation that is completely worthless…” discusses feelings of imposter syndrome both as a newer writer, and as an experienced one. Her second chapter, “I think in seeing other people’s work it’s reinforced in me what I find as an author to be valuable” elucidates her approach to refereeing. Her third chapter, “Peer review should be a formative feedback process”: collaboration and mentoring, discusses her view of peer review as mentorship.)

Amanda and I spoke in late July, 2020. She was the first person I interviewed this summer, after I re-opened a call for interviews. In this concluding chapter, Amanda reflects on power and privilege as it is reflected in peer review.

Amanda Nichols Hess

Amanda Nichols Hess

Pronouns: She/her/hers

Position: Coordinator of Instruction and Research Help/ Liaison Librarian for the School of Education & Human Services

Institution: Oakland University Libraries

Fun Fact: Today, November 3, 2020, Amanda will be working, for the first time ever, as a poll worker. She’ll be working at the county level to count absentee ballots, which will be critical in her battleground state.

Emily: You said something I want to circle back to. You said something about how your colleague mentioned that your writing style was more – you didn’t use the word informal, but you did say that it was just reversing sentences. And this is something that I think about a lot. I feel like in academic writing we have to perform with this privileged way of talking and I think it very much does come back to power and privilege. There’s one thing when a sentence isn’t clear and you are struggling with clarity. But the voice, to me that’s very frustrating for me to hear that it’s almost like you had to perform for a particular kind of writing style in order to get your work noticed or seen and published That, to me, is also back to that elitism of what we see in the top journals. And then who do we exclude when we do that? So I don’t know if you have any thoughts or reactions to that.

If I was a new librarian, a librarian who perhaps is in an underrepresented group who has heard their whole life that they “don’t write in an academic style” or they don’t have the voice that one is supposed to use or they’re not using correct grammar or whatever, I could see that having a much more detrimental effect or more long-lasting effect.

I do think there’s a distinction—whether that should be the case or not is arguable. If we talk about elitism and who’s being excluded by voice and tone and narrative style, I mean, I am a cis white woman in a very female-dominated profession. So you want to talk about privilege. I’m at one end of the spectrum, right? The very privileged end. If I’m being excluded because of my occasional way of flipping my sentences around, and that’s what people see as serious grammatical issues throughout an article, then I am sure that there are many voices that are being excluded because they can’t surmount some invisible hurdle that we don’t necessarily have justification for, or we don’t necessarily explain or state clearly. An idea that, “this is the way the writing should be done and this is why and this is the value that it has.” Because I think some of it’s just stylistic. It truly is stylistic. Maybe my peer reviewer who said that would have something different to say on the matter, but I agree with you. And for me as someone who, at that point had published a good amount of articles, had experience, it was something that I was able to rebound from. I could go into a toolbox and say, “okay, here are the personal reflective tools that I know work for me, so I’m going to take a step back. I’m going to take all their notes out and put it in a separate document where I’m compiling. Then I can see the comments that each reviewer had in summary about different sections, and I won’t see it as a person is attacking me in this way or criticizing me in that way. I’m going to talk to colleagues.” But these are tools that I have developed over a period of time. If I was a new librarian, a librarian who perhaps is in an underrepresented group who has heard their whole life that they “don’t write in an academic style” or they don’t have the voice that one is supposed to use or they’re not using correct grammar or whatever,  I could see that having a much more detrimental effect or more long-lasting effect. It wouldn’t make someone think, “I can publish peer-reviewed scholarship” or “I belong in this profession” or “I belong in this kind of a job. I can make it work in a position where I’m expected to do this kind of a thing.” Yeah, I agree with you and I can absolutely see the power dynamics there are maybe invisible to some, but probably just those who have privilege like me and like the reviewers of various journals.

Emily: I don’t have any data to back this up, but a way I’ve started thinking about it is that academic writing is a performance of whiteness—in our profession maybe not as much, but maybe—if you look at the gender breakdown in academic librarianship there are certainly more male-identified people in academic librarianship than in public librarianship. At least the last time I looked at the data. But I think it’s also a performance of maleness. So I feel like academic writing is a performance of white maleness, whiteness and maleness. I am also a cis whitish woman. There’s that Jewishness in there which is what I call ‘provisional whiteness.’ You’re only white until someone says you’re not white. But I did grow up with parents who are academics and so I’ve seen people performing this my entire life. I can’t imagine what it’s like to be someone who — regardless of race or ethnicity or gender identity– just didn’t see it performed their whole life. That is a barrier in and of itself and then you add all of the other things. Yes. We’re in agreement there.

Can you tell me what you know about open peer review and if you have an opinion of it, and where you see it fitting into our community?

I don’t know much about open peer review other than that I can figure out what open peer review is by context–knowing what peer review is. I think in a lot of academia in general — not just librarianship, because I’ve done peer review for different conferences whether it’s a university conference on teaching and learning or other conferences where they’re looking for peer reviewers for their submissions — I just think there’s this attitude sometimes that there’s a malicious intent that’s assumed and it’s not always the case. Now, in other disciplines maybe that is the case, but I actually feel in librarianship we’re so conscientious and we’re so, “Oh, I want to make sure there are no conflicts of interest here.” I would be totally fine if, let’s say tomorrow I submit to CIL and they say, “Your peer reviewers are this person and this person.” Or, “This person or this person, this article was written by Amanda Hess and she works at Oakland University.” I mean, the fallacy with peer review often is, for example when I talked about that one article that I reviewed that was all over the place, the first one that I did that helped me really think more critically about how a peer reviewer would look at my article. I was like okay, you’ve blinded this but you haven’t really blinded this.

All of the things that you’re saying about your institution I can figure out what your institution is. I can figure out who you all are. I’m a librarian. I know how to search the internet, okay? I can figure this out. So I think it’s like saying, “oh, I wrote my dissertation on Midwestern University.” We’re like yeah, I can see where you work so I know that’s where you’re doing your research. It’s this layer of distance that we think gives us some unbiased writing or some impartiality and that doesn’t exist. It doesn’t exist.

I just think if we said, “Okay Amanda, you’re reviewing this article by Emily Ford and here’s the article as it was written.” Then I can say, “Oh, let me see what she’s really talking about at Portland State or what else she has written on this topic.” And I could say “Oh, this gives me a fuller picture of this person’s work” and I could maybe give more targeted feedback, more beneficial feedback or I could understand her scholarship in context. So maybe that’s a one-way blind, so maybe you don’t know who I am. But then I think if you then know who I am, and I know who you are then we can have a conversation about it. So, I could say, “Here are my comments. Bob and I’m cc’ing Emily. Emily and Bob, if you guys want to talk together about it or Emily, if you want to talk with me about it you have what I have shared with him. He has what I’ve shared with you.” Everyone is looped in and I understand that maybe that would create more work for the reviewer but I also think it’s not necessarily not meaningful work. I think for me as someone who is a peer reviewer the value I get is thinking, “Okay, I’m helping these people.” Not every peer reviewer is like that, but I almost feel like if you opened it up and you said yeah, this might be a little bit more work for you if you have to be accountable to somebody else when you’re like, “Hey Emily, that was uncalled for and mean that comment that you made about me. That wasn’t constructive. That was just criticism.” I think you have to be more cognizant about what you say and you also have to be willing to stand behind your comments. Then you have to say, “Yes, I stand by it, and here’s why.”

I know that there are issues. I know that sometimes I’m very Pollyanna, and I’m seeing this as “this could be beneficial and I’m sure say we’re all conscientious, ethical librarians.” We’re not all conscientious, ethical librarians. There’s always baggage and issues — this person did this to my friend, or this person is this way, or this is what I’ve heard about this institution. I mean, I know there are issues and I know that there would be conflicts, but I think open peer review could really help address some of the issues that I think people complain about librarianship. Like,“We don’t have enough rigorous scholarship. It’s all these case studies or there’s not this real deep research like you see in other social science disciplines.” The way to build capacity, I think, is through processes that help people learn and grow and develop. And if you don’t want to build that up then you’ve got to value other forms of scholarship. If you want something different, you’ve got to do something different. Or if you don’t want something different, then you’ve got to accept what it is. I would be absolutely willing to submit to a peer review process that was open, and I would absolutely be willing to be an open peer reviewer, for people to know Amanda Hess is the one who reviewed my article. I think that would help me be probably more constructive in my comments even though I try and always be constructive. I could absolutely learn from that process, too, as a peer reviewer but also as someone who’s being peer reviewed.

Emily: I mean there’s so much behind that too. You have tenure now so it’s different for you. You mentioned that with something else where you could put the articles away for a while and think about them because you felt safe. There’s so much of that. I guess in my experience open peer review has been the more robust feedback. I can have a conversation with somebody. And I appreciate that. Certainly there’s issues with it as well. The fallacy of opacity. It’s not really true. I guess we’re coming up on time but I don’t know if you had any other things you wanted to say or thoughts….

I don’t think so. As you can see, I’m not shy. Maybe I’m a reviewer because I’m like, “I’ll share my opinion!” I joke with all my colleagues when they ask, “Will you look at my article?” I’m like “Yes, you know I love to give you my opinion on everything.” Even if you don’t ask for it, I’ll be like here, let me tell you what I think.

Emily: That’s also protected by tenure.

Yeah. You better integrate my feedback if you want a good letter of evaluation. [laughing] No, I think it’s really interesting, and I think the way that you are talking about sharing this information about peer review in a way that is like, “This is who said this. This is what this person’s experiences have been.” I think that’s really intriguing to me and, I think, really powerful.

Emily: I think one of the things that I am a little bit concerned about with this is that I’m opening up these conversations. I want to put your name on it. I want to be able to say “this is Amanda. Amanda works at this place.” I’m wondering if the fact that you’re willing to share your identity is because you have the privilege of all of your privileges. Your whiteness, your cis-ness, you’re a woman in a woman-dominated profession, your tenure. I’m struggling with that a little bit because my ideals of “well, we should all share openly!” Who am I excluding by doing that? I guess my hope is that by seeing people share openly and honestly that people who feel maybe less safe because of tenure status, because of being a minority in our profession or whatever, my hope is that they also feel comfortable sharing. I don’t know, we’ll see. I don’t know if I’m going about it the right way. I don’t know. I can change it up later.

I am comfortable talking about a lot of these things because I have tenure…

Well I absolutely agree. I think part of the reason that I am willing to share my name, where I work – I mean even when I’m talking about I’m a peer reviewer for this place, I mean, you could figure it out. I think it’s on my bio page. I am comfortable talking about a lot of these things because I have tenure, because I have gotten past the imposter syndrome, research insecurity kinds of feelings that I’ve had. But yes, five years ago even, certainly nine years ago I would not have been able to say, “Yes, please share my name or share my information. I don’t care.” I agree that there are probably people whose voices maybe are not being shared, but my hope is that people with privilege who share and who put themselves out there in everything. People with privilege need to put themselves out there to make it safer for people without that privilege. And to check that privilege and to hopefully diminish some of the privileges that come with various statuses and various identities.

“Peer review should be a formative feedback process”: collaboration and mentoring

This post is the third in a series of four sharing Amanda Nichols Hess’s story with publishing, peer review, and thoughts on open peer review. (Her first chapter, “Did I just get a dissertation that is completely worthless…” discusses feelings of imposter syndrome both as a newer writer, and as an experienced one. Her second chapter, “I think in seeing other people’s work it’s reinforced in me what I find as an author to be valuable” elucidates her approach to refereeing.)

Amanda and I spoke in late July, 2020. She was the first person I interviewed this summer, after I re-opened a call for interviews. In this chapter of her story, Amanda reflects on peer review as a collaborative process that should be approached as a mentoring relationship.

Amanda Nichols Hess

Amanda Nichols Hess

Pronouns: She/her/hers

Position: Coordinator of Instruction and Research Help/ Liaison Librarian for the School of Education & Human Services

Institution: Oakland University Libraries

Fun Fact: Amanda was once interviewed on the BBC about a creative nonfiction piece that focused on her Greek and Turkish family history.

What Amanda pointed out as a learning process in her previous chapters, I like to think of as part of mentoring and collaboration. She discusses early mentorships with colleagues, co-authoring articles at the beginning of her academic librarian career. But she also sees that there is learning as a referee and author, whether these are formal reviewing relationships or informal mentoring relationships with colleagues who read your work and offer supportive comments. There is also a role for Amanda to play as someone who acts as a mentor.

I think the emotions that I have felt around peer review—the fear, the insecurity, the frustration, the lack of clarity.  The lack of –I don’t even want to say transparency because part of the process is that it’s blind—but just the lack of understanding of the process sometimes, especially as someone who didn’t have experiences as a graduate student, as a researcher assistant who was working with a faculty member on publishing or working on research. I just think there’s so much assumed knowledge that we just think “Oh, well people think they know how this process works” and I don’t think I did. Even in seeing some of my colleagues who were hired after me, whom I’ve worked with, or who are working through our tenure and promotion process, I think it’s the same for them. And so I think there’s this level of secrecy, or I think sometimes it can be kind of scary to people or intimidating because there’s this unknown. I feel like I’m happy to share my experiences on both sides of the table to maybe make that a little less unknown or a little more known to people.

I feel like that would have helped me when I was first starting. I had some mentors, informal mentors or people who were willing to help me or work with me or talk with me about the process. I’ve published in different journals, some of which are more along the lines of, “We view peer review as a way to help authors improve. We try and work with authors as much as we can to get them to a point where their article can be published,” and some that use peer review as a real filtering tool to say, “This is our submission rate, but then we only publish a small percentage of those.” I really have found the former to be a lot more valuable than the latter, but I understand every publication and every editor and every editorial board has different motivations.

Emily: I think it’s really interesting that you mentioned the role of the reader. When I initially was doing my intake survey for interview participants, I asked folks how they identified in relationship to the literature. Reader was an option, but very few people marked that box.

The experiences that have stuck with you as more positive are those where peer review is approached as developmental or a collaboration, where it’s like “we really want to support you to get your work out there because we see something good in it.” You contrasted that with journals where peer review is treated as the filter. Maybe there are people that do want to help you with it, but the approach of that help is different. So in your view what should peer review do and what should it actually be for?

I really think of peer review — even when it’s blind — I can’t think of a better word than mentorship.

So I guess I skew more toward peer review should be a formative feedback process. I think for those journals where it’s a weeding or filtering out process, I feel like it’s a summative evaluation. It’s a yes or no. Or, of course, there are revisions. “Give us some revisions and then we’ll think about it again, we’ll reconsider.” I’ve had some of those, too, where it’s like, “we request revisions.” I do the revisions and then they say, “No, we don’t think so.” It seems like maybe you could have told me that in the first place if that’s where you were going to go. I really think of peer review — even when it’s blind — I can’t think of a better word than mentorship. Because I feel like mentorship connotes this idea of, “I’m an older, sage, wise, experienced person with a more junior, less experienced learner.” I think as a peer reviewer, as an editor, as an author, as a reader I’m always learning.

No matter what side of the equation I’m on or what role I’m in. I think that a peer review process should be a learning opportunity, because even if an article is outside of the scope of a publication they can just say, “yeah, this is outside of scope.” But I even think then there are opportunities to say, “these kinds of resources or these kinds of publications may be a good place for an article on this topic.” That says nothing about whether there are issues with the writing or how the research is done or the contents of the actual article necessarily. When peer review is most valuable to me, it is something that allows people to learn and grow and develop. It doesn’t necessarily have to be toward some magical ideal of, “This is what an academic librarian who publishes peer review scholarship, this is what their writing looks like.” Because I feel like here are all kinds of places to share one’s work. It doesn’t have to necessarily be through scholarly, peer-reviewed, top-tier publications or what we would say “top tier” whatever that means. I think at least in our library, and I think in my institution, and I think in other conversations that I’ve heard or been a part of, I think we are getting to the point where peer reviewed isn’t necessarily always the most impactful or doesn’t connote the most value. When we talk about value, what are we valuing? Are we valuing that this person can write in the conventions that we ask them to write in that reinforce various power structures? Or are we saying we want someone whose work really has a community impact, or impacts our profession more generally or increases equity, diversity, and inclusion, or increases access to information, or increases access to ideas? I think I’m fortunate at my library to have a colleague who really pushes those ideas of alt-metrics, which is really not the right way to really describe it, but that idea of measuring the impact or the value of our scholarship in different ways. It doesn’t have to be a peer-reviewed journal that’s behind a pay wall. For example, it’s interesting that we’re talking about this today because I recently – I think it was in 2016 – I published an article. I published an article in Communications in Information Literacy with a colleague in 2016 or 2017, and that’s an open access publication. I have a Google Scholar citation alert set up, and I get so many notifications that dissertations, articles, publications from all over the place are citing that piece because people can find it. People all across the world and at all different kinds of institutions. It’s funny, my sister set up the same alert so she’ll send me emails like, “Why are you being cited by this Ukrainian article?” I feel like at a very, very fundamental basic level that, to me, is a case for more open information because I can say, “Oh yeah, this article, which was published—in my mind—in a very high quality journal that’s open, it can have much more of an impact because people are able to find it and their institutions don’t have to maintain subscriptions.”

I’m getting way into the weeds now. I’m taking about something completely different, but I feel like peer review should be a tool that helps us advance, progress, move forward our profession. The professionals within the profession helps all of us to develop, whether that’s someone on the editorial side who says, “Well, this is how I think writing should be.” That person should be able to learn and take a step back and check their privilege and say, “is that really true? Do I understand what this person is saying? Am I being nitpicky?” And in turn, authors should be able to say, “Okay, well I’m learning that this place is really focused on helping growth and development and this is the place where I want to be publishing, not this place where they’re just worried about their acceptance rates or their impact factor” or whatever.

Peer-reviewed might not even be the best thing depending on the topic, depending on the kinds of scholarship being shared or the kinds of information being shared.

But I think it goes up a level to our profession or leaders in our profession having an understanding and knowledge of and acceptance that peer-reviewed isn’t the only thing. Peer-reviewed might not even be the best thing depending on the topic, depending on the kinds of scholarship being shared or the kinds of information being shared. And that other ideas and other ways of sharing knowledge and information have value. They just don’t fit into our established power structures. I know at our institution I think we have such a chip on our shoulders as librarians about the master’s as the terminal degree. We don’t know how we’re doing scholarship. We don’t produce the same kind of scholarship that other people do. We don’t get the million-dollar grants that other people get, that we’re so focused on peer review that I think other ways of disseminating ideas sometimes get shortchanged. We have to go through the faculty review and promotion board, so we’ve got to make sure that we are up to snuff with everybody else. I think we are changing in some ways but I think that’s in part because we have people who are in digital humanities and doing things in GitHub and they’re like, “This should be considered valuable. This is my scholarly output.” These things fall outside of peer review but they still have a lot of impact and value and really advance both librarianship but also other disciplines as well. I don’t know if that really answers your question.

Emily: I think it is peer review when you sit down with your colleagues informally. One of the things that in my work with peer review and researching it—and I am an advocate for opening up peer review to be much more transparent—it’s those conversations that, for me, have always been so fruitful. And it’s the relationships with journals where their aim is to help you improve your work where it’s more collaborative. I guess that’s my approach. That’s what makes our work better, is being able to have that honest dialogue and with someone you know-or maybe not even know-but someone you trust and it can be a more human experience than the robots with no identity.

The last chapter of Amanda’s story will be published in the coming weeks.

“I think in seeing other people’s work, it’s reinforced in me what I find as an author to be valuable”- Reflections on refereeing

This post is the second in a series of four sharing Amanda Nichols Hess’s story with publishing, peer review, and thoughts on open peer review. (Her first chapter, “Did I just get a dissertation that is completely worthless…” discusses feelings of imposter syndrome both as a newer writer, and as an experienced one.)

Amanda and I spoke in late July, 2020. She was the first person I interviewed this summer, after I re-opened a call for interviews. In this chapter of her story, Amanda reflects on her experiences serving as a referee, and how it has improved her own scholarly writing.

Amanda Nichols Hess

Amanda Nichols Hess

Pronouns: She/her/hers

Position: Coordinator of Instruction and Research Help/ Liaison Librarian for the School of Education & Human Services

Institution: Oakland University Libraries

Fun Fact: Amanda has two dogs, George and Gracie, who she dresses in costumes related to their names on Halloween.

After Amanda reviewed the transcript of our discussion, she said she felt like the reflective process was an important component to highlight, no matter what role you find yourself in. I agree. Amanda’s experiences with imposter syndrome show how she has used personal reflection as an author, but she has also had experience with reflection as a referee.

I remember the first submission that I reviewed, I was really struck by the lack of organization. As I was reading it, I thought, “Oh my gosh, this reads like this was an internal report that they’ve cobbled pieces together because someone said, ‘hey, you should share this.’” I also remember thinking, “This seems like something that’s been coauthored by a bunch of people – it has really good information, but it’s too all over the place and has no organization.” For my own practices as an author, I definitely thought, “this is really useful to see.” I remember at the time thinking that I could see myself writing something like this, so I needed to remember some of these things that I observed. I know the difficulties of being the first author and trying to make a consistent narrative voice throughout the paper or throughout an article, or of being a second or third or fourth author and being okay with the first author making it sound how they wanted it to sound. But I was just really struck by the lack of clarity and voice, and I really felt like it really took away from, or made it difficult for me to really get to, the really important contents.

I also think it’s like a three-headed monster. I’m a reader, I’m a reviewer, and I’m an author.

They had really good ideas and it was really meaningful, but I thought there were barriers that are in the way of that. And some of them were purely formatting. Like I said, “Okay, well if you can have a structure of headings or a structure of organization that helps me to say – as a reader – I’m going to jump to this section.” I had an editor hat, but also a reader hat, because whenever I’m a referee, I take the view of, “Okay, if I’m reading this article when it comes out, what am I going to think? What’s going to be difficult for me? Or what’s going to really stand out for me? Or what am I going to want more of?” So maybe it’s a duality, but I also think it’s like a three-headed monster. I’m a reader, I’m a reviewer, and I’m an author.

I would say I definitely bring my quirks and tendencies and preferences as an author to my refereeing or my editing. I think about what I appreciate from reviewers as an author, and I benefit from comments that are focused and concrete. I hate the comments of, “There’s a lot of literature on this topic out there” with no examples. It’s like you want me to go on a fishing expedition. Even if it’s an example or two – I reviewed an article or a submission recently and I thought, “oh, I have seen these couple of articles that might be useful for this author to consider,” so I included the citations. If I’m an author and I’m already working on trying to get a piece polished and published, and I have a full time job,  and I probably have a life outside of this, too, I cannot just pour through all of the literature ever written to try and fulfill some reviewer’s specific need that they really want, which is maybe that they really want me to cite them, but they don’t want to suggest it directly.

I think about myself as a reader.

I feel like this is how my head is when I’m reading something that I’m peer reviewing, or something that I’m blind peer reviewing, or something that I’m looking at for a colleague. I think about myself as a reader. What do I want to see? What really strikes a chord in me? What do I want to see more of? What is less interesting to me or what do I feel like gets too much in the weeds? What do I think could bring more clarity to their ideas and highlight those really important ideas? What do I think is maybe getting in the way of some of that clarity? And often that’s a heading structure. I know I’ve said headings, like, half a dozen times now. I love headings, obviously.

I also think that I go into reviewing any article that I read – whether it’s a colleague’s, whether it’s a submission to the journal where I’m a peer review – with the attitude of, “This is a good article. This person is a good writer. This is worthwhile. There is a place for this in the literature, whether it’s in the exact journal that they want to submit to, or whether they’re not sure where it should go.” I really view manuscripts in a positive light. I’m not someone who has a grading mindset, who’s trying to dock points or that the author has to earn something. I go in with an open mind because I also think if it’s coming through a journal that I’m working as a peer reviewer for, it’s already passed various markers of the editor saying, “yes, this fits within our scope.” I’m not going to see something that’s offensive. I’m not going to see something, I would hope, that would raise my hackles or make me feel like this is unethical or anything like that. Again, I think part of that is the fact that I really trust the editors and the editorial board of the journal that I’m a peer reviewer for. That may not be the case everywhere.

I think in seeing other people’s work, it’s reinforced in me what I find as an author to be valuable in getting things really ready for publication when I submit them.

When I think about being an author and giving peer review feedback, I really try and think, “how would I want to hear this if I was the author of this article?” I just try and be an empathetic person. I try and be kind while still being constructive. I really try and point out what’s working really well, especially if there’s some criticism or constructive feedback to give. I try and follow that hamburger approach. I want to make sure that even if I think, “Okay, this article is a hot mess” or “This article needs a lot of work,” there are always good things to build on or positive things to say. Comments like, “This is really interesting” or “This is where I would focus” or “I want to see more of this. I’m sure that this is there and maybe just make it more apparent to the reader.” Sometimes, I think that, as an author, you have things in your head that you think are clear and they are not necessarily clear to others. I think in seeing other people’s work, it’s reinforced in me what I find as an author to be valuable in getting things really ready for publication when I submit them. So having another set of eyes, really carefully looking through things. Going through an editorial process myself a couple of times to correct mistakes, find gaps, try and catch spelling errors. The simple stuff. It’s never perfect. I always have those anyway.

Maybe this goes back to power dynamics, but I’ve reviewed a couple of papers where there’s a page count or a word limit in the journal that I review for and I’m like, “You have widened the margins, you have shrunken the font size so that you can be to the page limit. Really, if you follow the other guidelines which are 12-point Times New Roman, one-inch margins, you’re like eight pages over.” At that point, I’m like, “Okay, authors, you really need to focus in on what’s the most important thing.” I think, again, that helps me to think about being clear as an author. Sometimes you’ve got to kill your darlings. Sometimes you’ve just got to cut out the things that are really personal to you or strike a personal chord for the interest of being clear and concise and focused for your readers. I guess I think of myself in three ways as I’m looking for a paper whether it’s mine, whether it’s a colleague’s or whether it’s for a journal it’s like author, editor and reader, potential reader.

Amanda’s story will continue in the coming weeks.

“Did I just get a dissertation that is completely worthless? Do I know what I’m doing?” – The pendulum swing of imposter syndrome

This post is the first in a series of four sharing Amanda Nichols Hess’s story with publishing, peer review, and thoughts on open peer review. Amanda and I spoke in late July, 2020. She was the first person I interviewed this summer, after I re-opened a call for interviews.

In this chapter of her story, Amanda talks about the pendulum swing of imposter syndrome, how these feelings were present for her after having an established career and history of publishing her writing.

Amanda Nichols Hess

Pronouns: She/her/hers

Amanda Nichols Hess

Position: Coordinator of Instruction and Research Help/ Liaison Librarian for the School of Education & Human Services

Institution: Oakland University Libraries

Fun Fact: Amanda’s very, very, very loud typing is infamous among coworkers and can be heard down hallways in a crowded library (when people used to work together and crowds gathered).

Amanda came to librarianship from a school librarian background and experience in the K-12 education system. After having been laid off and working in a non-librarian position, Amanda got a job at Oakland University Libraries. Because Amanda came from the K-12 system, and because she had never pursued writing and publication in her graduate education, she found the prospect of research and publishing intimidating.

I was not somebody in graduate school who had publication, writing for publication, writing scholarly articles or even professional articles on my radar at all. So when I first came to my institution…I didn’t realize what that meant, or what that would look like for me when I interviewed for this job… But the first year that I started, I had serious imposter syndrome. I was like, “I do not know what I’m doing. I don’t know how to do academic publishing kinds of writing.” … So, I was fortunate to have a couple of people who, not really formally mentored me, but were helpful to me as I got started. One was a colleague who was very experienced in writing articles and collaborating with other colleagues on scholarly writing. And then another was a former colleague who left K-12 education at the same time as I did and got a tenure-track position as an education faculty. So he dragged me along a little bit, and my work colleague helped me.

I feel like despite not knowing what I was doing, I learned as I went. And then once I got a couple of publications under my belt, I thought, “Oh, I can do this. I can figure out ways to find things that I’m doing or that I’m working on or that I’m interested in that are interesting to other librarians or to other even educators more broadly.” And so I think coming from a different environment, coming from an academic experience in my library graduate program where I was very focused on pedagogy and working on developing lesson plans, I felt very much like I had to compensate for my lack of experience or lack of knowledge in the scholarship area. I really focused heavily on that in the first couple of years of my job, and I feel like I started a ball rolling and then it just kept on going.

Emily: It sounds like your transition from school librarianship/teacher librarian to academic librarian was a little formative for you. Your success was largely in part to some good mentorship and collaborations with colleagues. And tenacity it sounds like.

[laughs] Yes, I think a healthy dose of fear, too. “I have to do this. I never want to be a disappointment.” These are deep-seated, childhood, baked-in ways of thinking. “Okay, I have to do this right. I have to send something in that’s polished, that’s well done. I don’t want to reflect poorly on my institution.” So yes, the first couple of years I was very, very aware of, “Okay, I have to make sure this is perfect before I submit it for a first pass.” I think I’ve let go of some of that now. The comments I get now are definitely not comments I would have gotten in my first couple of years where I was so focused on just trying to do it right and just trying to get something out there.

Sometimes when I see certain comments in reviewers’ feedback, it triggers that fear of imposter syndrome in me.

Sometimes when I see certain comments in reviewers’ feedback, it triggers that fear of imposter syndrome in me. I’m like, “Oh, what are they saying?” I have found effective strategies for my own mental processing on how to separate the content of the review comments and the emotion that I feel, or the knee-jerk reaction that I feel when it feels more critical than constructive. But it can be hard, I think, when you are not having a dialogue with somebody, when you’re just seeing words in an email, or words in a feedback system or on paper. You think, “But don’t they know how hard I worked on this?” and “Don’t they know that I considered that?” No, they don’t know any of that. So I figured out how I can manage separating the — very meaningful — constructive feedback with what feels like personal criticism, which it is not. That’s my own issue. [laughing]

If I think about my most recent work that I submitted for peer review, and I think about the first attempts that I made at peer review, they are different but they’re maybe not what you would expect.

The first couple of articles that I submitted and wrote weren’t analyzing big sets of data. I called them case studies, but they weren’t classical or traditional case studies. It was like, I’m going to describe what we did at our institution and back it up with research and literature. And when I submitted, I was very fearful.  I got a response back that was, “Yep, we accept your article.” There were no corrections, no changes needed. And I was like, “Well, this is easy!” I mean, it wasn’t easy because I really honed it and had people look at it and I was very critical on my own work of how am I going to submit this for publication…But the first thing that I wrote and submitted on my own for peer review, it was just the smoothest process probably I’ve ever experienced.

That was the case with my solo written articles the first couple of times. And then I feel like I settled into more of a groove of: have an article written, you’d submit it to a journal and they’d say, “Oh, this is really out of scope for us,” or they’d request revisions. But then more recently I had this very large dissertation, mixed methods, so I had interviews and some quantitative data. I took the interviews and more of the narrative data and was able to get a book out of it, but then I have this quantitative data that I was like, “Well, I think there’s meaning here” but it just didn’t work in a book. I was like, “Well, maybe I can make some articles out of this.” And so I’m riding high. At this point I think, “I actually know what I’m doing in research. I’ve taken research methods classes. I’ve written a dissertation. I’ve done all this. I can just clean it up a little bit and submit it.”

Well, first of all that wasn’t the case. I had to do some legwork, more legwork than I expected. I was like, “Oh, I’m redoing all of this work. I’m redoing this lit review.” So it was more effort than I had thought. I sent out inquiries to editors of two journals and I said, “Would you be interested in something like this? I think there’s a lot of data so it would be more than one article. I’m not sure if you would want it to be comprehensive.” Both editors said, “Oh yes, we’d be very interested.” I thought okay, great. I have real quantitative data, which is often not found in library literature and especially not on the topics that I’m researching and publishing on.. And I’m like, “Okay, great. I’m going to get some really top-tier journals to publish my work.”

And both of these articles were the ones where the reviewers were like, “This author is a terrible writer. This person does not know how to write. There are so many grammatical errors.” I’m like, “Excuse me?” They were just summarily rejected. I take that back. One journal editor was like, “Our reviewers really don’t think this fits, but I would be happy to work with you.” They were very constructive and they really tried. But at the end of the day they’re like, “We just can’t publish this.”

I felt very defeated…

I felt very defeated and I had to really put those away for a while. And when I was able to come back to the articles and to the comments, I was able to get more clarity. It’s not that they necessarily weren’t constructive, but I felt like it all just hit a little close to home.

That’s kind of personal because you’re really talking about my ability to write. I feel like at this point I’ve published enough that I feel like I can write for an academic audience. Or their comments were something to the effect of, “This person doesn’t know what they’re doing in terms of analyzing their data,” or “These statistics are wrong.” I’m like, “Did I just get a dissertation that is completely worthless? Do I know what I’m doing?” So it really led to some self-doubt and some questioning …So I gave myself a break. I took a break and I was like okay, I’m going to go back to some of the tactics that I had used in the past, or I had a colleague read a draft. I told them, “I’m really struggling with this. I think I’m too close to this. This is the feedback I’ve gotten. What do you think of this?” I asked people whose opinions I really value, whom I knew would be straight with me, who would sit down and talk with me about this.

In one case, that colleague and I talked it through and she said, “I think, for example, why they’re saying ‘this person has so many grammatical errors’ is that your writing style is more like how you speak, which actually makes it really easy to read, but it’s not really academese.” She said, “You’re flipping things around in your sentences, so if you go through and see a comma, go back around and try and change that.” I’m like, “Oh – that’s actually something I didn’t really realize I did,” but now that she pointed it out, I see a lot more now in my first drafts. So in talking with her and in looking at my articles, I could see the issues that I had. I could see this is why these were maybe rejected, and also because I was still trying to be too broad, or I was providing details in the way that really weren’t helpful or really weren’t targeted at a library audience.  I was still in that dissertation discipline, since that’s really its own thing, and that wasn’t useful. I was able to, again, finesse, revisit, and even break my research down into smaller, more manageable chunks. And when I did some of that I was able then to find a journal where an editor was willing to work with me. I got some really good constructive feedback from peer reviewers, and from there it was like the dominos fell. I was able to see more clearly these were the changes that I needed to make. These were the issues that people were having with how to understand what I was trying to say. Because it was dense. It was statistical research and I needed to put it in more practical language. I’m generally really applied in how I look at my research, and I was really not doing that. I was trying to get away with some laziness in how I was talking about my statistics and talking about how I did my analysis. And the reviewers were catching me on it. When I finally came around to talking with a colleague and finding a journal where they really were willing to work with me and they said, “We think this has value but we think it needs a lot of work,” I was able to really clearly see how I could clean up other pieces of data and present them in a way that was meaningful for other publications.

I was fortunate that I had the luxury to do that

I really struggled with the feedback that I got from the first round of reviewers and also the figuring out how to take that feedback and make it meaningful. So it really took me some time to sit with it, think about it, and even step away from it and not think about consciously, but turn it over in the back of my mind. I was fortunate that I had the luxury to do that because, at this point, I had gone through tenure and promotion at my institution, so I wasn’t as if I really had to push to try and get things out there. If I had that experience when I was still under review, I would have been a basket case. But I feel like I was able to take more risks and stick my neck out there a little bit more. I could let those things simmer and not just shelve them or throw them in the garbage can. I had the flexibility to go back to both peer review in a very formal sense of going through journals’ processes, but also talking with my own peers at my institution and saying, “Okay, where am I going wrong here? What am I missing? What’s not making sense? What doesn’t make sense to you?” I found that the ability to have a conversation with somebody across the table in our library café–when we used to be able to sit with people in the café–it was easier for them to tell me, “I’m not getting this,” and for me to say, “well, this is what I mean.” And then they could respond that, like “oh, okay, I understand it now but this is the area that I think is tripping you up.” That’s not really a peer review in the sense of what you’re probably looking at, but that kind of dialogue was just really helpful for me in surmounting the most challenging or upsetting peer review experience that I had, which was my most recent peer review experience. It’s very fresh in my mind.

Amanda’s story will continue in the coming weeks.

Why Stories?

Recently I was catching up on my podcast listening and was delighted to hear Radiolab’s Latif Nasser talking about how he finds stories. “I tell myself that there are 7.5 billion people on planet earth… and if you presume that one percent of those 7.5 billion people have those stories, there’s no way all those stories are getting told…There’s an infinity of stories all around us.” I truly believe that when we open ourselves to others’ experiences, we in turn reflect on our own. We have much to learn about ourselves by listening to others.

For years I have been fascinated by peer review and moving toward open peer review. I’ve thought about it, written about, and even helped develop some experimental open peer review processes. But I have truly come to believe that this work, while important, cannot have the same effects on others unless we sit down to listen to stories. To really listen. And really listening means that we will also reflect. And then we might take action.

But we cannot take action if the only way we can listen is confined. Allowing stories to filter into the open, where more people can listen, so more people can learn will better position us to take that action. It’s not just the sharing that’s important, it is that we openly share.

Peer review is essential to our work, but it is also imperfect. It introduces bias into the literature. Many publications lack clear reviewer guidelines, or reviewers do not use the guidelines to review. Reviewers in blinded review processes may not have accountability for their words or actions. Most peer-reviewing is hidden labor. Students may not learn how to provide peer review, nor do they learn what processes they may face as authors undergoing it. Our scholarly rewards system is broken and relies heavily on the tradition of peer review. This system introduces systemic bias. All this and more.

But we really don’t have that much evidence in LIS. We don’t know what people have experienced. We don’t know the challenges they have faced. We don’t know the wins. It will be in human experience. It will be in stories where we can discover evidence. That is why stories.

Welcome to Stories of Open

Stories of open began 3 years ago, when I sought to expand my research on open peer review. I was looking for a more involved qualitative research project that would gather and share personal experience narratives about peer review and open peer review in Library and Information Science.

As such I began to codify for myself a research project for an upcoming sabbatical. My aim with Stories of Open was always to gather and share personal experience stories. With an accepted book proposal, IRB approval, as well as a forthcoming sabbatical, I began to gather stories in the form of interviews. These stories, and their analysis using narrative inquiry methods, are forthcoming by ACRL Press.

But that is not enough. Everyone has a story to share. According to educational researcher Jeong-Hee Kim, narrative research’s purpose is to “… invite readers to a sphere of possible contact with a developing, incomplete and evolving situation, allowing them to re-think and re-evaluate their own views, prejudices, and experiences” (p. 235). Kim, Jeong-Hee. Understanding Narrative Inquiry. Thousand Oaks: SAGE Publications, 2016.

I am continuing to conduct interviews. My sincere hope is that I am able to share the transcripts of those interviews–our colleagues’ stories–on this website. The stories won’t be heavily analyzed, and the transcripts won’t be heavily edited. They’ll be offered in readable and digestible form.

My hope is that reading these stories inspires you to share your own, either here or with a colleague. My hope is that reading these stories makes you think about how our community can work to improve peer review. And my sincere hope is that the changes we collectively make to peer review will make it more just, more equitable, more inclusive, and our literature more diverse.

Please contact me if you would like to share your story or open or your experiences with peer review.