Stories of open began 3 years ago, when I sought to expand my research on open peer review. I was looking for a more involved qualitative research project that would gather and share personal experience narratives about peer review and open peer review in Library and Information Science.
As such I began to codify for myself a research project for an upcoming sabbatical. My aim with Stories of Open was always to gather and share personal experience stories. With an accepted book proposal, IRB approval, as well as a forthcoming sabbatical, I began to gather stories in the form of interviews. These stories, and their analysis using narrative inquiry methods, are forthcoming by ACRL Press.
But that is not enough. Everyone has a story to share. According to educational researcher Jeong-Hee Kim, narrative research’s purpose is to “… invite readers to a sphere of possible contact with a developing, incomplete and evolving situation, allowing them to re-think and re-evaluate their own views, prejudices, and experiences” (p. 235). Kim, Jeong-Hee. Understanding Narrative Inquiry. Thousand Oaks: SAGE Publications, 2016.
I am continuing to conduct interviews. My sincere hope is that I am able to share the transcripts of those interviews–our colleagues’ stories–on this website. The stories won’t be heavily analyzed, and the transcripts won’t be heavily edited. They’ll be offered in readable and digestible form.
My hope is that reading these stories inspires you to share your own, either here or with a colleague. My hope is that reading these stories makes you think about how our community can work to improve peer review. And my sincere hope is that the changes we collectively make to peer review will make it more just, more equitable, more inclusive, and our literature more diverse.
Please contact me if you would like to share your story or open or your experiences with peer review.
It’s been quiet around here for summer while I took two months off to drive around the country to visit friends and family I had not seen in years because of the pandemic. But I’m back at it and have plenty more stories to share with you. But before then, ACRL is hosting a webcast for #peerreviewweek21. The theme, Identity in Peer review, takes up a few chapters of Stories of Open, so it’s a perfect fit! (Need to read the OA version of the book? Find it at the bottom of the book’s description on the ALA Store.) While I, of course, think everyone should read the whole book, if you are most interested in identities, I recommend chapters: 4 – Roles of Peer Review; 5 – Dualities and Multiplicities in Peer Review; 9 – I Just Feel Like This Makes Sense to Me.
Opening Peer Review in LIS September 21, 2021 | 1:00-2:00pm CT
What does it mean to peer review in library and information science? What does it mean to be reviewed? How do our professional identities intersect with this vital research and publishing role? And what does it mean when peer review is opened to reveal these identities? In celebration of Peer Review Week 2021, this free webcast with Emily Ford will share insights into peer review in LIS as discussed in Stories of Open: Opening Peer Review through Narrative Inquiry, a newly published book from ACRL.
It is with excitement and a joyful anxiety that I break our regular programming to share this news: Stories of Open: Opening peer review through narrative inquiry is now available for purchase in the ALA Store! (It’s also an OA book, downloadable here.)
This book project is what prompted me to continue to gather and share stories on this website. From the introduction of the book:
I continue to work to edit and publish stories here, apologies for the lull and interruption, but things have been busy!
In this exercise of researching and writing—exploring ways of knowing through narrative of human experience—I have also come to be acquainted with our colleagues who bravely shared their thoughts. We theorized together, and together we explored their (and our collective) emotional experiences. Each and every conversation required folks to be vulnerable and trusting, and in return I have striven to offer you their experiences with loving kindness. This book is as much theirs as it is mine, and as it is yours. In the interest of protecting individual privacy, I have used pseudonyms for each individual mentioned below, with the exception of Stuart, who agreed to openly share their story. Stories from individuals with pseudonyms have been edited to omit personally identifiable information, and when needed, I worked with those individuals to edit their stories to include and exclude information as they felt comfortable. I would like to introduce you to ten incredibly generous and thoughtful individuals. The headings represent the title of each individual’s interpretive narrative.
This is the third in a series of three posts that comprise Hannah Gascho Rempel’s story. In her first story part, When you look at a body of literature…, Hannah discussed her recent ties to scholarly publishing, investigating its history, as well as her experiences as a journal editor. In her second story part, The system was meant for me, Hannah discusses her privilege and delves deeper into her experiences as an editor, hoping to make positive changes at the journal.
In this third and final story part, Hannah discusses her experiences withdrawing a journal article and publishing it elsewhere, as well her experiences with open peer review.
Hannah Gascho Rempel
Position: Professor and Science Librarian at Oregon State University Libraries
Fun Fact: Hannah teaches a Learning Through Play seminar class, in which students create games in small groups (and learn about research). This term the theme for their games is Space, and she’s really looking forward to seeing how they pull together learning opportunities for aliens and astronauts.
Emily: But what do you think peer review should be for? Like why are we doing this?
I was trying to think about that ahead of time, too. Ideally I would think of it as a conversation in a community that makes a work stronger. So, for example, there’s the open peer review style of reviewing at this one journal, it is a little bit of an apprenticing. I’m not sure if they’re still doing the Google Doc version but you see the comments along the way. I wouldn’t say the quality of the feedback I received via that particular venue was very good. I would say that because I don’t feel the reviewers I happened to be matched up with had either all that much experience or experience in my particular fields. And by fields I’m talking pretty broad like instruction librarianship. They just didn’t have that and so the feedback they gave was a little bit like, “okay, I will use my skills from my very first advisor and say the polite thank you for your helpful feedback. However, I know this and this and this too.”
So, the other side of that when it’s obviously closed and you get back the reviewer two because I have myself received the reviewer two-ish. Not a like horrible horrible version of that, but it’s this false community conversation then, [where you’re solely communicating via a written format, and there] are all these things about that written format that make it hard to [communicate] and they don’t quite get enough context and they don’t know you and they don’t know your history and that’s all by design. You’re not supposed to know all the things. So they give their feedback coming from up here and they’re like, “have you thought about this?” And you want say, “Totally. I’ve spent like half my career thinking about this. Like uh huh, yeah. But thanks.” And so the idea of that is of your hands motion in the air passing each other by and not having real conversation. The ideal of that [conversation in community] gets missed. Ideally you would be getting helpful feedback, because I do value much of the feedback that I have gotten. As with most people I, of course, don’t value it like two minutes after I’ve gotten it, but a day after I’ve gotten it then it’s situated a little more. You look back and you’re like, “Okay, it’s true.” How would they have known I have been thinking about that for half my career? I didn’t tell them that at all. How would anyone know? So yes, I should reframe that [idea I was writing about in my manuscript draft]. I should set it up better this way. Fair enough. And then you reread it, your own work later and you’re like, “Yeah, that was a better option. I’m glad I did that.”
Emily: It’s interesting to hear that you had not as robust of an experience with open review. In my experience open review has been more robust than closed review so that’s interesting that you had that experience. Peer review can – you never know. I’m wondering if that has colored what you think of open peer review now? Would you try again based on that experience?
I published twice in that open peer-reviewed journal, so I have tried it again although I wouldn’t say that the second time I tried it for the open peer review part of things. I’m still open to it. I’m open to a lot of things. I think in some – yeah, by having more experience in the profession just because I’ve been part of the profession longer, in theory I would then know people who I can get helpful feedback from. That said I don’t know all the people by any stretch. It’s a huge profession so I would always be missing out on some voices if I only ever sent my work to people that I know and value their feedback. So peer review can have that value when an editor is making some of those choices for you, finding new people and pulling in somebody who has a different background and maybe that would be helpful. I’m still open to having other approaches, but I think again if I could just give it to Janet Webster half the time I would probably get 90 percent of my helpful feedback from Janet Webster feedback and then 10 percent from the person that I don’t know behind, not even behind the wall of the open peer review but somebody that is totally new to me.
Emily: Yeah, interesting. Are you saying that because the journal asked you to say who are you going to ask to do this? I just want to connect back to that.
Right, because they do have the ask part of it. I did have one other more recent experience though of asking somebody. So the discipline that I, the science discipline that I grew up in used double-blind the whole way through, so recently I had a single-blind experience, -ish. I included a list of people to be included in the ask and then when I got the feedback I was like, “Oh, clearly they chose Greg.” Even though Greg’s name is nowhere on that. I was like, “Ha ha, wow. Greg, you can’t tone that part of you down at all.” But it was helpful so I guess having had a couple versions of that now, one at the openly reviewed journal and this other experience, asking someone – I guess what I’m getting at is there’s enough strategy in asking someone when it’s going to be for publication versus the informal Janet ask, when Janet is not going to be – I’m using Janet as shorthand for all the people, but when it’s not going to influence whether or not you get accepted or not.
Emily: What do you think given all these issues? You’ve seen issues of where on your end as author it’s supposedly opaque and you weren’t supposed to know it was Greg, but it was Greg. You have seen it as an editor where you have this reviewer two, where it’s not necessarily abusive, it’s just cursory and people just aren’t necessarily engaging deeply or robustly. How do we move past those issues of peer review—whether it’s an open peer review with transparent open identities or not—how do we fix the problems that we’re seeing in peer review?
That is a good question. I thought when I was an editor that a way that I would fix it was by including my feedback to the reviewers. So when a decision was made and the thank you for your service kind of email goes out to the reviewers and it included both of the reviewers’ feedback and then the content management system piped those two things in. It didn’t pipe in my feedback though, so I’d dutifully copy and paste my feedback as a “subtle” – Hannah is using air quotes – way of being like, “What you could have done was provide this kind of feedback.” I think for a few of the folks that did help, particularly people who I had invited to the editorial board—and not that it was about me inviting them, just maybe that they were newer and so were looking for inputs—so that did shape their feedback some. Just having an example of what was being looked for. But then the [peer reviewer feedback] form, I did change the form some over time. I changed the peer-review form to focus more on, to say, “Please look at…” It’s one of those things where we don’t always have a methods section in the kind of work we do, the kind of writing that we do, so it doesn’t follow the flow and so saying something on the form, like, “Do the results follow from the methods used?” Well that doesn’t even apply to a lot of the kinds of writing we do. So some of the checks that would be in a science writing just don’t follow. So, “Is it clear? Does it make logical sense?” sometimes is the best you can do and that feels so wide open. I tried to be a little more specific, but it was hard to do when there are all these forms of how people write in our scholarly conversation that are okay. I’m not saying that everything should be an empirical study by any stretch, but then it makes it harder to standardize and to know what you’re looking for. I will say the other thing I did try and get at in the form was that folks did not need to spend any time on copy editing and that grammatical decisions didn’t need to influence their feedback. And people did make that adjustment and I was glad. I wanted that for a couple of reasons that I’m guessing you can guess very easily. I didn’t want discrimination because of voice or tone across both, English as a first language, but also other kinds of writing that comes out. I didn’t want people to spend all their energy on this—as librarians we sometimes are wont to do that. “Is the Oxford comma here or not?” That’s on me [the editor] or that’s on actually Taylor & Francis [the publisher] folks. They [the publisher] did that level of things. So don’t spend all your energy there. What your energy is for is: does this fit in our conversation? Have the conversation.
Emily: Fit. It’s interesting that you use that word. Okay, so can you unpack that a little bit?
So, the particular journal has a scope. Not everything fits in that scope and so defining that scope is a changing conversation. And so the particular journal, the Journal of Web Librarianship, [they are involved in a changing conversation around:] what does web librarianship even mean? I mean what is that?
Emily: It’s everything we do now.
So then saying is it everything we do or is it a particular kind of thing we do? And what we do changes over time so where is that “fit” and where are we having that boundary? Because it’s shifting and here we are in 2020 and here it is still web librarianship? So, it’s a conversation to have about fit and so I did what editors typically do – they review it [an article submission] first and do a cursory check to see if the article matches the scope. So I can have some of those boundaries set for sure, and I can know this is like 100 miles from fit. But [it’s different if it is] 10 miles from fit and maybe our boundary should go out 10 miles. That I wanted to have other input on it.
Emily: Okay, that makes sense. I feel like there have been articles about they don’t “fit” in our workplace with a hiring things so it’s code for white supremacy. I just wanted to unpack that a little bit more.
Right. And it means there’s a community line and “there is a community line” is my understanding of the white supremacy version of fit too. To me fit still can have a purpose, but yes, if you don’t unpack it then you haven’t defined my meaning.
Emily: Did you have any particular story or anecdote that came to mind for you when you were thinking about your experiences?
I guess maybe two and I was going to say – I think they’re short and now you know that’s a lie. None of them – I don’t say anything short. So the first one I’ll [share] was on understanding that a rejection is not the end of the day. That’s something I would have learned from my plant sciences advisor. You get rejected by one journal, that’s nothing on you. You just move along to the next one. You take what feedback was helpful. Perhaps it wasn’t within the scope. Who knows. It could be any number of things but you move along and you’re going to get that thing published. It’s the outcome. I don’t know if it was my third or fourth article [in librarianship]. Anyway, fairly early on I got rejected and I’ve been rejected more than [that now], and that was my feeling about it. That’s how it works. It doesn’t always, you don’t always get all the things and you move along and I was able to get the article published then at the next place. That was not the experience that many of my colleagues had had, especially if they didn’t have a previous disciplinary experience in being part of a publishing community or an academic community. So having that as something that was more explicitly talked about and made okay that rejection wasn’t like close up shop, you’re never going to get tenure. Your life is over. Now you’re just moving along. That felt like another lack to me in a thing that was missing from maybe how librarians are, probably broader than that academics too – that it’s okay. You just move it along. So that was one [story/observation].
The other one is kind of related, but what I would consider my worst peer review feedback. It was for one of our more notable publications in librarianship, and what irritated me about that experience was the lack of editor interaction there. So I got a revise and resubmit on the first round after waiting like [at least] six months to get that feedback. The feedback was not fantastic, but okay. I took the feedback, made the changes, again waiting for forever, and it comes back with another revise and resubmit with even less substance to it and no mediation by the editor in either of those to say “it looks like you’re really just missing this one thing” or “it looks like you’ve failed to do that certain thing.” Nothing. And just kind of spinning it over and over again in this revise and resubmit cycle. And that was frustrating, especially because I was already an editor myself at that point and the quality of the feedback given on the second one would have been a conditional accept in my view, not a revise and resubmit. So it felt like there was just some laziness, over work, inattention. I don’t know what it was. And so I feel in our profession that if we’re not going to have a training system or this clear something – that it is on editors to take on some of that role and I felt that especially strongly at the time as an editor. That was puzzling to me.
Emily: Did you withdraw?
Totally. I withdrew.
Emily: Did you publish somewhere else?
Yes, and the editor at [the second place I submitted to] was like, “I have been an editor for five years. This is the first article that I’ve never had to give any feedback on. Neither reviewers had anything to say in terms of changes, blah, blah,” which isn’t so much to toot my own horn, because I had made changes based on that previous feedback so it had gotten feedback already. But it wasn’t very different at all from what I had resubmitted the second time there [at the previous journal]. I was like, “Well, exactly.”
Emily: What is your speculation of what happened at that journal where you did get this R&R twice?
I think the editor was kind of checked out.
Emily: Or swimming with submissions.
I’m sure that was true, yes. And that journal would have gotten many submissions.
Emily: Right. I think, too, the labor—again, I haven’t been an editor of a proprietary journal. I’ve only ever been on the editorial board and edited something that I invented with friends, so just take my comments with a grain of salt—but I would assume the labor that I’ve always put forth toward editing and things like that, that’s a labor of love. Sure, in a tenure-related position there is some recognition for that, but you just really need to put it on your CV and have a few examples of the quality and impact to point to. So where is the accountability in actually doing a really good job? And if something happens in your life like you’re splitting up with your partner or if you have a death in the family or if there’s a pandemic or you have a mental health crisis or anything like that, there’s no yeah – so there might be a flaw in the system. That’s very generous—maybe the editor had something happening and because the labor is donated a lot of times. What do you do?
Yeah, I know. That was my every evening for five years. That’s not true. That’s the other irritating thing about editing: it wasn’t consistent. I realized when I had interviewed for the role and asked the question of the previous editor “how many hours a week does this take?” And she sort of hemmed and hawed and I was like, “How can she not know?” And then as soon as I started I was like, “Ah, silly, silly girl. It just comes and goes and you can’t predict when everybody is going to submit five things at once.” Yes, it is a ton of work and nobody sees and nobody knows. This particular editor at the other journal – somebody sees and somebody knows, but yes, your general statements were all true.
Emily: If you were in charge of the peer review world in LIS what would be the first couple of things you would do to improve it?
Yeah, I’d have somebody shadow. I mean I’m pretty consistent with my apprenticing thing. I’d have somebody shadow, somebody else who had been identified as a strong peer reviewer, give them some examples of what was helpful over several ranges of article types, but I think practice is the only way that most things get better, so practice and then feedback and standardizing that as a thing, not as a “you’re in trouble because you’re getting feedback” or whatever, but just having a cycle there of practice and feedback, try it again and then you do the same for the next person in line and then demonstrate for them.
Emily: It’s really interesting that you said when we get feedback we feel like we’re in trouble. But feedback doesn’t mean you’re bad. But a lot of us have been taught or socialized that if we get feedback we’ve done bad and there’s shame associated with it.
And our ears close.
Emily: Yeah. All the feelings and like where is that coming from? Is that American? Is it whiteness? I guess I’m seeing a lot of this white supremacy and socialization. I mean I think that’s where we are culturally right now just with the grief of all of these murdered Black men mostly, some Black women. I think you hit it on the head, this, “I’m bad.”
Yeah, and the quick leap from my work is bad to I’m bad is amazingly fast. And it’s not anybody saying your work was bad. It’s just like, here are some other ideas. I was reading “White Fragility” earlier this winter. What I took away from her was that we need to do better with getting feedback about being racist. That’s kind of the core of what I took away. And where my brain stopped – and she gives multiple examples along the way. Where my brain stopped was like, “when have you seen anybody get feedback and then take productive action on it?” because it was just exactly what you’re saying. You give feedback about how you clean the sink, about how you did the whatever and people are up in arms about when you should have the door open or not.” And not to minimize the racism aspect, but I was like it’s not that it’s not about racism, but people aren’t taking any feedback well, so what will be the leap then that all of a sudden this will be the one thing they take feedback well on. Like, no. I’ve been wondering the same thing just culturally like what is the deal there. And so I was reading it when I was in Europe and then, of course, the world explodes after that. I’ve thought about collectivism and how that all plays out as well. And so you ask me whether it’s a U.S. thing. I think that it is somewhat of a U.S. thing. It’s not only U.S. but we’re such individuals both in taking everything on ourselves but also then everything is for ourselves. I don’t know.
Emily: So it’s like that kind of bootstraps mythology, individualism. This area, this fence around my yard, this is my property. And capitalism is part of that, too, I think. The U.S. capitalism. Lovely.
This post is the second in a series of three comprising Hannah Gascho Rempel’s story. The first post, When you look at a body of literature…, discussed some of Hannah’s current involvement and curiosities about scholarly publishing, stemming from the history of scholarly publishing, to her experiences as an editor.
In this second story part, Hannah unpacks her privilege and successes in scholarly publishing, and discusses limits of LIS education.
Hannah Gascho Rempel
Position: Professor and Science Librarian at Oregon State University Libraries
Fun Fact:Hannah was on sabbatical in the Czech Republic last year and enjoyed doing things like going to soccer matches and people watching while riding on trams and subways (and working).
Emily: You said 30 minutes ago that you were good at publishing. You have a book forthcoming that you’re working on. You have multiple articles. How do you think you got good at it?
I practiced it. I’m a white girl who school was meant for and designed for. I follow the directions. I know how to follow the genre rules and play by those rules and adjust myself accordingly. The system was meant for me.
Emily: How did you learn to serve as a referee?
So I was thinking about that ahead of time and ironically (or just whateverly) I actually have not refereed very many times. I have referred less than five times overall so more of my refereeing came during my [journal] editing days when I was actually the third referee on every single article—sometimes to a more or less degree—but every single article I reviewed. Which isn’t the way it looks on the outside, but at least at that journal that’s how it really happens on the inside. How I learned to do it: some through my first experience publishing back when I was still in horticulture; observing how my advisor—I can’t say that I ever observed her reviewing another article and how she gave that feedback, but I saw the feedback we got from the main article that came out of my thesis work and how she responded and taught me to respond to the feedback. And so [observing] what feedback was valuable and what could easily be dismissed. And so taking that logic then and trying to focus on the kinds of valuable feedback [in my refereeing]. Otherwise in librarianship all of the instances [journals] that I did review in had a [peer reviewer] form to fill out and so that was the guidance that was given there. But more of it, I would say, has come from the feedback I’ve gotten, not just in my plant sciences world but in librarianship, and what I’ve observed as being helpful feedback and then I tried to mimic that myself.
Emily: It sounds like there really wasn’t anything formal. It was just somebody who took an interest to be a good advisor and mentor you. Was that standard in horticulture and plant science?
No, I wouldn’t say so. I wouldn’t say that it was exceptional, that she was the only one, but there were certainly people who would not have had that experience. I would say it was common enough that other professors I’ve interacted with who would be a similar age to me, would have had that experience as well. Whether it was the advisor or a postdoc that you sat next to didn’t matter, but there was definitely some apprenticing sort of approach to it where you saw how somebody else did it, how they responded to feedback or gave feedback themselves, and then you learned that way.
Emily: If you didn’t have that background how would you have learned to do this if you just went to librarianship and not with the masters in plant science?
Yeah, that’s a good question because the closest I guess I can say that I would get is at Oregon State University. We have had a research and writing group that does meet regularly, although less so now, but we also have kind of informally [a network of] people that you know are [good at giving] certain kinds of feedback. So you would go to X person. You were coached early on that you go to X person if you want copy editing feedback. You go to X person if you want structural feedback or deep thoughts kind of feedback. And so I learned more obviously probably from the deep thought sort of feedback givers. There are a couple of people like Janet Webster would be one of those people so you get feedback from them a time or two and then you’re like, “I got it. Every time I should do the methods like this,” instead of whatever other way.
Emily: That’s interesting because it sounds like even though you have this group, whether it’s formal or informal, but it’s been an on-the-job experience. It’s not like in library school you’re learning about this process. As a journal editor do you have any stories of seeing reviews that were just totally out there where you had a reviewer two? How did you manage as the journal editor, the reviews you were seeing and then sending to authors?
Yes, I do know reviewer two. In my experience, I’ll start with – I saw my role as the editor to be a mentor to authors. I understood that in librarianship people haven’t had the training in this. So, when the question always comes up should you have learned X in library school. I have no expectations that anyone should have learned anything in library school. I don’t start with that premise at all. I’m amazed if we come out with anything. [laughs] Okay. I’m being flip; I know that’s not really true. What I mean is that library school represents a snapshot in time of some of the values of that discipline. And it isn’t possible that school will be able to predict all the twists and turns of an individual’s career, or all of the changes to a discipline or profession. It will always be a work in progress. So, I know that they haven’t come out with anything [training related to academic research writing that leads to published articles] so I saw my role to mentor the authors, which, for me, meant then that I would take the reviews and triangulate what was given, make sure to read the article myself (I would always read an article when it came in to see if it was appropriate and to see if it matched the scope). But then I would do a second read and particularly, depending on the feedback given by the reviewers, I’d take a deep read and give extensive feedback myself so that I knew in context what the reviewers’ feedback meant. I didn’t hide reviewers’ feedback ever. So, if there was a horrible reviewer two it wasn’t like I hid their feedback. But I would contextualize the feedback that was given and bullet out what I expected the actionable feedback to be for the author so that they were addressing that rather than two contradictory smorgasbord approaches of vanity items from the reviewers. So that all said – I am trying to think of an actual instance of a reviewer two. I am having a hard time coming up with reviewer two. What I had was so many cursory reviews that were so insubstantial as to be meaningless, and that’s what led me to need to do a really deep read. Because if there was nothing there I would have known based on my first read, that the content, the idea matched our scope. But the execution, whatever it might be, needed work and no info was given [from the referee]. So the more realistic situation was that, in my view, reviewers had no idea how to think like scholars in that way and provide feedback that could lead to action and improvement.
Emily: And why do you think that is?
Back to library school.
Emily: Okay, so it comes back to this lack of training?
Yeah, so lack of a research-driven approach to our profession. That leads to having that training whether that comes from library school or on the job or whatever it might be.
Emily: Who do you think should be responsible for that? You say keep going back to library school but obviously it’s not happening in library school.
Well, this kind of writing isn’t an end goal of many librarians. So for public librarians, for example, who have no clear need to publish, some might have a desire to publish and would work at it, but it’s definitely not an across the board [expectation], whereas the expectation in academia as we know it, is if you’re going to get your degree in physics, the jobs in physics will be professor. Here’s how that works and the apprenticing is very clear. Librarianship doesn’t necessarily end with this as the goal, so I think without having tracks in library school – so I went to the University of Washington’s iSchool [in the mid 2000s]. There wasn’t really an academic librarian track so it wasn’t a way to specialize along those lines. That’s the only way I guess I would see it happening is if you were required to do the work of apprenticing for that profession.
Emily: You keep using this word apprentice and I really like it. It’s not one that I’ve heard someone use before with librarianship as much. You hear internship but it’s not necessarily the same as apprentice. I had this idea while you were talking why don’t we have editorial apprentices? Why don’t we do that? It seems like it would be a really rich training ground to learn a lot of all of the other stuff.
This is the first of three parts of a conversation with with Hannah Gascho Rempel, Professor and Science Librarian at Oregon State University Libraries. Hannah came to librarianship as a second career, and has been a librarian for about 15 years. Her first was in the plant sciences, in horticulture. Her training in science afforded her the opportunity to have some training in scholarly publishing practices before coming to librarianship. She is the former editor of Journal of Web Librarianship. She left that role before she went on sabbatical to the Czech Republic as a Fulbright Scholar in 2019, and then made her way back to the U.S. as the 2020 COVID-19 pandemic hit. Hannah is also currently working on a co-edited book entitled “Teaching Critical Reading Skills: Strategies for Academic Librarians” that she expects to be published by ACRL Press in early 2022.
I should also note that I know Hannah through the small academic librarian community in Oregon.
Hannah Gascho Rempel
Position: Professor and Science Librarian at Oregon State University Libraries
Fun Fact: Hannah is looking forward to growing sunflowers this year rather than vegetables in her garden to give the deer something different to munch on for a change.
Throughout our conversation I notice the parallels Hannah is able to draw from her experience as a scientist and scholarly publishing in the sciences, and how she brings that knowledge to her career and endeavors as a librarian. What particularly strikes me is that Hannah is able to clearly articulate the different ways peer review is understood in some scientific fields versus the way she and I both see it as a mutable evolving process.
In this first part of Hannah’s story, we swing from the history to scholarly publishing to discussing present practices.
Let’s let Hannah introduce herself further.
I’m mid-career. I’m also midlife, mid-forties sort of person. I had another career before I was a librarian so that other start of a career was in the plant sciences in horticulture. I have a master’s degree from that field along with some of the publishing training that goes along with that and the interacting with other scientists that goes along with that. And then I switched gears to librarianship and have been in library land now for over a dozen years, 15 years. That’s a little bit of a shift, but for me it’s mattered for my publishing history and my thinking about academia because I started thinking about academia outside of librarianship and that definitely flavors how I continue to approach the research and writing process and how I train others whether it’s grad students at Oregon State University where I work or my fellow colleagues who are starting out in the profession so both of those things are influenced by my start.
Emily: Can you tell me a little bit about why you responded to the call for participation and why you wanted to participate in this particular conversation?
I, too, am interested in peer review. Not to your level. Mine has been in terms of being an editor of a journal as one aspect of it and then needing to think through what to do with the peer reviews that one gets. How to coach colleagues in the peer reviews that they’re giving without being too overbearing and wanting there to be this professional understanding, but there isn’t, I would say in our field, in the same way there isn’t an apprenticing that happens in the same way I would have experienced it in my previous sciences career. My advisor in that career was very clearly apprenticing me to something. She was demonstrating how she was responding to feedback, how one did that with the idea that I would continue in that way. But then my other part of the interest in peer review has been because I’ve been on a graduate student’s committee for the past year-and-a-half—she’s in forestry at Oregon State University—but her field would be a little bit more human dimensions of forestry kind of focus. So social science aspects of it, and part of her question has been looking at how people of color and underrepresented people are represented in the forestry literature of the past hundred years. And when you look at a body of literature as scholarly literature for the past hundred years it becomes really interesting. How to normalize that when you’re looking at how somebody was having a conversation in 1920; how is that different than the conversation in 1970 versus now? And to put things into categories like we want to do in academia, you start off by looking at the literature that was published by discipline. So what falls into fisheries versus forestry versus wildlife. And then what falls into peer-reviewed versus not peer-reviewed and opening that up and trying to work with her on this so she can ask questions. But the questions all start being like, well not nonsensical, but you’re just not having the same conversation anymore so looking at what peer review meant to someone pre-1970. And then on top of that having an understanding that she can communicate to her advisors and the rest of her committee who would tend to be more traditional scientists who think peer review does mean a very particular thing and that it is a stable conversation across time. And this isn’t to say they’re stupid intractable people at all. That’s not it. It’s just they haven’t thought about it as something that wasn’t always there. So I’ve done some more reading on the history. I don’t know all the things about history of peer review but it’s fascinating and I like it that you’re documenting how it is now because it won’t always be this way and it is in flux so that as we move along the way in the profession we can see that.
Emily: There is a scholar named Aileen Fyfe out of the UK who has done some interesting work on the history of scholarly publishing. Fyfe and their colleagues kind of go into this history of peer review noting that it was in the 1960s and 70s that brought about peer review to the proprietary publishing market. What we think of as traditional peer review today was actually market driven[see Untangling Academic Publishing].
The other version that I’ve heard, too, that I’m sure you know about as well is the government regulatory aspect of it. So they were getting input that congress was saying “oh, well you’re making this report about X. We think you should do this and they’re like if we want to be able to say as a community no, that idea is bonkers and you have nothing, we need to have something very official looking about it so it was manufactured to get that. It was sort of yes, depending on what function you think congress serves if you think they are representing a community it was sort of suppressing a community input which I don’t know that I’m going to stand up for that one very hard but it was to protect the scientific community was the way that I’ve read it too.
My grad student, whose disciplinary home is natural resources—I was trying to figure out okay, well Journal of Forestry, Journal of Wildlife Sciences, to figure out those journals’ histories and look back. We have their journals. I can look in our catalog and find them and so looking at their table of contents and how things are described over time, but it’s fuzzy. It’s not at all transparent and it’s not like there’s this narrative for each of the disciplines. It’s like there were some of these overarching scientific conversations, but then it must have filtered down somehow and so was that at a conference, was that just in somebody’s hallway? How did it actually happen? That I can’t figure out and how long it took to filter from this 1960s mark to all of these other areas. I can’t tell.
Yeah. But it’s interesting just like when the trail is not – and I think like genuinely it’s interesting. It wouldn’t be as interesting if you were like “oh, the answer is on page two.” Anyway, I’ve been thinking about it for a while both professionally, and I have a curiosity from the historical side, too.
Emily: Your interest is a whole other layer of intellectual labor that’s related to our job serving patrons, which is interesting. Can you tell me a little bit about your relationship with academic publishing?
Sure. At the institution where I work we are tenure-track so some of my relationship with publishing is driven by needing to pursue tenure and promotion in order to be employed. So I started publishing in the library land stuff as soon as I started in libraries and – do with what you want but I’m good at this. This is a thing I know how to do and I’m good at it. So I’ve published 15-20 peer-reviewed journal articles. I’ve written two books. I also have an ACRL publication in the pipeline [“Teaching Critical Reading Skills: Strategies for Academic Librarians”]. It will be an edited collection though.
I’ll back up and just say a little bit on the tenure track. I’m now a full professor so in each of my stages I’ve experimented with some different approaches. Pre-tenure I was probably what you would term a little more conservative. I focused more on getting my articles published and targeted venues that I viewed as higher impact. So that stage of focusing on College & Research Libraries or Journal of Academic Librarianship kinds of places. And then after I got tenure I focused more on publishing in open access venues with less concern for the prestige factor of them and playing around a little bit more with some different research methods and styles as well as trying out some books, which I had never done before. As a science person I’m very article focused so it was entertaining to try out books for a change. I got full professor a year ago and so in my mind that moved me more strongly into a mentoring role so we’ll see what that looks like in terms of publishing myself. I don’t see any need to push myself to the first author position so I’m not sure where that trajectory will go. I’ve written things and I see it as a way to communicate with a larger community for sure, but then the other aspect of publishing it was being a journal editor and so that was not with – so it was sort of a two-facedness about myself. I started that after I got tenure but it was a Taylor & Francis publication. I myself was publishing in open access venues but was not an editor for an open access publication. [sigh] All the choices. I got to see a little bit more of what publishing looked like and interacted with a publisher and what other people’s goals may or may not have been from that vantage point.
Emily: You have goals as an editor. The Taylor & Francis as a company maybe has goals and then the editorial board or the author have goals. I don’t know what your experience was but is there a story or two?
Yeah, or at least notes. Story makes it sound more interesting than it felt. I guess I would say to start that Taylor & Francis was pretty hands off during my whole five year term. The managing editor for the library sciences wing of publishing (if that was their title), the person who was initially in that role switched jobs within several months after me starting. Based on [conversations I had with] the previous editor, that the previous relationship [with the managing editor] would have been a little bit more in depth. And then throughout time there were multiple changes in staff of people I interacted with so– there was no thinking about or strategizing about direction or journal choices or anything. More of just getting the issue off to press. That kind of work. Those people changed many times so the consistency there was very low and it was a small – well it was just me. As the editor I didn’t have an associate editor for most of that time. Most of the work in terms of interacting with Taylor & Francis was just to keep the thing functioning in terms of getting another issue out. So they had no interest whatsoever in giving input.
Here’s the little note story part, I did have questions about open access options, because at Taylor & Francis you can, of course, you can pay a gazillion dollars and have a fully open access version of your article which, of course no one’s going to do. But then you can also have a preprint, [so I asked the publisher] is that true, what would they allow? Would they allow a post-print? You couldn’t put up the actual journal formatted post-print but you could put up your version after a certain amount of time and in an institutional repository. The whole not very fantastic but at least an option.
Emily: It was embargoed.
Not very clearly embargoed. A very short embargo, but yeah. That was an option and so I did interact with them some to make sure that I was clear on the rules, that the linking happened from the journal website which they managed. I couldn’t change anything on the journal website without interacting with a person. I always had to have somebody make all those changes—the this is not centered correctly, etc. Somebody had to do that for me. So I did put up the author instructions for how to post a pre-print.
And then in terms of author goals, I made sure in the article acceptance notices to put in info about here’s what next steps to take; to always include the directions very explicitly on how authors could at least put things in their own institutional repository. I started tracking them by doing Google Scholar searches of what had been published in the journal and seeing what showed up in Google Scholar as in repositories—almost nothing—which is just fascinating to me that folks in this profession are doing so little work themselves to put their own work in institutional repositories! And I know that it’s not as simple everywhere and that not all libraries do have their own institutional repositories so I had started including a link to a more general repository folks could use. Anyway, it just didn’t feel like that was a goal that folks who were choosing to publish there had – that it wasn’t that important to them.
You asked about the editorial board. Also, it was an international journal. There were folks on the board from all over the world and I didn’t recruit all of them because I was the second editor for this journal. Some of them were legacy. I did recruit some of them though, and we didn’t meet very much at all. They didn’t give a lot of input. They served as reviewers on a somewhat regular basis, but in terms of giving direction that was pretty much all on me and it was only if there was a particular need, like we need articles for a special issue. We need help on something. That’s when that would happen but it wasn’t a direction providing body.
Emily: How do you make sense of that authors weren’t really interested in posting their articles in an OA repository?
I can’t make sense of it very much. Sometimes I wonder, so while I’ve been a librarian for what starts feeling like quite a while, I’ve only worked at one institution and I start to feel some of the limitations of that in terms of understanding culture across multiple places. So at my own institution, very soon after I started working there, we passed an OA mandate in our library and then [an OA mandate was passed for] the whole institution—which doesn’t have all that much teeth to it but it was talked about very regularly. It was a high priority. It was something that we were clearly meant to value and I can only assume that doesn’t happen really at all that many places. And the people who were publishing during my tenure [as an editor] came from a mix of places so it wasn’t all R1s by any stretch and that’s great. I’m happy for that. Some comprehensive places, a couple public libraries, not very much, but if the message just isn’t held all that strongly across different parts of our community.
Emily: It seems a little ironic to me, especially given the journal that they were publishing in. [laughs] I just want to note that you rolled your eyes.
Rightfully so. Can I roll them back the other way?
Emily: Despite the OA mandate at OSU, you mentioned that you didn’t really start publishing OA articles until you had achieved tenure. What led you to not pursue that as much until after tenure? Because I would just assume again that your guidelines or your documents for promotion would have something about OA in them if there’s an OA mandate.
So, all of my works are in our repository and for our mandate that was what was requested. So pre-tenure, all that stuff is still in the repository. It was post-tenure that I made the choice to go with fully open access journals that wouldn’t require repository mediation.
Emily: Why is that?
In part to make [long pause] – it sounds overly snooty or something to say I’m helping out those journals, but I guess to make them more of an accessible choice if people are clearly trying to publish in them instead of everybody only trying to publish in Journal of Academic Librarianship, for example, or trying out other venues instead.
Emily: Pre-tenure were you worried about the impact of the journal? What was your concern? Or was it just something you didn’t consider?
I would have had some [articles] that were actually in open access journals but that would have linked more towards the community there. So, for example, a science librarian journal that I published in early on that is online that had open access. It is online. It’s only online. It’s clunky. It’s gotten better so it made sense for the community and that was my goal, so I would say that it was actually more of an outgrowth of me transitioning from my science career—where you have this topic so logically you match it up with this outlet that gets you the most bang for your buck and has the correct scope for what you’re talking about. So those two things being very valuable, that’s what you do and that’s how you move forward [in the sciences]. So that was the messaging I feel like I had most recently and strongly, and that’s what made logical sense to me then. And so it would have been more that it made sense to try and publish in C&RL, which is still open but not quite as easy. So getting in a bigger place was the value I brought from that past experience.
Position: Associate Professor, Simmons University School of Library and Information Science
Fun Fact: Laura likes to sing around the house, and sometimes her dog joins in.
Emily: Earlier you said something about how you tell students what peer review is for. I would like to hear more about what you think that actually is. Why should we be doing peer review and what is its function? What should it be for?
That’s a good question. What I usually tell the students it that it’s supposed to be a quality check so that it’s supposed to be that people with some expertise in the general field of the article or the book are able to look at the manuscript and say first of all overall this is good quality, but in particular to really look at things like what methods were used. Again, assuming this is a research article: What methods were used to study the research questions? What sort of analysis was used, were these appropriate to the questions that were being asked, were they implemented correctly or appropriately? And then whatever conclusions or inferences are drawn from that data, do they seem to line up with the data? So the idea is that someone who should know enough about all of these questions is able to look at that and make some call about whether all of those things were done well enough. But like I said I think that it doesn’t always work the way it’s supposed to.
Emily: Before, when you talked about how you approach refereeing, it doesn’t sound to me like you approach it as just a quality check. It sounds to me like you want to be helpful and collaborative. Does that ring true for you? So you’re telling your students that it’s about this quality check, but in practice you’re doing something different.
Actually that’s a really good point and I think you’re right that my understanding of the peer review, and so again how I relate that to my students is that it’s all about this quality check. Trying to make sure that things don’t get to print that shouldn’t be in print because there are major issues with them. But I think you’re absolutely right. That especially over time I think I have more and more tried to be more almost like a mentor and taking it on almost like a mentoring of trying to give some really helpful feedback with people with this sense of I really want you to get published. Here are the things that I think will help and I don’t know that I’ve really talked about that aspect with it to my students. And yet it’s not like I think that I’m alone in how I approach that either. Even though it wasn’t necessarily how it was ever presented to me either. I think that most of us do sort of try to take on that role and I think that’s important and probably something we should talk about more.
Emily: I’ve never heard someone say that they actually just do a quality check. At least in our field. Most people want to be supportive. They want to be collaborative. You said you had that one grant review and it was like two words. Maybe that person was just doing a quality check. Not to say that your work was of poor quality, but I’m just trying to think maybe their understanding of peer review is different than yours and that’s why. What are you actually expecting from this process?
Yeah, I guess that’s a good point and like I said, we’re never really taught how to do this. At least in my experiences we’re not taught how to do it and we don’t have a really lengthy conversation about it and so I think yeah, I’m sure there’s a lot of different perspectives on what’s the right way to do it or what are we supposed to be focusing on and that’s really problematic.
Let me put it this way. It’s not that I have extensively read the guidelines and aims and scopes and things like that of all these different journals, but I will say of the ones that I’ve read portal: Libraries and the Academy is the only one where I’ve actually seen them say that they take a mentoring approach to their feedback. So they actually have it written into their guidelines which I think is really pretty cool.
Emily: Does it make you feel differently about what you think peer review should do and what it should be for?
Yeah, like I said now that we’ve been talking about it I think that I will be talking about this a little bit differently in my classes, because I think it’s important like you said, at least in our field, to talk to people about this and to just understand the peer review a little bit more broadly and to think about the extent to which we do or do not approach it as mentoring and constructive feedback. I think it’s also important because there’s a push, especially for academic librarians who are on tenure-track, to publish and yet we don’t teach them how to do that usually in our master’s programs. And so if you happen to come from a social science background or a hard science background then you might be all set even without the MLIS preparing you for that. But for a lot of the other students—and something like 70 percent of our students at Simmons come from a humanities background—it can be really challenging in a lot of cases to go out and rate those articles not having been taught how to do it. And so if the peer reviewers take a little bit more of a mentoring approach you might actually really be doing a bigger service to the field even than just what the review itself does.
Emily: I don’t know if you’ve seen any of these peer reviewer courses that have been offered by some of these commercial entities like Publons. I guess I just wondered if you had any thoughts about what it means for a company or a publisher, a commercial publisher, to be filling the gap?
Yeah, it’s a great question because obviously there’s a part of me that there is a gap and somebody does need to address it so I think that’s a good thing. Without having taken any of the courses, I guess I would wonder if they’re going to be replicating certain systems. And if they’re going to be taking a very particular and narrow view of how peer review should be carried out, and again whether they would consider those mentoring aspects of it. I think maybe my sense is that many of us are adopting that perspective. I would just wonder. I think it could be a good thing and again, without having done it I can’t really say, I guess maybe I’m just skeptical of big publishers in general and maybe I would have to really check it out and see what I thought.
Emily: Have you ever thought about or experienced open peer review? What does that phrase bring to mind?
Yeah. I mean my understanding is I guess it could probably be carried out in a couple of different ways, but my understanding is that open peer review would be when either the peer reviewers, the names of the peer reviewers are shared with the author and the authors’ name is shared with the peer reviewers, and/or a third option is that it’s all actually put out there openly so that it can be publicly viewed by the peer reviewer.
I know there was at least one journal that asked me at some point at the end of my review if I was willing to share my name. I’ve never been on the receiving end of that, and I’ve never reviewed someone knowing who had written the paper. In terms of my feelings about it. I’m pretty torn, especially in terms of the idea of sharing the author’s name because just learning what I’ve learned about implicit bias, I feel like it could work maybe either way. The happy thought would be that this would be a way for us to be more inclusive in publishing by being aware of: are we constantly approving these texts that are coming from certain perspectives? But the flip side would be that we’re going to actually consciously or unconsciously block more publications from people of color, knowing that’s how implicit bias works. So I’m torn. I would be happy to do it if I thought it was going to work the way it was supposed to.
Emily: Yes, but opaque peer review is also not working the way it’s supposed to.
Yeah, I mean it just raises that question of which would be the better approach or the lesser of two evils or whatever.
Emily: Well, if they’re both evil why are we doing it at all? Because of implicit bias I wonder, can we require implicit bias training? Can we, in a rubric for a referee, include some thinking reflective questions about bias as we’re reviewing something? In opaque review are replicating the voices that are familiar and so is there privilege inherent in that this person knows how to phrase a question in the performative academic way? And then is someone being excluded from that?
I think probably. Even though I wasn’t necessarily taught how to be a peer reviewer, I was taught how to write a research paper and there was a lot of things implicit in that and I wasn’t even taught that until I got to the PhD program. It’s not just the structure. It’s not just that the lit review comes before the findings. It’s really the phrasing, the style. Consciously or unconsciously when I’m reviewing papers, that’s what’s in the back of my mind. And I imagine that’s what’s in the back of the mind of many reviewers: this is what an academic paper sounds like. Even just as an example, I have one PhD student right now who is using grounded theory for her dissertation. And so she’s using first person and someone on the committee came back and was like “but you’re never supposed to use first person.” Just as one very simple example. So yeah, I think that the system probably does replicate those things. And then two, even in opaque reviews if someone’s first language isn’t English that is often obvious from the writing. I’ve reviewed for journals that have explicitly said we’re really looking to increase our global participation or whatever so yeah, I think there’s a lot there.
Emily: There’s a lot. Can you think of what the ideal model would be?
That’s a great question. I don’t know. Off the top of my head I would really have to give it more thought. I guess I do feel like some sort of review makes sense. You wouldn’t want to just say everybody publish everything and then we all have to try to wade through it. I do think that a more structured approach where people somewhere get training on, first of all, what’s the purpose of this approach, how do we do it, etc. But then really thinking more broadly about what qualifies as an academic paper, what questions are appropriate, what methodologies are sanctioned. All of that kind of stuff, and figuring out what biases we all are bringing to that when we do those reviews. Because even in LIS I think there’s some bias towards certain kinds of methodologies over others. There’s certain methodologies being more empiricist or more objective. And I think we need to have a broader conversation within the field. I think that goes beyond just peer review and it’s really kind of thinking within the field. What are we trying to do here?
Emily: Why do you think we have that?
I mean my sense is that it’s not just an LIS problem. It’s a problem across all different fields and I think that these are things that sadly I am just really beginning to become aware of. It wasn’t something that I necessarily thought a lot about as a student coming up through the ranks, but just the idea that for so long so much of the power structure was white and male and that within that structure, then, certain decisions were made about what is a good academic paper, what is formal academic writing, what are the sanctioned questions and the sanctioned methods? And then once that happened, many of us were just kind of replicating the system without ever questioning “why is this particular methodology superior to that methodology?” or “why are we not interested in these questions?” It’s something that I have been talking a bit about with some of my PhD students, because a number of my advisees have gotten push back in different areas where they’ve been told “oh, that’s not really a good question for your dissertation” or “you really can use that method but you have to pair it with this other method.” And my approach has really been, “this is your project and we want it to be the best it can be, but I want to work with you to make it your project,” if that makes sense. And that’s getting off on a little bit of a tangent, but I think if we could start thinking more broadly about just what kinds of things are worth pursuing or what methods are acceptable to pursue them with, I think it would be better off for it. We would have a richer field of literature.
Emily: People who are working towards tenure can’t bite off a project that is a longitudinal study and get promotion and tenure necessarily because you have a timeline. Maybe people also don’t understand qualitative methods. Quantitative methods are sometimes arguably quicker to write about. At what level do we reinvent the culture of promotion and tenure because it is so limiting (and in my view a performance of whiteness and a performance of maleness in a lot of places). On the other hand, being tenured I feel secure enough to be able to say no. I feel secure enough to be able to say “I think that is a very bad idea.” Are there ways that we can change the culture of promotion and tenure and keep the good things about it?
I don’t know but it is an excellent question. And in some ways I think I may have been in a little bit of a sweet spot at Simmons because we’re not a research intensive university. Of course I felt pressure to publish and things like that, absolutely. But I feel like I probably had a little bit more leeway in terms of what counted or how much was expected in comparison to maybe some of my colleagues who are at R1 universities. I guess I’m not exactly sure where I’m going with that, but just to say that yeah, I think the culture is pervasive for sure but even if there’s some gradations and I think that when so much emphasis is put on research and on quantity within a certain time period, it really does drive people to focus on things that they know they’re going to be able to push out and that are going to have enough traction to get accepted. Like you said there are some really good things about the tenure system in the sense that once you are on the other side you have a lot more leverage to push back on some of those things.
Emily: Yes. I feel that replication too in terms of if you achieving tenure means that you are writing the article that the reviewers want because you know you need it published. And so maybe that is dictating your research methodology. Maybe that’s dictating the voice in which you’re writing your article. You’re not able to subvert the system, but still do really good work because the system wants this and you have to play with it in order to get to that place of power. I’m privileged because I have some power now because I am tenured. I think that’s the only power I have. These are the things I like to ruminate about and I appreciate you listening to me.
It’s fascinating. I’m really glad we got a chance to talk. I really appreciate having this time to talk about it because I think in a lot of ways, I feel like I’ve been approaching all of this and even teaching about all of this. Even though I felt like I was thinking about it a lot, this has really gotten me to think about it more and to articulate some ideas that maybe I was putting into practice without ever having articulated. So I think it’s great. I think it’s really helped me to think a little bit about how I will share these things with my students going forward, certainly, and even as I’m peer reviewing continuing to think about how I do that. I also just think, I mean you’ve raised some really great questions about the process – things that I know as a field we’ve been grappling with. But there’s such a shift in scholarly communication and publishing right now, that it does make me wonder if we might not be coming up on a good time to try and push for some kinds of changes. I’m not exactly sure how you do it, but I mean I published two textbooks. They are both coming out, and one is going to be coming out next month in open access.
Yeah, but that whole process, going through that process has made me think about just the whole idea of traditional publishing and again, the advantages that it brings and the disadvantages that it brings and I think peer review is, especially where it’s been attached in a lot of ways to traditional publishing, I think it’s at an inflection point. So it’s a good time for us to be having these conversations.
Today’s post is the second in a series of three parts that comprise Laura’s story. The first part,Just trying to get them to think about the nitty gritty of the process, unpacked her approach to teaching peer review in the classroom. The portion of our conversation shared in this post discusses Laura’s experiences as a referee, and her reflections on how she learned to do refereeing work.
Position: Associate Professor, Simmons University School of Library and Information Science
Fun Fact: Laura just got elected Secretary of the Board of Trustees for the Somerville Public Library.
Emily: So how did you learn to referee? Was it just practice over time? Did anyone mentor you?
I think it was a little bit of a few things. I wouldn’t say I was really mentored per se, except that I did work with two faculty members who were editors of a journal—even just by observing some of the things that they did. Both of them gave me feedback on some of my early drafts of articles. So kind of seeing how they did it I think actually was helpful. I guess there was some mentoring. I think the more that I was teaching—I don’t want to say that peer review is like grading papers because it’s not exactly—but there was some sort of give and take there where I also learned over time how to give better feedback on papers, just again by practicing and doing more of it. And so I feel like over time I just learned how to give better feedback in general. Things that were more constructive, more specific, etc. And then I think also some of it came from being reviewed myself and recognizing the things that were helpful and the things that were less helpful.
Emily: It sounds like you were self-directed in learning about it. It was just experience.
Yes, I think so. I mean I would say there wasn’t really anything in any of my formal training as a master’s student or PhD student where we were actually really taught how to do that. I mean there may be some places where this is happening, where it’s being done better now because I know a lot of my colleagues, for instance, are having their students peer review each other more, and they’re doing it with more guidance. Again, reviewing your peer student on a paper they’re writing for class is not exactly the same as peer reviewing a journal article, but at least you learn a little about giving feedback. I had been an English major and so how you write papers as an English major is pretty different than how you’re supposed to write research articles. And once I took the research methods class as a PhD student, essentially where they really broke down a research article, a few things clicked for me. It was like okay, now that I understand the structure of this article, I also know what I’m looking for. It was helpful for me in writing articles, but then over time as I reflected on it, it became helpful in terms of how do I analyze an article.
Emily: Do you have any experiences or anecdotes as a referee that are sticking out in your mind?
There have been one or two times where I’ve gotten some articles that started off strong, so they posed a question that I thought was really interesting, but then it just really seemed to fall apart. And those are hard because I want to be supportive in my feedback—and I’m always thinking about the fact that these are my colleagues even if they’re not people that I know directly. These are my colleagues who, like me, are trying to just do their work and do it well. So I think what’s sticking out is in some of these cases where I feel like there’s really some major issues, but trying to think about: do I just say reject this, forget about it? Or do I say, “here are all the things that you could do to try and fix this article.” Like I said I’ve been leaning more towards just giving the more detailed feedback and trying to encourage people. Again, we’re also working in a field where in most cases the issues that we’re looking at are either not executing the methodology as well as they should, or not doing a good job of analysis. It’s not like clinical trials or medical research or anything like that where anyone is going to be harmed necessarily by this. I feel like there’s more opportunity to say “look, just keep working on it. I think there’s something here that you really need to tease it out, give it more support.”
I do remember one time, and this was quite a while ago, I did suggest rejecting a particular article and it was mainly because it was incredibly, it seemed to me to be a very narrowly researched question with an incredible small sample size. And what’s interesting is that article did get published [and I don’t know that it went through a second round of review] or if it did it didn’t come back to me.
…. it really kind of made me think of was I using really looking at it from the perspective of this particular journal…. So basically I rejected something but it got published by that journal anyway, and it really got me thinking about whether I was not understanding what the perspective of the journal was. And it’s not as though I review always for the same journal either and so trying to remember and think about who is the readership of this journal and what is really appropriate for them I guess.
Emily: How do you think we could solve that problem? What would you as a reviewer need to be able to do that work better?
I think part of me worries that the information is all there and maybe I’m not just taking enough time to think about it. Like I said a lot of times they will send you some guidelines and things like that about the journal. In some cases I’m just thinking like “oh, I know this journal. I’ve written for this journal.” To an extent I think that the information in a lot of cases is there and we just need to take the time. I also think we probably could do a better job of packaging and sharing that information. And I think one of the problems – like I said we’re already volunteering our time to do the peer review. It takes a substantial amount of time to do it well, and if I have to add another – even if it’s another 20 minutes or half an hour to read through all of these guidelines and get this particular perspective—it is a challenge. It makes it harder for me to agree to do it, so I wonder if there’s some way without losing the quality and all the nuance. If there’s a way to just really, in a streamlined way, just say, “here are the things we want you to look at when you’re looking at this article. Here are the things you really need to know about the readership of our particular journal.”
This is a little bit tangential, but the other thing I have found really frustrating is that some journals give you a template. Often it’s set up like a survey where there’s a few closed-ended questions that you have to fill in the dots. Sometimes the questions do not line up with the article that you have just read. That’s really challenging and I think that’s something they need to think about. If you publish a range of types of articles, maybe research articles and more philosophical kinds of things, you might have to have a couple of different templates for people to respond to.
This story comes from Laura Saunders, a professor at the Simmons University School of Library and Information Science. We spoke in late July of 2020. This story will be published in three parts. This first chapter discusses Laura’s approach to teaching peer review in the classroom.
As you read through Laura’s story, you may find tension in her telling of how she teaches peer review in the classroom and how she herself approaches peer review when she practices it. (I know I noticed it.) We got to think about that tension together. Maybe her story will help you uncover the tensions in your own teaching practices as they relate to the practice of the thing itself.
Position: Associate Professor, Simmons University School of Library and Information Science
Fun Fact: Laura is currently studying Italian
In addition to her own research and publishing, Laura works to elucidate the peer-review process for her students. Among the classes that she regularly teaches, she teaches information Sources and Services, a user instruction and information literacy class, and an academic libraries class. Prior to becoming a professor, Laura was managing a small career resource library. It was affiliated with an academic library that also housed career counselors. Half of Laura’s time was spent providing reference and instruction services as well. She then pursued a PhD as a part-time student while working, and landed at Simmons. In our discussion Laura talked about learning to referee and refereeing, teaching peer review to students, and we also ruminated together on editorial power and open review.
Emily: Can you tell me a little bit about why you wanted to participate in this interview?
I think it’s a few reasons. One reason of course is just because I have been a peer reviewer and been the subject of peer review. So the process definitely interests me, but it also interests me in a little bit more of an academic sense for a couple of different reasons. One being because I teach librarians, emerging professional librarians and so part of what we talk about is peer review in the sense that some of them, especially the ones who want to be academic librarians, are going to have work with patrons who will have to look at peer-reviewed papers for their research, and so part of what they need to be able to do is not just locate those papers, but really understand what peer review is and what the process is so they can help their patrons understand that. Some of them are going to go on to publish themselves, and so they might be involved in that.
One of my PhD students who recently graduated, did her dissertation on financial conflicts of interest in scientific publications. And part of what she looked at—even though it was a little bit outside of the scope—she ended up including in her review some information about scientific things that had been retracted, and so failures of peer review. In that regard it’s kind of an interesting question.
Emily: How do you teach peer review in the classroom?
I actually it’s a little bit of a mix of trying to just give as good an overview as I can of what the process is and how it’s supposed to work. So just kind of explaining to my students why do we do peer review and what is it that we’re hoping to accomplish by this and then how does it work. It’s this idea that I focus a lot on peer-reviewed journal articles, so I’ll talk about that process of somebody writing an article, sending it out and that it will get reviewed by two experts in the field who will give feedback, and then you’ve got this range of possibilities from accept to reject and everything in between, with major revisions and minor revisions and so on. I’ll kind of go through that on a really basic level, and then from there I’ll usually just share with them some of my personal experiences because I feel like it helps them to understand, especially those who haven’t ever been through it.
I’ll talk to them a little bit about when peer review fails, and at times they’ll get really, I think rightfully, kind of annoyed with the idea that people aren’t doing their jobs. But then I’ll often ask them, “Well how much do you think peer reviewers get paid to do this?” And there’s this look. And people are looking around. And when I tell them, “Nothing. We do this out of professional courtesy,” it gets them thinking more about what is really involved, what the labor is that’s involved, and why it may not always be done as thoroughly as we would hope and expect. So just trying to get them to think about the nitty gritty of the process. Because on the surface level it sounds like something that should work really well, and it’s a nice setup and whatever, but thinking about all the little pieces and how they fit together and the fact that most of the time reviewers are doing this work without getting paid—and you do get some professional return on it in the sense that you can say you did this service to the profession, but since most of it is blind peer-reviewed no one is ever really going to know what you did and how well you did it. So just talking through all of that with them and sharing some of my experiences.
Emily: Do you have particular experiences that you find yourself sharing over and over with them?
I suppose so, yeah. There’s one, and it’s funny because I don’t really remember what the review itself was, but it was one of those things where the advice really stuck with me. It was while I was working on my PhD I was collaborating with my advisor on a paper. We sent it out and it came back with a request for revisions. I was reading through it and talking through it with him and he said something along the lines of, “you have to understand, sometimes people are reviewing the paper they wish you wrote instead of the one you actually wrote.” And it made something click with me that I have found to be helpful going forward. Sometimes when I get reviews now, instead of reading them as being overly critical or they just didn’t understand, I’ll start thinking about, “okay, what might have been in their mind? What were they looking for that was different from what I was actually giving them?” I share that story quite a bit.
I was recently collaborating with a master’s student who ran a survey for an independent study that I oversaw. I thought it was a really interesting research question, and I encouraged her to publish it, and I worked with her and we ended up going through two full rounds of revisions and at one point she asked me “is it normal for this to happen?” I could tell that she was feeling maybe a little bit like they were saying it wasn’t good work or something like that. As we talked through it and then as we got to the final round, she said to me “wow, I really see how much better this paper is now and how it was really worth it.” The paper is going to come out pretty soon.
I’ve seen that full range of the feedback that really isn’t particularly helpful. I submitted a grant proposal two years ago and it was reviewed by five people and they sent all five reviewers scores. It was interesting because the person who gave me the lowest score gave me two words of feedback. I have shared that story with my classes since then because I said, “what am I supposed to do with this?” I mean they told me nothing. If you’re going to volunteer to review you should at least put some effort into it.
Emily: At a journal you might have an editor that will tell you where to focus, what to work on. Does a grant officer play that same role?
Yes, at least for the one that I applied for they do. You’re able to meet with them and they can give you some high-level feedback. With regard to that particular grant proposal that did not get funded, she couldn’t go into a whole lot of detail, but she was able to give me a sense of what was the general consensus. What are some of the areas that I might work on to strengthen it the next time around and stuff like that.
Emily: You mentioned something about this whole idea of reviewers reviewing the article that they thought that they wanted to read versus actually reading the article that was submitted. Can you talk a little bit more about that?
I can remember in particular one study that I did where in two rounds of feedback one of the reviewers both times said something like, “well, you really should have asked this question as part of the survey.” And it was kind of like well, sure. To be honest I’m not sure I even agreed that that question necessarily should have been there, but whether it should have been or not, it wasn’t, so there was nothing we could do about it at this point. I think the first time around I kind of glossed over it, but then the second time around in my response to the reviewers I tried to say something like I appreciate this point and I mentioned in the area for further research that that could be something that someone could look at in the future. I responded, in reality it wasn’t there and here are some of the reasons why it may have been just as well not to have it there. I tried to give a response that I hoped would help them to understand what my thinking was.
Emily: Do you think the reviewer even got to see that comment?
I am not sure. It’s a really good question because I think it depends on the journal. When I have been a reviewer and I’ve gotten a second round, it has worked both ways. In some cases I just get a new revised article and in some cases I get the revised article along with my original comments and any response that the author wrote so I think it really depends on the journal.
Emily: Are you screaming into an abyss where you’re offering this response to feedback and the only person reading it is an editor? Where does that conversation go?
It’s a great question and, in fact, that made me think, too, just recently I submitted an article. I’ve been trying to look at how faculty members in LIS teach soft skills in their classrooms, so things like interpersonal skills, and I’m working with a collaborator, and we ended up deciding to survey people outside of LIS, too, so we could do some cross-disciplinary comparisons. So we sent the article out, the one that looks at these several fields that we surveyed. I sent it to one journal in higher education that I thought looked like a good match. We made a few points about active learning, that in order to learn these kinds of personal and interpersonal skills you really do need to practice, because interestingly and probably not surprisingly, a lot of the top methods the people said they used were readings and lectures. How well is someone going to learn interpersonal communication if all they’re doing is sitting and listening to a lecture? So it seemed like a good match, and the editors sent it back and said, “I’m sorry but I’m not going to forward this to the reviewers because your suggestions” – she said something along the lines of “you’re making suggestions about how to change classes that instructors could not use in their individual classes. Those kinds of changes would have to be made at a program level.” I was thinking, “no, they wouldn’t actually.” But again, it was kind of this awkward position where I definitely disagreed with what the editor was saying, and she wrote me this really long thoughtful response but I kind of felt like, “no, I really don’t think this is the case,” but I also felt like she had made her decision and the way it was written it just kind of seemed like it was pretty clearly a no and it didn’t seem like it was worth it to write back and argue. But there was a part of me that thought this could have actually been an interesting conversation and who knows, maybe even one that might have been of interest to the readers of that publication.
Emily: I was thinking about the power that this editor had in terms of selection bias of what’s getting published in that journal and what’s actually getting out in the literature and how it’s approached. So I feel like peer review is one of those where, without practicing it, how do you learn it?
Well I think you raise a really good point about the power that the editor wields. Definitely this was interesting so it was one of the very few times that I had a general editor reject something out of hand. And I did think that was interesting because like you said it does mean that there’s definitely a lot of power there in terms of just deciding what gets through the – it’s a gate-keeping function, right? And so what actually gets through the gate and when one person is doing all of that or most of that, there’s probably going to be some skew towards whatever their particular areas of interest might be, even unconsciously. I think that’s interesting and I think that could have been a case where maybe if it had gone out to a couple of different people, their responses or reactions may have been slightly different. Who knows? It is something that I’ve thought about a lot and again something that I’ve brought back to the classroom.
I remember quite a few years ago when I was first starting to teach in LIS, hearing a story on NPR that they had done a study of articles in health and medical journals. What they found is that, and I’m not sure if I’m remembering all the statistics correctly, but in clinical trials when the results were positive, the articles got published something like 60 percent of the time. When the results were negative, they only got published 12 percent of the time. I remember that really striking me because those negative results could be just as important to know about. But, of course, for lots of different reasons they’re not getting published and there’s a reason to look at those kinds of publishing trends in medicine, but we’ve got to figure it’s happening in other fields, too. I’m sure in LIS there aren’t a lot of people writing articles about how we failed or how it didn’t work for us. Again, you can think of all the reasons why people may not want to write those things, but you do wonder what are all those things that are out there that would be valuable knowledge that aren’t getting published. And then it raises questions, too, about not just the content necessarily, but the voices that are being stifled and the idea that often there’s going to be some bias towards different questions, different kind of methodology, different content, different authors, all of that kind of stuff. I think this idea of this objective blind peer review suggests that all of those biases get pulled out. But, of course, we know that’s not the case.
So as you said I think that does come back to what I try to teach in the classroom, and just really getting folks to think about, because so often and it’s been a long time since I’ve been in the library—teaching as a librarian as opposed to a library faculty member. I remember back when I was in classes where I would see a librarian say to the class, “okay, I know you need to look for peer-reviewed articles for this class, so in this database you can check this little box and get peer-reviewed articles.” And the implication seemed to be, or I was at least afraid that the message the students were getting in these cases were, “once I click that box, I can trust anything that comes back.” I felt like rarely did we have a discussion about how what you get might still be problematic. I feel like those discussions might be happening more now than they were 15 years ago or so. But trying to get my students to think about that and then finally I think it’s absolutely true: we’re not really taught how to peer review. Sometimes the journal will send you some guidelines when they ask you to peer review and they’ll go throw a set of guidelines at you or something. It is absolutely something that I had to learn over time and I think I probably do a much better job of it now. Do you know what I mean? I think about some of those people who got my first reviews who probably weren’t feeling too much like they were more useful than some of the ones that I’ve been complaining about because I didn’t really know what I was doing.
In this installment we talk more explicitly about what peer review should be for. Why do we do this thing? What can we do to improve it? Sarah shares her thoughts on what a hybrid open and opaque process could be and do.
Position: Associate Professor, Outreach Librarian
Institution: Hunter College – City University of New York (CUNY)
Fun Fact: Sarah’s favorite way to get around the city, aside from the subway, is by bicycle (especially during a pandemic). She’s never owned a car.
Emily: You mentioned the word gate keeping in terms of peer review and I’m wondering if that’s really what peer review should be doing. Do you have thoughts on why we do this process with publishing and what it should be? What should peer review do, ideally?
That’s a really good question and I don’t know that I have a good answer for it. I’ve experienced traditional peer review that I’ve talked about, I’ve also experienced open peer review, which is a different thing entirely.
I published something two years ago In the Library with the Lead Pipe, which is open peer reviewed. I knew who the reviewers were, they knew who we were, and that was a totally different experience. And I liked it because I felt like more care was taken in the exchanges because we knew who each other were. Now, I understand that one of the reasons for traditional peer review is allegedly to eliminate or reduce bias in the review process—you don’t know who the people are who wrote or whatever—but I feel like in certain fields like you can still guess, so it’s sort of artificial. I don’t have a fully formed thought on this, but I feel like academia in general is about gate keeping: who is in and who is out, whose voices are represented, whose are not? So peer review traditionally has obviously favored the white western male cis-gender heterosexual perspective; that’s who academia publishes. But that’s opening up and more and more voices are being heard.
I feel like at its best, peer review should be an honest review by colleagues or peers of the value of your work. I don’t necessarily feel like that should be shrouded in mystery. I don’t feel like there should be unwritten rules that you have to follow to get through that process. I feel like it can be more transparent. And I don’t know if totally open is the way to go, maybe a hybrid version of some anonymous review and some open review so that you have an open exchange with somebody and so that you can have a dialogue. I feel like the reviewer comments just go into the void and you never get to personally respond to them to say, “okay can you tell me more about why you said this,” or have a conversation so that there’s more of a human understanding between people about the reasons for doing things. I feel like there’s a value in being able to talk it out to say, “oh I misinterpreted that; you could make it clearer by doing these things.” That would be really valuable for somebody for revisions, to be able to say “I did it this way on purpose” or “thank you for that, that really helps me think about another way to write my work.” I’m not a scholarly communications person so I don’t have a lot of really well-founded arguments about all of this. I really liked that open peer review process, but I also see the problems that could come along with it if you’re reviewing people you know and you don’t really like those people, or you do like those people, as if you’re going to overlook things if they’re your friends or whatever. But I feel like there’s a place for a hybrid model that would still maintain the integrity of the process but also enable a little bit more of a dialogue between reviewers and researchers.
Emily: It sounds like it was a good experience and you liked it, at Lead Pipe, but you said that it could be problematic. Did you feel like you experienced some of those problems with that process?
I don’t know. I had met one of the reviewers at a conference once, so I knew who she was and I knew the names of the other people and so there was maybe a little bit of intimidation on my part. I really respected their work, but that could be said for any reviewer even if you don’t know who they are they should be someone whose work is valid in the field and they need due respect or whatever. I didn’t run into any of those problems really with that process. And like I said, it was a co-authored piece so we had the ability to have a conversation with each other about the process. It was interesting and it was totally different than anything we had done before.
Emily: Why did you and your collaborator decide to submit to Lead Pipe?
This article that was in Lead Pipe was another bit of my research that I care very much about actually, because this work was about this ongoing program that we had developed with a summer bridge program at our school. We developed a curriculum for students to be delivered over the summer while they were between high school and college. We had written and presented about developing the curriculum and about all this other stuff up until this point and then we felt like we had one more thing to say about this process. We interviewed a number of former students and asked them about their experiences with this curriculum. We wanted it to be a back and forth, if not fully co-authored, but certainly collaborative process between the students and us. And so we thought Lead Pipe was a good place for this because, stylistically, they publish lots of different types of articles. They publish qualitative research studies, they publish quantitative research, they publish theoretical stuff, they publish all different types of things. We decided to submit to them because we had submitted to a couple of other journals and had been rejected for lack of formality again, so this comes back to this voice that we wanted to tell it in. We wanted to tell this story and we wanted to use the student’s voices and we wanted to incorporate our own voices. So pitched it to them and they said, “yeah we’ll work with you on this” and we shaped it for their publication. I felt pretty good about how that ended up. We arrived at that because we knew that what we had written was not another traditionally academic style article so that was why we submitted there.
Emily: Thank you for sharing that. I don’t know if you know this but I was one of the co-founders of In the Library with the Library Pipe. It’s a place where we can have our own voice shine through and they don’t have to look traditionally academic but they still can be academic, there’s that flexibility. The open peer review process that we tried to create is still working. So, I just want to reflect that my heart is just full to hear that.
I was really pleased and I agree with you like it’s one of the few bits of professional reading that I keep up with because I think like the content is so interesting and so varied and I feel like a lot of it is still relevant to my work as well. So, thank you for founding the journal. I appreciate it.
Emily: Thank you for sharing that. I want to unpack this idea of a hybrid open since you mentioned it. I know you say you haven’t really codified for yourself what that hybrid would look like, but what should peer review look like with a degree of more transparency?
I mean I don’t have fully formed thoughts about this, but as I was talking it out with you, the idea of an anonymous reviewer has power because then you don’t have that sort of personal, “oh this person knows who I am therefore they can trash me if I make bad comments about them.” So I feel that there’s power in that, but I feel like it’s that humanity piece that’s lacking. And I know that’s partially the job of the editor, to communicate with writers and between reviewers and writers, but I feel like there’s an interim step. If you had an article that was reviewed by an anonymous reviewer and an open reviewer, then an editor or a co-editor could facilitate a dialogue between the reviewers and the writer to say, “so I’ve spoken with the anonymous reviewer and here are the things that they have commented on and I see that the other reviewer has commented on these things, now let’s talk about what all of this means for you.” If the editor could really make that a conversation rather than a one way: I’m going to give you comments and you have to do it or don’t do it. Let’s make it an exchange. There’s always value in talking about your research and writing rather than just writing and submitting, but to actually have a conversation with somebody about it and about your process and to get feedback to clarify your thinking. When I’m with students that’s what I say, too. “Talk with somebody about it, you should be able to talk about your research in a way that makes sense to somebody else, and if it doesn’t, then you have some work to do.” To be able to build that into the process—the reflective discursive process between an anonymous reviewer, an open reviewer, an editor, and an author. It’s a lot more work for an editor and I think it’s a heavier lift, but I think systems need to change. I think that there’s a place for that now, given the move toward open review and open scholarship.
Emily: I like the collaborative process. The approach that frustrates me – the purely gate keeping approach – makes it feel almost robotic. I feel like research is a human endeavor and what we’re doing as librarians is a human endeavor and I feel like peer review practices should reflect the humanity, just like you said, it should be a human interaction, it should be approached with care and love and collaboration and support.
Particularly in this field. I think maybe in the hard sciences it’s a little different when everyone’s trying to submit to Nature and get published in these high impact journals, but I feel like in librarianship our humanity is part of what makes us good librarians. That, I think, should be reflected in our literature and the process by which this stuff gets published should also reflect the humanity of the profession. Same thing with social work other of these professions that are caring professions or human centered professions—I think there’s space for that. I don’t think we need to mimic the hard sciences to be valid in the research, I think we can carve our own path.
Emily: But why do we mimic the hard sciences or why have we in the past?
I think that some of that has to do with the inferiority complex in academia that librarians have with “we’re not really faculty but we are called faculty but we don’t have PhD’s all the time.” We walk that line between service providers and researchers and I think that there’s always a bit of a struggle there. Some folks purely identify as academics, where some people say we really are in a service profession. I think that there is space for both of those things to be true, but I think that there’s that conflict. And I think that the people—particularly who I know in CUNY—who fought for faculty status for librarians are not willing to give that up and are not willing to bend that to mean anything other than, “I am a faculty member, I am a researcher, therefore I will mimic the processes that have been in place for however many years even if I don’t think that they’re appropriate for our field.” I respect that and I respect the folks who came before who fought those fights. I think it was really important for people to do that and to speak up about the fact that we are researchers and academics as well, but I also think that times have changed and the profession is necessarily changing and evolving and academia also is necessarily changing and evolving and that there is room for more conversation about what is the purpose of this now. Do we need to mimic these systems of oppression? (For a dramatic turn of phrase.) Do we need to continue in this vein or can we say, “okay so we did that for a while and now we see a different way for ourselves forward?”
Emily: I’ve been calling it our insecurity problem and actually it comes through in my book quite a bit, but I’ve been pretty influenced by the introduction to this bookThe Self as Subject. Ann-Marie’s introduction, she posits that if we as librarians – she’s quoting someone or paraphrasing someone – if we don’t really have a certain guiding theory so where we have this insecurity – are we faculty? Are we not faculty? Are we service providers? Are we not? How do we exist in this world? When that happens our research will be guided not by what we want, but by what our institutions want. And so if our institutions want us to be doing quantitative research with methodologies that are “tried and true” or positivist or whatever, then that’s the direction that we’re going to go because we’re uncertain of who we are. And so, we’re letting the institution dictate our research, our peer-review processes, et cetera.
We’ve been having those conversations on our teams. We are revising our tenure guidelines now and I’m on the committee that’s doing that. We’re talking about the need to push for giving weight and value to the things that we value in this profession rather than mimicking what other, differently focused departments have done. I think those are really important conversations to have. Then also engaging the administrators who are reviewing files in order to be able to make a case that these things are important. I remember a while ago somebody at the college saying, “oh you need to have solo authored blah, blah, blah X number of solo authored peer-reviewed articles,” and I pushed back. I said, “we don’t do anything solo. I have had one solo article. Everything I’ve been writing, all of my work, is collaborative. I have to work with other people.” That’s not the point or the realm in our profession; we don’t have to write a book, we don’t have to do a solo research study, we worked together. I publish with the same people a lot. You’ll see on my CV that early on I had these two colleagues I worked with them two or three different studies, and then later on it was two different colleagues that I worked with on a bunch of other things; that’s how we work. We work in teams and we have to reflect that in our literature. It’s disingenuous to do it otherwise, I think.
Institution: Hunter College – City University of New York (CUNY)
Fun Fact: Sarah is from Illinois but New York is her adopted home. She says, “I can’t imagine living anywhere else, and I’m so glad to be raising my kids here (even during a pandemic).”
Let’s pick up where we left off, with feelings.
Emily: How did you find the emotional fortitude to move forward after feeling so – I mean you didn’t use this word but it sounds to me like you felt really devastated by that.
It did. I think that’s an adequate description. It took a moment. I read the comments, I got really upset about it and then I thought, “well if this is important enough to me to work on then I’m going to take the comments in the spirit of constructive criticism.” And I also talked to colleagues. I said, “look I’ve got these conflicting comments from three different reviewers, some of them were helpful, some of them are harsh, what do you do with this information?” And a couple of my colleagues who were tenured who had gone through this process before told me “you have to figure out what’s important to you to stick to your guns on and you have to figure out what you’re willing to bend on and how you can still maintain your integrity and make the revisions that they’re asking for.” They said when you send the revisions in you can go reviewer by reviewer or comment by comment and say here’s how I modified it according to his comment. I didn’t agree with this one so I didn’t make any changes based on the this one. You can go and you can justify that it’s not just like someone is grading your paper that you have to do the things they ask you to do, you can actually push back and say, “no actually it’s written in this journalistic style because that’s the way that I wanted to present the work and I maintain that this is an appropriate way to tell this story.” So I think talking to other people and understanding that it was okay for me to do that was really important. I was able to, after my initial emotional reaction, go back and say okay now that my feelings are in check, I can see the value of these comments and I will then address the ones that I feel are worth addressing and I will tell them why I didn’t address the other ones. And that’s exactly what I did.
Emily: Was that the first time you had ever learned that you could do that? You had no idea what to do and that kind of negotiation of the process was totally new for you?
Yup. It was totally new for me. I had no idea what to do. I thanked the reviewer and said they gave me a lot to think about. To address the comments I outlined the changes I made. And to the reviewer number two I said, “I was unsure how to respond to the comments from reviewer number two, as it seemed to me like he or she wanted me to write an entirely different article than the one I submitted.” I said, “I do intend to continue my research in the broader area of these areas, but this is not the focus of this particular manuscript.” So, that’s how I addressed it. But yeah that was the first time that anybody had told me that it’s okay for you to push back on this and say I’m not doing this because it doesn’t maintain the integrity of my research. That was powerful to learn and I think and that’s something that I try to pass on when I’m asked to informally review others’ writings. If you get comments back that you disagree with, you are allowed to say that you disagree with those comments.
Emily: It’s such a black box. We’re thrown in as librarians into these academic positions, and we have these promotion and tenure requirements, and we don’t know what we’re doing when it comes to this stuff. Wow do we learn?
There was basically no mentorship in my institution. CUNY is a huge place, there’s a lot of colleges; Hunter has a pretty big library. When I first started I asked if there were mentorship opportunities for people who are new in the tenure track. And there wasn’t really so I started a committee of junior faculty who get together and talk about their research, but it’s similar to the peer-review process. The tenure process is so wonky and so ill-defined at most campuses; each one has different requirements and each chief librarian has a little bit different advice. Nobody was willing to state “here’s what you need to do” because they don’t want the liability. They want to have flexibility and so nobody is really willing to mentor in a formal way because they don’t also want to be in trouble over telling people the wrong thing. So, there’s this real culture of fear around it, which I think sucks, but at least I had some colleagues who were willing to talk to me about it and have these informal opportunities to talk about my research.
Looking outside the library I had other faculty colleagues that I spoke with. The chief librarian at the time gave me some good advice. He was not an academic, he came from the corporate world and so it was interesting to hear his perspective on it. I think so much in academia, there’s so much gate keeping involved, you have to know the rules before you even begin, you have to just figure out the rules and no one is going to tell you what the rules are until you screw up and then they’re like, “oh by the way you could have done it this way.” Well, you could’ve told me that in the beginning and set me up to succeed rather than setting me up to fail. I just think it’s unfair. So, I’ve tried to pay forward the lessons that I’ve learned when other people come and ask me about things.
Emily: So, how did this experience compare with other experiences you’ve had in peer review? Has it informed how you serve as a referee? Has it in formed how you submit or how you write?
I’ve only refereed a handful of times but it’s definitely had an impact. I feel as a reviewer you get this feeling of, “I have the power to make or break this thing,” and it can be a power trip. Knowing what I know about it now, my approach was to be very careful with the work because it was somebody’s very hard work. Just think about the way you word things. It’s like when you’re talking to people who you care about, you don’t speak harshly to them, you try to gently encourage them in a particular direction. It’s really informed that for me that I don’t make mean comments. I feel like some reviewers take it as an opportunity to be mean. I don’t know why, I guess it’s a power trip or human nature—sometimes people need somebody to take stuff out on. But for me there’s value in the fact that somebody put in the work to submit this thing and so I need to take that in the spirit in which it was submitted and give thoughtful feedback that would help them rather than tear them down. I teach a one-credit class for the library and it’s helped me giving student feedback as well. Students are trying to do their best. So don’t be mean, just tell them what you want them to do, and tell them when they did it right, and tell them when they didn’t, and offer them an opportunity to revise their work. I think it’s pretty simple.
In terms of submitting my work, everything else I’ve authored has been collaborative and so that’s been a totally different process. I think the reason this article about the Puerto Rican prints was so personal was because it was my only solo-authored piece. When you get rejected for a group effort there’s other people to experience it with. “Okay well clearly that wasn’t a good fit, so let’s figure out how to move on.” It doesn’t feel as personal to me; I don’t care about that research as much. I haven’t solo authored anything since this, if that’s any indication of how rough it was. I would love to, but I haven’t had the time. I’m working on another big archival research project now, sort of slowly, obviously during COVID I can’t get into the archives, but I’m a little nervous about that one, too, because it feels awfully personal because I care about this research as well.
Emily: You said that you didn’t care as much about your other research projects, but I’m going to interpret that a little bit differently so please tell me if I’m wrong. I’m going to posit that maybe when you’ve had rejection of works that were collaboratively authored it’s that there was a sense of camaraderie in the rejection and that there was a space for processing that.
I think that’s a fair way to say it and that definitely was the feeling. Saying that I didn’t care about it as much feels flippant, but quite frankly it’s true to a point; I’m not as invested in academic library literature as I was in this one particular research study. I have to publish for tenure, and I got tenure, and I’m really pleased with that, but I know that the stuff that I have written—aside from this one bit of research that I did—is not going to be as impactful on the world in any meaningful way as. That’s why I say I didn’t care about it as much, because I wasn’t as personally invested in it. There are things, obviously, that I have cared about since then and that I do still care about. Actually, now that I think about it there was one chunk of my research I do care very deeply about as well, but that was also collaborative and so when that stuff would get rejected, we would come back to the team and say, “okay so how can we make this better because we do think this needs to be told.” But yeah, I guess that’s the danger of writing about something that you really care about. It’s not that I think the other work is bad work, it’s just that I don’t think it’s necessarily as impactful or important.