This post is the third and final in a series telling Debbie’s story. The first part, Clear about process, discusses two of Debbie’s recent experiences submitting journal articles. The second part, It has to be about the material, delves further in depth into Debbie’s thoughts about why we peer review. In this third and final part of Debbie’s story, we explore her thoughts on open peer review, and explore what Debbie thinks would be a good approach to training for early career professionals or those inexperienced in peer review. We also explore together critical approaches to peer review and discuss what’s happening in our communities in regards to critical reflection about power, research, and reviewing.
Debbie Schachter
Pronouns: she/her/hers
Position: Director, Library Services & Learning Commons, Langara College; Director, CAPER-BC
Fun Fact: Debbie recently got her first electric bicycle, which she named Silver (it’s a RadRunner Plus).
Emily: I wanted to explore open peer review a little bit with you. Open peer review can be any number of things. There have been attempts to define it but there’s not really one agreed upon way that people define it. Essentially an all too basic definition is that open peer review is when a reviewer and an author can know who one another are. At a more robust implementation it could be that there’s actual engagement or back and forth in conversation between reviewers and authors. Hearing that and realizing that this might be the first time you’ve thought about it, do you have any reactions or thoughts about challenges with that or positives of something that would enable that?
When you know who they are it can feel hurtful…
I think that it could be very positive. I think it would definitely help the growth of the individual who’s writing, who maybe is new to research or new to publishing. It reminds me of the experience that you have with a supervisor—and my supervisors were very good. But I know that there were people who had supervisors who were very bad and not helpful; who were not able to communicate what their vision is very effectively and not able to provide helpful feedback. It does take two, so whatever the reviewer or whatever your supervisor is telling you, you have to understand and reflect that back to them so you are both clear about what you are talking about. It can be challenging depending on the individuals. When you know who they are it can feel hurtful if you are someone who’s inexperienced and your writing [you think] this represents you—which it doesn’t—but it can be hard to take feedback. Sometimes it’s easier when it’s behind that anonymity shield, but I think as we’re trying to learn and engage in that scholarly conversation it makes sense. It’s kind of like having a mentor or a supervisor. “Yes, I see that you’ve written or you’ve researched on this interesting topic. In order to make this a really successful paper for publication here are some of the things that you need to do.” I think that is really valuable. That is the whole point so again, it depends on the people and how they would approach it. You would think that the person who’s submitting a paper would necessarily have to understand that they have to be open to feedback. The weirdest thing would be you get something back and it’s like oh yeah, it’s fine. It’s like really? There’s nothing?
Emily: Have you had that experience?
No.
Emily: I have.
Doesn’t it feel like, “did you read it?”
Emily: Yeah, it did because when it happened to me it felt like I worked really hard at this, but I know there’s something that can be improved. I just hit my wall. I’ve had that happen a few times and it felt very frustrating to me.
I’ve had it in different contexts so I know it’s like if I wrote a report at work and “oh yeah, that’s fine.” It’s like really? Did you read it? Do you have anything to add? That’s just what I mean if I were being open like you were. You want to improve it. You want it to be critiqued because it’s going to be published.
Emily: I know you had really good experiences going to the ECIL conference and that you had some experiences as a doctoral student. What would your ideal be for folks who aren’t in doctoral programs? How could they learn about the peer review process? What do you think would be the best way for that to happen?
In addition to those two areas I mentioned, I did go to a peer at my own university. She was an AVP who publishes a lot. (She’s not with us anymore.) She wrote a lot of articles. You know what, I just talked to her about what it’s like, what her experiences were. She also reviews. She gave me an example that was very similar to the scenario I had with one of my papers and I thought “oh, okay, well if her experience is similar and here she’s been publishing for 20 years or longer, I don’t feel so bad.” So, talking to someone who does publish, I found it to be quite revealing and really helpful. “So, what kinds of things are you looking at? What’s your experience? How do you navigate it?” And everyone is going to have a different experience so finding more than one person who might act as a mentor even would be ideal. I always hesitate to ask people to read articles, but people do offer so that’s the other side where if someone is willing to read and give it a bit of a critique or give you some feedback before you submit, that’s also a great way to start the process. But again, you don’t always find willing readers.
Emily: Your doctoral work was on critical information literacy. Do you see a connection between critical pedagogy or critical information literacy and peer review?
[long pause] I’m not really sure actually. Because the thing about critical information literacy or critical pedagogy is we’re asking the individual to be bringing themselves wholeheartedly into the learning process. Reaching down into their own experiences and making sense of what we’re talking about if we are providing a bit of a framework or some information. We’re actually having them draw out from themselves. So perhaps if peer review is done effectively it’s kind of triggering that learning experience. I could see it from that perspective, but the other side is it could just simply be like your teacher marking a paper where you’re not having any kind of critical reflection. You’re not bringing anything further to it. You’re correcting grammar or you’re redoing structure. You’re not necessarily having a fulsome conversation about a topic and reflecting on it and arguing your own point of view through the review process necessarily. That’s what you’re doing possibly in the paper itself, but in the review process it might feel entirely one-sided depending on who’s doing the review. Other than that yes, it’s a critique. It’s a critique, but it’s a critique that I don’t know. That someone else is critiquing your work really. So it’s not quite, I don’t think.
Emily: The way that I’ve thought about it is that perhaps it’s not isolated just to that back and forth between the referee and the author. I think I’ve been looking at it critically—about the whole system. One of the questions that we ask in critical information literacy and critical pedagogy is: who has power and who does not have the power? How much power does an editor have? How much power does a referee have? How much power does an author have? And is it an appropriate balance? Particularly in the United States right now with all of the movement towards racial justice and antiracism or thinking about whether—not just the act of peer reviewing, the one-on-one looking at the article—but the whole system suffers from perhaps white supremacy.
In Vancouver the same questioning is happening. You’re absolutely right. Questioning of what’s happening to indigenous people and the incarceration rates and in communities such as Toronto where there is a larger Black population what’s happening with the BIPOC communities. And also in the Maritimes. Yes, the same questions and whose voices are we hearing and whose voices are we not hearing. No one ever stops, right.
Emily: Exactly. And so I’ve been trying to think about the peer-review process, not just the one-on-one or the focus at that small level but at the macro level—whose voices are not there? And for me the question comes back to the culture of higher education—whose voice isn’t present in the culture of higher education or not as present.
Maybe things are improving, but you still have those same voices excluded.
The good thing about the blind review is you don’t know the authors’ names, so you don’t have any sense about like, this is where whether women’s voices are being heard or not. You can set that aside because you’re not making assumptions based on a name. That’s a good thing. But depending on how the article is written it can be very clear. I’m talking about my personal experience as a woman. It’s like okay, what kind of bias is the reviewer bringing? So I can certainly see in that sense as you were describing the whole process for peer review yes, we’re not taking a critical approach to publishing. I think we’re probably still in exactly the same situation we have been for a number of years. Maybe things are improving, but you still have those same voices excluded. Either they don’t have the opportunity, or they don’t have the wherewithal to publish. So it goes back to those well, you know, we review everything that comes in, but what if it never gets to the publishers. What if it doesn’t get to the journals? That’s where our fundamental problems still reside. The review process, if everything were done effectively, the editors would be out seeking input. And you know calls go out but it’s not quite the same thing is it? Just getting the call. And then working with people so that they can actually publish. That’s a whole other supportive structure that doesn’t exist.
Emily: This reminds me of my former colleague Bob’s experience. When he started discovering autoethnography as a method and using it he faced some bias from journal editors saying this method isn’t acceptable.
That is really interesting. The only reason why I find it so interesting is because we had a lot of conversation about it at the University of Edinburgh where I was doing my doctorate. So I was at the Moray School of Education so that was the school. Autoethnography was an interesting topic that we examined because we were exploring the different ways we were going to conduct our research. There was a faculty member there who had done her doctorate and her thesis in autoethnography research. So we had a chance to look at that particular form of research and I thought “oh, this will be so excellent.” We looked at examples where it was maybe in Scotland, kind of underprivileged children experiences, articles that were autoethnographic or put together different components of that and how that did very much bring voice to people who are underrepresented. It’s not like Paolo Freire was doing autoethnography, but some of his work with I guess ill-educated peoples could easily lead to that kind of bringing students in to be able to conduct their own and to be able to publish their own story. So that linked to critical literacy itself. I found it really interesting. That’s an area where I’m probably going to look at in the near future.
Emily: Bob and his colleagues were trying to get autoethnography into LIS and it just kept being met with this bias and so that’s another thing that I think about—if you’re thinking about a critical view of our whole peer review system. If an editor isn’t even interested, if that selection bias is introduced, I see that as wielding great power as to what voices or approaches aren’t getting heard.
But we have to start talking about it because again it’s like “oh yeah,” when you talk about these things people go, “oh yeah, I’m sure that’s true.” Unless you’re actually actively talking about it, it’s never going to change. You say implicit often, sometimes explicit bias is really, really important. I totally agree. It is interesting. That’s why I did an Educational Doctorate, because LIS has been very much focused on quantitative and practical research rather than qualitative. In social sciences and the move towards accepting the validity of qualitative research we’ve come a long way in the past several years. The validity of that form of research now is usually accepted, especially when you’re dealing with people. For LIS to still be resistant is a little bit disingenuous. This isn’t a business degree and even if it were you’re still qualitative.
Emily: Yes, this conversation that we’re having comes up a lot for me when I’m talking with folks about qualitative versus quantitative. I flippantly call it our “insecurity problem” in LIS because a lot of us only have masters’ degrees and yet we’re in these faculty positions or we’re academics, we’re teaching and we have to somehow prove that we’re as good as the other faculty. And one way that’s been more historically accepted is to be doing quantitative research.
You look at how the data is collected. You look at the validity of the data and you’re like really. Okay, and then what? You still have to put the story together and that’s the qualitative. That’s what gives meaning to the numbers.
Emily: Absolutely. I believe in stories obviously since I’m asking for people’s stories. Do you have any other thoughts or things that are coming to mind about peer review or your experience or a story that you wanted to share today?
I don’t think I have anything else to share at this point. Like I said, as I teach mostly masters students now and library technicians, it is interesting to work with students and I could see some – I can just see that feedback mechanism that happens when you’re teaching and you’re working with students and how you might take that away and apply that into your research as well. But I don’t think I have anything further other than I look to support anyone that works for me or that is a student of mine to consider research and consider writing and publishing just to, like you say, share their voices and those voices that we’re not hearing. I just hope to be able to support others in their own aspirations. That’s kind of my aim out of all of this. Not that I necessarily will be publishing much more myself in the future. The research itself is the interesting part I think.
Stories of Open 