From the journal impact factor to the h-index, there has been plenty of discussion and debate among scholarly circles about ranking policies and practices on the article, journal, and even the scholar level in the last year or so. As scholarly blog posts give academics an ideal platform to tackle – and work through – hot-button issues through less-formal commentary and analysis, we get a nice glimpse in the ACI Scholarly Blog Index of what those in academia are currently talking about and/or directly experiencing. It gifts us with a “bug on the wall” view of the faculty lounge, framing certain experiences that can’t necessarily be inferred from the publications section of a CV. And because such posts often reflect an even broader view of what is being (or will be) published in academic journals, we always like to see how the commentary and data points parallel the peer-reviewed literature.

Layering both offers an interesting view of the research behind the researchers, and vice versa, and of the experience of academia behind the academics. All of this can tell us what’s important to faculty and staff, and discussions on impact and ranking in the measurement of contribution or significance is, most definitely, a significant issue to – and with significant impacts on – academic researchers.

There are many papers that touch on impact and ranking policies and standards. While such articles often appear in publications that cover broader topics or reach a wider net of subject experts, like journals on “higher education”, it’s fascinating how broad-reaching articles in more niched publications can be. One such example is an article entitled Journal ranking effects on junior academics: Identity fragmentation and politicization, written by Bertrand Malsch, PhD and Sophie Tessier, PhD and published this spring in Critical Perspectives on Accounting. In the paper, authors Malsch (Queen’s University) and Tessier (HEC Montreal) take a combination of ethnographical and autoethnographical approaches to look at how a shift in policies regarding ranking and measured significance would have an impact on those early-career researchers who are still working to establish a strong foothold in their fields.

According to Malsch and Tessier, the implementation of the Article Influence Score presented some unique challenges for their junior colleagues. While the journal impact factor is based on what Malsch and Tessier describe as the “repeated inclusion of articles in referential chains”, the Eigenfactor Score is a measurement that attempts to quantify a “journal’s total importance to the scientific community”. The Article Influence Score, then, is a score calculated by dividing the Eigenfactor score by the number of articles published in a given journal. (You can read more about both of these here.) In their paper, the authors describe how the impetus for the topic came about. Malsch and Tessier write that, following the adoption of the Article Influence Score standards, “the criteria for tenure changed. Whereas before A publications mattered only for material reward, they now mattered for professional (e.g.: tenure) and symbolic (e.g.: professorships) recognition as well.”

Ultimately, the authors found that while the ranking system may pose challenges that junior researchers need to overcome, the consequences of overcoming those challenges can lead to greater diversity and an increased sustainability in academic research – given the continued consideration of ranking design and faculty support. Ranking policies will likely always play some role in the research (and the funding of that research) in most universities, so in Journal ranking effects on junior academics: Identity fragmentation and politicization, they look to ways to minimize negative impact on, and maximize potential benefits for, early-career researchers. As Dr. Malsch notes in ACI’s interview with the authors below, “The key point is not to fight against ranking but to make sure that the making of rankings is open and transparent, and that their design can help increase and maintain intellectual diversity.”

bertrand abstract
Malsch & Tessier, Journal ranking effects on junior academics: Identity fragmentation and politicization. Critical Perspectives on Accounting V 26, Feb 2015, 84–98.

Akin to ecosystems, research fields are fragile entities. Just as introducing non-native species can cause substantial shifts in the functioning of an ecosystem, adopting a new research incentive policy or modifying journal ranking processes can cause substantial shifts in field research dynamics and affect its sustainability.
Malsch & Tessier, Journal ranking effects on junior academics: Identity fragmentation and politicization

How did the new ranking process come about? What gaps or needs was this change initially supposed to fill?

Sophie Tessier: The ranking itself is not new. It’s the sorting criteria that changed. The Impact Factor was replaced by the Article Influence Score. The aim was to use a score that did not consider self-citation.

Bertrand Malsch: The original version of the ranking was supposed to help change the research culture of the school by making it more ‘productive’ and encouraging professors to concentrate mainly on a certain set of academic journals.

While the policies implemented would affect all researchers, your paper highlights the impact to junior researchers. Is that due to the likelihood of being affected, or the intensity? How did this awareness first come about?

Tessier: The main reason these policies have such a big impact on junior researchers is that they are very often linked to tenure. Hence, I would say it is both: since one’s position is at stake, it is a matter of intensity and since more and more universities use ranking for tenure decision, the likelihood that junior researchers will be affected increases.

Malsch: There is also another reason which is connected to the fact that junior researchers are, by definition, at the beginning of their academic career.

A dominant view emerging from this body of research is that rankings represent a threat to diversity as they tend to impoverish the originality and practical relevance of accounting research. Our personal confrontation with the research incentive policy at our school and a new journal ranking process tends to reinforce that point of view.
Malsch & Tessier on the present body of available research in Journal ranking effects on junior academics: Identity fragmentation and politicization

While the journal publishing your paper was discipline-specific, your paper would resonate with academics in all disciplines. Have you talked to colleagues in other fields?

Tessier: Yes, I have discussed this paper with colleagues in management. The idea was to question the image of helplessness and passivity often associated to junior researchers.

Malsch: Not specifically. But the increase of ‘publish or perish’ kinds of pressures is a phenomenon that has been well documented across a wide variety of fields. There is no doubt that our experience will make sense for other young academics engaged in perilous tenure track positions…

Any thoughts on a future paper on this topic for a different or broader audience, for example a higher education journal?

Malsch: The writing of this paper took significant emotional energy (at least for me). Academics are not used to publish that kind of stories and to reveal publicly the background of their institution. I hope I won’t have to do it another time in my career.

Tessier: I agree with Bertrand. This was an emotional paper to write and for the moment, I do not plan to write on this subject again.

We do not defend, in this paper, a radical view against journal rankings… [I]nstead, if used with caution (for instance, developing different sub-discipline rankings for non-mainstream research fields such as accounting history, social and environmental accounting or taxation), journal rankings may be one instrument, among others, that serve to promote diversity and achieve greater sustainability.
Malsch & Tessier, Journal ranking effects on junior academics: Identity fragmentation and politicization

Your paper points to how early career researchers can use these types of ranking standards to their benefit. Can you touch on that?

Malsch: It is difficult to imagine an academic world where rankings would disappear. Competition and comparison belong to human activities. The key point is not to fight against ranking but to make sure that the making of rankings is open and transparent, and that their design can help increase and maintain intellectual diversity.

Tessier: I would add that junior researcher need to keep in mind their university’s ranking, but also have a general idea of how different universities rank the same journals differently. Rankings vary a lot.

What can senior researchers, and institutions, do to support that realignment in junior researchers?

Malsch: Senior researchers should be at the forefront of research institutions to defend junior researchers. Especially tenured professors, since the latter benefit from a situation where they can take risks. It’s a matter of generational solidarity. On the other hand, institutions should be very careful with the manipulation and creation of rankings. They might produce unintended effects by demotivating young researchers.

Tessier: I think it’s important for senior researchers to listen to junior researchers’ preoccupations about ranking and be proactive in fixing any inconsistencies.

To remain alive, the ideal of diversity needs debates and controversies between ideologies, paradigms and ideas. However, what we should aim for is a ‘civilized conflict’ between opponents respectful of one another and aware of the inherent limitations of the intellectual perspectives they support.
Malsch & Tessier, Journal ranking effects on junior academics: Identity fragmentation and politicization