tenure

Reconsidering research impact

I have a complicated relationship with bibliometrics (i.e. “research impact metrics”).

The silhouette of a person looking at a huge swirling modern art piece on a digital screen. — Photo by Gunnar Ridderström on Unsplash

I have a complicated relationship with bibliometrics (i.e. “research impact metrics”), as does the field of academic librarianship and academia itself, if we’re honest. On one hand, there is something undeniably fascinating about how statistical formulas can help us understand “scholarship as conversation” at a macro-level. As a field, bibliometrics has been heavily influenced by information scientists. Eugene Garfield, often cited as a founder of modern bibliometrics and cocreator of Journal Citation Reports, held a degree in Library Science from Columbia University, for example, and created the infamous journal impact factor not to assess the quality of a researcher’s CV, but to help librarians make decisions about which journal subscriptions to purchase. This relationship with librarianship makes me feel some level of responsibility to/for the field of bibliometrics even if I’m not a bibliometric scholar per se. Librarians’ collection development responsibilities have traditionally positioned us as arbiters and assessors of information quality, for better or worse, which tracks with the project of research assessment.

On the other hand, several high-profile bibliometric formulas have been coopted by corporate entities to serve for-profit purposes and have been misapplied in the academe as markers of excellence for individual researchers. In the 2010s, research assessment experts denounced the use of Clarivate’s journal impact factor and overreliance on quantitative citation-based evaluation of research “impact” through statements like the San Francisco Declaration on Research Assessment (now operating as DORA) and the Leiden Manifesto for research metrics. I share concerns cited in those statements about whether folks who use these numbers in retention, promotion, and tenure (RPT) processes can accurately explain the insight each formula is supposed to convey. Journal impact factor and Elsevier’s “CiteScore” are not generic proxies for journal quality. They aggregate citation rates over a (relatively short) period. In a vacuum, that seems like potentially useful information to consider as one part of a comprehensive assessment of a journal’s and/or a researcher’s portfolio. As a key determiner of whether a journal or its authors offer any value to the scholarly conversation, it becomes dangerous.

It is also well established that these popular metrics don’t map well across academic disciplines because different disciplines have significantly different publishing rates and citation practices, whereas metrics that normalize citation attention between disciplines like field-weighted citation impact are not commonly used in RPT procedures (from my understanding). Consequently, the perceived value of scholarship in the arts, humanities, and social sciences is diminished when their citation numbers are compared to STEM fields. In the fields that bibliometrics favor, their outsized influence in RPT processes forces faculty to “study for the test,” thereby limiting the list of suitable publication venues to a select, well-established few (often owned by one of the big five publishers). From a simple assessment standpoint, Goodhart’s law reminds us “when a measure becomes a target, it ceases to be a good measure.”

Although Goodhart’s law is certainly applicable to the current use of bibliometrics in academia, I think the problem requires us to zoom out a bit. The core issue leading to the misapplication of bibliometrics is academia’s attempt to assess a concept as sweeping and nebulous as “impact” using a few numbers that address one specific type of impact—contribution to the scholarly conversation. Furthering the progress of knowledge is a valid and core reason why many scholars do research, but it certainly isn’t the only reason why people do research, and it’s not the only value that research provides to the world, so it shouldn’t be the only aspect of research impact we evaluate and thereby ascribe value to.

Of course, making any kind of structural change in academia is easier said than done for many reasons, including the entrenchment of bibliometric-based evaluation in the publishing industry and in academic tenure processes. But, there are folks exploring alternatives: in 2023, the University of Illinois Urbana-Champaign introduced a Public Engagement Research Option (PERO) for Promotion and Tenure, designed for faculty who “address a societal problem to contribute to the public good; produce traditional and non-traditional outputs; collaborate with communities or organizations; [and] engage in a mutually beneficial exchange of knowledge and resources with community partners.” The HumetricsHSS Humane Metrics Initiative provides workshops and toolkit resources for academic departments to consider how they can embed their stated values in their evaluation of scholarly life, reminding us, “if we don’t measure what we value, we will only value what we can measure.” Just this month, DORA published their Practical Guide to Implementing Responsible Research Assessment at Research Performing Organizations.

I heard about that DORA resource while attending a new (to me) conference in April. The Center for Advancing Research Impact in Society hosts an annual summit around “broader impacts”—the National Science Foundation’s terminology for research’s “potential to benefit society and contribute to the achievement of specific, desired societal outcomes.” I’d been debating which research impact professional development opportunities I should explore, given that “research impact” is included in my job title. This one intrigued me because of its necessarily broad conception of impact, and I came away energized by the creative research development work happening around the world. Research development offices are logically very interested in understanding and tracking the impact of research conducted at their institutions, and they clearly spend a lot of time thinking about how to do research assessment well, and how to encourage researchers to embed the public good in their plans for research outputs. Two favorite sessions included the Association of Public & Land-grant Universities’ walk-through of “Modernizing Scholarship for the Public Good: An Action Framework for Public Research Universities,” and the Research Development team at University of Texas at Austin’s presentation on their Research-to-Policy workshops that teach faculty how to make their work accessible to legislators.

Although librarians are welcomed at the ARIS Summit, I didn’t meet any other library folks wandering between conference rooms. I mention this not to admonish anyone or toot my own horn, but rather to raise the question: why shouldn’t we participate at these types of conferences, contributing to and learning from conversations about research impact/assessment? I would argue the prevailing (mis)conception of research impact is an information literacy issue. As a librarian, I feel a responsibility to remind scholars of the ways “authority is constructed and contextual” and how our interpretations of information (like bibliometrics) influence the way those specific pieces of information come to have value. These issues are especially relevant to those of us in the scholarly communication subfield, as we play a particular role in helping scholars decide where to communicate and share their work. I now see the “open knowledge” and “research impact” components of my job title as much more intertwined that I originally thought, but it required me to conceptualize a holistic vision of research impact that incorporates the societal contribution we believe many types of scholarship can make. Academic libraries are increasingly involved at every stage of the knowledge lifecycle, and as such, I think research assessment/evaluation is an area where librarians could reconceptualize our approach, training, responsibilities, and campus partnerships.

Getting started with professional development

Last week, I did my first conference presentation as a tenure-track academic librarian! I’m actively resisting the urge to qualify or minimize that statement – it was virtual, it wasn’t about my own hardcore research, etc. I did it, and I’m proud of that! It got me thinking about professional activity as it relates to tenure (or in Salisbury University’s case for librarians, permanent status). I am not someone who comes from a family of academics; I distinctly remember getting “librarian” on one of those career profiles in high school, and immediately thinking, “Oh, no. That requires a master’s degree.” I never thought I’d be here, entrenched in academia and needing to think about publishing research. I’ll talk a bit about my most recent presentation, then some of my broader thoughts.

Thank you to the North American Virtual Reference Conference for the opportunity to speak. My talk was titled, Supervisor at a distance: supporting undergraduate reference workers. At Salisbury University, I am the Research Help Desk Coordinator. I am responsible for hiring and supervising 4-5 student workers in a given semester; they are tasked with answering all sorts of questions, including reference and research help ones. The presentation focused on current training and feedback strategies as well as initiatives I’d like to implement in the future. I don’t ever work on the desk with them – the research help desk only has one person at any given time, which is why I refer to myself as “at a distance.” I am not that far removed from my own student worker experience, so I’m constantly thinking about what I had or wished I had for their experience, too. For this first year, I made some minor changes – giving students more consistent feedback on their work and implementing a “Chat Transcript of the Month” email – but for the most part, I’ve been trying to see how the desk runs now before making drastic changes. These are my slides and references, though I’m happy to talk and answer any questions.

What was cool about this particular conference was that I’d actually presented here before; my supervisor as a graduate assistant gave me the opportunity to co-present here about my own entirely remote training as a result of COVID. Additionally, while I was a senior in undergrad, I presented at a statewide conference for writing centers. I hope to offer a similar collaboration to my student workers at some point. I try to make sure they know that I’m invested in their success, not just as workers but also as students. I’m positive that having that previous experience as a student gave me the confidence to submit now.

Even still, I find it hard sometimes to pursue broader research opportunities. Publishing in something like a journal still feels enigmatic or nebulous, even though I am intimately familiar with different publications, given the nature of my daily work as a research librarian. I think part of this is personal; I can be a true champion of others’ work and cheer on students and faculty alike with their research topics, but when it comes to my own, it’s harder to do. The imposter syndrome can be really intense. My inner critic questions how I could possibly add to the already bustling academic conversation, or my attempts at writing something like a journal article get held up in the research phase, wherein I try to consume everything possible about the topic. (My Zotero library is… robust, to say the least. Thank goodness for collections!) I also have so many different interests that it’s hard to narrow my focus on one research topic; I’ve heard this sentiment over and over from librarians, too. I often set out to learn more about one thing and find myself down an entirely different pathway.

In that vein, I’d like to turn it over to you, readers: what did your first foray into research or conferences look like? How did it come about? Did you have collaborators, or was it a solo venture? Do you have advice for new academic librarians who are navigating what “professional activity” means for them in their job expectations?

Lost in Exigency, Baby.

As a researcher of organizational communication, I have spent much of the past year observing and analyzing crisis communication by university and library leadership. Often I’m in the minority, praising communication and transparency I’ve observed from university responses to the pandemic, where others have been quick to find fault. Far from an apologist, my unpopular opinion just as often puts me at odds with leadership, especially when presenting views of those further removed from the hierarchy.

Kansas stirred up quite a bit of crisis communication last week as its Board of Regents (KBOR) unanimously passed a policy suspending University tenure protections aimed at addressing immediate budget challenges due to COVID-19. While no University is required to implement the new policy, KBOR gave Universities 45-days to respond with criteria and a process for invoking such a policy.

University administrators’ reaction to the policy was swift, either seeking broader input or acknowledging they would not implement. An open letter also quickly circulated, criticizing the policy as well as administrations who hadn’t joined in denouncing it outright. Without diminishing this alarmed response — the policy and its adoption are certainly worthy of it – my first and strongest impulse is always to ask questions.

One important question being asked is “why is a new policy needed?”

When KBOR’s Governance Board met to present and discuss this policy (around 19 minutes in), Regent Kiblinger asked this very question, in context and comparison to KBOR’s existing “financial exigency” policy. The response she got highlights several distinctions, while justifying it against other COVID-specific policy changes the Board has made. The first key distinction, though, really struck me.

“It treats everyone the same.”
KBOR General Counsel, Julene Miller

The new KBOR policy (6.b.ii) creates a COVID-specific exigency (if you will) applicable to any state university employee without the declaration of financial exigency as a prerequisite. Who is affected is the current focus of both the response from Universities and indeed the essence of how this policy “treats everyone the same”. It is admittedly more complicated , but deceptively that simple.

This sentiment — a takeaway from Kathleen Fitzpatrick’s very timely visit to my campus this week discussing Generous Thinking: A Radical Approach to Saving the University (John Hopkins University Press) – has bolstered an uncomfortable hunch I have about this current situation in Kansas. Fitzpatrick calls for a radical rethinking of how we participate in our collective values as communities of learning with a “duty of care”. While faculty loudly address the failure of leadership to make clear or understand the value of academic freedom and shared governance promised by tenure, we fail recognize and acknowledge how its structures like tenure and governance have also been built and broken down in ways that no longer live up to the values they intended to secure.

The key differences in the policy getting the most attention have to do with who has input throughout the processes and changes to the advance notice, process of appeal, and the affected employees. These explicit limitations on who is consulted, their representation and due process do raise justifiable concerns on a number of levels, one of which may be the further erosion of academic tenure in and beyond Kansas. But, remedying these concerns through a narrow lens of tenured faculty rights, just hits wrong. It misses the larger point, an opportunity even, that could be driving new and more equitable policy. Ironically, it misuses the privilege afforded by that which actually needs defending here.

What’s driving the policy?

The next set of distinctions the KBOR Governance Board outlined in its meeting mention the expedited timeline the new policy provides and an executive-centric role in the process. While financial exigency lacks as explicitly specific a process, it does ensure the executive role is shared with governance. We’ll come back to this.

But, first, the gist of the financial exigency. The Kansas Reflector summarized the policy even more succinctly [my emphasis inserted]:

Under existing Board of Regents’ policy, a state university must formally recognize a financial exigency that [has already] required elimination of non-tenured positions and operating expenditures. With the declaration, the universities could move ahead with reductions in tenured faculty positions.

By the open letter responding to this new policy asserting at the outset that, “procedures already exist to make decisions according to financial exigency as part of shared governance”, its authors acknowledge the similarly dire circumstances that concern both policies. KU’s rules and regulations for shared governance of financial exigency further emphasizes the gravity.

“Only as a last resort after all possible alternatives calculated to preserve the survival of the University as a quality institution of higher learning have been in good faith examined, and utilized or rejected, should the University consider the release of any tenured member of the faculty on the basis of Financial Exigency.
https://policy.ku.edu/governance/USRR#ArticleVII

If urgency of the circumstances are justifiably similar and this new COVID-19 specific policy is something clearly different than financial exigency, then how we talk about those differences matter.

The positive difference in the policy of financial exigency (shared governance) has already been noted. My question is why would we defend a policy that requires Universities to explicitly begin eliminating non-tenured lines before any tenured lines?

Because…tenure?

Like crises before it, COVID-19 has been both a global equalizer and a stark reminder of disparities resulting from all kinds of privilege. For KU employees COVID-19 has already forced temporary salary reductions across all employee types, additional hiring freezes, and top-down realignments that have resulted in administrative promotions and demotions, unpaid additional labor, and the voluntary and involuntary loss of good people and expertise.

Values and standards can certainly be upheld during unprecedented upheaval. The open letter reminds us “the statements of core values and standard practices [of academic freedom and tenure] were composed during moments of extraordinary societal upheaval and unrest—during the worst economic depression and the deadliest world war”. We must acknowledge they also occurred when “in some Northern cities, whites called for African Americans to be fired from any jobs as long as there were whites out of work.” Can we honestly say today, under circumstances similar or unique from crises of the past, that explicitly prioritizing the elimination of non-tenured faculty or staff actually “preserves the survival of the University as a quality institution of higher learning”, but eliminating any tenured faculty doesn’t?

What actually needs defending here?

What if Universities responded to the call for developing this framework with a generosity inherent in “it treats everyone the same”? What if a framework did include shared governance, adequate notice and appeal privileges, and did so (as the policy requires) for any state university employee?

Wouldn’t that be better than defending currently inequitable alternatives like financial exigency, or narrowly focusing on the academic freedom afforded only to tenured faculty? Wouldn’t it be better than an institution publicly declaring they are (for now) not planning to submit criteria or implement the new policy as is?

These questions prevent me from defending an alternative of financial exigency as it exists. In posing these questions, I also acknowledge that KBOR’s approval of this policy without shared governance and transparency is indeed unprecedented in a manner that “degrades the working conditions of the entire university and the learning conditions for all of our students”. At the same time, I can see the issues it is attempting to address gives unprecedented opportunity for more equitable approaches to “save the university”.

Having worked for the State of Kansas in the KBOR system for more than 20 years, I have experienced the political and budgetary challenges to employee salaries and benefits from the vantage point of different employee statuses – classified and unclassified staff, tenure track and non-tenure track, and tenured faculty administration in which I have supervised or mentored all of the former. This given me access to perspectives some of my colleagues may not as readily share.

Clearly this issue has shaken more than the foundational principles of tenure. It has us questioning ourselves, our professional relationships, and our leadership at all levels. Ultimately, though, this is a good thing. And while manifested poorly, both these polices, and at all times our privilege, should be open to further critique.

Shared governance is a privilege students, staff, and faculty can and should exercise on behalf of the most vulnerable. It calls us to use our collective voices not in self- or self-identifying preservation, but for giving unprecedented voice, participation, and due process for all employees. KU’s chancellor has publicly and internally sought input from governance, administrators, faculty, and staff across the university to determine this process, despite any requirement of the KBOR policy to do so – as he should! In addition to this invitation, it is our right and obligation to demonstrate how equity becomes and remains a prerequisite part of any policy.

Let’s get to it.

Weaving It Together

Image from "Technology of textile design. Being a practical treatise on the construction and application of weaves for all textile fabrics, with minute reference to the latest inventions for weaving" by E.A. Posselt is in the public domain. Courtesy of the Internet Archive via Flickr. — Image from “Technology of textile design. Being a practical treatise on the construction and application of weaves for all textile fabrics, with minute reference to the latest inventions for weaving” by E.A. Posselt (1899) is in the public domain. Courtesy of the Internet Archive via Flickr.

I recently finished writing my narrative statement for my second year tenure review file. It felt like pulling teeth. The statement required me to weave together the aspects of my work as well as my research and service to tell a meaningful personal story about my professional purpose and goals. The other sections of the file–the description of accomplishments, presentations and publications, committee work, etc.–were a piece of cake by comparison. I’m not sure why the statement felt quite so difficult, but, boy, did it ever.

All my teeth-gnashing about my narrative statement made me think about a program I developed with colleagues this semester, a series of panel discussions that we called “From Concept to Creation: Uncovering the Making of Scholarly and Creative Accomplishments.” We developed this program in order to celebrate the work of our faculty and staff. Even more importantly, though, the idea for this series grew out of a desire to share stories within our campus community about how we engage in research and creative work. We wanted to host conversations about process, not just product. In sharing a behind-the-scenes look at their work, we were hoping panelists would reveal their steps and stages, but also the information literacy and digital literacy skills, habits, and attitudes that were important to each project. I was excited about the potential of this panel series because I think uncovering process is not just interesting, but empowering. And by increasing the transparency of their component parts, we hoped these kinds of research experiences might feel more approachable to our students.

In conversations with panelists as we prepared for the series, we offered guiding questions they might consider as they prepared their remarks like the following: How did you take your first steps?, How did you ask questions?, How did you identify a path for your research?, How did you engage with other scholars’ work on the topic to develop your own?, How did your work change course during the process?, What attitudes were important to your process?, What skills and tools were key to your process?, How did you gather/organize/analyze data?, How did you draw conclusions?, and What did you learn along the way?.

I had imagined panelists would likely select a particular publication or project and discuss some aspects of its development. Instead, most chose to talk about their undergraduate experiences and their entry into graduate work or their field. Panelists described choices they made, challenges they encountered, and how their paths changed over time. Embedded in each of their stories, too, were practices and perspectives related to information literacy that seemed to me to have been crucial to their process.

What strikes me most now, though, is how each panelist interpreted the program theme and the guiding questions and how they chose to tell the story of their work. When my collaborator and I asked our colleagues to talk about their research processes, I didn’t give much thought to how personal their stories might be. As I reflect on the difficulty I felt in drafting my narrative statement, I’m thinking about the balance I, too, was trying to strike. I’m thinking now about how we weave together process and purpose, personal and professional to help focus and understand our work.

Grappling with My Faculty Identity

Please welcome our new First Year Academic Librarian Experience blogger Melissa DeWitt, Research and Instruction Librarian at the Regis University, Denver, Colorado.

When I began applying for academic library jobs late last year, I was introduced to the wild world of academic personnel classification. First, I figured out what tenure was and what that might mean for my own career. I then discovered that there are tenure track librarians on 9 or 12 month contracts, faculty librarians who are not tenure track, librarians who are not classified as faculty, and everything in between. It was complicated, and I didn’t quite understand what my status might mean in the greater context of the university or college I would work for. In fact, I’m still grappling with my identity on campus, what it means in relation to my colleagues, and how rank influences interactions.

In April, I became a faculty research and instruction librarian at a small university. We do not have tenure, but we are on a rank and promotion track, which means that I have service requirements to students and the university and requirements to contribute to the profession through presentations, publications, and research so that I can be promoted. There are a mix of tenure track and non tenure track positions at my university, and the designation depends heavily on the college. Complicating these designations are the histories behind them. There used to be a faculty union for some of the colleges, but that union dissolved. The library used to have tenure, and some librarians still have tenure, but new librarians do not. I learn new information every day about my university’s history, college structures and classifications, and I’m still confused. It’s like a jumbled bowl of faculty spaghetti. I don’t know if I’ll ever figure it out, and this is just one college campus.

I’m not usually one to care about titles, what I’m called, or what my status is; however, I’m discovering that other people really do care and that has implications for how they interact with me. What I’ve started to care about is how people perceive what I do. It seems that a large part of my job is fixing people’s perceptions of my job and advocating for it in the first place. Veronica Arellano Douglas recently posted an article that helped me reflect on some of my feelings. A lot of times, I think I’m seen as a resource or as something that can be used by others instead of as an educator and expert in my own field of study. There was a conversation on Twitter recently about this idea that librarians are seen as helpers rather than colleagues. This comes across in emails where colleagues ask me to present an impossible list of things in a 20 minute period or introduce me as a magician who can pull resources out of thin air instead of a colleague that studied and practiced and is an expert in her field. Or, there are comments of surprise that the librarians attenda so many events (it’s literally our job and we want to support our students and colleagues), though attending campus events can sometimes be a struggle when we’re left off the invite list.

Which brings me back to labels. What does it mean if I’m classified as faculty but am not always treated as one? What I think it means is that I have to do a bit more work to build relationships and collaborate with other faculty members. When a colleague remarked that it’s nice the library is invited to faculty events, we had a conversation about how librarians are invited because we’re also faculty. If I receive an instruction request that includes too many topics or doesn’t give me enough time to teach, I can push back and inform them how I approach instruction. I’m trying to find my boundaries and tell colleagues “this is what I do” instead of “is it okay if I do this?” because I don’t need permission to do my job. I’m having conversations about my role in educating students, pointing out opportunities for collaboration, and valuing my own skills and expertise. Many of my colleagues are really receptive and great to work with. I’ve had success teaching classes that go beyond the one-shot model because colleagues were open to the idea that we could have more than one class. I’ve also helped colleagues create research projects for students and brainstorm ideas to teach students information literacy topics. Those moments feel like a win.

I’d also like to point out that, as a new librarian, this is all very terrifying. For one, I feel like an imposter half the time, and I’m also a young female that’s mistaken for a student more often than not so talking to seasoned academics is intimidating. However, I’d like to believe, even if I weren’t classified as faculty, that I’d still be having these conversations and building these relationships with my colleagues. I have to remind myself that I know a lot of stuff and I can make a substantial contribution to my university, our students, and the library profession. For now, I’m going to keep learning all that I can about my university, continue building meaningful relationships with my colleagues, and perhaps one day, figure this academia thing out.

How are librarians classified on your campus? What do your relationships with faculty look like?