Assessment

How Many Books? In What Formats?

This post comes from a guest poster, Scarlet Galvan. Scarlet is the Collection Strategist Librarian at Grand Valley State University.

A blogger for Inside Higher Ed recently published a plea for a more nuanced understanding of IPEDS data on academic libraries. Like that blogger, I also wish data about academic libraries offered more detail about their work and position as necessary infrastructure. I remember the failure to meet each other as colleagues has collective outcomes.

IPEDS is not designed to capture data about academic libraries accurately. This is a problem because of how familiar IPEDS data is to the rest of higher education, how our colleagues across the university will reach for this data with trust and certainty. Tools like IPEDS flatten complexity, offering a rather facile view into deeply complicated systems. There are many qualifiers. ACRL’s 2015 alignment with a portion of IPEDS questions to streamline reporting means how we conceptualize the work of other institutions faces similar constraints.

After skimming IPEDS data, the blogger’s suggestion, “…maybe academic libraries can avoid layoffs in lean budget times by cutting down on the roughly 40 percent of spending that goes to materials and services” comes down to individual approaches. I can only reflect on my experiences and note the marked difference in organizational capacity when leaders view challenges as opportunities as opposed to choosing what is comfortable.

Even when reducing a collections budget like this is an option, it’s not a decision that comes from a place of strength and would destabilize our teaching, learning, and research missions. That said, subscription-based resources make up the majority of collections expenditures in academic libraries. These carry a yearly inflation rate of 5-7% or more, though a growing number of library workers are working to recalibrate these contract terms. It’s cheaper to keep people. Most university employees do not receive similar increases to their salary and benefits each year. Still, an actionable response is to better understand how our university budgets work in order to clearly see them for what they are: articulated values.

Statistics describing the collection shift continuously as the way we define those things evolves along with scholarly communications ecosystems. IPEDS could offer a snapshot at best. Meanwhile, content providers engage in decades-long campaigns to consolidate and monopolize research infrastructure, impact metrics, search algorithms, mining user data, and even hiring processes.

What metrics might be used instead? What questions could we ask for a richer understanding of the impact of libraries and library workers in higher education? These things are much harder to get at than numerical totals in broad categories like budget allocations and collection totals. For example, we might consider what percentage of research output is Open Access, measure institutional investments toward open infrastructures, acknowledge and document the expertise to make archival collections findable, build research data management plans, measure student outcomes based on faculty authored and adopted open publications, or capture the incentives we offer for such work for accreditation, tenure, and promotion. Until different metrics can be included, we’re left to specific definitions of value and indigent questions. “How many books? In what format?”

Additional Reading:

Lamdan, S. (2022). Data Cartels: The Companies that Control and Monopolize Our Information. Stanford University Press. https://www.sup.org/books/title/?id=33205

Wilson, M., & Cronk, L. (2022). The NERL Playbook. Commonplace. https://doi.org/10.21428/6ffd8432.2b8579b0

Scholarly Publishing and Academic Research Coalition: Elsevier’s Acquisition of Interfolio: Risks and Responses. https://infrastructure.sparcopen.org/interfolio-acquisition

Burning with Your Own Passions

Since 2008, ACRLog’s “First Year Academic Librarian (FYAL) Experience” series has annually featured 1-2 academic librarians in their first year on the job in an academic library. This new series, “Where Are They Now? Former FYALs Reflect,” features posts from past FYAL bloggers as they look back on their trajectories since their first year. This month, we welcome a post from Kimberly Miller, Assessment Librarian and Liaison to Psychology at Towson University.

“Where are they now?”

Right now? Like many of you, right now I am at home seeking quiet and solitude away from the chaos of managing work, family, school, and self-care during a global health crisis. Thinking back to my first-year librarian experience, I can’t help but laugh and think, as our ALSC colleagues already reminded us, responding to a global pandemic was definitely not covered in library school.

What’s Happened?

In 2012, shortly before joining ACRLog as an First Year Academic Library Experience (FYALE) blogger, I was hired as Emerging Technologies Librarian & Liaison to Psychology at Towson University. In that role I provided technology leadership within the library’s Research and Instruction Department. I also taught information literacy workshops, provided student and faculty research help, and worked with the Psychology-related collection. While the open-ended nature of the role was sometimes daunting (what exactly “counts” as an “emerging technology” still remains a mystery to me), all-in-all it was a great first position because the diversity in my responsibilities provided a lot of areas for exploration and growth. And some of that growth, particularly around risk taking and experimentation, is captured in my FYALE blog posts.

Over time, as I began to articulate my expertise and vision, I successfully advocated to narrow my position to focus specifically on “learning technologies” necessary to support formal and informal learning within the library. Other highlights between my first year and now include:

Changing my job description (twice)
Applying for, and achieving, rank promotion and permanent status
Participating in ACRL’s Immersion and the Institute for Research Design in Librarianship
Attending too many conferences and serving on too many committees
Starting an instructional technology doctoral program and, recently, transfering to the masters program (graduating May 2020!)
Becoming a parent
Serving in leadership positions within regional and national professional organizations
Collaborating with senior library leadership as librarian representative to the library’s Leadership Council

Turning Point

As I reflect on those experiences it’s clear to me that the month I had a child and was notified that I was awarded permanent status marked a significant turning point for me personally and professionally. When I returned to work, I realized I was spending more time managing projects and, indirectly, the people associated with those projects than I was exploring and creating technology-based instruction itself. And I was good at it. I loved my job and the people I worked with, and I had developed a talent for leading people to solve interesting problems. As a doctoral student, I also gained a deeper expertise in educational leadership and professional development necessary to take on new challenges. At the same time, I was growing tired of running into the same roadblocks and questioning whether what I did really mattered while seeing little opportunity to grow into new professional areas.

In my cubicle, a now-faded handwritten quote reminds me that “People who do not blaze with their own passions burn out.” This quote has been my guiding principle as I’ve made decisions, both small and large, about how I spend my time. Throughout my career, one of my driving forces has been a desire to deeply understand the rationale behind our work and the evidence needed to help make that work a success. With this in mind, I proposed that my experiences and interests made me a good fit for the new Assessment Librarian position our Dean of University Libraries announced in the Fall of 2018. After several conversations and some final job description editing, I transitioned into my new role as TU’s Assessment Librarian in January 2019.

Now and the Future

I’ll admit that, unlike technology, assessment initiatives are not high on the list of exciting or flashy library projects. But I would argue that’s because assessment is best when it is infused within all other work that we do on a day-to-day basis. Assessment is not just counting, number crunching, and correlating. The flashiest project will fizzle if we don’t know how or why it was successful. And that’s what assessment is about to me – it is being curious and asking questions about the way our services, systems, and collections support our community. Academic libraries make profound differences within and beyond our campuses, and the best way to continue doing so is to continually learn from our work.

As an Assessment Librarian, I find meaning in dispelling myths about assessment while building our library staff and faculty’s capacity to apply evidence within their specific domains to provide excellent user support and services. While I help everyone learn about the nuts-and-bolts of assessment, I also get to tie assessment to how we explore new possibilities for serving our users. For example, in November I spoke to our staff as part of our library’s Inclusion, Diversity, Equity, and Accessibility (IDEA) Spark series about using assessment to dispel the “myth of average” when designing library services, spaces, and resources. I’m excited to explore how I can continue to support this work in the next stage of my career.

While the jump from instructional technology to assessment may seem strange to some, for me it was a chance to lean into new skills and solve new challenges while leveraging the talents I cultivated in my previous role. I also continue to learn a lot about navigating the politics of projects that require working both horizontally and vertically within the library’s organizational chart. As the first person in this new role, I have come full circle and once again find myself with an open-ended opportunity to shape our library’s path forward on key strategic initiatives. This time, I get the unique and exciting privilege of a front row seat to the amazing work happening in nearly every area of our library. I can’t wait to see what else I didn’t learn in library school!

When Did Efficiency Become the End-Goal?

Earlier this week I read the latest Library Trends article by Karen P. Nicholson, Nicole Pagowsky, and Maura Seale, Just-in-Time or Just-in-Case? Time, Learning Analytics, and the Academic Library (also available via the University of Arizona Repository. If you haven’t read it yet, stop reading this blog post and head on over to that article because it is well worth your time.

It’s an exploration of time, in fact, and examines the relationship between academic libraries’ adoption of learning analytics as a crisis response to the “future of academic libraries” discourse that has been around as long as libraries. One of the very first blog posts I ever wrote was in response to this constant state of crisis and dire warnings of the future. Nicholson, Pagowsky, and Seale describe this existential fear as the “timescape of a present-future, whose primary value lies in staving off the risk of a library-less future” (2019, p.4). By existing in this “present-future” we seem to be responding to a known-future, one that we must make changes to adapt to fit, rather than a future of our own making that we have the power to shape through organizing, taking actions based on values, and a concerted effort to create change as a profession.

I so appreciate the authors linking the notion of time to power, because time is being used in such a way that renders us powerless. We’re somehow always working against a constantly ticking clock, trying to be more productive and more effective and more efficient. But when did education become about efficiency? When did we collectively decide that our library instruction programs should be about teaching the most classes, reaching the most students, providing badges, or highlighting major initiatives. Learning is messy. Teaching have can impacts that are small but significant. If we are constantly living in a present-future of what our libraries will or will not be then we are unable to exist in the moment in our libraries, classrooms, and interactions with those around us.

The irony of the popularity of future-casting in libraries AND mindfulness is not lost on me. One is constantly urging us to look forward, mitigate risk, and plan against predictions. The other asks us to be present in our current state, maintain awareness of ourselves and those around us, and work to cultivate a sense of balance with the world. Is the push towards mindfulness a response to our ever-anxious existence as libraries looking toward the future? Is it the answer? Or do we need something more?

I suspect that mindfulness / awareness of the present is a start, but that it should then lead toward present action. What can I do in this moment to make things meaningful for myself, my colleagues, my library? The push towards making work, particularly instruction work, more sustainable tends to edge towards standardization, or, as Nicholson writes, the McDonaldization of Academic Libraries, again because we are looking towards a future rife with cost efficiency concerns, doing more with less, and proving value. It may appear to be programmatically sustainable, but ultimately sustainability relies on people, and people burn out. People get tired of teaching the same lesson plan over and over again. People get fed up with the distance between themselves and the students at the other end of that online lesson. For our work to be truly sustainable it needs to also be sustaining to our needs as people who entered the work of librarianship, specifically teaching librarianship, to help others.

So what can a present-aware, meaningful practice of librarianship look like in the current academic library?

Assessment as Care

water from watering can falling on small plants - Photo by Markus Spiske on Unsplash — Photo by Markus Spiske on Unsplash

I’m in a weird head space at the moment. I attended the Critical Librarianship and Pedagogy Symposium (CLAPS) last month and am about to attend the Library Assessment Conference (LAC) later this week. Based on what I experienced at CLAPS and what I’ve read about LAC, the two conferences couldn’t feel more different. I am very curious about how we continue to conceptualize and shape the idea of assessment in libraries.

At CLAPS, I had the pleasure of attending a workshop led by Anne Cong-Huyen, Digital Scholarship Strategist, and Kush Patel, Digital Pedagogy Librarian, at the University of Michigan. Their exploration of critical digital pedagogy in librarianship was a wonderful mix of writing, reflection, and discussion on the ways in which we can build critical and queer feminist communities in our classes. As part of the session, Anne and Kush asked participants to read five different excerpts of selected texts on care, praxis, technologies, design, and assessment, and then write our reflections on these excerpts as they apply to our own teaching and the learning we want to facilitate in our classes. (You can read the excerpts on the slides they’ve graciously shared online). The excerpt that resonated with me the most was from Critical Generosity by Jill Dolan, which illustrates a generous and caring approach to the criticism of dramatic performances and artists. It was presented as a model for assessment in teaching and learning, one that recast–in my mind, anyway–assessment as care and sustenance.

The current narrative of assessment in libraries is that of justification. We prove our value, show our impact, and demonstrate our connection to student learning and student success. I know we are working and teaching at a time when higher education funding and academic jobs are precarious and departments and faculty are constantly being asked to prove their worth. I am sympathetic to our attempts to demonstrate, through assessment, that our work in libraries is important. I’ve done and published this kind of assessment myself! But because our assessment is done with the intent to appease an external audience, we are constantly in a position to validate our own existence, rather than support the learning realities of our students and teaching librarians. Our assessment is an act of survival, in our minds, rather than something that enriches and feeds ourselves and our students. I’ve shared my professional angst about librarianship not having a seat at the academic table and the ways that influences interactions between librarians and faculty. Our library assessment culture reflects this reality, but it also continues the narrative that we need to prove ourselves worthy of trust and acceptance.

Dolan writes about her first encounter with “critical generosity” in David Roman’s book, Acts of Intervention. Roman describes caring for friends who were HIV positive during a showing of the famously long play, Angels in America. Throughout the performance, he conducted frequent interpersonal assessments: Is everyone doing ok? Does someone need to take their medication? Is there enough food and water? Do people need a break/rest? The root and ongoing narrative of this assessment was care, sustenance, and really, love.

I recognize I’m asking for what many may view as a stretch: making a connection between the interpersonal care Roman and Dolan write about and institutional library assessment. But our teaching and learning in higher education and libraries is about the students we teach and the interpersonal connections we make everyday. So many of our attempts at assessment stay away from “messiness.” We want numbers that make good stories, and we want those good stories to make the library look good. But in staying away from messiness we are erasing the people at the center our work–their complications, needs, bodies, etc. In short, we’re staying away from the “gore” of learning. I don’t mean to be graphic, but I do think our proclivity for neatness is in direct conflict with the process of learning. In my own attempts at large-scale, summative, value-focused assessment, the best I’ve been able to show is that learning takes time, and our own work as teaching librarians is never-ending.

Yes, I know we have annual reports to write and numbers to share with our directors, deans, provosts, and presidents. I do too. But we have power within our profession with the papers we write, the kind of assessment we advocate for and practice, and the care that we exhibit within our work. What would it mean to embrace a critical practice of assessment? What could that look like?

Going to ACRL Immersion 2017: Reflection and more reflection.

First, I do apologize for the late blog, but I wanted some days to collect my thoughts.

A couple of months ago, I found that I had been accepted to the Teacher Track for ACRL Immersion. This program was to take place in Burlington, Vermont. Last Sunday, I arrived and the program concluded last Friday. I have had some days to think about the last week, what I learned, what I need to do, and how I need to do it. There were so many things about Immersion that I could focus on, but instead, I will list what I learned (or didn’t).

The theme of the week was transformation and reflection. I had colleagues who had gone to Immersion many years ago and they told me that I needed to be prepared, because it would be a transformative experience. I agree, but my whole self (body and mind) was not completely transformed. Instead, I found myself thinking back and forth about an idea or concept and then being left with either a clear idea or continuing to be baffled. I would reflect about certain things during and outside of Immersion, which brings me to the next point.
I found that like Jon Snow, I know nothing.
Wait, maybe I do. A lot about what you thought about your teaching will be challenged. You will definitely doubt yourself, but that’s what the week is for. You’re there to doubt, reflect, hopefully transform your ideas, and reflect again (you’ll find that reflection is a huge thing at Immersion).
I learned that everyone has a different assessment technique and activity. I found myself pretty excited to take these ideas to American University Library. For those of you who know me, you know that I was not a huge fan of assessment…but I have clearly changed my ways.
The activities, exercises, and discussions that we did throughout the week, are meant to not only challenge the way you teach, but to look at things in a different light. For example, I know myself and a couple of other participants were definitely challenged when it came to active learning. Just because you have your students do some type of movement while in class, does not mean that it’s active learning. I know that might be a “duh” for some people, but definitely not for me. It is something I am still reflecting on.
I was comforted to know that some people at Immersion share the same insecurities I do. Whether it’s about teaching or being a librarian.
Through informal and casual conversations with other librarians at Immersion, I found us discussing some struggles we share or have. Whether it’s struggles with being a librarian, teaching, or other frustrations, it was clearly topics that we simply cannot share with our colleagues back at our own institutions.
No one leaves the program suddenly transformed into the “perfect teacher.” I do not believe that is the point of the program. What is the point, is gaining a new set of understanding, comprehension, reflection, and motivation of the topics taught at Immersion.

On a personal note, knowing I was coming to Immersion was exciting because I lived in Burlington when I was a child. It had been 20 years and Immersion gave me the opportunity to come back. I was in awe over how much the town had changed and kept asking myself, “how could my parents ever leave such a beautiful town.” They had their reasons, but it was a delight to have great Immersion faculty, great Immersion participants, and a great space to share ideas, thoughts, doubts, and breakthroughs.

I was also able to have breakfast with my first grade teacher and so that was the cherry on top of a great week. Until next time, Burlington.