Tag Archives: Assessment

Let’s Assess!

The California State University system has been considering a move to performance-based funding. Librarians here have expressed a lot of consternation about how we can show we’re worth funding, let alone why we should have to show it at all. At the same time, Cal State Fullerton is making a big push towards implementing and assessing High Impact Practices (HIPs) as part of our focus on increasing student engagement, retention, and graduation, so as a library we have to figure out how we can incorporate HIPs as well.

I think there’s a tendency for librarians to worry a lot about how we’re going to assess what we do, both because it’s not something we’ve done formally and we’re inexperienced with it, and because librarians worry that their academic freedom is going to be impinged.

As an Instructional Design Librarian, I see the push towards assessment a little more optimistically (though I’m not at all onboard with performance-based funding!). However, I am interested in being effective at what I do, whether facilitating student learning, or providing outreach at campus events. I want to know if I’m making a difference. I also feel pretty well equipped to perform assessment activities since instructional design is my specialty.

What do you want to know?

Before we should start worrying about how we’re going to assess what we do, we have to decide what we want to assess. I think that there are two paths to decide on what we want to assess. There are our campus priorities, which of course we have to support, especially with the possibility of performance-based funding. There are also our own priorities – what kind of library do we want to be? What kinds of services and collections do we want to provide? What do we want students to learn from our one-shots?

I’m an Instructional Design Librarian, so naturally my assessment focus has an instructional bent. I like to write learning objectives for every session I teach, and each learning object that I create. Learning objectives ought to be SMART: Specific, Measurable, Attainable, Relevant, and (sometimes) Time-bound. I write objectives for one-shots like “students will be able to give two examples of why a scholarly database is a better choice for university research than Google,” or they’ll be able to “describe the process for finding books using the Books & eBooks tab on the Pollak Library home page.” I’m not likely to demand that students write essays on how to search for books at our library so that I can formally measure their mastery, but I am likely to do an informal assessment to verify that students are learning.

Both formal and informal assessment will give us useful information on what works and what doesn’t, possibly saving us from using time and money on instructional efforts that aren’t effective. Formal assessment is just the quizzes and tests we give, or other graded assignments from which we can collect data. Informal assessment might just consist of chatting with your students to see if they “get it,” or if students are actually accomplishing a given task that they were assigned. I use a lot of informal assessment in my one-shot sessions due to limited time. However, I’m also working on a couple of assessment-related projects at my university that will yield useful data.

ID Workshops, ACRL Assessment in Action, and Badges

As this post falls under the cateogory “First Year Academic Librarian Experience,” you might assume that I’m still a new librarian. You’d be correct. That means I haven’t yet accomplished any meaningful assessment efforts, but I’m on my way. Last month I planned and delivered an instructional design workshop for librarians, wherein I introduced the concepts of Backward Design and how to write learning objectives. We definitely need to work more on writing measurable learning objectives, and I plan to deliver more workshops in the future. (FYI, I assessed my “students’” learning through Plickrs and Padlets in the class, but it was definitely informal assessment). I want to help librarians discover ways they can be as effective as possible in their work.

I’ll be doing some major assessing for the next 14 months since I’m now the proud Librarian Team Leader of the Pollak Library Assessment in Action (AiA) Team, part of the third cohort of ACRL’s AiA program. We’re going to embed our Human Services Librarian into an online class this fall, and assess his impact on student learning. This particular Human Services class used to come in for in-person instruction, but since it’s gone totally online the instructors have done without library instruction. This is a trend at our university, so I’d like to learn how we can better serve our online students and now just let them fall off our radar.

Finally, I’m contemplating the mechanics of implementing a badges program here at our university. I recently managed to get a really simple (beta) eLearning webpage going for us on a WordPress platform. There are several WordPress plugins that can be purchased that turn your WordPress site into an LMS – like LearnDash and Sensei. With the site magically transformed into an LMS, we can award badges based on quiz and tutorial completion. How cool would it be to tally up the number of Super Searcher badge-holders? If we were able to track our badge-holding students’ retention and graduation rates, we’d have some really nice information on the library’s correlation with student success!

After The Values Study

ACRL has received a considerable amount of positive feedback about the Values of Academic Libraries Study. Perhaps you’ve had an opportunity to catch one of the presentations about the study that Megan Oakleaf, author of the study, or ACRL President Lisa Hinchliffe, have conducted at a number of different conferences.

At the Midwinter conference, during a meeting of ACRL’s Leadership Council (the Board, section chairs, and other miscellaneous representatives), a question was raised about what we do next with the Values study, or rather what comes after the study. If anyone at the meeting had a good idea, he or she chose not to share it because there were no responses to the question – and perhaps folks just had not yet had much time to give thought to that particular question. The study provides abundant information, from a mix of qualitative and quantitative studies, to help academic librarians provide evidence of the ways in which our libraries make valuable contributions to student and faculty success, and help to improve higher education. But the report itself is not a research study that provides concrete documentation of the value of academic libraries. What it does well is provide ammunition for library leaders who will want to argue for the value of academic libraries, and use it to make a case for institutional support. So the question about what comes next – what more can be done to create a strong connection between academic librarians and the value they provide – is a good one. I suspect ACRL is already cooking up some plans for next steps to extend the “value of academic libraries” initiative, but I’m not sure what they are.

As I’ve been thinking about this “what comes next” question, two possibilities have come to mind. I continue to believe that some of the most essential areas in which we can demonstrate the value of our work are student retention, persistence to graduation and student success beyond graduation. How do we connect our contributions to these higher education performance issues? I wanted to share some thoughts about this, and would like to hear what you think might make a good follow-up to the values study. One inspiration for a next step is the recently released book Academically Adrift that has created quite a stir in higher education circles with its finding that for many of our students there is little learning in their four years of college. The findings are based on data collected from a sample of 2,000 students from 24 four-year colleges. The students took standardized learning assessment tests three times during their college years.

That approach could offer some possibilities for a next step. With enough grant money a sample of students could be tracked in order to assess changes in their research skills. As seniors would they still be starting their research at Google? If asked, to what extent would they point to the librarians at their institution as playing a role in their academic success? Did the librarians have any impact on their ability to stay enrolled? The authors of Academically Adrift are already moving on to the “next step” in their research on student learning, and they’ll be looking more closely at alumni and what happens after college. Targeting alumni might even work better as a way to document the value of the academic library. If asked, what would alumni have to say about their library experience? I could see that as a more qualitative study, interviewing alumni to get more in depth information about their library experience, what value it provided and whether it was making a difference for them in their careers (assuming they’ve started careers).

A few colleagues and I previously did some quasi-experimental research on the use of LibGuides and whether, by examining the annotated bibliographies produced by the students in control and experimental groups, we could ascertain if the LibGuides made a difference in the use of library resources. While it was difficult to determine if higher quality work could be attributed to having access to the LibGuide, one thing we did notice is that there were clear outliers within the study groups. Some students performed far better, and perhaps that’s not unusual in any academic setting. Looking specifically at library research skills though, especially evaluation of content, what leads some students to excel? Another possible follow-up to Values Study could track the outliers into their post-graduate years to determine whether or not they still use their learned library skills in the workplace – and can any post-graduate success with work that involves research and/or writing be attributed to library research skills education. If we could link library research skill building with positive post-graduate or career performance that could definitely speak volumes about the value of academic librarians. There’s no question that these types of research projects are involved, somewhat complicated and almost a full-time job in themselves. That’s where ACRL’s connection with LIS educators to conduct the research makes good sense.

I’m not sure what will come after the Values Study. Given its success and value as a starting point, there is strong support in the library community for further research into the value of academic librarians and their libraries. In this post I focused on student retention and persistence to graduation. The Values Study also points to the academic librarian’s contribution to faculty research and productivity, as well as institutional prestige. There are important areas too for “next steps” research. ACRL is open to ideas for what comes next. Let ACRL know what you think would be a good next step. A great idea for what comes after the Values Study could come from anywhere in our profession.