Assessment

I Liked, I Wished, I Wondered: A One-Year Review

One year ago today I flew one-way from ORD to LAX for my first real librarian job (and obviously for the weather). I’m going to take an assessment nugget I once learned from Jennifer Brown, Emerging Technologies Librarian at Columbia University and reflect upon this time using the following measurements: I Liked, I Wished, I Wondered.

I Liked

I liked plenty thus far as a Health & Life Sciences Librarian at UCLA. Most importantly, I am grateful for my work colleagues. I work with people that truly care about learning and how it is reflected within library practices. I work with inspiring and supportive people of color. I work with people that have more to talk about than libraries (this is so important!). While I didn’t necessarily imagine myself working as a librarian in the sciences, I like working in this domain! While I have health sciences experience from working as a speech-language pathologist,, I didn’t appreciate scientific research, its importance, its limitations, and its possibilities, as much as I do now. The sciences seemed a bit intimidating in the beginning, but I have been pleasantly surprised at how accessible it can be, even if someone doesn’t have a sciences background (or even an interest…I am curious how much these are linked). I also like the new matrixed organizational structure within the UCLA Library. It allows for librarians to do a little bit of everything while focusing on a specific area: Collections, Outreach, Research Assistance, Research Partnerships, or Teaching and Learning. This encourages communication across units. For example, I am on the Teaching and Learning Team with the Visual Arts Librarians. This is not a librarian with whom I would typically interact, however, this allows for collaboration, transparency, and information dissemination in seemingly unrelated functions and subject areas. Did I mention that I also like (LOVE) the weather? UCLA is a gorgeous campus all, come visit!

I Wished

I wished I came into my position having a better grasp of collections and scholarly communication. These are essential parts of my everyday duties, and while I have learned these functions over time, I think I would have hit the ground running a bit faster if I did a better job of taking a collections class or participating in a collections and/or scholarly communication focused internship during my MLIS.

I wished I had more time! There are moments where it’s hard to stay focused. This is likely due to a combination of my slightly average organizational skills and saying yes to opportunities. I do think I have been saying yes for the right reasons. I want to be of service, test my capacity in my role, and see what I liked (see above). The good news is that certain responsibilities do not last forever, and now I do have a better idea about what I would like to keep pursuing, what might make sense to stop in a year or two, and what to say yes/no to next time around. I want to be mindful of librarian burnout, so while I’m happy to try it all out, I don’t want to resent the profession either.

I Wondered

I wondered how things would be different if what I wished and what I liked had worked in concert. I wonder where I would be if I hadn’t come to UCLA. I wonder if I prefer to manage others or work as a subject or functional liaison. Will I stay in health sciences librarianship or would I branch out to other areas? I have truly enjoyed diving into medical librarianship, but I have wondered if a I would be better suited to focus upon a functional area. I enjoy pedagogy, active learning, outreach, and connecting different campus partners – perhaps there is a place for me in these areas? I enjoy wondering about this all at UCLA because the matrixed organization and professional development opportunities allow me to explore. I have also wondered if I will stay at an R1 institution, make the jump to a community college, or even try my hand in public libraries.

What Now?

I have always disliked the idea of having a 5-year or 10-year plan. I believe in intention, serendipitous moments, and blending that with your personal drive and abilities. I did not come to librarianship through a straight path, and, while I don’t want to change my career again, I am open to different possibilities that can harness and enhance my skill set. Writing this out has definitely forced me to reflect upon the past year, see how far I have come and what the future might hold. One year down and many more to go!

What are some different ways you taken assessment of your career path as a librarian?

Following the road of assessment

This Fall semester has been taking off like a rocket. It’s been a little less than a month, but library instruction has been taking up a good chunk of my time. At my institution, American University, we have a program called College Writing. This program requires all incoming freshman to take at least one section of College Writing.

Every faculty member that teaches College Writing is paired with a librarian. At least one library instruction session is required and it’s up to us to shape the lesson so that it’s relevant to the student’s’ current assignment.

This semester is a bit different. I had a total of 18 sections of College Writing, compared to the nine sections I had last Fall. I was prepared for a busy semester. Oh boy, has it been busy and it’s only been 2 weeks!

I could be as detailed as I want about my routine, but it’s basically a chain of communication. I ask the faculty member about learning outcomes, what they want out of this library instruction day, what skill level their students are at, and are the students quiet? Do they participate? Details like these help me out a lot, since I will only see the students in the classroom once or twice in the semester.

As I scheduled classes, reserved rooms, and worked on my class outlines, I struggled with how I would incorporate assessment into my lessons. Assessment is a topic I have been thinking about for a while. To be honest, this was a subject that I had been avoiding because it was something that made me uneasy. I have always told myself “I’ll do it next semester” or “I’ll find more information about it later.”

However, it’s been a year since I have started my job at American and decided that this semester it was time to incorporate assessment into my library instruction. When I think of assessment, I tend to think of a ton of data, a desk full of papers everywhere, and an endless amount of work (OK, I like to exaggerate). Now, I do have some forms of assessment in my classes, but it’s in the form of the questions I ask the students in order to evaluate their familiarity with not only the library, but the resources that we are using in class.

Assessment comes in many forms, but I specifically had one method in mind. Over the summer, I worked with another colleague in doing library instruction for the Summer Transition Enrichment Program (STEP). This program provides incoming freshman with preparation for academic success. STEP is a 7 week residential program that helps students with the transition from high school to college. They have a class that is very similar to a College Writing class, meaning, they have a research paper due by the end of the program. One of the components of that class is a library instruction day. As my colleague and I started preparing to co-teach one of the classes, she asked what form of assessment I do for my College Writing classes.

Immediately, I felt ashamed. All the time I had put assessment off and this was the moment where I finally had to own up to it. However, I have awesome colleagues who don’t poke (too much) fun at me. She talked about the post class questionnaire that she usually did with her students. Together, we came up with a couple of questions for the students in the STEP class. It was not a long process whatsoever, but I came to see that there is actually nothing scary about it, like I had thought.

There are many different types of assessment, ones more complicated and time consuming than my little questionnaire. However, I wanted to start small and with something I was comfortable with. My library instruction classes only started last week, but I remember getting back the questionnaires and leaving them on my desk for a couple of hours. I was afraid to look at them. What if the students did not learn anything? What if they hated me? What if I was the worst librarian ever?

After a couple hours, I needed to log my classes into our stats. I counted the questionnaires and look through them. To my surprise, the students did well. Now, this is an assessment to help me analyze what the students had trouble comprehending and also the areas where I need to do better.

And guess what happened? I found one area where I realized I needed to explain better and spend a little more time on. It’s only the beginning of the semester and I have already found ways to improve upon and this is what it’s really about. To me, assessment is an opportunity to learn about your teaching and improve as you go along.

As someone who is new to this, I want to continue to learn about assessment. There are a couple of resources that one can turn to:

-Look at your own institution to see if they offer any workshops on assessment. What resources do they offer to help their staff or faculty?

-Research other institutions to see if they have assessment in place or an assessment toolkit

-Research the literature on instruction and assessment to see how other institutions go about it

Finally, your colleagues will be your most valuable tools. What assessment do they do? Take them out for coffee and ask them!

I still have a couple more College Writing classes, but I am going to make it my goal to incorporate even more assessment for next semester’s classes. In other words, I am going to make myself accountable. For next semester, I will write another post on how I plan to incorporate more assessment into my teaching, but I also want to know from our readers, what assessment do you do for library instruction? Stay tuned!

Let’s Assess!

The California State University system has been considering a move to performance-based funding. Librarians here have expressed a lot of consternation about how we can show we’re worth funding, let alone why we should have to show it at all. At the same time, Cal State Fullerton is making a big push towards implementing and assessing High Impact Practices (HIPs) as part of our focus on increasing student engagement, retention, and graduation, so as a library we have to figure out how we can incorporate HIPs as well.

I think there’s a tendency for librarians to worry a lot about how we’re going to assess what we do, both because it’s not something we’ve done formally and we’re inexperienced with it, and because librarians worry that their academic freedom is going to be impinged.

As an Instructional Design Librarian, I see the push towards assessment a little more optimistically (though I’m not at all onboard with performance-based funding!). However, I am interested in being effective at what I do, whether facilitating student learning, or providing outreach at campus events. I want to know if I’m making a difference. I also feel pretty well equipped to perform assessment activities since instructional design is my specialty.

What do you want to know?

Before we should start worrying about how we’re going to assess what we do, we have to decide what we want to assess. I think that there are two paths to decide on what we want to assess. There are our campus priorities, which of course we have to support, especially with the possibility of performance-based funding. There are also our own priorities – what kind of library do we want to be? What kinds of services and collections do we want to provide? What do we want students to learn from our one-shots?

I’m an Instructional Design Librarian, so naturally my assessment focus has an instructional bent. I like to write learning objectives for every session I teach, and each learning object that I create. Learning objectives ought to be SMART: Specific, Measurable, Attainable, Relevant, and (sometimes) Time-bound. I write objectives for one-shots like “students will be able to give two examples of why a scholarly database is a better choice for university research than Google,” or they’ll be able to “describe the process for finding books using the Books & eBooks tab on the Pollak Library home page.” I’m not likely to demand that students write essays on how to search for books at our library so that I can formally measure their mastery, but I am likely to do an informal assessment to verify that students are learning.

Both formal and informal assessment will give us useful information on what works and what doesn’t, possibly saving us from using time and money on instructional efforts that aren’t effective. Formal assessment is just the quizzes and tests we give, or other graded assignments from which we can collect data. Informal assessment might just consist of chatting with your students to see if they “get it,” or if students are actually accomplishing a given task that they were assigned. I use a lot of informal assessment in my one-shot sessions due to limited time. However, I’m also working on a couple of assessment-related projects at my university that will yield useful data.

ID Workshops, ACRL Assessment in Action, and Badges

As this post falls under the cateogory “First Year Academic Librarian Experience,” you might assume that I’m still a new librarian. You’d be correct. That means I haven’t yet accomplished any meaningful assessment efforts, but I’m on my way. Last month I planned and delivered an instructional design workshop for librarians, wherein I introduced the concepts of Backward Design and how to write learning objectives. We definitely need to work more on writing measurable learning objectives, and I plan to deliver more workshops in the future. (FYI, I assessed my “students’” learning through Plickrs and Padlets in the class, but it was definitely informal assessment). I want to help librarians discover ways they can be as effective as possible in their work.

I’ll be doing some major assessing for the next 14 months since I’m now the proud Librarian Team Leader of the Pollak Library Assessment in Action (AiA) Team, part of the third cohort of ACRL’s AiA program. We’re going to embed our Human Services Librarian into an online class this fall, and assess his impact on student learning. This particular Human Services class used to come in for in-person instruction, but since it’s gone totally online the instructors have done without library instruction. This is a trend at our university, so I’d like to learn how we can better serve our online students and now just let them fall off our radar.

Finally, I’m contemplating the mechanics of implementing a badges program here at our university. I recently managed to get a really simple (beta) eLearning webpage going for us on a WordPress platform. There are several WordPress plugins that can be purchased that turn your WordPress site into an LMS – like LearnDash and Sensei. With the site magically transformed into an LMS, we can award badges based on quiz and tutorial completion. How cool would it be to tally up the number of Super Searcher badge-holders? If we were able to track our badge-holding students’ retention and graduation rates, we’d have some really nice information on the library’s correlation with student success!

After The Values Study

ACRL has received a considerable amount of positive feedback about the Values of Academic Libraries Study. Perhaps you’ve had an opportunity to catch one of the presentations about the study that Megan Oakleaf, author of the study, or ACRL President Lisa Hinchliffe, have conducted at a number of different conferences.

At the Midwinter conference, during a meeting of ACRL’s Leadership Council (the Board, section chairs, and other miscellaneous representatives), a question was raised about what we do next with the Values study, or rather what comes after the study. If anyone at the meeting had a good idea, he or she chose not to share it because there were no responses to the question – and perhaps folks just had not yet had much time to give thought to that particular question. The study provides abundant information, from a mix of qualitative and quantitative studies, to help academic librarians provide evidence of the ways in which our libraries make valuable contributions to student and faculty success, and help to improve higher education. But the report itself is not a research study that provides concrete documentation of the value of academic libraries. What it does well is provide ammunition for library leaders who will want to argue for the value of academic libraries, and use it to make a case for institutional support. So the question about what comes next – what more can be done to create a strong connection between academic librarians and the value they provide – is a good one. I suspect ACRL is already cooking up some plans for next steps to extend the “value of academic libraries” initiative, but I’m not sure what they are.

As I’ve been thinking about this “what comes next” question, two possibilities have come to mind. I continue to believe that some of the most essential areas in which we can demonstrate the value of our work are student retention, persistence to graduation and student success beyond graduation. How do we connect our contributions to these higher education performance issues? I wanted to share some thoughts about this, and would like to hear what you think might make a good follow-up to the values study. One inspiration for a next step is the recently released book Academically Adrift that has created quite a stir in higher education circles with its finding that for many of our students there is little learning in their four years of college. The findings are based on data collected from a sample of 2,000 students from 24 four-year colleges. The students took standardized learning assessment tests three times during their college years.

That approach could offer some possibilities for a next step. With enough grant money a sample of students could be tracked in order to assess changes in their research skills. As seniors would they still be starting their research at Google? If asked, to what extent would they point to the librarians at their institution as playing a role in their academic success? Did the librarians have any impact on their ability to stay enrolled? The authors of Academically Adrift are already moving on to the “next step” in their research on student learning, and they’ll be looking more closely at alumni and what happens after college. Targeting alumni might even work better as a way to document the value of the academic library. If asked, what would alumni have to say about their library experience? I could see that as a more qualitative study, interviewing alumni to get more in depth information about their library experience, what value it provided and whether it was making a difference for them in their careers (assuming they’ve started careers).

A few colleagues and I previously did some quasi-experimental research on the use of LibGuides and whether, by examining the annotated bibliographies produced by the students in control and experimental groups, we could ascertain if the LibGuides made a difference in the use of library resources. While it was difficult to determine if higher quality work could be attributed to having access to the LibGuide, one thing we did notice is that there were clear outliers within the study groups. Some students performed far better, and perhaps that’s not unusual in any academic setting. Looking specifically at library research skills though, especially evaluation of content, what leads some students to excel? Another possible follow-up to Values Study could track the outliers into their post-graduate years to determine whether or not they still use their learned library skills in the workplace – and can any post-graduate success with work that involves research and/or writing be attributed to library research skills education. If we could link library research skill building with positive post-graduate or career performance that could definitely speak volumes about the value of academic librarians. There’s no question that these types of research projects are involved, somewhat complicated and almost a full-time job in themselves. That’s where ACRL’s connection with LIS educators to conduct the research makes good sense.

I’m not sure what will come after the Values Study. Given its success and value as a starting point, there is strong support in the library community for further research into the value of academic librarians and their libraries. In this post I focused on student retention and persistence to graduation. The Values Study also points to the academic librarian’s contribution to faculty research and productivity, as well as institutional prestige. There are important areas too for “next steps” research. ACRL is open to ideas for what comes next. Let ACRL know what you think would be a good next step. A great idea for what comes after the Values Study could come from anywhere in our profession.