Tag Archives: library instruction

If At First You Don’t Assess, Try, Try Again

ACRLog welcomes a guest post from Katelyn Tucker & Alyssa Archer, Instruction Librarians at Radford University.

Instruction librarians are always looking for new & flashy ways to engage our students in the classroom. New teaching methods are exciting, but how do we know if they’re working? Here at Radford University, we’ve been flipping and using games for one-shot instruction sessions for a while, and our Assessment Librarian wasn’t going to accept anecdotal evidence of success any longer. We decided that the best way to see if our flipped and gamified lessons were accomplishing our goals was to evaluate the students’ completed assignments. We tried to think of every possible issue in designing the study. Our results, however, had issues that could have been prevented in hindsight. We want you to learn from our mistakes so you are not doomed to repeat them.

Our process

Identifying classes to include in this assessment of flipped versus gamified lessons was a no-brainer for us. A cohort of four sections of the same course that use identical assignment descriptions, assignment sheets, and grading rubrics meant that we had an optimal sample population. All students in the four sections created annotated bibliographies based on these same syllabi and assignment instructions. We randomly assigned two classes to receive flipped information literacy instruction and two to play a library game. After final grades had been submitted for the semester, the teaching faculty members of each section stripped identifying information from their students’ annotated bibliographies and sent them to us. We assigned each bibliography a number and then assigned two librarian coders to each paper. We felt confident that we had a failsafe study design.

Using a basic rubric (see image below, click to enlarge), librarians coded each bibliography for three outcomes using a binary scale. Since our curriculum lists APA documentation style, scholarly source evaluation, and search strategy as outcomes for the program, we coded for competency in these 3 areas. This process took about two months to complete, as coding student work is a time-consuming process.

assessmentchart

The challenges

After two librarians independently coded each bibliography, our assessment librarian ran inter-rater reliability statistics, and… we failed. We had previously used rubrics to code annotated bibliographies for another assessment project, so we didn’t spend any time explaining the process with our experienced coders. As we only hit around 30% agreement between coders, it is obvious that we should have done a better job with training.

Because we had such low agreement between coders, we weren’t confident in our success with each outcome. When we compared the flipped sections to the gamified ones, we didn’t find any significant differences in any of our outcomes. Students who played the game did just as well as those who were part of the flipped sections. However, our low inter-rater reliability threw a wrench in those results.

What we’ve learned

We came to understand the importance of norming, discussing among coders what the rubric means, and incorporating meaningful conversations on how to interpret assessment data into the norming process. Our inter-rater reliability issues could have been avoided with detailed training and discussion. Even though we thought we were safe on this project, because of earlier coding projects, the length of time between assessments created some large inconsistencies.

We haven’t given up on norming: including multiple coders may be time-intensive, but when done well, gives our team confidence in the results. The same applies to qualitative methodologies. As a side part of this project, one librarian looked at research narratives written by some participants, and decided to bravely go it alone on coding the students’ text using Dedoose. While it was an interesting experiment, the key point learned was to bring in more coders! While qualitative software can help identify patterns, it’s nothing compared to a partner looking at the same data and discussing as a team.

We also still believe in assessing output. As librarians, we don’t get too many opportunities to see how students use their information literacy skills in their written work. By assessing student output, we can actually track competency in our learning outcomes. We believe that students’ papers provide the best evidence of success or failure in the library classroom, and we feel lucky that our teaching faculty partners have given us access to graded work for our assessment projects.

Embedding, Flipping, and More at LOEX 2014

I was fortunate to be able to attend the LOEX Conference this year, which took place May 8-10 in Grand Rapids, MI. I have only ever heard great things about this conference, and accordingly, I had a great experience.

This was my first time attending the LOEX Conference and I only became aware of it recently (within the past year). Many readers here will likely be familiar with LOEX, but for those who aren’t, LOEX stands for Library Orientation Exchange and it is a “self-supporting, non-profit educational clearinghouse for library instruction and information literacy information.” The annual conference has earned a reputation for being particularly relevant and exciting for instruction and information literacy librarians, as attested by the many people I met who were either multiple-time attendees, or thrilled to finally get to go to the conference.

I went into the conference with high expectations, which were met and exceeded. The two days were full of presentations and workshops that are extremely relevant to my work, and with ideas I can incorporate by making small changes. I love coming away with new ideas that are practical, so I can actually implement them myself. Here are some things that the LOEX Conference got me thinking about:

Embedding

A few presentations focused on embedded librarianship in one way or another. “Embedded” often refers to being embedded in an online course, but these conversations also brought up ways to extend the library’s presence beyond the one-shot, without necessarily being embedded online. For example, librarians can collaborate with faculty to redesign a course or a central assignment. That sounds like it can be a huge task (to me, at least), but some possibilities for integrating information literacy outside of the one-shot could be having students do a reflection paper about their research process, introducing concept mapping to develop literature reviews, or discussing with faculty how information literacy fits in with their own disciplinary content and pedagogical goals.

Another opportunity to be more embedded comes as a solution to a common problem – when you receive a request for instruction at the very beginning of the semester, clearly not at the point-of-need. Of course, try to schedule the instruction session at a time when students will benefit more from the information, but you could also visit the classroom at the initial request for a short 5-10 minute introduction of yourself and the library. This would increase students’ familiarity with a librarian and allow you to build a relationship with students prior to the one-shot session, an important connection which I think can go a long way. If the initial classroom visit gets too time-intensive, it could be replaced by a re-usable introduction video.

Flipping

During the interactive session on the flipped classroom, my group ended up talking about student buy-in and accountability: what do you do when students come to class having not done the pre-assignment or reading? One answer is to plan ahead with faculty so that the pre-assignment can be added to their syllabus, thus adding more accountability. My first reaction to this idea was that there is no way I can have instruction sessions planned out far enough in advance to be added to a syllabus. However, I now think this could take the form of a more general statement, for example:  “At least one class session will be led by a librarian to introduce you to library resources and assist with research skills. This may require a pre-assignment.” This leads to another point that the presenters stressed as important for a successful flipped classroom: identifying faculty who will be supportive. It’s less likely that students will see value where their instructor doesn’t.

That session also served as a great reminder for me that flipping the class should not be an opportunity to cram in more information, but an opportunity to cover a topic more in depth through the use of a pre-assignment and in-class active learning. I realized that the one time I somewhat-flipped the classroom, it was because I didn’t have time to cover everything I wanted to. I sent a video tutorial for them to watch ahead of time, and it was just an add-on, rather than an enhancement.

And More

These are really just a few things that I came away with after LOEX, and it’s already my longest post here yet. Some other useful ideas I picked up had to do with active learning assessment, design recommendations for online tutorials, and reflecting on and improving teaching strategies.

I constantly had a tough time deciding which session to attend, because they all looked good. By scrolling through the conference hashtag (#loex2014) on Twitter, I could tell that was the case. One thing I didn’t expect was how much I enjoyed the interactive sessions. Although I didn’t think I would want to interact very much, I ended up loving how they facilitated conversation and sharing of ideas with new people.

It was great to attend a conference for academic librarians that was so focused on instruction and information literacy, and I definitely hope to go to the LOEX Conference again sometime.

Getting Started with Instruction

This semester marks a significant step for me as I’m finally getting into doing instruction sessions on my own. Throughout last fall, I observed a lot of instruction sessions from several librarians and across a range of subjects. I also co-taught a handful of classes with a colleague, but it wasn’t until this month that I took on my own instruction sessions. I’m really glad I did some co-teaching already, because I was definitely nervous at the time and it’s good to have that out of the way now (for the most part).

In a short span of time I have done a handful of sessions, and not one of them the same. I started writing detailed reflections of all the instruction I have done so far – what I did, what worked, what didn’t work, what I would do differently next time, etc. – and while that is incredibly useful for me personally, I will refrain from posting the entire detailed accounts here! However, I will give a quick run-down:

  • So far I have done one-shots for two sections of Rhetoric, a course that’s required of all undergraduate students, but which can vary a lot depending on the instructor. For one section, their assignment was concept-mapping and researching potential careers based on their majors; the other section needed to find images to use for a visual analysis. Like I said, interesting stuff going on that was fun to work with!
  • I did a workshop in collaboration with TRiO, an organization that works with first-generation students. Part of the goal was to send them out into the stacks in a safe, no-pressure situation, so that they can avoid the “panic moment” later on when they really need to find something. Attendance was pretty low as expected, because it wasn’t required for a course, but some good discussion came out of it nonetheless.
  • Large groups of middle school students visit our library throughout the year to do primary research for the National History Day competition, and on one occasion I gave a 15-minute introduction. I kept it simple with just basic information and demonstrating SmartSearch – it was fun to switch gears for a bit for a much different audience than usual.
  • And most recently I gave an Express Workshop on how to use and make infographics. Express Workshops are weekly 30-minute workshops held in an open area in the Learning Commons, with a different topic and presenter every week.

I’m glad to have such a variety of classes to work with – for one thing, it keeps things interesting, and for another, I think it’s more challenging (in a good way) than if I were repeating basically the same session. However, the planning has been difficult at times.

A lot of the difficulties may come down to time management and figuring out my own process. I planned ahead as much as possible, but often felt like I was really getting prepared when time was down to the wire. I wanted to have lesson plans laid out a good deal ahead of time and prevent the stress of procrastination, but it was difficult for me to focus on future sessions when there were others to take place first – especially since these were my actual first instruction sessions ever. I think my planning problems stem in part from the fact that this is a much busier time of year than I expected it would be!

I can’t wait to get to the point where I’ve done enough instruction that I’m more confident with the whole process, from planning, to delivery, and assessment. When planning a session I consider many possible options and what would be most effective, and then still tend to question my decisions on what to include and how to conduct the session. I already feel a little more confident in my teaching abilities than I did even a month ago, and I know that the rest will take some more time and practice.

Does anyone else have similar concerns? Do you plan ahead, or do you work better under pressure? How much time does it take to plan a session?

Focusing the Mind, Practicing Attention in the One-Shot Library Session

ACRLog welcomes a guest post from Jill E. Luedke, Reference & Instruction Librarian at Temple University.

Scenario: Students arrive at the library instruction session, get seated, and log on to a computer. Where is their attention? Is it on what I have to teach them? More likely, they’re distracted by competing priorities like assignments, rent, relationships, work, or the allure of some electronic device. It seemed no matter how I would package the content, many of them were still unable or unwilling to receive what I was presenting. I realized that to be more effective, I first needed to focus the students’ attention.

As a teacher of lifelong research skills, it’s part of my responsibility to give students tools to help them handle their frustrations and preconceptions about research. How could I expect students to process what I was saying if their brains weren’t ready to receive the information? I began the experiment of devoting a few minutes of my sessions to guided mindful meditation. My intention by having students meditate at the beginning of class was not to turn them all into Buddhists. It was to help clear their mind-clutter and reduce their research stress. This practice in mindfulness was about preparing them to be receptive learners.

That may sound like quite a feat, but as a practitioner of yoga and meditation I had experience with the immediate and lasting benefits of these types of practices. Whenever I was stressed or feeling overwhelmed, I could take a few moments in my office to close my eyes, breathe, and “let go” before heading out the door to teach a class.

In class, I avoid the stigmas and stereotypes associated with meditation by referring to it as an “exercise” or a “practice.” I frame it in the context of addressing research stress. Watching the students, sitting with their eyes closed, is sometimes my only opportunity to know whether or not they are actually paying attention to me. Afterwards, we’re more ready to move forward with the rest of the curriculum.

I’ve noticed that engagement in my classroom activities has improved through the incorporation of meditation, especially when they notice their instructor participating. I’ve also found it to be a useful way to form a connection with students in the one-shot class. The responses I’ve received so far have been anecdotal, but positive. I frequently have one or two students who thank me or comment how much they liked the “meditation” (they give it that name). Inevitably, one or two students don’t participate in the activity, but they still sit, quietly, waiting patiently. One instructor told me, “At first, I thought, this is way too hippy dippy for me, but then I just went with it, and it was awesome.”

Good instruction may require incorporating unconventional pedagogical practices. For me, my teaching was influenced by a learning environment that wasn’t a traditional classroom. Trying something off-beat could appear misplaced. However, if this new technique is applied with authentic intention it can transform the classroom experience for both the teacher and the student.

I discovered that by leading meditation, my authentic self is a little brighter in these instructional sessions. Conducting something so “hippy dippy” in this unexpected context leaves me a bit exposed, but I’ve noticed it’s been a way for me to offer a little of myself to my students. I’ve found that this type of vulnerable offering says more about me than a story I could tell about myself in an effort to “connect” with my audience. I continue the personal mindful practices that help me be more present for my students. Complementing this, I’ve found the more I lead mindful practices for my students, the more focused and attentive we all are to each other. If deviating from the traditional notion of class time results in a more productive learning experience, then this is an experiment I intend to continue.

Further Reading:

Brown, P.L. (June 16, 2007). In the classroom, a new focus on quieting the mind. The New York Times. Retrieved from http://www.nytimes.com/2007/06/16/us/16mindful.html?pagewanted=all&_r=0.

Parry, M. (March 24, 2013). You’re distracted. This Professor can help. The Chronicle of Higher Education. Retrieved from http://chronicle.com/article/Youre-Distracted-This/138079/.

Tugend, A. (March 22, 2013). In mindfulness, a method to sharpen focus and open minds. The New York Times. Retrieved from http://www.nytimes.com/2013/03/23/your-money/mindfulness-requires-practice-and-purpose.html?smid=pl-share

Strategies for That Time Again

It’s that time of the semester again, the time when I find myself responding to requests by saying “When is this due? It’s that time again.” And beginning conversations with the same phrase: “How are you?” “Busy,” is usually the response. “Me too — it’s that time again.”

At my university the weeks between Halloween and Thanksgiving are usually the busiest time for library instruction, the time just after midterms and when students are beginning to work on their final research assignments. This year enrollment is up at the college so we have an unexpectedly large number of library sessions for our introductory English Comp course. It’s a good thing — we love it when students come to the library! — though our Instruction Team is perhaps stretched a bit thin this semester, our classroom nearly constantly booked.

With so much instruction this semester it’s easy to feel somewhat out of control, like we’re spending our time being more reactive than active and less intentional about instruction than we’d like. Our Instruction Team’s usual strategy for instruction is to tie it closely to students’ course assignment, to allow students time to work on their course-related research during the library session, to try to incorporate active learning whenever possible. But when things get busy it can be challenging to meet these goals. With all of the additional sections there are a large number of adjunct faculty who are new to the college, and it can sometimes be difficult to get in touch with them to discuss the session beforehand. Sometimes an instructor’s schedule will change; what seemed at the beginning of the semester like a library session date that fit well with students’ work on research assignments suddenly isn’t anymore. And sometimes, despite our best efforts, a class comes in without an assignment, the instructor requesting an orientation lecture that’s not closely tied to their research for the course.

My colleagues and I have given lots of thought to these intro English Comp sessions, the backbone of our library instruction program. We’ve created student learning outcomes, we have a short assessment, we think hard about how the session can meet the needs of our students as they begin to build their information literacy competencies in college. But when the classroom is booked straight through from 9am-5pm most weekdays, when we can’t find an hour during the week for our whole team to meet, I wonder how we can preserve some time for reflection and intention. What strategies do you use to build in time for thinking on and discussing instruction at your library, even when the semester’s at its most scheduled?