Resolutions for Failure

How bout that January, eh?  Lots of memes out lately about the longest month ever.  Yet, like this reddit thread, I don’t really get it. I mean, despite my Oklahoma-born, summer-loving upbringing, I do expect that January is supposed to be snowy and damn cold.

I also don’t love, but expect annual evaluations.  They provide a time to reflect on the highlights of the year and set goals for the next.  Most often I approach this task (and leadership generally) from a strengths-based perspective, which has its roots in positive psychology research.  I encourage people to own what they are best at, even using it to build areas at which they feel not so great.   But, as January has brought a lot of harsh realities to the fore, it feels necessary to juxtapose this month’s normal, optimistic resolution with a page from Brene Brown and ponder what didn’t go quite right this year.

My acceptance into the 2018 cohort of the Institute of Research Design in Librarianship (IRDL) certainly put a postive move on a long-stuck research agenda, and in all respects (except one) it was an a-ma-zing experience. That same week, I was also furtively struggling to complete editor changes for a book chapter on knowledge management in libraries (ala this past post).  Trying to do research while learning how little you actually know about research is one thing.  Working on two research project simultaneously with that fragile skill set is another.  Working against an already extended deadline on a near-complete redo of said research and writing certainly takes one down a peg or two.  But wait!  There’s more.  None of these humiliations can beat the crushing horror four (4) months after submitting the final revised draft, realizing that I’d attached the wrong file.

Yes. Epic. Fail.

I have never asked for an extension I couldn’t meet. I have never wanted to write about a topic more than I wanted to write about meetings and knowledge management in library organizations.  Needless to say, the editors confirmed they’d moved forward without my chapter included. But if we’re being honest, while I was satisfied with the final draft I thought I’d submitted, this blunder was a blessing in disguise that helped me realize how far my cart was in front of this particular horse.

My actual and ongoing research for IRDL has been more like an extremely long January. I’ve progressed in some ways with ease and others with more groping at the dark.  Navigating my mentoring and research  network, I’ve partnered with a friend and colleague who is familiar with my topic and who has strengths in areas that I need to grow.  She and I have spent most of the year sorting out data after messy, incomplete data, just trying to figure out how to approach a sample to use for our analysis.  It’s been frustrating, paving over the same paths and feeling you’ve come up no further along.  We met again this week to pave with our local hub of digital research librarians. In the process we made breakthrough.  A face-palming breakthrough, but a breakthrough nonetheless.

I like to think Winston Churchill, as he’s often quoted, understood the better that lies ahead of the struggle.  Better even than the adage that this too shall pass (because, kidney stones?),  I prefer to remind myself and others that research is just messy until it’s not messy. This is what we teach as librarians, but sometimes forget to tell ourselves.

If I hadn’t been introduced to Brene Brown’s research, or learned what I did from IRDL, or had this particular editorial experience, or the practice of using my strengths, I don’t know that I could as easily take fails forward into something better and more genuine.  That I can say moving through vulnerability has become easier for me, is precisely because that is what the concept of strengths brings to bear for anyone’s vulnerabilities.  My top five Gallup strengths – Learner, Activator, Strategic, Analytical, and Individualization — help me more easily learn from my mistakes, analyze and strategize new paths, know myself and who to go to for help, and take action to keep going!  But even if you can’t yet  see your own strengths this way, research has shown vulnerability is a necessary part of personal and professional growth.

When I complete my current IRDL research, and when (not if) I  get back to research and writing about meetings and knowledge management in libraries, you and I both want it to be good and valuable and cleaner than the path it takes to get there.  So, I embrace the mess!  It may not always be pretty, but it’s a path that moves you forward if you let it.

Failure and Feelings

This semester I’m co-teaching a graduate class at my university in a certificate program in interactive technology and pedagogy. It’s a course I’ve taught before, though it changes somewhat each time I teach it, in part because it rotates between several different faculty members every year or so. The course focuses on the practice of teaching and learning with technology, and this week, our last “content” week before our students’ final presentations, the topic was failure.

We’ve had a session on failure during the other times I’ve taught the course, though I believe this is the first time that failure is leading us into the final presentations (and papers due soon after). Our discussion this week was terrific — the students and my co-teacher and I brought our own experiences with failure inside and outside the classroom to bear on our conversation, and we talked through both logistical/practical and emotional aspects of failure in academic contexts generally as well as around their projects specifically. An article by Alison Carr, In Support of Failure, was the focus of much of our discussion, especially about the emotions around failure.

It’s not yet the end of the semester (my college’s semesters go very late — 11 more days!), and I’m thinking about failure too. I’d meant to write more often on ACRLog this semester, but failed to do it. I’d meant to find something more interesting or relevant to write a post about today, but failed to do it. I’m thinking about failure that’s both general and specific: it’s not that I don’t have ideas for topics to write on, but that the topics seem either too well-trodden or too local. There’s a lot going on right now, though I do have time to write, but I’ve failed to take that time to write. I’m feeling all kinds of emotions around these kinds of failure, most of them of the mopey variety, though I also realize that here near the end of the semester it’s not unusual to have more feelings than usual.

When we were talking through classroom failures in class earlier this week, we talked a bit about failure in research and library instruction and how some experiences that might initially seem like failures can actually be pretty valuable for students. The pre-planned vs. spontaneous approach to teaching about keyword searching is a great example of the way a failure can be a useful learning experience. Students are unlikely to find exactly what they’re searching for the first time around, and for a librarian to model (in front of the whole class) that process of searching, not getting useful results, and refining your keywords and strategy to search again is much more realistic for students to see. And (I hope) it makes them feel less anxious about doing their searching “the right way.”

Thinking on this more today I’ve realized that a successful spontaneous search in an instruction session is somewhat choreographed, and still has some measure of control that prevents it from being a true failure. There is that element of uncertainty — it brings me some discomfort to be spontaneous in front of an entire class because what if it doesn’t actually work? What if we refine keywords again and again and still don’t find anything useful? There’s a right way and a wrong way to fail in the library classroom, which seems tied to control. I wonder, if we’re willing to give up some of that need for control, is it still possible to fail in the right way?

Failure is an Option or When Things Go Wrong

Several times I thought about writing about my experiences as a first-year librarian at ACRL in Baltimore, but many others wrote about the controversies, the twitter fights, and the OA panel better than I could have. Zoe Fisher’s post about the twitter fiasco is a must read as well as Veronica’s post on ACRL.

As a new librarian, something stuck with me though, and that is the idea of failure and works in progress at the conference. A conversation with Katlyn Griffin, a fellow new librarian,  and I had via twitter and in person about the idea of “works in progress” or even “failed projects” as learning opportunities at conferences. ACRL, as I’m sure many of you know, isn’t the venue for unfinished or failed products.

As a field that feels on the brink, it is difficult to talk about projects that fall flat. No one, least of all me, wants to broadcast failures. Unfortunately, if you’re a new person, it is often the failed projects or the work-in-progress projects are all that you have to contribute. I think there is a difference between work-in-progress and failures, but they exist at this periphery of prepared and completed national conference level discussion. I presented a work-in-progress at ACRL and it actually went very well; I learned a lot about the process and it helped me move towards a conclusion, and I believe that conferences should encourage unfinished work outside of the lightning rounds where the blur makes the projects blend in with the background. But where does failure stand?

Scenes from NASA

“Failure is Not an Option” Gene Kranz, Flight Director of Apollo 13

It is difficult to broadcast or talk about failing, especially in competitive circles, but I am a product of failure.

Prior to library school I was on a traditional academic path. I interviewed at PRESTIGIOUS UNIVERSITY for a PhD program in Media Studies. After three days of being wined and/or dined, tours of campus, meetings with students and faculty, talking about my future there, I was not given a spot in the program. I felt rejected, like all that I had been working toward was squashed on the whims of a committee. I thought a lot about why I was drawn to the field and where I was the happiest when I was a student. That place for me was the library. I was interested in memory and media but more than that I was interested in knowledge as an academic field.

Ultimately, the library was the better and wiser choice for me. I am thankful that prestigious university did not pick me for their PhD program, as much as it hurt at the time.  I’m able to write and publish about topics I feel passionate about, teach and work with students, support faculty, and I’m extremely happy to be where I am. Libraries are my home, but still I felt like it was a “plan b.” I’m certain that I’m not the only one out there in our field who has a similar story.

Even writing this paragraph was difficult because failures, or the perceptions of failure or disappointment, are difficult to talk about.  It would be even more difficult to stand in front of an audience of colleagues and say “this is what went wrong.” Imagine that during a staff meeting… now imagine it during a National conference.

Given the response that accompanied the polished, but controversial, papers presented at ACRL, I wonder what the response would be if a paper concluded with “this project failed.” Would it be seen as a waste of time? Would the twitter-sphere explode in rage at the failed project? Currently, we all struggle silently alone.

In the past year I’ve had a few projects go awry. I had our Research Week Student Research Symposium hosted entirely by our Institutional Repository, it did not go well. In the end, our research office decided to go another direction but will now require students to deposit their materials. This was my first professional set back, but it allowed me to grow and see where the edges were in our partnerships across campus. I worked extremely hard on getting this to work, and in the end it just could not do what our research office wanted. Does this reflect poorly on me as a librarian? I thought so at the time. When I took a step back and thought about it, I really began to believe that this was a bump on a long road. Sharing this experience with the other librarians and saying “here is what didn’t work,” allowed all of us to learn from this experience. Getting all of the student projects deposited in the IR after the fact is all that I wanted in the first place, so this failure ended up as a slight win.

Projects fail. We do not know when that they will fail when we start them, if we did we probably wouldn’t repeat the mistakes that caused the problems, but these are valuable opportunities to learn about the process and the problems we all share. If creativity and experimentation are valued in our field then there should be expectations of failure and we should talk about them openly. I firmly believe that opening forums like ACRL or ALA to “failure talks” could be a great asset for new and old librarians. Talking frankly about what went wrong instead of sweeping it under the rug should be a goal of the larger library community.

More importantly, we fail. Failure has formed me as a librarian. We sometimes do not get the jobs we desperately wanted, or the promotion, or the book chapter. Right now the culture tells us to be quiet about these instances and shames; because being a sore loser or being upset about losing out on a fellowship or project is unbecoming and might jeopardize our future goals. I don’t agree with this because I think as a field we should grow to a place where we are not ashamed of our baggage and our failures and drop the feelings of animosity and competition. Especially for new librarians where the road upwards is the most difficult, letting  individual failures be known is a powerful reminder of what we have to gain and lose as a field. Ultimately, this means that successful librarians must lift up those around us when projects or goals fail. We should be open about our failures to serve as a guide to those who will follow, and lift up those who are not as lucky as we. 

 

When is the Struggle TOO Real?

One of the advantages of having a partner who happens to be a math professor is that we can talk academic shop. A few weeks ago, over a serious dishwasher unloading, we started talking about a recurring theme manifesting itself in our college’s faculty Facebook group: toughening up college students. From debates about trigger warnings to conversations about cultivating students’ grit and comfort with failure, our colleagues are consistently inconsistent about how we should help college students succeed in academia and life. I’ll lump myself and my partner into this group, too. As a faculty we want to be sensitive to student needs and life experiences, but we also don’t want them to fall apart if they get a bad grade on an exam. We want them to make a real attempt at solving a difficult problem or tackling a challenging project on their own before asking for help, but we also recognize that many students have serious outside stressors (economic, familial, emotional, etc.) that might prevent them from giving their all to their studies.

For years librarians have been chanting that “failure is good” because it is a signal of attempted innovation, creative practice, and learning (particularly when applied to information literacy instruction). We want our students to learn from their mistakes, which means they have to make them first. Math education is no different. There’s a small but mighty push for experiential and problem-based learning within the discipline that wants students to learn from their mistakes. As my partner and I discussed this we couldn’t help but wonder:

At what point is the struggle too much?

Earlier in the day he’d met with a student who claimed she was working on one homework problem for 4 hours. Earlier that semester I’d met with a student who spent an entire weekend looking for research in the wrong places with the wrong search terms. I’m all for giving it the old college try, but in both cases, this just plain excessive struggle for little reward. As a librarian who has been doing this job for a while, I have a good sense of when I’ve tapped my intellectual well. I know when to ask for help. My partner does, too. Most academics know when to take a step back, take another approach, or ask a colleague for suggestions. But this is a learned skill. We like to think of it as tacit knowledge–students have to experience failure to know when they are failing the right way as opposed to just struggling unnecessarily–but is it really? Does the experience alone help them gain this knowledge? Or can the struggle just be too real for some students, leading them to eventually equate math or research with pointless stress?

I think the key in the library classroom is not to focus on failure but to focus on process: Model, practice, repeat–over and over again. It’s a challenge when so much of students’ grades depend on a final product (an exam, a paper, a presentation, etc.) and often requires a shift in emphasis from the professor. By modeling a process–a step I think we (and I know I) often overlook in our attempts to make our classrooms spaces for active learning–we give students a sense of what struggle can look like. Granted, there’s no one standard process for research, and we don’t want to imply that there is one, but making our thinking and doing visible to our students can go a long way towards demystifying research. We get stuck, we back-track, we try again, we struggle, but we are never alone when we do so. It’s something I try to stress to all my students in hopes that they too feel like they never have to struggle alone.

If At First You Don’t Assess, Try, Try Again

ACRLog welcomes a guest post from Katelyn Tucker & Alyssa Archer, Instruction Librarians at Radford University.

Instruction librarians are always looking for new & flashy ways to engage our students in the classroom. New teaching methods are exciting, but how do we know if they’re working? Here at Radford University, we’ve been flipping and using games for one-shot instruction sessions for a while, and our Assessment Librarian wasn’t going to accept anecdotal evidence of success any longer. We decided that the best way to see if our flipped and gamified lessons were accomplishing our goals was to evaluate the students’ completed assignments. We tried to think of every possible issue in designing the study. Our results, however, had issues that could have been prevented in hindsight. We want you to learn from our mistakes so you are not doomed to repeat them.

Our process

Identifying classes to include in this assessment of flipped versus gamified lessons was a no-brainer for us. A cohort of four sections of the same course that use identical assignment descriptions, assignment sheets, and grading rubrics meant that we had an optimal sample population. All students in the four sections created annotated bibliographies based on these same syllabi and assignment instructions. We randomly assigned two classes to receive flipped information literacy instruction and two to play a library game. After final grades had been submitted for the semester, the teaching faculty members of each section stripped identifying information from their students’ annotated bibliographies and sent them to us. We assigned each bibliography a number and then assigned two librarian coders to each paper. We felt confident that we had a failsafe study design.

Using a basic rubric (see image below, click to enlarge), librarians coded each bibliography for three outcomes using a binary scale. Since our curriculum lists APA documentation style, scholarly source evaluation, and search strategy as outcomes for the program, we coded for competency in these 3 areas. This process took about two months to complete, as coding student work is a time-consuming process.

assessmentchart

The challenges

After two librarians independently coded each bibliography, our assessment librarian ran inter-rater reliability statistics, and… we failed. We had previously used rubrics to code annotated bibliographies for another assessment project, so we didn’t spend any time explaining the process with our experienced coders. As we only hit around 30% agreement between coders, it is obvious that we should have done a better job with training.

Because we had such low agreement between coders, we weren’t confident in our success with each outcome. When we compared the flipped sections to the gamified ones, we didn’t find any significant differences in any of our outcomes. Students who played the game did just as well as those who were part of the flipped sections. However, our low inter-rater reliability threw a wrench in those results.

What we’ve learned

We came to understand the importance of norming, discussing among coders what the rubric means, and incorporating meaningful conversations on how to interpret assessment data into the norming process. Our inter-rater reliability issues could have been avoided with detailed training and discussion. Even though we thought we were safe on this project, because of earlier coding projects, the length of time between assessments created some large inconsistencies.

We haven’t given up on norming: including multiple coders may be time-intensive, but when done well, gives our team confidence in the results. The same applies to qualitative methodologies. As a side part of this project, one librarian looked at research narratives written by some participants, and decided to bravely go it alone on coding the students’ text using Dedoose. While it was an interesting experiment, the key point learned was to bring in more coders! While qualitative software can help identify patterns, it’s nothing compared to a partner looking at the same data and discussing as a team.

We also still believe in assessing output. As librarians, we don’t get too many opportunities to see how students use their information literacy skills in their written work. By assessing student output, we can actually track competency in our learning outcomes. We believe that students’ papers provide the best evidence of success or failure in the library classroom, and we feel lucky that our teaching faculty partners have given us access to graded work for our assessment projects.