“Sunrise, Sunset”: A Reflection on Assessment and the Framework for Information Literacy for Higher Education

ACRLog welcomes a guest post from Donna Witek, Associate Professor and Public Services Librarian at the University of Scranton.

url

Photo by Moyan Brenn on Flickr

When I first learned about assessment at the very beginning of my professional work as a librarian, there was one aspect of the process that made complete sense to me. I was instructed that an assessment plan is just that–a plan–and that it is not only OK but expected for the plan to change at some point, either during or after it’s been put into action.

Now, the specifics on how these changes happen, what are best practices in altering an assessment plan, and the relationship between the integrity of the assessment data gathered and any changes made, are all complex questions. I am in my seventh year working as an instruction librarian in an academic library, and I consider myself at best an engaged learner-practitioner when it comes to assessment–I am by no means an expert, and I offer this as a disclaimer as I share some thoughts on assessment and the Framework for Information Literacy for Higher Education [pdf].

In the years since I was first trained in basic assessment practices, I still find the recursive, cyclical nature of assessment to be the aspect of the process that legitimizes the rest. Learning is a messy process, and as instructors we understand that there are multiple ways to reach the same goal–or learning outcome–and that different learners learn differently. It could mean our approach to teaching (i.e., our pedagogy) needs to be adapted–sometimes on the fly!–to meet the needs of the students in front of us. Or, maybe the way I articulated one of the learning outcomes for an instruction session turns out to be way too ambitious for the scope of the instruction, and ten minutes in I realize I need to change the formulation of the outcome in my mind in order for my teaching and the students’ learning to harmonize.

What I love about the principle that an assessment plan is meant to be changed (at some point) is that it means the above scenarios are not failures, but part of an authentic teaching and learning process. This is empowering for teachers and students alike.

Now, it is my understanding that all assessment plans change eventually. In the case of an assessment plan that from the outset is harmonized perfectly to the learning context to which it is applied, it isn’t changed until the end of the assessment cycle, but it still changes and develops in response to the information (call it data if you’d like) gathered throughout the process.

At the end of this week and after almost two years of development and review by the profession, the Framework will be considered for adoption by the ACRL Board of Directors during ALA Midwinter. The Framework is not conceived as an assessment document, as it “is based on a cluster of interconnected core concepts, with flexible options for implementation, rather than on a set of standards or learning outcomes or any prescriptive enumeration of skills” (Framework [pdf], p.2).

This begs the questions: What is the relationship between the Framework and assessment? And how does this in turn relate to the revision task force’s recommendation that the Information Literacy Competency Standards for Higher Education be sunsetted in July 2016 (Board of Directors Action Form [pdf], p.3)?

Before I share some ideas in response to these questions, Megan Oakleaf offers to the profession “A Roadmap for Assessing Student Learning Using the New Framework for Information Literacy for Higher Education” [pdf] (JAL 40.5 2014). I highly recommend reading Oakleaf’s roadmap, as my own ideas touch on many of the same points found in her “Ok, So Now What?” section, though I want to fold into the discussion the relationship between this process and the proposed sunsetting of the Standards.

Here I offer just one of many possible paths toward incorporating the Framework into your local information literacy instructional practice. It is a theoretical model, because it has to be at this point: the Framework is not yet adopted. As will hopefully be made clear, not enough time has passed for this model to have been fully implemented, though some libraries have begun the process. (1)

The first step I would recommend, based on evidence from libraries that have taken this approach and found it fruitful and impactful on both student learning and programmatic practices, is to read the Framework, both individually and as a group with colleagues in your instruction program, and through reflection and discussion identify intersections between the Framework and the information literacy instruction work you are already doing. (2) Rather than feel pressured to overhaul an entire instruction program overnight, instead use the Framework as a new way to understand and build upon the things you’re already doing on both the individual and programmatic levels.

If your current practices are heavily situated within the Standards, I think this exercise will surprise by unearthing the connections that do in fact exist between the Standards and the Framework, even as the latter represents a significant shift in our collective approach to teaching and learning. (3)

The next step would be to review your learning outcomes for individual instruction sessions in light of the Framework, to be inspired by the connections, and to be challenged by the gaps–and to rewrite these outcomes based on both your engagement with the Framework and your recent assessment of your own students’ learning using these outcomes. The cycle of assessment for learning outcomes tied to individual instruction is short–these outcomes can and should be reviewed and revised in the period of reflection that immediately follows each instruction session.

In many ways, this makes individual instruction the most immediately fertile context in which to use the Framework to be inspired and to transform your instructional practice, keeping in mind the complex concepts that anchor the frames require learners to engage them in multiple learning contexts throughout the curriculum in order to be fully grasped. Still, even a one-shot can incorporate learning outcomes that will help learners progress toward understanding of these concepts in a manner appropriate to the learner’s current level of training in a discipline or disciplines.

But what of your programmatic information literacy learning outcomes? What about the places where information literacy has been integrated into curricular programs within or across the disciplines? And what about those (fortunate!) institutional contexts in which information literacy is integrated explicitly into the learning outcomes for the institution as a whole?

The beauty of assessment, as I suggest above, is that it is cyclical. Just as all ACRL guidelines and standards undergo cyclical review, so too do our local assessment and curriculum plans–or at least, they should. As each assessment plan comes up for review, librarians who have been engaging the Framework in their individual instructional practice can share “upwards” their experiences and the impact on student learning they observed through that engagement, and so fold the concepts underpinning the Framework into each broader level of assessment.

In this way, the Framework’s influence will cascade upwards within a local institutional context according to a timeline that is determined by the review cycles of that institution. While the revision task force’s recommendation to the ACRL Board is for the Standards to be sunsetted a year and a half after the Framework’s recommended adoption, I would argue that it is in the spirit of the Framework for local timelines to necessarily trump ACRL’s: as long as librarians are engaging the Framework, both individually (in instruction) and collaboratively (as local assessment plans and curricular documents come up for review), and doing so in light of the information literacy instruction work your library has been doing since (or even prior to) the adoption of the Standards fifteen years ago, (4) then the worry associated with sunsetting the Standards on the national level will be eclipsed by the particular, robust influence the Framework is having on your own campus, with your own students.

And anyway, we do our best work when we’re focusing on the students in front of us. So, let’s get to work.

 

Notes:

(1) Nicole Pagowsky shares the first steps of a similar process underway at the University of Arizona.

(2) The first example of this I’ve encountered is at Trinity College; librarians leading in different areas of Trinity’s information literacy instruction program presented at the 2014 Connecticut Information Literacy Conference their success with this initial approach to implementing the Framework (video and prezi).

(3) Amanda Hovious has created a helpful series of Alignment Charts for ACRL Standards and Proposed Framework, which represent one practitioner’s approach to connecting these two documents. I would argue that there are as many potential charts/models for connecting the Standards to the Framework as there are practitioners interpreting the meaning and content of each. It is for this reason I believe it was prudent for the revision task force to abstain from developing a model for alignment themselves, as such a model would run the danger of being wrongly interpreted as “canonical” because of its association with the task force that developed the Framework. That being said, Hovious’ charts are informed by her training as an instructional designer, and coupled with her notes for interpretation at the beginning of the document, represent a valuable perspective on how these two approaches to information literacy instruction relate. Another example that is equally compelling, in this case because the alignment is anchored to locally developed core competencies, is offered by Emily Krug, King University. It is compelling because it models (literally) the notion that information literacy is locally situated, by using King University’s core competencies as the concrete bridge between Standards and Framework.

(4) Barbara Fister offers an historical perspective in which she recalls the anticipated reception of the Standards when they were first adopted in 2000, and the remarkably similar conversations we are having now in relation to the Framework.

Scholarship as Conversation: The Response to the Framework for Information Literacy

ACRLog welcomes a guest post from Jacob Berg, Director of Library Services at Trinity Washington University.

The Association of College and Research Library’s (ACRL) Framework for Information Literacy in Higher Education (pdf) has gone through three drafts, and was sent to the ACRL Board of Directors for approval earlier this month.

It was possible to do an excellent job of teaching information literacy (IL) under the old Standards, and that remains the case. It was also possible to do a lousy job. Nothing has changed. The same is true of the Framework; some campuses will thrive under it, while others will not. In all these instances, neither the Standards nor the Framework was or is sufficient or necessary to successfully teach information literacy.

And yet the discourse around the third and final draft should make many academic librarians pause. Conversations in blog posts, listservs, and social media reveal straw men, ad hominem attacks, and a lack of understanding of educational psychology and pedagogy, among other issues. Observing these discussions, we should reflect on how we interact with each other to create knowledge regimes and epistemic communities (1). Here I will focus on blog posts.

I.

In the last few months, we’ve seen an Open Letter from some New Jersey academic librarians, since signed by others, ask the ACRL to not sunset the Standards, as well as a fierce rebuttal from two academic librarians in New York City, among other works.

The former accuses the ACRL Framework Task Force of being “tone deaf to the politics of Higher Ed.” It also lacks any evidence of information literacy “success,” however defined.

  • What did information literacy look like in New Jersey academic libraries prior to implementing the Standards, and how have the Standards helped?
  • Who did these Standards work for? Librarians? Professors? Administrators? How, and why, or why not?
  • What would change in New Jersey under the Framework?

The answers to these questions go unmentioned.

In addition, the Open Letter mentions the political stakes for a shift from Standards to a Framework, but fails to show what those stakes are. I would very much like to hear more about this. (For what it’s worth, at my place of work I will spend my meager political capital elsewhere, as the administration prefers the American Association of Universities and Colleges IL rubric, and I believe there are many roads to information literate Damascus.)

Maybe the Framework is “tone deaf to the politics of higher education.” But maybe the politics of higher education are tone deaf to what educators, librarians included, are trying to accomplish in classrooms and on campuses. No doubt that politics is powerful, more powerful than academic library and information science (LIS) professionals, but given what I see of said politics, I’d much rather be against it than with it, and some push back is healthy.

Meanwhile, Ian Beilin and Nancy Foasberg mount a powerful defense of the Framework in a rebuttal to the Open Letter:

The Standards understand information as a commodity external to the student; something that can be obtained and subsequently “used.[i]” When we look at information in this way, we are thinking of information literate students as consumers who must choose among many options, like shoppers selecting goods from among those placed before them in the market. The Framework instead aims at a more social understanding of information and information literacy. Most notably, it uses the explicit metaphor of a conversation, but it is also interested in the ways that authority is constructed and the ways that information artifacts are produced. Research is thus framed as an interaction among people rather than a choice among artifacts.

Yet their article maligns standards everywhere with the specter of Common Core, a case of guilt by association (though to be fair, the Open Letter mentions Common Core first). To Beilin and Foasberg, the move to return to the standards is “a conservative, backward-looking disposition,” never mind that one reason Common Core is so reviled in some circles is how radical it is.

Writ large, their defenses of localized learning and the role of theory in library and information science inadvertently expose Threshold Concepts (TCs), mentioned only once in their article, for what they are: a loose collection of pedagogically unsound and empirically untested practices. To wit:

  • If localization is a worthy goal of the Framework why do Threshold Concepts come from a Delphi study as opposed to individual institutions?
  • To what extent are these Threshold Concepts like, and unlike, Standards?
  • Theories gain acceptance when tested. What are the tests for Threshold Concepts? Where are they? (2)

It is interesting that an article so focused on theory should ignore the theoretical issues that make up the bedrock of the Framework.

II.

Responses garnered from the most recent feedback form (pdf) that accompanied the third draft in November showed that, of the 206 surveys received,

  • 91% were satisfied with the opportunities to provide feedback to the Task Force on drafts of the Framework
  • 67.4% support the new Framework
  • 63% were satisfied with the proposed definition of information literacy
  • A majority of respondents were satisfied with the new frames (satisfaction ranged from 71% for Information Creation as a Process to 83% for Scholarship as Conversation).

I do not know if 206 responses is a good number or not, but one jarring realization to emerge from this process is that while many academic librarians are faculty and/or instructors on their campuses, we lack a grounding in educational psychology and pedagogy. (3) How else would we have come to either embrace or tolerate Threshold Concepts?

“What do you wish your students were able to do?” “What kind of work do you think they could create?” “What do they come to this school being able to do?” “What does a graduate of X college look like?”

Those are questions one library director asks faculty at her place of work. They are good questions, but neither Standards nor a Framework makes those questions possible. If the current discussion has enabled or validated one to ask them at a place of work, that is excellent, but as I see it, those questions were always there for the asking. There is nothing in LIS education that prevents this discursive formation under the Standards, or before their adoption in 1999.

III.

The upcoming ACRL meeting at the American Library Association Midwinter meeting in Chicago will have a spirited discussion on the Framework, featuring the Board of Directors and a question and answer session. Because scholarship is indeed a conversation, at least part of the time, it is my hope that the discussions provoked by the above links, including those in the footnotes, shed some light on how librarians and information professionals interact to create knowledge and knowledge practices in the profession. I think we can do better. I will not be able to attend Midwinter, and I hope it’s free of some of the discourse we’ve seen leading up to this point.

Meanwhile, absent a set of Standards, or a Framework, strong work in information literacy will continue to take place.

 

Notes:

(1) “Knowledge regimes are sets of actors, organizations, and institutions that produce and disseminate policy ideas that affect how policy-making and production regimes are organized and operate in the first place.” John L. Campbell and Ove K. Pederson, “Knowledge Regimes and Comparative Political Economy,” 2007 (pdf).

On epistemic communities, see Wikipedia.

(2) The Women and Gender Studies Section of ACRL will be the first to test this Framework.

Again, I point to Darrell Patrick Rowbottom’s “Demystifying Threshold Concepts,” Journal of Philosophy of Education (2007), in which he argues that one can test for abilities, but not concepts; that it is empirically difficult, if not impossible to show multiple conceptual routes to the same ability; and that thresholds differ from person to person, among others.

See also, Lane Wilkinson’s “The Problem With Threshold Concepts,” Sense and Reference, (2014), and Patrick K. Morgan’s “Pausing at the Threshold,” portal: Libraries and the Academy (2015).

A similar critique can be applied to Task Force committee member Troy Swanson’s defense of the Framework; instead of shoehorning Standards into lesson plans and learning outcomes, we can now do the same with Threshold Concepts.

(3) Again, see Dani Brecher and Kevin Michael Klipfel’s “Education Training for Instruction Librarians: A Shared Perspective,” (2014) and Kimberly Davies-Hoffman, et al.’s “Keeping Pace with Information Literacy Instruction in the Real World,” (2013), both in Communications in Information Literacy.

For a good example of how educational psychology can effect academic librarianship, see Jessica Olin’s “Not Mutants nor Ninjas nor Turtles, but Teenagers,” Letters to a Young Librarian, (2015).

Using the New Framework to Teach Ferguson

In moments like the reaction to Brown’s death, we need more engagement, not less, and each of us has something to offer. –#FergusonSyllabus by David M. Perry

Last week, on November 24th, the grand jury of St Louis County announced their decision not to indict Darren Wilson for the August 9th murder of Michael Brown. A flurry of conversation and protest started. People began tweeting and media outlets started covering multiple cities across the nation (and the world) that were protesting in solidarity with Ferguson. London, Atlanta, Boston, New York, and Chicago were just a few that participated.

Amidst the tweets expressing outrage and shock about the decision, a conversation began about education, pedagogy, and the nation’s youth. Marcia Chatelain, an Assistant History Professor at Georgetown, had already started the conversation in late August with #FergusonSyllabus.  But the decision not to indict revitalized the conversation. More educators—of any level, from elementary school teachers to college professors—added suggestions under the hashtag. Moreover, blogs and media outlets started to curate the resources being shared and interview other educators about best practices for starting the Ferguson conversation.

A passage from Dissent illustrates the complexity and magnitude of the effort:

“A middle school teacher in Madison, Wisconsin had students review the grand jury evidence. Meanwhile, I had my students in Washington, D.C. connect the Ferguson decision to Rosa Parks’s activism in seeking trials for black women raped by white men in the South. Volunteers in Ferguson read books from #FergusonSyllabus to children—unexpectedly out of school again—at the local public library.”

Some of the most profound teaching recommendations (I think) came from instructors that were utilizing inquiry-based teaching models. By encouraging students to construct their own meaning and giving them a space to do, these instructors stimulated critical thinking, metacognition, and deep self-reflection in their students. One such example, from an instructor named Melissa, was featured in the New York Times Blog. Here are some of the insightful, open-ended questions she posed to her students:

  • What is justice?
  • How can we enforce it?
  • Who should enforce it?
  • What factors stand in the way of justice?
  • Do we need police? If so, what should be their job?
  • What role does/should the media play?
  • How did the media frame Michael Brown’s shooting and why? (Looking at various media outlets, including the New York Times obituary, which surprised me…)
  • Why do humans hold prejudices and how can we acknowledge them and move on?

The variety of topics introduced range from racism, housing inequality, and militarization of the police to international human rights. This movement has even gone beyond the humanities. PBS reported that science teachers were also challenging their students with issues surrounding Ferguson. One example included an instructor asking his students to learn more about tear gas and its effects on the human body.

Yet, the conversation—at least publically—on education and Ferguson has been almost silent in the library world. Some of the incredible #critlib folks mentioned it in passing in regards to critical pedagogy, but otherwise it is difficult to find other conversations. Many have rightfully acknowledged the Ferguson Public Library and their instrumental support of the community. But there still seems to be a gap in librarians’ conversation and sharing of resources, specifically among those of us that do instruction and work directly with students on information literacy.

And there is a need for our voices! The same PBS transcript featured the following conversation:

Jeffery Brown: You know, Liz Collins, you just said something a minute ago about determining the truth. When information is coming at us so quickly, especially in social media, there’s misinformation, right? How do you — how have you dealt with that?

Liz Collins: That’s so tricky and something that teenagers deal with all the time, because they love Twitter, and Facebook, and Instagram, and the information moves faster than the fact-checking. So, I think that’s an important lesson for them to learn across the board.  Just because you’re getting this information, who’s the source? How trustworthy is it? What’s that person or organization’s bias? What do they want you to think and why?And I think teaching them to challenge that and think about that goes beyond this issue, but also gives them a lens with which to approach this issue as more and more facts come out.  And every day, we’re learning more and more about what happened and having to sift through all those facts.  And I think teaching that skill is valuable in any subject and easily transferable. 

If this isn’t information literacy, folks, I don’t know what is. It might be a coincidence that ACRL released the third version of the Framework for Information Literacy for Higher Education exactly two weeks prior to the grand jury’s decision, but I think that we should see it as an opportunity.

Several pieces of the new Framework challenge us to teach students these exact skills. The issues in Ferguson can be a current, relevant, and important vehicle for students to explore their information literacy skills in. Here is an introductory list of the more salient examples in the newest document that could reflect issues specific to Ferguson as well as questions/ starting points librarians might use to form learning outcomes and activities:

Experts understand the need to determine the validity of the information created by different authorities and to acknowledge biases that privilege some sources of authority over others, especially in terms of others’ worldviews, gender, sexual orientation, and cultural orientations (lines 160-164) 

  • What type and/ or medium of information is privileged? What are some structural reasons for this?
  • How might you be an active member of the information ecosystem in combatting this privilege? What specific forms of communication might you use?
  • What modes of communication lend themselves to bias? How can you detect bias about current events like the issues in Ferguson?

Recognize that authoritative content may be packaged formally or informally and may include audio, visual, and other nonprint sources (lines 185-186)

  • Find an example of authoritative, visual content about Ferguson. What type of source is this (primary or secondary)? Why do you consider it authoritative?
  • Compare a tweet, blog post, and news source about Ferguson. Which one(s) are authoritative and why? Does authority always correlate with medium?

Understand the increasingly social nature of the information ecosystem where authorities actively connect with one another and sources develop over time (lines 191-192)

  • Find one conversation on Twitter that includes more than two people and has more than ten tweets. How were opinions changed? Were beliefs confirmed or challenged? Was anything cited and if so what impact did that have? 

Understand how and why some individuals or groups of individuals may be underrepresented or systematically marginalized within the systems that produce and disseminate information (lines 276-278)

  • Find one source where a protester is interviewed (not just photographed). How difficult was it? How is the protestor portrayed? How does this portrayal relate to the medium and/or the article’s author’s affiliation?

Employ critical skills to evaluate information; effectively resolve conflicting information; monitor gathered information and assess for gaps or weaknesses (lines 319-321)

  • Find two sources that express conflicting information about what happened between Michael Brown and Darren Wilson. Where are gaps present in either? What sources do they have (eye witnesses, forensic evidence, etc.)? What conclusions did you reach and why?

This work is hard, especially if you only have one-shot sessions. But it’s still important. For those of us that would rather stick with looking at peer review or studying the information cycle, David B. Cohen has some wise words to offer. He challenges us to think about this singular time in history as well as what we want and expect our students, the future leaders of this country, to be able to do:

And for some students who will most certainly remember this time, we’ll have to explain why this particular event—and the tragic pattern in which it fits—that mattered so much to them was not worth our time, not considered educationally relevant (emphasis mine)

We must remind our students that both stories and information are not one-sided but instead very complex and contextual. Despite some technical flaws I (and others) might see in the Framework, it is clear our profession is moving in this direction. Chimamanda Ngozi Adichie articulates what happens when people engage with only one story or perspective:

The consequence of the single story is this: It robs people of dignity. It makes our recognition of our equal humanity difficult. It emphasizes how we are different rather than how we are similar- “The Danger of a Single Story”

As librarians, we must still continually remember that failing to teach students to be perceptive, empathetic critical thinkers has immense consequences for our entire society.

Note: It would be impossible for me to cover all of the brilliant blogs, tweets, and summaries of teaching material covering Michael Brown’s death within this forum. This is in no way an exhaustive list. Please feel free to explore more sources on your own and tailor the pedagogical conversation to your area of expertise.

Likewise, the ideas generated from passages in the Framework are merely starting points intended to start conversation. They are not meant to be prescriptive or exhaustive. Librarians should adapt these ideas (and other parts of the Framework) to best accommodate their teaching constraints and style.