Meredith Farkas has a thoughtful post at Information Wants to be Free on our love of numbers and how little they tell us without context. Less traffic at the reference desk: what does that mean? It could mean that students don’t find the help they get there useful, or that your redesigned website or new signage has solved problems that used to require human intervention. More instruction sessions? Maybe more faculty attended conferences and needed a babysitter.

Meredith’s post made me think about the statistics I recently compiled for our annual report. Many of them are things we count in order to share that information with others through national surveys. We dutifully count how much microfiche and microfilm we have added to the collection (seriously?) and how many print periodicals we have (fewer all the time, but our growing access to electronic full text is virtually impossible to measure; does a title that has a 12 month embargo count?). We haven’t used this report to share how much use our databases are getting and which journals in those databases are getting downloaded most often, or what Google Analytics tells us about which web pages attract the most attention. We use that information for decision-making, but it doesn’t become part of the record because the time series we use was started back when the earth’s crust was still cooling. (Guess what: acquisition of papyrus scrolls, clay tablets and wax cylinders is way down.)

In the end, I’m not all that interested in the numbers. The really interesting data is usually the hardest to gather. How do students decide which sources to use, and does their ability to make good choices improve over time? When they read a news item that someone has posted to Facebook, are they better prepared after our sessions to determine whether it’s accurate? Do students who figured out how to use their college library transfer those skills to unfamiliar settings after they graduate? Do students grow in their ability to reason based on evidence? Have they developed a respect for arguments that arrive at conclusions with information that isn’t cherry-picked or taken out of context? Can they make decisions quickly without neglecting to check the facts? The kind of literacy we’re hoping to foster goes far beyond being able to write a term paper. And knowing how many microfiche we own doesn’t have anything to do with it.

Now I have a question for our readers. Are there ways you regularly assess the kinds of deep learning that we hope to encourage? What measures of learning, direct and indirect, do you use at your library? Have you conducted studies that have had an impact on your programs? Are you gathering statistics that seem particularly pointless? Should we start an Awful Library Statistics blog? The floor is open for comments.

photo courtesy of Leo Reynolds.

Another piece of the project involved interviewing faculty to tease out some of the thinking behind them, to see how faculty supplement assignment prompts with in-class instruction, and what issues they see students struggle with. While it was clear in the interviews that faculty are frustrated by students’ lack of preparation, and that they spend lots of time explaining how to carry out the task, the assignments themselves don’t address the problem.

PIL’s previous study of student experiences found that virtually all students use the Internet in their research, but very nearly all of them also use library databases. Not so many used books in their research. In contrast, of the assignment prompts analyzed in the study, 60% required or encouraged use of materials on the shelves in the library, 43% suggested that students use library databases (though few specified which ones would be most useful), and 26% suggested students might find good sources through the Web. Fifteen percent discouraged or forbade the use of Internet sources, and 10% specifically forbade the use of Wikipedia. The authors seem correct to describe the approach to research laid out in these assignments as “tradition bound” – not just in terms of where students were likely to find the appropriate sources, but in that 83% of the assignments asked students to write traditional research papers. (When collecting these prompts, the researchers asked for assignments that asked students to find and use sources; they didn’t ask for research paper assignments, but that seems to be the primary way faculty engage students in using sources.)

One final intriguing connection between the report on student practices and on assignments: few students turned to librarians for help with their research, though they did look to their teachers for guidance. And though the majority of assignments recommended students use print resources in the library, very few of them suggested consulting with a librarian.

Here’s the abstract:

A report of findings from a content analysis of 191 course-related research assignment handouts distributed to undergraduates on 28 college campuses across the U.S., as part of Project Information Literacy. A majority of handouts in the sample emphasized standards about the mechanics of compiling college research papers, more so than guiding students to finding and using sources for research. Most frequently, handouts advised students to use their campus library shelves and/or online library sources when conducting research for assignments, though most handouts lacked specific details about which of he library’s hundreds of databases to search. Few handouts advised students about using Internet sources, even though many of today’s students almost always integrate the Web into their research activities. Very few handouts recommended consulting a librarian about research assignments. Details about evaluating information, plagiarism, and instructor availability appeared in only a minority of the handouts analyzed. The findings suggest that handouts for academic research assignments provide students with more how-to procedures and conventions for preparing a final product for submission, than guidance about conducting research and finding and using information in the digital age.

There’s also a short video summarizing the results available as well as an interview with Andrea Lunsford, the goddess of writing instruction and a principal investigator behind the massive Stanford Study of Writing.

Note: edited to correct a few numbers that I’d reported incorrectly. (D’oh!)

photo courtesy of monica, nic

That should resonate with librarians. Welcome to the club!

Even better, he plans to make improving communication with librarians, who he calls a “kindred community,” a priority this coming year. He recognizes how we are dependent on one another, and points out that open access isn’t free; it takes money to select, organize, make editorial improvements, and make scholarly work discoverable. (Doesn’t most of that sound eerily familiar?) Though some discussion at the conference focused on joining forces to make e-books available to libraries, it seems as if we’re still seen as a revenue source, as customers, not as partners in publishing. I’d much rather invest my money in books that my students and faculty can use without the hassle of DRM, that won’t disappear if I have a bad budget year and have to cancel a subscription, and that are available to everyone in the world. Chances are I’d still buy some of the books in print – for those that will be read closely, not just harvested for quotes, the cost of printing a copy is worth it. I just don’t want to invest in collections of e-books nobody uses. (I know some libraries have had success with e-books; most of our students don’t like reading anything longer than a paragraph unless it’s on paper or can be printed. No, I don’t want to pay for a database and pay a second time for printing. Google, I’m looking at you.) And until e-readers are affordable, platform-agnostic, and embraced by our students and faculty, I don’t see them as significant change agents; in any case, they’re design is based on the consumer market, not on the kinds of sharing and sampling that scholars need to be able to do.

The reason we need university presses is because they put their books through a far more rigorous peer review process than trade publishers and so have earned enormous prestige among scholars. They also publish research that may seem entirely without value to commercial publishers, to whom the only value is market value. For university presses, their work is a mission, not just a business, but it’s work that needs funding. We need to be more than customers; we need to be working together, making the best use of our pooled resources.

Jennifer Howard (she has been busy lately) also recently wrote a long piece about institutional repositories. It’s fascinating reading, and suggests that various models are meeting with some success, if libraries are willing to put a lot of time and energy into it. But while IRs are great for local materials, niche information (test reports on tractors – who knew how many people were eager to get their hands on that!) and gray literature, they are not the fix for the scholarly communication crisis, no matter how many institutions adopt open access mandates.

Rather than have university presses look for lessons from trade publishing while we try to coax faculty into using open access platforms, I’d like to see librarians sit down with university presses and talk about where our missions and our skills align, figure out how to fund publishing of quality scholarship, and embrace open access.

Is that so hard? Don’t answer that question.

But I thought I’d share something fun we’ve been doing this spring at my library – we started a journal club. A couple of times a month, we gather for breakfast in the college cafeteria on a Friday morning to discuss a common reading chosen by one of us. These include preprints of College and Research Libraries articles, articles from Communications in Information Literacy, or (most recently) the Taiga Provocative Statements coupled with the Darien Statements.

We’ve been joined by an intern, who brings a fresh perspective from a student who is about to go to library school but is still close to the undergraduate experience. (Maura, we’ll miss you when your internship is over!) We also have recently-hatched MLS who has a sharp mind and has been an excellent sabbatical replacement. (Anyone looking for a top-notch young librarian? Let me know.)

These have been such fun conversations, and they have been productive, too. Out of one of these informal get-togethers, we come up with a plan to hire and train some peer tutors to work at the reference desk between ten pm and midnight. Because we’ve had a lot of interest from students in doing internships, and we have a good example of peer tutoring in our Writing Center, we think we adapt some of our materials for interns into training, and provide some reference service at a time when the librarians are ready to call it a night but our students are finally getting a stretch of time when they can concentrate on their research.

Our journal club has proven to be a low-stakes, simple, and fun way to do a bit of professional development. Are there things you do at your library to foster good discussions among the librarians or share new ideas? Do tell.

(photo courtesy of arvindgrover)

Discovery services, as you can imagine, were a big topic at ALA Midwinter this year. EBSCO discussed their new product at both the LITA Electronic Resources Management Interest Group on Friday night and at their own Academic Lunch on Saturday; Cal State Web Services Librarian David Walker discussed them at the LITA Top Tech Trends forum on Sunday, and my own ALA committee, the RUSA MARS Local Systems & Services Committee, hosted a discussion forum about them on Sunday afternoon.

These services were born in response to librarians’ exasperation with isolated content and disappointment with federated search technology, as well as the continued realization that our students want the library to work like Google. But according to Senator Joe Lieberman, libraries are not alone: the Senate Committee on Homeland Security and Government Affairs not only recognizes a similar problem in intelligence databases, but is saying the same thing: Why doesn’t it work like Google?

Wednesday, January 20, 2010, on NPR’s Morning Edition, Lieberman told Renee Montagne what librarians have been telling each other about students for years. “I’m concerned that they [employees of the National Counterterrorism Center, in this case] don’t have the easy ability to draw linkages between the various databases.” He continued: “when we go into Google…Google immediately searches an enormous number of databases. It’s not clear to me that, at the National Counter Terrorism Center today, if you put in the name ‘Umar Farouk’ or even Nigerian it will automatically cross-search all the intelligence and law enforcement databases it has. I want to find out whether that exists, and I’m afraid that it doesn’t.”

Montagne couched this as a “computers” problem. “Is that computers?” she asked. “Is that, literally, you cannot go in there and put ‘Abdul Farouk, Nigerian, Yemen’ and…bring everything together?” Of course, saying it’s a problem of computers, or even one of search, simplifies it greatly. It’s a problem of not only bringing together, but accurately searching, de-duping and ranking results from databases designed on different platforms using different descriptive standards (from bare-bones MARC to full-text and everything in between) to fulfill very different information needs (think MEDLINE versus Web of Science versus MLA). It’s also a problem of getting information providers to agree to work together, especially when doing so potentially violates their core business, which is to provide value-added, premium information at a price. EBSCO’s Sam Brooks described the problem well when discussing vendor efforts to get indexing services to agree to let products like EBSCO Discovery Service and Summon (Serials Solutions) search their full files, not just the top layer of metadata. His description (which ended with, of course, his telling us how using EBSCO solves this problem) brought home the complexity of this endeavor and how far, with so many information providers working at cross purposes for profit, we probably still truly are from that one Google-like search box, despite all vendor claims.

So far, I haven’t heard anything negative from libraries about discovery services, and user testing at the University of Minnesota, the University of Chicago, and Dartmouth College (as described by our panelists, Cody Hanson, Frances McNamara and Barbara DeFelice) was, also, largely positive (while pointing towards directions for refinement). David Walker cautioned that the true measure of these products remains to be taken, but I am cautiously optimistic and very excited – as long as libraries and vendors (like our law enforcement agencies) can keep our shared goals in view.

In this respect the even more recent fallout between EBSCO and Gale over mainstream magazines is disheartening: with each telling such different stories I fear that we will never learn the whole truth. Will “one search box to rule them all” become “one vendor to rule them all”? It seems contrary to the spirit of cooperation that the library community has fostered since books were unchained centuries ago, but the true measure of this possibility, like that of discovery services, remains to be taken.

Amy Fry

–They spend a lot of time getting a grasp of context: the big picture, the words being used to describe what they’re investigating, what they’re supposed to produce as a finished product. (This, it seems to me, is particularly true of novice researchers – or any researcher who is investigating something they know little about.)

–They don’t report using searching Google as their first step in starting a research project; they consult course readings to get their grounding. (Google and Wikipedia come first for non-classroom research needs.)

–Most of them don’t seek help from librarians. They seek it from their professors. Only about 20% consult librarians, and that is most often for help with search terms and with finding full text sources already identified.

–They consistently use a limited number of sources and strategies based on what has worked before. In large part their problem isn’t finding sources, it’s limiting the number of sources available so they can complete a project.

–putting off research because of “library anxiety” seems to have been replaced by confident procrastination.

–In addition to Google, almost all students report using library databases. Databases are useful for locating credible sources, and credibility matters to them (though brevity is also appreciated); Google is helpful in understanding context and figuring out what those sources mean.

–Most students also consult the catalog as part of their research process.

–The traditional “research strategy” still found on some library websites – moving from general to specific by means of reference books, then books, then articles,then the web – bears no relationship to student research practices. (I can’t resist adding that I thought that “research strategy” was bogus twenty years ago.)

The authors raise some thought-provoking conclusions which mirror some of my concerns. Does the kind of work these students do using library resources contribute to life-long learning, or are they preforming tasks that will get them through college and then be abandoned? If they are taking their cues from faculty, shouldn’t we be sending cues to faculty? Maybe rather than providing library services most students find unimportant to them, we should spend more time working with their research mentors: their teachers.

More will be coming from this project – including an analysis of instructor assignments. Which reminds me – I’ll bet faculty would be interested in the findings of this survey. See if you can use a few nuggets from it to start a conversation.

photo courtesy of oceandesetoile and the Flickr Creative Commons pool.

The authors describe the way Oregon State University has adopted Boyer’s definition of scholarship – which embraces not just discovery of new knowledge, but application, teaching, and integration. After examining what librarians have been doing, they concluded the problem isn’t being productive, it’s explaining the “breadth and impact” of librarians’ scholarly work. This includes not only traditionally-published research, but additional modes of communicating ideas.

Blogs are vehicles to teach and communicate to both broad and specific audiences. Their format precludes them being taken seriously as scholarship in current tenure review processes, but their content often demonstrates engagement and suggests impact in ways rarely seen in the print library journal. This raises questions about the concept of format and vehicle. Expanding acceptance of new forms of communication along with reconsidering what constitutes scholarship will benefit librarianship as a whole. A first step is accepting open-access, peer reviewed journals as outlets of high impact and validity. The next step will be integrating non-traditional peer reviewed work such as blogs that have an active readership and generate comments and commentary.

The outsourcing of faculty evaluation by peers – relying on university presses and journal rankings to determine whether a colleague is worthy or not – has contributed to the problem libraries find themselves in: having to somehow fund access to a bloated body of research, much of which is only produced to gain job security. (Two years ago an MLA survey found a third of institutions required progress toward publishing a second book. This, when libraries’ budgets can’t keep up with bare necessities.)

Maybe in a backhanded way the work we do, documented in a way that people in other disciplines can understand, could provide a model for sanity.

CC-licensed image courtesy of Kristina B.

The new ECAR study on students and technology has just come out (thanks for the tip via Collib-L, Bill Drew!) and as usual, there are interesting findings. Nearly 90% of students come to college with a laptop now, and an even higher percentage of them use the library’s Website at least once a week. That’s a higher percentage than those who download music or videos (86%). Update: Bernie Sloan at Collib-L points out an interesting tidbit from the report: “…the percentage of students who reported using the library website daily has increased from 7.1% in 2006 to 16.9% in 2009.”

Texting and use of social networks are nearly ubiquitous, but instant messaging is dropping, which has interesting reference implications. The number who report they contribute content to the Internet through videos, wikis, or blogs is under half, and when asked about their use of these technologies for academic purposes, the percents drop into the single digits. Sorry, blogs and podcasts – they’re just not that into you. But they figure they know their way around searching. Eight out of ten say they’re proficient Internet searchers; about a third say they’re experts.

One finding that hasn’t changed much over the past few years – students don’t want a whole lot of technology in their courses. About 60% prefer a “moderate” amount of technology; only a small percentage wanted no technology, but they outnumbered the even smaller percentage that wanted their courses delivered entirely through technology.

In their responses to the final open-ended question of our survey, students wrote explicitly about a preference for “real books and people” and said “shiny new tech is still no substitute for well-trained, passionate instructors.” Of the many comments expressing this sentiment, perhaps this one sums it up best: “There is still a big disparity among academic staff when it comes to use of IT in class. Some professors are obsessed with their technology and some don’t like to use it at all. There needs to be a balance between human interaction and IT-based learning.

This is one of those studies that I read each year, a useful snapshot of emerging technologies and the role they play in the lives of our student. This one makes me think about ways to add texting to our reference repertoire, and reassures me that our Website is important to students. It reminds me that students thing they’re pretty good at searching and that I will need to persuade them they could be better. But it also reminds that these “digital natives” are not full assimilated into the Borg; they still prefer face-to-face learning with some, but not too much, technology involved.

CC-licensed photo courtesy of Frank Farm (frankfarm.org)

As if health care reform, the mess in Afghanistan, and H1N1 weren’t enough to ruin your day, having to cope with new editions of two major style manuals (neither of which actually keeps up with new information formats because they keep changing) is one of those “in the cosmic scale of thing it’s really incredibly trivial but ARRRGGGGHHHHH!!!!!” events.

MLA has finally decided it doesn’t matter what library you were in when retrieved an article or what “service” happens to be selling your library a particular database this contract year. Ten points for the rationality team. But leaving out URLs because anyone can do a search and find a website? One with no discernible author and several phrases at the top of the page, any of which might be the title – or the site name – or the sponsor? All of which are commonplace phrases that retrieve 5 million possible URLs? Okaaaay…. Deduct five points and go stand in the corner.

But it’s undeniably APA that has won the World Series of Stupid Style Manual Changes. DOIs? Not a bad idea. Citing the web site of the journal? Bad idea. Issuing seven pages of corrections and making excuses by saying they are “nonsignificant” errors?

Priceless.

This could be the tipping point. The time has come for faculty and librarians working with undergraduates to loosen up. In the cosmic scale of things, this manual labor really is trivial, but it carries a huge carbon footprint. For every hour spent writing a paper, at least an equivalent hour is spent trying to figure out whether you need a comma or a period here, which city out of the six on the title page is the one to use, what database you printed that article out of, or trying to identify the website of a journal for an article published in 1986 that you printed off JSTOR, given the publication changed titles three times and switched publishers five time since then. As this activity always happens in the wee hours of the morning on the day the paper is due, lights and computers have to be running, so we’re talking about a major energy drain. That’s not counting the environmental damage caused while creating and shipping the large amounts of carbonated caffienated beverages consumed in the process. Or the evening hours of the professors who are doing much the same while marking papers. Or the librarians trying to update their websites, guides, and class materials.

And what exactly are the learning outcomes of creating an error-free list of references? You learn that research is a pain in the butt. You learn that it’s really, really important to follow pointless rules with utter scrupulousness. You learn that, at the end of the day, you’ll get points off because you didn’t follow the pointless rules – unless, of course, you’re making a bundle off book sales, in which case “nonsignificant” is a valid defense.

I recommend that librarians stop teaching citation styles. (Why did we get stuck with that job, anyway?) That professors stop spending hours trying to correct student work using new style manuals as unfamiliar to them as to their students and go play with the baby or take a walk instead. That students are told “the reason we cite sources is because they serve as your expert witnesses; people need to know who these witnesses are, so provide their credentials, ones that readers can use to find the sources themselves, because they may want to learn more about the subject. That’s why we cite things. Oh, and to give credit where it’s due and avoid a plagiarism rap. That’s important, too.”

As for all those arcane rules? “Don’t worry about it. They’re nonsignificant. Just give me the information I need to find the source, and make it easy to read. That’s all I ask.”

We might not save the planet, but we would save a lot of pointless aggravation. Not to mention a few bucks buying updated style manuals.

CC-licensed photo courtesy of Jonno Witts; part of the Writer’s Block set.

Not long ago, a columnist at AAC&U’s Leap project blog, liberal.eduation nation, complained about the increasing crowd of literacies clamoring for our attention and suggesting that apart from the problem of all the newcomers – digital media literacy, spacial literacy, diaspora literacy – most new kinds of literacy have a short shelf life.

The “literacy” that seems most to vex educators and students alike is the one that takes aim at the moving target of technology. Indeed, the very terms used to name this elusive yet much-coveted literacy—computer literacy, information literacy, technological literacy, digital literacy, etc., etc.—are no more stable than the knowledge, skills, competencies they’re meant to describe.

Er, but . . . this seems to me to confuse information with information technology, learning how to think with learning how to use tools. Sadly, the only reference to an exploration of what we mean when we use the term “information literacy” is Stanley Wilder’s complaint that it is useless and misses the point. (ACRLog included the storm of debate it kicked up as a top story of 2005.) Given that most of us involved in teaching and learning in libraries thought Wilder didn’t actually grasp the concept he was criticizing, it’s kind of sad to see an organization that mostly wants to do what we are trying to do give him the last word.

The other big problem is linking information literacy with technology. Somehow once the techies took on the name “information technology” people forgot that information also resides in technologies that are centuries old. And they lose sight of the fact that it’s not about technology, it’s about what the student does with what he or she encounters. True, we get sucked into explaining how the library works, but the ultimate aim is to get students working in the library so that they can become part of the process of making knowledge, not just absorbing it as a finished product.

This morning I read Robert J. Nash’s essay in Inside Higher Ed, “Resist the Pedagogical Far Right” where he argues (as does AAC&U and as do most libraries serving a student population) that our focus should be on learning, not teaching, and on how the student can learn to as her own questions and solve problems rather than be exposed to a body of core knowledge in the hope some of it sticks. I’m not entirely sure why he characterizes this as a “far right” pedagogy – unless he’s thinking of ACTA, which, among other things, wants to return to basics and defend students from David Horowitz’s dangerous academics.He also seems to be taking a dig at those who feel youth have had their brains stolen by Web 2.0 and turned into ignorant and shallow-minded zombies. Or perhaps he’s thinking of the funding cuts and business strategies that have turned the professoriate into work-for-hire temps paid by the course. Whatever he means, he doesn’t really make a strong case for why this has a left-right dimension. What he says afterward, though, is a good defense of liberal (as in “liberating”) learning that focuses on the student.

The key is to remember that the most important part of the word evaluation is value. The best way to evaluate the outcomes of meaning-making learning is to ask students themselves what the value of their experience has been. According to [Ken] Bain’s research, the best evaluation stresses learning rather than performance. Performance means living up to others’ expectations and requirements. Learning means that students take full responsibility for their own intellectual, emotional, kinesthetic, and personal development. Performance is mainly about acquisition, storing information, and taking tests. Learning is developmental and an end in itself. . . .

So much of what I’ve learned about teaching in the academy for over four decades can be summarized in this way: often when I teach less, I find that I actually teach more. I call this a “pedagogy of ironic minimalism.” Whenever I take the time to call forth what it is my students actually know, and whenever I intentionally minimize the “endless breadth and depth” of my own “vast wisdom and knowledge,” then my students learn the most. This, dear readers, is why I keep coming back to the classroom — for lo these many years.

I think that’s actually our best argument for academic libraries and for giving students a chance to learn in them. Not learn about them, not learn how to manipulate the tools, but to be actively seeking, sorting, sifting, and making meaning as a central part of their education. If we want to define information literacy, we need to make it clear that practicing it – exploring ideas independently – is a big part of the high-impact practices that we know make learning stick.

And in the meantime, maybe we can also reflect on our own teaching practices. Can we teach less to make more room for learning? Do we do some of the things that Stanley Wilder believes is information literacy (information-seeking training done exclusively by librarians)? Can we put the focus on not just learning how the library works but learning where knowledge comes form and how it’s made? Can we work with faculty to make this happen more often?

Maybe if we can do that, Stanley Wilder will figure out what we mean when we talk about information literacy as a critical habit of mind and the AAC&U will realize it’s not about technology that will change next week.