Terri Fishel

Demonstrating Value Through Assessment and Surveys

 Posted by at 2:24 pm  Communicating Value, General  No Responses »
May 122013
 

At last month’s ACRL conference, there were more than 50 sessions that focused on assessment and/or the value of academic libraries.  While it was not possible to attend all the presentations, there were a number of poster sessions that provided examples of how libraries are gathering information to present on their value.  I focused on several of the poster sessions that had been publicized as focusing on value in the ACRL conference program.

One poster, “Demonstrating the Value of Academic Libraries with the MISO Survey” presented information on data collected from over 80,000 respondents between 2005 and 2012.  The survey has just completed its eighth year.  In terms of full disclosure, our institution participated in this survey for the first time this spring.   The presenters for this poster session were from a variety of  institutions including Bryn Mawr, Earlham College, Lafayette College, Spring Hll College, University of Richmond, and Brandeis University. MISO stands for “Measuring Information Services Outcomes” and the web-based survey instrument includes questions that measure use for both library and IT services.  The survey instrument collects feedback from faculty, staff, and students on services to measure the frequency of use, what services are considered most important, as well as measuring satisfaction by the respondents.   By evaluating those services that are frequently used as well as those that are not, as mentioned in their poster, the survey can be used both for improving services and for strategic planning.  With high response rates from faculty and students, libraries can assess those services that are both high in use and high in satisfaction as well as those services that are less frequently used and/or have low satisfaction.  For libraries looking to find information on what services are of the most value to faculty, it will provide a measurement of those services that faculty value highly and also use frequently.  With the data, libraries should be able to craft messages on high use/high value services in terms of demonstrating their overall contribution to the institution as well as the value of their support for faculty.  Two of the questions addressed by the survey are:

  • What services and resources are important to our constituents, and how successfully do our organizations deliver them?

  • How effectively do we communicate with our campus constituencies about our services and resources?

As a means of making improvements, the results should be able to provide data to show that changes may need to be made if  there are services that the librarians value highly, but faculty or students aren’t using.  This is where improvements in communication could provide an increase in underutilized services.

An additional feature of the survey is that MISO allows participating institutions to collect data on their campus and compare their results with the peer institutions who participate in the annual survey.  It is not the same group of institutions every year.  However, as a tool it not only measures satisfaction, but also can help identify trends.  You can see a copy of the poster presented at ACRL at

http://www.misosurvey.org/wp-content/uploads/2013/04/MISO_ACRL_Poster_FINAL.pdf

 

More details about the MISO survey will be found at the general information page for ACRL -

http://www.misosurvey.org/acrl/

 

There were a number of excellent posters and sessions on work being done by libraries to both use the Value report and to collect data to help demonstrate value.  Other posters will be featured in future blog postings.

 

Library sessions improve student scores

 Posted by at 9:36 am  Communicating Value, Library Impact on Students  No Responses »
Apr 032013
 

Guest post by Debbie Krahmer, Learning Commons Librarian, Colgate University

[A version of this post was originally printed in the Fall 2012 Colgate University Libraries newsletter.]

 

The Core Curriculum is a key part of Colgate University’s liberal arts education. Only one category, Core Communities & Identities, explicitly states information literacy in their top 3 learning goals. Every Core CI class involves a research project of some kind, in the form of posters, videos, or research papers. The variety of research projects has been an obstacle to designing a suitable assessment instrument. How can you assess the impact of information literacy instruction across a diverse set of projects, geographical areas, and disciplines?

Grounded in ACRL’s Information Literacy guidelines, we based our assessment on “Project Information Literacy,” the national survey conducted by Alison Head and Michael Eisenberg. I worked with three teaching faculty from Core CI to survey over 500 students to determine how they were using and evaluating resources, and if attending a library session had any influence on the outcomes at the end of the semester.

The survey was administered at the beginning and end of the Fall 2011 and Spring 2012 semesters, and is being repeated for the 2012-2013 academic year. It consisted of 7-8 select PIL 2010 questions, administered through Google Forms. Using Google Forms helped to cut down on the time it took to collect the data, and the shortness of the survey allowed students to finish it in about 15 minutes. The majority of the 574 respondents were first year or sophomore-level students.

The broad reach of the PIL survey made it the perfect model for developing a simple assessment tool that could be used for any course. Colgate was one of the participants in the 2010 survey, so we had not only national data but also Colgate-specific benchmarks to measure against.

Perhaps the most exciting result of the survey was that attending a library information literacy session showed a far more positive change in using and evaluating research materials from the start of the semester. While all students tended to move from non-scholarly to scholarly sources over the semester, the students who had a library session reported using scholarly materials more than students who didn’t have a session. These students consulted online journal databases much more often, and generic Internet search engines much less often. Nearly 50% of the students who had a library session reported that they consulted a librarian in the course of the research project, while only 13% of the students without a library session reported consulting librarians.

Overall, the survey supported the faculty’s perception that Core CI has a positive effect on students’ information literacy skill development. It also serves to support the need for hands-on, face-to-face interactions within the classroom between students and librarians. The simplicity of the survey doesn’t allow us to determine why the library sessions had such a positive influence, but it does give us a very simple way to show that there is value in inviting librarians into the classroom.

We’ve used the results, as well as the growing research around the PIL study, to determine areas where instruction should be improved. Perhaps more importantly than that, the conversation around information literacy standards and expectations has improved since the survey. There are more frank, open discussions of what faculty and librarians expect from students, a key component for continuing to improve our support of student learning.

You can view a poster demonstrating the results of the 2011-2012 Core CI Information Literacy Assessment at the Colgate University Digital Commons.

_________________________________________________

If you have similar studies that you would like to see highlighted in the Value blog, please submit using this form.

Assessment 101: Getting Started

 Posted by at 12:35 pm  General  No Responses »
Mar 252013
 

In The Value of Academic Libraries, page 93, “What to do next” includes, “Get Started.”

Whether you start with a small project or use existing data, the most important step, is to start.  If you haven’t started already, what are the barriers or challenges facing you?  Are you uncertain as to what the exact challenges are that are keeping you from embarking on a project?  The following are a few possible resources that may be of help to get you started.

A freely accessible resource, “Assessment 101 for Librarians: A Guidebook”, by Tina Chrzastowski is available from IDEALS (Illinois Digital Environment for Access to Learning and Scholarship) [http://hdl.handle.net/2142/28582] This guide is an excellent starting point for anyone who just wants to get a brief introduction to the topic.  Librarians often cite a lack of skills as being one of the major obstacles for starting an assessment program, but Ms. Chrzastowski states:

“Experience is helpful, but not absolutely required.  In time, today’s library assessment beginners become tomorrow’s experts.  This is a skill that can be learned, and one that does improve with experience, learning from mistakes and taking risks.  Librarians and staff willing to shoulder the responsibilities of assessment need to be allowed some room to learn, grow, make mistakes and forge ahead.  

The important piece is ‘this is a skill that can be learned.’  In addition to the guide, Ms. Chrzastowski provides a list of additional resources to help one start the process of learning.

In Megan Oakleaf’s workbook, Academic Library Value:  Impact Starter Kit, Activity #25 is focused on Assessment Fears, Challenges, and Barriers.  Megan has made a copy of this activity freely available at her website.  http://meganoakleaf.info/ALVactivity25.pdf  Perhaps completing the exercise can assist your organization in identifying not only what is preventing you from starting, but perhaps help you identify strategies for overcoming those barriers.  Is it a question of time, or resources?  Is it a lack of skills or lack of interest?  Maybe you could challenge your assumptions and identify a strategy that will get you started on assessment in your own institution.

Lastly, the most recent issue of Portal: Libraries and the Academy (January 2013) includes, “A Review of the Literature on Assessment in Academic and Research Libraries, 2005 to August 2011” by Jon R. Hufford.   As pointed out by the author, “the use of the words assessment and evaluation among librarians is sometimes ambiguous, or has changed over the years.”  He goes to explain his selection criteria for inclusion in his review of the literature.  As a survey of the literature over six years, he has provided a distillation of books, articles, and conference proceedings that all provide additional resources to help one develop a better understanding of assessment in libraries.

If one keeps in mind that a primary emphasis of assessment is to analyze for the purpose of providing improvements when needed, the most important part of assessment is to get started.

What’s holding you back from getting started with assessment?

 Older Entries
Site Admin

© 2010-2012 Association of College & Research Libraries, a division of the American Library Association

 Suffusion theme by Sayontan Sinha