Demonstrating Value Through Assessment and Surveys

 Posted by at 2:24 pm  Communicating Value, General  No Responses »
May 122013
 

At last month’s ACRL conference, there were more than 50 sessions that focused on assessment and/or the value of academic libraries.  While it was not possible to attend all the presentations, there were a number of poster sessions that provided examples of how libraries are gathering information to present on their value.  I focused on several of the poster sessions that had been publicized as focusing on value in the ACRL conference program.

One poster, “Demonstrating the Value of Academic Libraries with the MISO Survey” presented information on data collected from over 80,000 respondents between 2005 and 2012.  The survey has just completed its eighth year.  In terms of full disclosure, our institution participated in this survey for the first time this spring.   The presenters for this poster session were from a variety of  institutions including Bryn Mawr, Earlham College, Lafayette College, Spring Hll College, University of Richmond, and Brandeis University. MISO stands for “Measuring Information Services Outcomes” and the web-based survey instrument includes questions that measure use for both library and IT services.  The survey instrument collects feedback from faculty, staff, and students on services to measure the frequency of use, what services are considered most important, as well as measuring satisfaction by the respondents.   By evaluating those services that are frequently used as well as those that are not, as mentioned in their poster, the survey can be used both for improving services and for strategic planning.  With high response rates from faculty and students, libraries can assess those services that are both high in use and high in satisfaction as well as those services that are less frequently used and/or have low satisfaction.  For libraries looking to find information on what services are of the most value to faculty, it will provide a measurement of those services that faculty value highly and also use frequently.  With the data, libraries should be able to craft messages on high use/high value services in terms of demonstrating their overall contribution to the institution as well as the value of their support for faculty.  Two of the questions addressed by the survey are:

  • What services and resources are important to our constituents, and how successfully do our organizations deliver them?

  • How effectively do we communicate with our campus constituencies about our services and resources?

As a means of making improvements, the results should be able to provide data to show that changes may need to be made if  there are services that the librarians value highly, but faculty or students aren’t using.  This is where improvements in communication could provide an increase in underutilized services.

An additional feature of the survey is that MISO allows participating institutions to collect data on their campus and compare their results with the peer institutions who participate in the annual survey.  It is not the same group of institutions every year.  However, as a tool it not only measures satisfaction, but also can help identify trends.  You can see a copy of the poster presented at ACRL at

http://www.misosurvey.org/wp-content/uploads/2013/04/MISO_ACRL_Poster_FINAL.pdf

 

More details about the MISO survey will be found at the general information page for ACRL -

http://www.misosurvey.org/acrl/

 

There were a number of excellent posters and sessions on work being done by libraries to both use the Value report and to collect data to help demonstrate value.  Other posters will be featured in future blog postings.

 

Teams Selected for ACRL Assessment in Action Learning Community

 Posted by at 11:00 am  Uncategorized  1 Response »
Apr 092013
 

Assessment in Action Program LogoACRL has selected 75 institutional teams from a pool of 98 applicants to participate in the first year of the program Assessment in Action: Academic Libraries and Student Success (AiA). The program is made possible by the Institute of Museum and Library Services and carried out in partnership with the Association for Institutional Research and the Association of Public and Land-grant Universities. The teams, representing all types of institutions, come from 29 states and 3 Canadian provinces. For a list of currently confirmed institutions, see the AiA program webpage.

In their applications each institution identified a team, consisting of a librarian and at least two additional team members as determined by the campus (e.g., faculty member, student affairs representative, institutional researchers or academic administrator). They also identified goals for their action learning projects.

“The top applications were distinguished by the team composition, their readiness and the quality of their project goals. We also looked for strong institutional support to help the teams see their projects through to completion.” said Terri Fishel, vice chair of ACRL’s Value of Academic Libraries Committee and library director at Macalester College. “The application reviewers sought action learning projects with the greatest potential to contribute to the greater library and higher education community.”

The proposed topics of selected institutions include:

  • Do students who attend information or media literacy sessions attain higher grades than students who did not?
  • How does students’ work with special collections materials affect their ability to think critically and develop intellectual curiosity?
  • Do re-admitted students (who have appealed dismissal) improve their academic performance and persist at a higher rate due to mandatory meetings with a librarian for research assistance?
  • Does our new library/learning resource center facility have an impact on the student community, contributing to student enrollment and excitement about completing skills sessions and library orientations?
  • Do library contributions to a program for at-risk students (empowering them by connecting more deeply to local community issues through faculty partnerships and learning projects) enhance research, critical thinking, problem solving, and analytical skills? Are library efforts to support this program helping to increase student success and retention?
  • Are library interventions to increase students’ media fluency skills effective at improving their digital storytelling abilities?
  • In what ways does library participation in a targeted sophomore year program lead to higher graduation and retention rates, improved student engagement and satisfaction, as well as post-graduation success?
  • What is the impact of embedding librarians in our student scholar program?

To ensure project results are disseminated to the broader community, each institutional team will submit a final report and each librarian team leader will prepare and deliver a poster at the 2014 ALA Annual Conference. The AiA program, part of ACRL’s Value of Academic Libraries initiative, employs a blended learning environment and a peer-to-peer network over the course of the 14-month long program, which runs from April 2013-June 2014. The librarians will participate as cohort members in a one-year professional development program that includes team-based activities carried out on their campuses. An important component of the AiA program is establishing a learning community where librarian team leaders have the freedom to connect, risk and learn together.

“I am thrilled to be working with such a diverse group of institutions pursuing these very interesting project ideas,” said Lisa Hinchliffe, co-chair of ACRL’s Value of Academic Libraries Committee, a lead facilitator in the AiA program and professor at the University of Illinois at Urbana-Champaign. “The program facilitators are strongly committed to establishing an environment which supports collaborative learning and shared competence. We can’t wait to get started!”

Learn more about the AiA program at the ACRL 2013 conference during the session Update on Value of Academic Libraries Initiative from 1:30 – 2:30 p.m. on Friday, April 12.

AiA is a three year program. ACRL will be selecting 100 additional institutions to participate in the 2014-2015 class. Stay tuned for an announcement in January 2014 with more details on how to apply for the next round.

Library sessions improve student scores

 Posted by at 9:36 am  Communicating Value, Library Impact on Students  No Responses »
Apr 032013
 

Guest post by Debbie Krahmer, Learning Commons Librarian, Colgate University

[A version of this post was originally printed in the Fall 2012 Colgate University Libraries newsletter.]

 

The Core Curriculum is a key part of Colgate University’s liberal arts education. Only one category, Core Communities & Identities, explicitly states information literacy in their top 3 learning goals. Every Core CI class involves a research project of some kind, in the form of posters, videos, or research papers. The variety of research projects has been an obstacle to designing a suitable assessment instrument. How can you assess the impact of information literacy instruction across a diverse set of projects, geographical areas, and disciplines?

Grounded in ACRL’s Information Literacy guidelines, we based our assessment on “Project Information Literacy,” the national survey conducted by Alison Head and Michael Eisenberg. I worked with three teaching faculty from Core CI to survey over 500 students to determine how they were using and evaluating resources, and if attending a library session had any influence on the outcomes at the end of the semester.

The survey was administered at the beginning and end of the Fall 2011 and Spring 2012 semesters, and is being repeated for the 2012-2013 academic year. It consisted of 7-8 select PIL 2010 questions, administered through Google Forms. Using Google Forms helped to cut down on the time it took to collect the data, and the shortness of the survey allowed students to finish it in about 15 minutes. The majority of the 574 respondents were first year or sophomore-level students.

The broad reach of the PIL survey made it the perfect model for developing a simple assessment tool that could be used for any course. Colgate was one of the participants in the 2010 survey, so we had not only national data but also Colgate-specific benchmarks to measure against.

Perhaps the most exciting result of the survey was that attending a library information literacy session showed a far more positive change in using and evaluating research materials from the start of the semester. While all students tended to move from non-scholarly to scholarly sources over the semester, the students who had a library session reported using scholarly materials more than students who didn’t have a session. These students consulted online journal databases much more often, and generic Internet search engines much less often. Nearly 50% of the students who had a library session reported that they consulted a librarian in the course of the research project, while only 13% of the students without a library session reported consulting librarians.

Overall, the survey supported the faculty’s perception that Core CI has a positive effect on students’ information literacy skill development. It also serves to support the need for hands-on, face-to-face interactions within the classroom between students and librarians. The simplicity of the survey doesn’t allow us to determine why the library sessions had such a positive influence, but it does give us a very simple way to show that there is value in inviting librarians into the classroom.

We’ve used the results, as well as the growing research around the PIL study, to determine areas where instruction should be improved. Perhaps more importantly than that, the conversation around information literacy standards and expectations has improved since the survey. There are more frank, open discussions of what faculty and librarians expect from students, a key component for continuing to improve our support of student learning.

You can view a poster demonstrating the results of the 2011-2012 Core CI Information Literacy Assessment at the Colgate University Digital Commons.

_________________________________________________

If you have similar studies that you would like to see highlighted in the Value blog, please submit using this form.

 Older Entries
Site Admin

© 2010-2012 Association of College & Research Libraries, a division of the American Library Association

 Suffusion theme by Sayontan Sinha