Monday, 31 July 2017 – Parallel 2: Institutional Comparisons
Author(s)
Ms. Siobhan Burke (Jisc),
Mr. Lee Baylis (Jisc)
Title
Insights from Jisc & HESA Analytics Lab: An agile, cross-institutional approach
Abstract
Purpose The purpose of this session is to provide an overview of the Jisc and HESA Library Data Labs project and its outputs. Library Data Labs ran as a special instance of the three-month Jisc and HESA Analytics Labs phases, which bring together cross-institutional teams to develop proof of concept data visualised dashboards. Analytics Labs normally draws from University strategic planning departments, but Library Data Labs set out to to discover whether the process could be successfully replicated for a library audience, using library data to make library-specific insights, and develop dashboards suitable for delivery via HESA’s Heidi-Plus service. Design, methodology or approach The session will take the form of a presentation explaining the Library Data Labs project and its outputs. Findings The session will present the outputs from the five cross-institutional library teams, and one Jisc team. An overview will be given of the ‘User Story’ approach whereby teams defined the questions to be addressed by the dashboards, and the user stories which teams defined will be described. A selection of dashboards will be presented and the next steps for the project explained. Conclusions The continued success of the approach developed through the Labs projects is borne out by the feedback received from participants, the continued interest from an increasing range of applicants for future development cycles, and the growing interest in gaining access to the prototype dashboards already produced. Originality and value The session offers insight into the Library Data Labs project including a summary of all the user stories developed by participants. The value of the session will be in learning about the latest library analytics developments happening in the UK, as well as learning how institutions will be able to access some of the dashboard outputs of the process.
Biography
Siobhan Burke: I am currently responsible for leading the transformation of Jisc’s Library Support Services. The Analytics Lab was an opportunity to expolre data visualisation in a library context. I am a qualified librarian, having previously worked as Electronic Resources Co-ordinator at the University of Manchester Library.
Lee Baylis: I am currently working on Jisc’s Learning Analytics project, helping universities to leverage their student data to make more informed decisions. Previous roles included applications developer and systems administrator and my background is in physics and data analysis.
Author(s)
Dr. Rachel Fleming-May (School of Information Sciences, The University of Tennessee),
Ms. Regina Mays (University of Tennessee Libraries),
Ms. Amy Forrester (University of Tennessee Center for Information and Communication Studies),
Dr. Carol Tenopir (School of Information Sciences, The University of Tennessee),
Dr. Dania Bilal (School of Information Sciences, The University of Tennessee),
Ms. Teresa Walker (University of Tennessee Libraries),
Dr. Suzie Allard (College of Communication and Information/School of Information Sciences)
Title
Experience assessment: Designing an innovative curriculum for assessment and UX professionals
Abstract
Purpose While assessment and user experience expertise have been identified as areas of growing focus and need in all types of libraries, research has demonstrated that formal Library and Information Science (LIS) education has not yet developed a uniform infrastructure to prepare students for employment in these roles (Applegate, 2016; Askew & Theodore-Shusta, 2013; Nitecki et al., 2015; Oakleaf, 2013; Passonneau & Erickson, 2014). This paper presentation will report on development, structure, and execution of a new model for preparing information professionals with assessment and user experience expertise. Funded by the United States Institute for Museum and Library Services (IMLS) 21st Century Librarians program, this new model combines workplace experiences, focused and standard course content, and intensive mentoring by academics and practitioners. Design, methodology or approach In the summer of 2016, the user experience and assessment (UX-A) team at the University of Tennessee School of Information Sciences solicited applications for a cohort of twelve students who would earn the American Library Association-accredited Master’s degree while completing a two-year co-curricular program to strengthen their skills in assessment- and usability-related areas. Twelve applicants were selected and began the program in August, 2016. They are currently enrolled in their third semester of coursework and are participating in a structured series of workshops, research projects, and intensive mentoring through on-site placement in several academic libraries and specialized information centers. This presentation will feature a representative from each of the stakeholder groups involved in the project:
- One of the Co-Principal Investigators, who is also an instructor in the program in question;
- A member of the project’s senior staff, also an assessment librarian in the home institution’s library;
- One of the program’s on-site placement mentors;
- The project’s Program Manager; and
- At least one of the participating students.
Findings Project success is tracked through a system of rigorously-designed assessment tools created to measure students’ progress as well as the experience of the students, on-site mentors, and other stakeholders. The paper presenters will share preliminary findings from these tools. Research or practical limitations or implications Although this project is supported by grant funding, which provides tuition and stipend for the twelve students, the presenters will share insights and suggestions for implementing aspects of the program at low- or no-cost and for sustainability of the program. Conclusions; Originality and value At a time in which the library practitioner and LIS educator communities are contemplating how best to prepare professionals with much-needed expertise in assessment and user experience, this project represents a new and innovative approach to educating future information professionals in this area. Attendees will learn about the structure and design of the program, assessment tools used, as well as how to address practical concerns such as participant recruitment, scheduling, and other aspects of implementation. References Applegate, R. (2016). Educating assessors: Preparing librarians with micro and macro skills. Evidence Based Library and Information Practice, 11(2), 74-86. Askew, C., & Theodore-Shusta, E. (2013). How Do Librarians Learn Assessment? Library Leadership & Management, 28(1), 1-9. Retrieved from https://journals.tdl.org/llm/index.php/llm/article/view/7026. Nitecki, D. A., Wiggins, J., and Turner, N. B. (2015). Assessment is not enough for libraries to be valued. Performance Measurement and Metrics, 16(3), 197–210. Retrieved from https://doi.org/10.1108/PMM-10-2015-0032.
Oakleaf, M. (2013). Building the assessment librarian guildhall: criteria and skills for quality assessment librarian. Journal of Academic Librarianship 39(2), 126-128. Passonneau, S. and Erickson, S. (2014). Core competencies for assessment in libraries: A review and analysis of job postings. Library Leadership & Management, 28(4), 1-19.
Biography
Rachel Fleming-May is an Associate Professor in the School of Information Sciences at the University of Tennessee. She is Co-Principal Investigator for “Experience Assessment,” an Institute of Museum and Library Services 21st Century Librarian-funded program to prepare Master’s students for careers in assessment and user experience.
Author(s)
Prof. Simone Fühles-Ubach (Cologne University of Applied Sciences)
Title
What is your contribution to the funder’s strategy? Relationship and impact of research libraries
Abstract
Purpose The paper describes the different positions and roles that research libraries may have in their funding organizations: center or periphery, making a difference respecting the overall strategy or having just a strategy of their own; having strong communication links to the top management as well or just to their users. Design, methodology or approach Since 2008 the Joint Science Conference (Gemeinsame Wissenschaftskonferenz – GWK) in Germany deals with all questions of research funding, science and research policy strategies and the science system which jointly affect the Federal Government and the Länder. For the big four German national research societies (Max-Planck, Helmhold, Fraunhofer, Leibnitz) the GWK formulated a set a performance indicators that are used to compare the different societies in a monitoring report published every year. Interestingly there are a several aspects affecting the libraries and their services and products, but the libraries (approximately 200) are not mentioned in any way. While customer orientation has become increasingly important in research libraries for many years, the contribution to the overall corporate strategy has played a rather subordinate role. Libraries started to define their own library strategy very much orientated on user needs and changing library services and products. What attracted less attention was the contribution of the library to achieve the overall corporate objectives of the funding organizations, i.e., the contribution to the research society’s strategy. Findings An exemplary matrix is developed, that can be adapted by all research libraries, to show the extent to which the research library is able to support and influence the level of different performance indicators and thereby to contribute to the overall goal of the organization. The matrix is developed using the example from the strategic objectives and the respecting performance indicators of the sponsoring institution. A graduated traffic light system is used to make visible which contributions in which areas can be expected from the research library. A visible and clear demonstration of the library performance characterizes the perception by the top management. In spite of the great variety in research libraries, it is possible to find methods for demonstrating the influence and the contribution of research libraries on their funding organizations. Research limitations Research libraries are usually quite different from each other and there are not many tools to compare their contributions and to show their performance to the management level. This is a method of showing influence and impact in the German research context, which may not be transmitted to other countries with other research structures. Conclusions The value of research libraries has usually been seen as supporting the research process for the users in the best possible way. As more competition arises even between different research institutions or societies, the role of the library respecting the contribution to the stakeholder’s strategy opens up another perspective. Originality and value The paper identifies the importance of playing an essential role in the fulfillment of the overall strategic goals of the funding organization, especially for research libraries, as there are usually special metrics to compare the big scientific research organizations in Germany.
Biography
Simone Fühles-Ubach studied library and information science. For almost 10 years, she worked in the administration of the German Bundestag. She did her PhD at the Humboldt University in Berlin. Since 1998, she has been working as a professor for library management. Her research interests: user research and performance measurement.
Author(s)
Dr. Darlene Parrish (Florida Atlantic University),
Ms. Simone Clunie (Florida Atlantic University)
Title
How assessment websites of academic libraries convey information and show value
Abstract
Purpose The purpose of this lightning paper is to share the results of an analysis of assessment websites from a sample of academic libraries from four countries, including the United Kingdom, Canada, Australia and the United States, to determine how assessment information is conveyed through websites and how this information may help to demonstrate value to constituents. In a recent article about “best practices” for assessment committees (Brannen et al., 2016), it was recommended that more investigation be done regarding the impact and use of assessment websites. Recently, a working group was formed at the Florida Atlantic University Libraries to try to identify “best practices” as found on assessment websites of academic libraries. This working group was also charged with recommending assessment websites that report in a user friendly way the results of assessment studies and other related information and to look for common practices or any trends and innovations. As former members of the working group, the endeavour peaked our interest from some of the things we found and discussions we had, and this has inspired us to further investigate a wider sample base and types of academic libraries. Methodology A sample of assessment websites of academic libraries from the four countries to be evaluated will be selected from the Association of Research Libraries list, which includes academic libraries in the United States and Canada. In order to have a more representative sample of the academic libraries from the United States, given the USA’s complex historical germination of tertiary institutions, a sample of academic libraries will be reviewed from the lists of Historically Black Colleges & Universities Library Alliance along with the American Indian Higher Education Consortium. In the case of Canada, the UK and Australia sample libraries will be taken from the lists of the Canadian Association of Research Libraries (CARL), the Research Libraries of the United Kingdom (RLUK), and the Council of Australian University Libraries (CAUL), respectively. Any commonalities in the type of information conveyed through assessment websites will be identified along with trends and innovations contrasting the current assessment culture.
Findings We do not have any findings yet, as the analysis has not been conducted. When the analysis is completed, conclusions will be drawn regarding the type of information that academic library assessment websites report in common, if they have instituted new trends and innovations, and how this information may help to convey value to constituents. Research or practical limitations or implications Only a sample of academic libraries from each of the four countries, including United Kingdom, Canada, Australia and the United States, will be analysed for this review of websites. Conclusions As libraries are required to become more accountable and demonstrate that they are meeting performance metrics for funding and accreditation, an assessment website can be a means for providing data for evidence based decision making and reporting other strategic information. An online presence can also be an important indicator of how a particular library treats assessment. In this brief presentation, we hope to share our observations of this review of the websites and provide suggestions in conveying assessment information using a website, to help create a picture that will demonstrate a library’s relevance and value. Originality and value
This informal review and presentation about assessment websites is an attempt to add to the ongoing broader discussion about “best practices” for academic library assessment. References Brannen, M., Cunningham, S., and Mays, R. (2016) Assessment committees: Good practices from ARL libraries, Performance Measurement and Metrics, 17(3), 224-240.
Biography
Darlene Ann Parrish is a University Librarian at the S.E. Wimberly Library, Florida Atlantic University, Boca Raton, Florida. She worked for many years as the Business Librarian and since 2009 has been working with assessment. Darlene received her Ph.D. in Library and Information Studies from Florida State University in 2001.
Simone Clunie is a cataloger with the S.E. Wimberly Library, Florida Atlantic University, Boca Raton, Florida. She is currently working on her Master’s of Science in Information at Florida State University.
Author(s)
Mr. Alan Oliver (Ex Libris (UK) Ltd.)
Title
Attaining key performance indicators through benchmark analytics
Abstract
Purpose
Libraries are increasingly asked by their institutions and governing agencies to demonstrate that they provide value while still optimizing their collections. Existing tools provide analytics that allow libraries to track KPIs such as collection usage, workflow processing times, number of patron requests filled, and even number of patron searches. Nevertheless, there is a growing need for industry-wide benchmarks that will allow libraries to compare themselves with others in their class. Such benchmarking already exists for various library associations, such as the analytics available through SCONUL in the UK, DBS in Germany, and ARL in the United States. But for libraries that require more granular or, on the contrary, broader analytics in order to encompass libraries in similar situations, other means of creating benchmark KPIs are necessary.
Design, methodology or approach In order to provide such a platform for benchmark analytics and KPIs, Ex Libris surveyed a wide range of libraries to determine:
- The key attributes of libraries relevant to benchmark analytics, such as size categories, staffing, type, and budget;
- Relevant measures to include regarding collections, expenditures, service activities, and usage.
Findings
In this presentation we will present the resulting benchmark KPIs available in Ex Libris Alma and how they can be used to demonstrate library value and efficiency.
Biography
Alan Oliver undertakes thought leadership and business development activities throughout the Europe, Middle East, and Africa regions.
Monday, 31 July 2017 – Parallel 2: Innovative Methods I
Author(s)
Dr. Frankie Wilson (University of Oxford)
Title
Following our students’ home: Using a cultural probe to investigate the experiences of new Doctoral students
Abstract
Purpose The purpose of this research was threefold. (1) To test whether the cultural probe methodology was appropriate, applicable and workable as a methodology for investigating the impact of information skills teaching. (2) To investigate the impact of information skills teaching on new doctoral students’ information seeking behaviour. (3) To seek insight into the wider picture of the research landscape that new doctoral students occupy, so that library services and interventions could be designed to fit into students’ workflow, rather than expecting them to fit into ‘the library way’. Design The design of the research was a longitudinal study over the first term (10 weeks) of the doctoral programme experience, using a cultural probe method. Developed in 1999 by three designers [1] a cultural probe is a collection of activities and exercises that are given to members of a community with the purpose of obtaining a rich understanding of their lives, including their motivations, thoughts and behaviours, alongside their cultural and social contexts. A key feature of the cultural probe kits is that they are designed to promote engagement and inspiration in participants in order to get them to consider their day-to-day behaviours in detail, and evaluate and reflect on their choices and actions. Cultural probes are an exploratory methodology, in the ethnography cannon, usually used without any specific research questions in mind. This research experimented with the methodology to determine if it could be used in a more targeted way. The researcher adapted the cultural probe methodology developed by Priestner & Marshall for the Cambridge University FutureLib Project [2]. The Cambridge study employed the methodology over an intensive two-week period, requiring 20 minutes per day of participants’ time, followed by a 90 minute interview. This study adapted the methodology to be less intensive, requiring 10 minutes of participants’ time three days a week, and follow up interviews only to clarify aspects of the participants responses post-analysis. Methodology Five students were recruited from the Education Department, as it was felt that such students would be more open to the non-traditional aspects of the methodology. In addition, there is a large cohort of doctoral students in this department, who all start at the same time (October), so it was felt it would be easy to recruit the required number. Each participant was provided with a study pack – a brightly coloured A5 lined notebook, acting as a diary; a bright red flash drive; a selection of creative materials; and an introduction sheet. The ‘diary’ contained 30 questions – three each week. Every Monday featured the same question: “How did you discover what you read today?” Every Wednesday there was a task. Half of the tasks were photo assignments such as “Take a photo of the best part of your day” “Take a photo of something surprising”. The other five tasks were based on ethnographic interviewing techniques advocated by proponents of UX in libraries [3], such as “Write a love letter to a library”. On Fridays there was a reflective question, such as “What motivates you to work?” “What do you want to have achieved by Christmas?”. Each ‘diary’ was handwritten by the researcher, and decorated with images (such as a pumpkin on 31st October) and stickers, and personalised, including a handwritten recipe for the researcher’s favourite pumpkin pie, and a Christmas card from the researcher to the participant. Findings Every participant completed the diary and tasks in full and in detail, and reported that they enjoyed taking part in the study. Participants felt that the process of undertaking the reflective (Friday) questions had helped them with their own research process. Apart from the somewhat cramp-inducing hand-writing of the diary questions, the methodology was low effort for the researcher, and the results were both exciting and simple to analyse using standard qualitative techniques. There was a clear evolution in the information-seeking behaviour of participants as they progressed throughout the term. However, there was no disruptive change from attending the Information Skills session. It is hypothesised that this is because both of the content of the session, and also the timing of it (week 2 of the study). Conclusions The cultural probe methodology can be applied to a less-intensive longitudinal study in order to effectively gain insights into the research landscape that new doctoral students occupy. These insights can then be used to design University (not just library) services and interventions to fit into students’ workflow and assist them through the adjustment to working on a doctoral programme. With one small alteration, namely explicitly requiring that the cultural probe participants attend the personalised information skills consultation provided for all doctoral students during the course of the study rather than assuming they would, the cultural probe methodology is appropriate, applicable and workable for investigating the impact of information skills teaching. Originality and value This research is the first time the cultural probe methodology has been applied both in a lower-intensity longitudinal way, and to try to illuminate a specific research question.
There are a number of documented methods for obtaining evidence for the impact of information skills teaching on the information seeking behaviour of students. However, these methods are either extremely time-intensive for the researcher (bibliography analysis), only give an isolated snapshot of library-focused behaviour (interviews), or provide results where causal relationships are unable to be drawn (GPA). This methodology has none of these disadvantages, with the possibility of providing a rich, context-situated picture of information-seeking behaviour before and after intervention. References Gaver, B., Dunne, T. and Pacenti, E. (1999) Cultural Probes, Interactions, 6(1), 21-29. Priestner, A. and Marshall, D. (2016) Snapshot: A Cultural Probe Study Exploring the Research and Information Behaviour of Postdocs and PhD Students at the University of Cambridge. Retrieved from https://futurelib.files.wordpress.com/2016/09/the-snapshot-project.pdf. Priestner, A. and Borg, M. (2016) User Experience in Libraries: Applying Ethnography and Human-Centred Design. Abingdon, UK: Routledge.
Biography
Frankie Wilson is the Head of Assessment & Secretariat at the Bodleian Libraries, University of Oxford. She received her Doctorate for the development of the Quality Maturity Model and her research interests include quality in libraries, performance measurement, organisational culture, and research methodologies.
Author(s)
Ms. Rachel Lewellen (Harvard University),
Ms. Suenita Berube (University of Massachusetts Amherst)
Title
University of Massachusetts Amherst Libraries portfolio project
Abstract
Purpose The Portfolio Project is a longitudinal case study that analyzes multiple interactions between the Libraries and undergraduates from the class of 2020 at the University of Massachusetts Amherst. Using a portfolio model, this project combines qualitative and quantitative data to produce a more nuanced and complex understanding of library impact by focusing on the breadth and depth of library contributions to the student experience. It spans interactions related to services, collections, resources, and facilities. This study seeks to understand and articulate library contributions to student learning and success. Design, methodology or approach This study assesses, communicates about, and reflects upon the UMass Amherst Libraries impact on the class of 2020. It follows the undergraduate class of 2021 over 5 years (September 2016 to May 2021). Student data provided by the registrar is combined with library data including collections, equipment, instruction, and other sources. Additional data may include the National Survey of Student Engagement (NSSE), LibQUAL+, a local senior year survey, local anthropological studies, interviews, artifact analysis, and data from partners sharing library spaces such as the Writing Center and Information Technology. A class profile including demographics and the identification of special populations was created (international students, transfer students, veteran status, etc.). Analysis seeks to identify sub-populations that are under-served so that programmatic changes can address gaps. For example, programs or student populations that are not receiving library instruction may receive additional outreach. In future years analysis may also include grade point average, retention, and time taken to graduate. A collaborative team advises the project, reviews data from a variety of sources to provide context, and supports the implementation of change. Findings The presentation will report on the first year analysis of the use of library collections, equipment, instruction and outreach. It examines the relationship between the class profile and the profile of users of library collections and services with implications for change. Questions for future analysis will be identified. Research or practical limitations or implications This research has a number of limitations. The definitions of student success and impact are varied. The relationship between student and library interactions do not imply causation. Data management challenges and the availability of robust data sources are additional limitations. Conclusions Conclusions will focus on year-one analysis, reflection, change, and the identification of additional research questions for a limited number of data sets. It will outline the next four years and identify potential portfolio components.
Originality and value This approach is unique in that few libraries have used a portfolio model to holistically represent their impact. This study aims to look broadly and deeply at the connection between the library and students. The reflection component of the portfolio promotes the iterative cycle of assessment as it leads directly to change. Opportunities to discover new paths of inquiry are encouraged through the reflective component. Unanticipated outcomes and questions are embraced.
Biography
Rachel Lewellen was the assessment librarian at the University of Massachusetts Amherst from 2005-July 2017. She conceptualized and launched the Portfolio Project in the fall of 2016. In July she started as Head of Assessment for Harvard Library.
Author(s)
Dr. Kris Markman (Harvard University)
Title
Insightful assessment: Screen-based eye tracking as user experience research
Abstract
Purpose and Design While a variety of quantitative and qualitative methods, such as surveys, interviews, and ethnographic observation, have been used to measure library performance for some time, new technologies have emerged that are expanding the range of data that can be collected about library users’ experiences. One such technology is eye tracking. Eye tracking devices use infrared sensors and high speed cameras to track and capture a users’ eye movements across a target object, for example a computer screen or mobile device. While earlier eye tracking technology was quite cumbersome and usually required the participant’s head to be restrained, recent developments have resulted in a range of robust yet lightweight eye tracking devices that can be used in a wider range of settings inside or outside of a lab. This has greatly expanded the range of possible research uses for eye tracking, including for measuring library performance. A recent literature review found that while the use of eye tracking methods in the broader field of library and information science has increased over the last ten years, it is still relatively rare, particularly in applied research within libraries (Lund, 2016). Screen Based Eye Tracking: One of the most common uses for eye tracking methods within library and information science is as part of usability testing of websites or other digital interfaces (Lund, 2016). For example, Kules and Capra (2012) were interested in whether training videos or contextual help links would be effective in helping students make better use of facets in a library catalog search. They combined eye tracking with a method called Retrospective Think-Aloud (RTA) to determine which parts of the search interface attracted users’ attention over the course of six different search tasks. RTA is similar to traditional think-aloud methods of usability testing, except that the participant’s narrative occurs during a replay of the eye tracking video. They found that while overall participants looked at the search results more than any other part of the interface, users who saw the training video did look at the facets section of the interface more than users who did not see the video. They also found that none of the participants noticed or used the contextual help link, which was confirmed through both the eye tracking results and an analysis of the server logs. Another library usability study used eye tracking as part of an examination of interlibrary loan options in a consortial OPAC (Jones, Pritting, & Morgan, 2014). In this case, eye tracking findings revealed how users read the results list, scanning the titles of records and generally ignoring thumbnails of book covers and subject headings. These are just two examples of the kinds of previously hidden data that eye tracking methods can uncover for libraries. In addition to studying search interfaces, screen based eye tracking can be used to study library tutorials, websites, and mobile apps and interfaces. This short paper will discuss how screen-based eye tracking technologies can be used to gain deeper insights about user experience and user behavior in an academic library. I will combine findings from the literature with examples drawn from multiple eye tracking studies conducted at a large academic research library in the United States. I will outline the benefits of eye tracking as an innovative method, with an emphasis on the practical uses for measuring different aspects of library performance. I will also discuss some of the challenges for implementing eye tracking in libraries. I will conclude with an overview of the major eye tracking vendors and discuss some of the options for libraries of different sizes. Conclusions Eye tracking is a new technology that allows for innovative data collection in multiple library settings. By combining eye tracking with other research methods, such as usability studies, log analysis, or interviews, libraries have the potential to dramatically increase their understanding of a range of user behaviors in their physical spaces and across their digital interfaces. Eye tracking technologies provide deeper insights into previously hidden user behavior, and can illuminate alignments or conflicts with expressed user preferences.
References Jones III, W. E., Pritting, S., and Morgan, B. (2014) Understanding availability: usability testing of a consortial interlibrary loan catalog, Journal of Web Librarianship, 8(1), 69-87. Retrieved from http://dx.doi.org/10.1080/19322909.2014.872967. Kules, B., and Capra, R. (2012) Influence of training and stage of search on gaze behavior in a library catalog faceted search interface, Journal of the American Society for Information Science and Technology, 63(1), 114-138. Lund, H. (2016) Eye tracking in library and information science: a literature review, Library Hi Tech, 34(4), 585-614. Retrieved from https://doi.org/10.1108/LHT-07-2016-0085.
Biography
Kris Markman is the Director of Digital Learning & User Experience (DLUX) for the Harvard Library, where she leads collaborative efforts to develop and support proficiency in innovative digital pedagogy and evidence-based decision-making across the Library. She holds a Ph.D. in Communication Studies from The University of Texas at Austin.
Author(s)
Mrs. Erika Rood (North-West University),
Ms. Marieta Buys (University of Pretoria)
Title
Keeping up quality Information Services during transformation
Abstract
Purpose To investigate future scenarios of assessment of Information Services staff performance in South African academic libraries in times of chaos.
- How chaos affected performance agreements and assessment in specific academic libraries?
- How to address change in service delivery performance assessment in SA academic libraries?
- Establish the role of middle management of academic libraries in maintaining quality information services.
Design, methodology or approach Exploratory study regarding available literature and a survey of some academic libraries in South Africa. Innovation or creativity follows times of chaos and we need to establish how South African academic libraries can contribute to service delivery in times of chaos. Quality service need to be maintained and middle management need to know how to assess performance during turmoil and drastic change. Perceptions about current trends and actions will be identified with a short survey. Findings We anticipate findings from our literature review and survey. As the current status of drastic change is dynamic and ongoing findings are not yet conclusive.
Conclusions Certain recommendations and practical implications may emerge into future scenarios:
- Nothing changes – keep on doing what we do.
- Small adaptions – “We’re just trying to fit the old things into the new form, instead of asking what is the new form going to do to all the assumptions we had before.” (Hichert, 2016).
- Radical change – The extent of change needs to be explored?
Originality and value We took our cue from the 14th Annual Library Symposium, November 2016 (Stellenbosch, South Africa), where the theme was “Shaping the academic library of the future: adapt, empower, partner, engage”. We realise that #feesmustfall is reshaping the higher education landscape in South Africa and we need to adapt our processes, empower our students and staff, partner with all stakeholders and engage in discourse on issues that determine the fate of academic libraries. The ensuing chaos/uncertainty is also impacting on the service delivery of academic libraries and in particular service delivery staff performance assessment. Therefore new and innovative ways to assess staff performance need to be identified and be part of the conversation in this time of chaos. References Hichert, T. (2016). How to be ‘future fit’ and manage complexity and uncertainty. Paper presented at the 14th Annual Library Symposium. Stellenbosch, South Africa. Neerputh, S. (2004). Developing guidelines for performance appraisal of subject librarians in KwaZulu-Natal academic libraries. Pietermarizburg: University of kwaZulu-Natal. (Thesis M.I.S.). Riaz-Mohamed, A. (2015). Uncomfortable about current protests? Embrace your discomfort and be part of social change. Thought Leader. http://thoughtleader.co.za/psyssa/2015/10/22/uncomfortable- about-current-protests-embrace-your-discomfort-and-be-part-of-social-change/ Seeman, C. (2015). “No One Gets Left Behind”: managing library Human Resources during dramatic shifts in Academic Libraries. In Hines, S.S. & Simons, M. (Eds.), Library Staffing for the Future. Bingley: Emerald, pp. 101 – 125. (Advances in Library Administration and Organization, 34). Stilwell, C., Bats, R., and Lor, P.J. (2016). Introduction: redefining the role of libraries in the political process and in conflict situations. Library Trends, 65(2), 93-107.
Biography
Mrs Erika Rood, Manager Information Services, North-West University
Ms Marieta Buys, Faculty Library Manager: Humanities and Theology, University of Pretoria
Monday, 31 July 2017 – Parallel 3: Campus Collaboration
Author(s)
Dr. Scott Walter (DePaul University)
Title
Communicating value across the university: Library assessment across academic, student, and administrative affairs
Abstract
Purpose The purpose of this paper is to share the approach taken by library staff at DePaul University (Chicago, IL, USA) to adopt new approaches to integrating library assessment into student learning support and learning assessment programs across the university, including first-year instruction, co-curricular programs, and learning analytics. Design, methodology or approach The approach taken for this presentation is primarily descriptive, exploring innovative approaches taken by DePaul University library staff to establish partnerships across campus that demonstrate the value of the library to strategic initiatives of the university. While individual initiatives to be described are at different phases in terms of implementation, the presentation will include an exploration of the design of assessment activities for those initiatives in which formal assessment has been conducted. Findings For all initiatives described, a primary “finding” will be the library’s success in terms of integrating library assessment (or library contributions to institutional assessment efforts) into traditional (first-year instruction) and non-traditional (co-curricular learning) areas of institutional concern. For initiatives that have progressed to the point of formal assessment, e.g., through assessment of student work products, the findings of those studies will be shared. Research or practical limitations or implications There are limitations of the work to be presented, as well as implications for future practice. Among the limitations is the fact that some of the initiatives to be described are still in their early stages, while others have run a more complete course of design, delivery, and assessment. The implications of the work to be presented, however, far outweigh the limitations, as the approach taken by DePaul University library staff may serve to introduce to participants several new opportunities for strategic partnerships in the assessment of student learning, support for innovation in teaching, and use of campus facilities. Initiatives currently in progress at DePaul that have implications for broader practice include: assessment of first-year information literacy instruction; assessment of “embedded” information literacy instruction; assessment of library contributions to co-curricular education; assessment of library contributions to institutional investment in learning analytics; assessment of library contributions to institutional investments in the articulation of “transferrable skills” relevant to entry into the workplace; and assessment of investments in library space as a mechanism for promoting innovation in teaching and learning. Conclusions This presentation will draw conclusions regarding the opportunity to take a broader view of the library contribution to institutional efforts to assess student learning and the value of educational programs made available across the campus, and across academic and non-academic divisions of the university. conclusions may likewise be drawn, based on the DePaul University experience, regarding the value of taking this broader view of library assessment as an under-appreciated facet of institutional assessment efforts to bring greater attention to the library as a strategic investment for the university. Originality and value This proposal will provide a case study based on 5 years of innovation and relationship-building across a university with a documented record in supporting student success, curricular innovation, and the adoption of new technologies in support of teaching, learning, and scholarship. Local experience in advocating for greater attention to the library as a component of co-curricular education has exposed a considerable opportunity for alignment with student affairs assessment initiatives in the United States. Local experience integrating library data into an enterprise-system for use of learning analytics has exposed a critical need for alignment with institutional research, enrollment management, and student success initiatives with academic and administrative affairs divisions. While DePaul University examples have been highlighted in recent programs sponsored by the Association of College & Research Libraries and bepress Scholarly Publishing Services, this will be the first presentation of a high-level view of the strategic opportunities pursued by library staff across the university over the past 5 years.
Biography
Scott Walter is University Librarian at DePaul University, and an adjunct faculty member at San Jose State University’s School of Information and Dominican University’s School of Information Studies. He is currently serving as Vice Chair of Chicago Collections and as an Americas Regional Council representative to the OCLC Global Council.
Author(s)
Prof. Karin DeJager (University of Cape Town),
Prof. Mary Nassimbeni (University of Cape Town),
Mr. William Daniels (University of Cape Town),
Mr. Alexander D’Angelo (University of Cape Town)
Title
The use of academic libraries in turbulent times: Student library behaviour and academic performance at the University of Cape Town
Abstract
Purpose This paper will explore how an innovation in the University Management Information System, the construction of a Data Warehouse, was leveraged to incorporate library data by an initially sceptical and grudging strategic management team. The rationale for our request had been that we would then be able to extract evidence of correlations between library use and student achievement – thus far an almost impossible task because the different systems do not speak to each other. The diverse nature of the student body at the University of Cape Town (UCT) made this new instrument particularly useful, as it promised to enable us to isolate records by student demographic characteristics and to compare the library behaviour of different groups. This kind of information is of particular interest to the institution which is at present dealing with crises which have been precipitated by nationwide student protests, popularly summarised in the call for “#FeesMustFall” among students who suffer from the effects of poverty and exclusion in higher education. Approach We approach the data extracted from the Data Warehouse from the comparative demographic perspectives of students’ degrees of disadvantage in an effort to uncover any hitherto hidden patterns of library use. Findings Preliminary findings show that it is possible to map the use of the library as expressed by footfall, database use and loans, against students’ pass rates and their collective grade point averages, indicating that increased library use correlates positively with better academic performance. In the paper we’ll also comment on some of the initial correlations between student library behaviour before, during and after the nationwide disruptions that destabilized universities and threatened their survival at the end of 2016 just before the final examination period. We shall interrogate the effects that library closures (under threat of damage) at a critical time in the academic year might have had on library use and also on student performance. Conclusions Students on financial aid, our indicator of disadvantage, come from schools and environments where access to information technology and libraries is very limited, with the result that library habits are either poorly established or not established at all. At the University of Cape Town, considerable support is in place for students to encourage the development of library habits, for example assistance in information commons venues and in the provision of online training tools. Initial analysis of available data is indicating that students who have acquired library habits regardless of unfavourable financial circumstances do not exhibit behaviour and academic outcomes markedly different from that of their more privileged peers. Originality and value Combining library data with data from the University Data Warehouse is a new approach in South Africa. It is an approach that is of value both to the library and the institution at large and has brought meaningful insights into the role the academic library might be seen to play in promoting student academic achievement.
Biography
Karin de Jager is Emeritus Associate Professor in the Library and Information Studies Centre at the University of Cape Town. She still acts as a research supervisor and mentor. Her research interests include library performance assessment in academic libraries and information literacy.
Mary Nassimbeni is Emeritus Associate Professor in the Library and Information Studies Centre at the University of Cape Town. She still acts as a research supervisor and mentor. Her research interests include library policy and performance assessment in both public and academic libraries.
Author(s)
Mr. Boyd Rodger (Bodleian Libraries, University of Oxford)
Title
A green approach to reducing library storage costs
Abstract
Purpose The Bodleian Libraries opened a new high-density storage depository in the form of a warehouse in 2010. It is similar to the warehouses at Harvard, Princeton and Yale universities. The cost of maintaining the environment was expensive and an alternative was sought for the energy intense model of conserving materials. Design, methodology or approach A series of incremental experiments with the Building Management System (the software that manages the environment in a building) of the Bodleian Libraries large storage depository was conducted between 2012 and 2016. The research methodology was that of action research. This involved the collaboration of consultant engineers, conservators, and the author of this paper as joint co-ordinator of the group. Each experiment was conducted in an agreed sequence with the option of a senior library manager (myself) being able to cancel any experiment if there was any concern about the temperature or the humidity levels. One of the experiments involved switching off the machinery that heats and chills the environment between 16:30 and 19:00. This was the most expensive electricity charging period where costs are 872% higher. It proved successful with no adverse effect on the environment. Findings The experiments resulted in significantly reducing the amount of energy required to maintain the temperature and relative humidity for the preservation of the books. It produced the following savings:
- A saving of £90k per annum in utilities costs;
- A reduction of 412 tonnes of carbon;
- A reduction of 283k Kilowatt Hours in electricity per annum.
Research or practical limitations or implications Library, conservation, and facilities management professionals tend to focus on the results of environment management in large library depositories (i.e., the consistency of temperature and humidity levels) rather than outcomes (i.e., how much energy is required to achieve that consistent temperature and humidity). Working towards a more green and cost effective alternative to conventional depository environment management, therefore, requires a radically different use of measurement methods. Conclusions The use of measurement methods in the preservation care of library material requires a more sophisticated approach than simply measuring temperature and humidity. Collating and analysing data on the environment management will inform which parts of system are using the most energy and could be subjected to experimentation to make them operate efficiently. It is also important to continuously ask probing questions about how the environment management system works even though the questioner is not an engineering professional. Originality and value The experiments at the Bodleian Libraries storage depository have been acknowledged within the University of Oxford as best practice for other buildings in the University. In November 2016 this work was recognised at the Green Gown Awards which identify exceptional sustainability initiatives in universities and colleges. The University of Oxford was the winner under the category of carbon reduction.
Biography
Boyd joined the Bodleian Library when it was beginning a major period of transformation. He led the start-up of the new library storage warehouse in 2010 and since 2014 has managed improvement projects. Boyd also has experience of policing, business consulting, and performing at the Edinburgh Festival.
Author(s)
Mr. Aaron Tay (Singapore Management University),
Mr. Galvin Soh (Singapore Management University)
Title
Library collaborations within the University on analytics and user studies
Abstract
Purpose The twin rise in interest in library analytics (Showers, 2015) and UX (user experience) (Appleton, 2016) in libraries has led to new ideas and methods for studying the library and its users for evaluation and assessment. While librarians can and should acquire skills to employ techniques such as data mining and ethnography arguably working with non-librarians who have a different viewpoint can possibly yield fresh insights. At the Singapore Management University (SMU), a University with a strong focus on experiential learning pedagogy and business intelligence (Gottipati & Shankararaman, 2016) collaboration with the library seems to be an obvious win-win strategy for both students who need to practice their skills in a real-world situation as well as researchers as a test-bed for new technologies. The author who was hired as Library Analytics Manager in 2015, juggles a number of responsibilities including but not limited to ad hoc operational analysis, management of library business dashboard, library-student outcome correlation studies (Soria, Fransen, & Nackerud, 2013) as well as serves as a point person for collaborations on analytics within and outside the library (Tay, 2016). The collaborations that he has been involved in span a range of quantitative areas such as analytics and data mining on electronic resource usage and gate counts (with various student groups majoring in analytics), deploying new analytics using wifi tracking (with research centers), measuring value of information literacy (with writing centres) and in qualitative areas involving in-depth interviews and ethnographic studies (with students in research method classes). They also differ in complexity from collaborations with students in 101 classes on business performance improvement studies to multi-year collaborations with Ph.D. students and research analytics labs. Design, methodology or approach The author proposes a model for academic libraries on managing collaborations in analytics and performance management with different participants based on level of expertise, purpose of collaboration and sensitivity of data. Findings Analytics and performance assessment is a multi-faceted area, with many opportunities for collaboration. The challenge is with finding the best fit for collaboration that maximises the benefits for both the library and the collaborators. Originality and value While collaborations with faculty on information literacy (Wijayasundara, 2008) are not new, collaborations on analytics is a new unexplored area and as great potential for synergy. References Appleton, L. (2016). User experience (UX) in libraries: Let’s get physical (and digital), Insights, 29(3), 224-227. Gottipati, S. and Shankararaman, V. (2016). Designing a data warehousing and business analytics course using experiential learning pedagogy, Proceedings of SIGEd 2016 Conference.
Showers, B. (Ed.). (2015). Library Analytics and Metrics: Using data to drive decisions and services. Facet Publishing. Soria, K. M., Fransen, J., & Nackerud, S. (2013). Library use and undergraduate student outcomes: New evidence for students’ retention and academic success, portal: Libraries and the Academy, 13(2), 147-164.
Tay, A. (2016, November 18). 5 reasons why library analytics is on the rise [Web log post]. Retrieved from http://musingsaboutlibrarianship.blogspot.sg/2016/11/5-reasons-why-library-analytics-is-on.html Wijayasundara, N. and Singh, D. (2008). An initiative to enhance the linking of information literacy in teaching through faculty-library collaboration, In Abrizah Abdullah, et al. (Eds.), Towards an information literate society: Proceedings of the International Conference on Libraries, Information and Society, 59-68.
Biography
Aaron Tay is library analytics manager at Singapore Management University and has about 10 years experience in academic libraries. He has a keen interest in all things libraries and blogs at Musings about librarianship (https://musingsaboutlibrarianship.blogspot.com) and he can be found on Twitter at @aarontay.
Author(s)
Ms. Asa Forsberg (Lund University)
Title
Implementing a model for service development in libraries: Collaboration and empowerment
Abstract
Purpose This lightning paper describes a work in progress. Lund University is a comprehensive university with eight faculties. Each faculty has a library organisation, and the faculty libraries form together with the centrally financed University Library a decentralized network, with around 200 staff members. The library network is directed by a management group, consisting of the faculty librarians and the library director. The management agrees on a yearly operational plan. According to the quality policy taken in 2014, a separate activity plan for quality assessment has also been agreed on every year. The library network is strongly engaged in developing the services for students and researchers and throughout the years many projects, both large and small and many of them successful, have been carried out. However, it is felt that the service development is not systematic enough: documentation is not always consistent and can be difficult to find, and the outcomes of projects and activities are not always evaluated. In consequence, the organisation does not benefit as much as it could; the risk is that we reinvent the wheel instead of progressing thanks to knowledge gained.
In Autumn 2016 a small group was commissioned to write a new quality policy. Considering the problems described above we decided not to write a general policy. Instead, the new quality policy consists of a model for service development, designed to be used practically, and a short text explaining how to use the model. The purpose is to achieve a more systematic service development and to facilitate the development process. Another important factor is that the quality assessment will be integrated in the library network activity plan. The library network management approved of the policy, and in December 2016 it was adopted by the Library Board, the formal head of the Lund University Library network. Design, methodology or approach The quality policy applies as from the 1st January 2017, and should now be implemented. The implementation will be a process during 2017, and we view this process as a learning experience and evaluation of the model. The model describes a cyclic process of five phases: Current status, Planning, Development work, Implementation, and Evaluation. The model should be generic, i.e., possible to use in both larger and smaller development projects and activities, and lead to a systematic approach. The central part of the model is a toolbox, with methods and other tools to be used in the different phases of the model. It is digital (a LibGuide), and the emphasis is on methods and processes which have been used in the library organisation, varying from quantitative surveys to UX (User Experience). Each tool is briefly described, with links to further reading/material, and, when applicable, information about the colleagues having used it and the context. It has been decided that the model will be used when implementing the joint activity plan for the library network 2017, with several activities ranging from planning for a new library system to setting up a staff training program for the next three years. Swedish academic libraries do not have a tradition of large assessment departments. In general, there may be one assessment librarian, which is the case at Lund University Libraries. Consequently we have small resources to support the management and the staff who are going to accomplish the activities. Our goal is to propose a model and a toolbox to facilitate and support the processes, and make them systematic. Staff members should be able to use the model when planning the different phases in a project, and thus making the process more solid and valid. We practise as we preach: in a “project about the projects,” we will test and evaluate the model and its components during 2017. In March we will gather the staff responsible for the different activities in the activity plan to do a workshop. This will be an occasion for the participants to begin the planning of each activity, and also to discuss and express the need of further support and training. With this information we will make a program for 2017, with seminars, workshops and training. We will use agile methods to continuously evaluate and adjust the toolbox and supporting activities and in June key persons will be interviewed. In the end of 2017 we will make a larger evaluation. The results will give us evidence to adjust and improve the model and the toolbox.
In April we will also as a small benchmarking project, together with Malmö University Library, arrange a “project meet” where participants give lightning papers about projects and methods. In May we will arrange a workshop about systematic service development at a national Library and Information Science conference, to discuss best practices and share knowledge. Findings This being a work in progress there are no findings at this moment. In July, we will have accomplished the initial workshop, at least one training session and a seminar, and also made the first evaluation with stakeholders involved. The findings from the interview study will be presented. Conclusions We are in the beginning of the process to implement and evaluate the model. So far the model and the concept of a systematic, cyclic and continuous process to change and improve the library services has been well received, by the Library board, the management and other colleagues. It will be used when implementing the joint activity plan for the library network. We consider this as an evaluation and learning process that will lead to improvements in the model and the toolbox. It is our expectation that the model will also be used in other contexts, such as projects and activities initiated by individual faculty libraries or the University Library. Originality and value The value of the paper is that it describes a practical effort to change and improve service development within an organisation.
Biography
Åsa Forsberg is a librarian, working as an assessment librarian at the Lund University Library.
Monday, 31 July 2017 – Parallel 3: Strategic Planning
Author(s)
Ms. Jennifer Peasley (La Trobe University)
Title
One small step: Developing a framework for assessing Australian higher education libraries
Abstract
Purpose The Council of Australian University Librarians (CAUL) has developed a framework for describing and assessing the role and functioning of contemporary university libraries. The framework, Principles and Guidelines for Australian Higher Education Libraries (2016), acknowledges the leadership and professional expertise necessary for the delivery of high quality services and resources for research, teaching and learning and presents guidelines for assessing and articulating the contribution of the library in the context of institutional priorities. The Principles and Guidelines framework addresses a range of uses, including internal and external review, accreditation, and focused self-assessment to guide continuous improvement. The framework acknowledges the continuing evolution of university libraries and the unique professional skills and capabilities held within libraries that contribute to institutional and national higher education performance outcomes, and helps to articulate how libraries add value through these skills and capabilities. Design, methodology or approach The methodology used to develop the Principles and Guidelines for Australian Higher Education Libraries (2016) involved three techniques: the gathering and analysis of information to inform the development of a framework; consultation with external stakeholders to further gather information and test proposed options; and review of the options with the users of the Guidelines to determine whether the framework would meet their needs. The development process involved:
- desktop research: a review of existing higher education and library standards and measures;
- extensive consultation with leaders within Australian higher education and libraries, involving thirty one-on-one interviews with a diverse range of University library stakeholders (university librarians, senior university academic and administrative staff, affiliated library and university bodies and peak bodies representing students);
- in-depth consultation with staff in Australian and New Zealand higher education libraries.
A consultant was engaged to undertake the research and interviews to ensure that the principles would be externally focused. Findings The insights from leaders within Australian higher education shaped the development of the Principles and Guidelines, highlighting consistent themes: the importance of libraries’ client-centred approach, their provision of access to information, their ability to optimise the usability of information and to collaborate within the university and beyond the university. It was critical that the Principles and Guidelines aligned with and referenced the Australian Higher Education Standards Framework (Threshold Standards) 2015, and Quality Indicators for Learning and Teaching benchmarking process. While the development of the Principles and Guidelines was acknowledged as important to university libraries, expectations of what would be delivered through this process varied greatly among the university library community. Extensive consultation with CAUL members was undertaken to refine the document to ensure that it would meet the needs of the majority of university libraries, and it was agreed to undertake further work to develop more detailed performance indicators for those libraries who wish to use them. While there has not been universal agreement on all aspects of the Principles and Guidelines, there has been great benefit in having wide-ranging robust discussions about measuring and articulating value and impact. These discussions have highlighted the complexity of demonstrating the value and impact of university libraries and the need for a range of approaches and tools to do so; the Principles and Guidelines are just one tool. The further development of maturity indicators for each Guideline will assist libraries to benchmark their libraries and provide a path for future development. Conclusions The Principles and Guidelines for Australian Higher Education Libraries can enable the proactive assessment of university library performance across a range of priority areas:
- Strengthening learning, teaching and research outcomes
- Fostering the creation and dissemination of new knowledge
- Growing a dynamic, sustainable and accountable organisation
They provide CAUL members with a framework to assist with planning and strategic alignment, and articulating:
- What is a contemporary university library?
- The alignment of library activity with that of the university
- How libraries add value through their professional skills and capability
- How university libraries are distinguished from other services and facilities – their visibility, impact and independence
- How university libraries can influence and support government and university policy development and decision-making.
The framework is a starting point for higher education libraries in measuring their performance; as a community, higher education libraries in Australia and New Zealand will contribute to the further development of the framework through case studies, detailed indicators, and the inclusion of new guidelines which reflect our response to the changing environment of higher education libraries. Originality and value
The development of the Principles and Guidelines for Australian Higher Education Libraries is an important first step in having a common approach to measurement which moves beyond input and output measures to quality and impact. The collegiate approach taken to develop the framework, including the input of stakeholders outside libraries, enabled the capture of a wide range of views and provided an important external perspective. While the framework does not solve the issue of articulating the value of academic libraries, it has prompted a more focused consideration of quality and value and provides a starting point for the development and sharing of tools and methodologies among higher education libraries in Australia and New Zealand.
Biography
Jennifer Peasley is University Librarian at La Trobe University Library, responsible for leading library services and partnerships across the university’s five campuses. Jennifer is Chair of the Council of Australian University Librarians’ Quality and Assessment Advisory Committee and has a strong interest in quality assurance, evaluation and organisational development.
Author(s)
Ms. Kathryn Ball (McMaster University Library),
Dr. Lorie Kloda (Concordia University Library),
Ms. Elizabeth Hayden (University of Ottawa Library)
Title
Linking assessment to the strategic plan in academic libraries
Abstract
Purpose The authors’ current research is exploring the use of metrics in strategic plans in Canadian academic libraries. Design, methodology or approach
The initial phase of the research project involves examining the current strategic plans of member libraries of the Canadian Association of Research Libraries (CARL), checking for the use of a specific strategic planning framework and the use of performance measures as part of the plan. The second phase will involve follow-up interviews with assessment librarians and senior leaders in the organizations to further explore the use (or non-use) of metrics in the strategic planning process, and the link between assessment processes and the strategic planning process. The approach is similar to a study conducted by Rayna Bowlby on member libraries of the Association of Research Libraries (ARL) (Bowlby, 2010). 52% of CARL libraries are also members of ARL. A comparison of the results of the two studies will be a part of the research. Findings An initial scan indicates that 79% of Canadian academic libraries have a strategic plan posted on their website, of which 26% have associated metrics. Research or practical limitations or implications Although the study focuses on Canadian academic libraries their strategic planning processes and performance measures may be of use to other academic libraries.
Conclusions The study will provide current information on the state of strategic planning in Canadian academic libraries highlighting similarities, differences and gaps in types of metrics. Originality and value We hope to provide insight into the challenges of the strategic planning process and the development and use performance measures. It will be interesting to note if the situation has changed in those Canadian libraries that were also part of the Bowlby study. References Bowlby, R. (2010). “Are We There Yet? Aligning Planning and Metrics — Strategically”, Retrieved from http://libraryassessment.org/bm~doc/bowlby_raynna-2.pdf
Biography
Kathryn Ball’s research interests include the use of the Balanced Scorecard in academic libraries, the development of performance metrics and qualitative methods in library assessment. She has spoken at provincial and national conferences and written several articles on these topics.
Lorie Kloda’s research interests include evidence-based practice, library assessment, and information needs of health professionals. She was awarded a Doctor of Philosophy, Information Studies from McGill University in 2012. She currently serves as editor in chief of Evidence Based Library and Information Practice.
Author(s)
Ms. Lisa Horowitz (Massachusetts Institute of Technology)
Title
From vision to success: Assessment workflows for major strategic initiatives
Abstract
Purpose
In October 2016, Chris Bourg, director of MIT Libraries, released the preliminary report of the Task Force on the Future of Academic Libraries (https://future-of-libraries.mit.edu). The report is a first step in describing “how the MIT Libraries ought to evolve to best advance the creation, dissemination, and preservation of knowledge; and to serve as a leader in the reinvention of research libraries.” The report’s value is not only in its thought-provoking propositions and impacts in implementation, but in the underlying necessity that in order to work, its assessment must be forward-thinking, not simply evaluations of current activities. At the same time, librarians and library staff who are accustomed to working in a very tactical operational environment where practical considerations are first and foremost (how can we better serve the customer with our reference service? how can we improve our current processes for handling reserves materials? what is the best training that we can offer our staff so that they can support our new GIS lab? etc.) can sometimes struggle to translate the not-so-tangible implications of strategic visions and plans like this one into measureable action. In this lightning talk, participants will hear
- how to apply assessment principles to help staff understand how to get from an 80,000 foot big-picture vision to a practical determination of what makes that vision successful;
- how to break down a vision into clear descriptive implementable goals that can be measured to show that progress is being made or success has been achieved;
- how to think about outcomes as distinct from deliverables, which allows flexibility to change goals based on new information or activities while still focusing on the original vision.
The talk will end with some suggestions for how libraries can work together to develop assessment tools and models to make intangible strategic visions like social justice, diversity and inclusion, or open access for all, into goals that are measurable, in order to enable libraries to offer evidence to the world of our impact on society.
Biography
Lisa R. Horowitz is the MIT Libraries’ assessment librarian. She develops, coordinates, oversees and consults on assessment efforts across the Libraries. For nearly two decades, she has been writing about assessment practices. Most recently, she served on the ACRL Task Force on the Proficiencies for Assessment Librarians and Coordinators.
Author(s)
Mrs. Satu Bohm (Uppsala University Library),
Mrs. Karin Byström (Uppsala University Library),
Mr. Petter Nerelius (Uppsala University Library),
Mrs. Linda Thorn (Uppsala University Library)
Title
Negotiating change
Abstract
Purpose
Uppsala University Library is one of the largest in Sweden, with a wide range of specialties – old collections, digital repositories and current support to faculty and students. In the last 18 month the library has gone through a re-organization with the aim to make the library more united – one library. The change is among the most comprehensive that the 400-year-old library has undergone.
The purpose of this paper is to describe how the library has developed ways of leading change by involving employees and managers at different levels in the change process. We focus on change management based on our experiences as change leaders. We will describe the major success factors and failures.
Design, methodology or approach
The re-organization was done in several steps. Library units were merged and three processes introduced; Media and Collections, Study and Research Support and Information Services.
Within these processes, three working groups were established with the goal to present new workflows and services for the digitization, working methods in library teaching and management of the virtual reference desk.
Although the three areas for the process mappings were quite different from each other, our experiences as change leaders and use of methods for change management show many similarities. We therefore believe that they can be used in a broader context.
To gain approval for the changes, all processes also had reference groups where new ideas and proposals were discussed. Information meetings for all staff were also arranged.
Findings
In anchoring change at all levels we believe negotiation is a key concept. Negotiating change guarantees quality and substantiated decisions from both a user perspective and an organizational perspective. Negotiation has also been a way of reflective practice around goals, effectiveness, working methods and leadership. It is not a fast track to results but we believe that anchoring change at all levels is a key concept for long-term success. Another success factor has been a joint change leadership.
Conclusions
The library has developed new ways of leading change by involving employees and managers at different levels in the change process.
Changes can result in lot of tension and resistance in the organization and the new methods have been an important part in avoiding this – to include concerned groups in the process and to take time to anchor the change.
Biography
Satu Bohm is the process manager for Study and Research Support. She works with developing and coordinating research support and courses in information literacy.
Karin Byström is the process manager for Media and Collections. She works with developing and coordinating acquisitions, cataloguing, digitization and preservation of the print and electronic collections of the library.
Petter Nerelius is the Program Leader for the library reorganization. He works with change management and support Process Managers, library management and the HR-department in their work with co-workers and groups in the different departments of the library. Petter is very experienced in the field of project management as well as change management and has worked with numerous organisations in both large and small projects.
Linda Thorn is the process manager for Information Services. She works with physical and virtual reference services, user experience and the library space.
Author(s)
Mr. Damon Jaggars (The Ohio State University),
Ms. DeEtta Jones (DeEtta Jones and Associates)
Title
An agile planning framework: Design, implementation, and assessment
Abstract
Purpose Following a transition in leadership in 2016, the Ohio State University Libraries embarked on an ambitious project to re-envision and renew its strategic intent. Prior to designing a planning process, library leaders reflected on past experiences with strategic planning, focusing on what worked well and what did not. This reflection on past planning efforts exposed a common experience of the conflation of the strategic with the operational. The plans resulting from past efforts often confused statements about strategic intent (what an organization aspires to be) with implementation plans (how that organization will get there), which usually led to somewhat unfocused, overstuffed inventories of all the good things a research library should do and muddied attempts to provide clear articulations of where it would actually place its strategic effort, supported by real financial and human resources. Leadership also reflected on how traditional strategic planning processes can often consume libraries in extended, burdensome planning activities, which often result in static 3 to 5-year plans that can bring on the unfortunate side effect of pushing organizations into collective psychologies of task list completion. Instead of traveling this well-worn path, the Libraries chose to design and implement an agile planning framework, designed to facilitate an ongoing organizational conversation about its strategic intent and how it plans to move from intention to reality. Such a framework would necessitate a lighter-weight, open-ended process, allowing for increased flexibility and openness to unforeseen opportunities in its implementation. If implemented successfully, such an approach would ensure the continuing integration of library faculty, staff, and external stakeholder voices into planning, management, and assessment discussions because the framework itself is conceived as an ongoing, integrated discussion with and between these groups. DeEtta Jones and Associates, a consulting firm with deep experience in organizational transformation, was contracted to help facilitate the overall design process and initial stakeholder engagement activities. The authors will describe the design and implementation processes through the lenses of both a library administrator and a process consultant, sharing the methods utilized, results, as well as practical considerations. Design, methodology or approach The basis of the agile planning framework is a concise statement of the Libraries’ strategic intent: articulations of who we are, what we value, and our view of the future, along with focused direction statements flowing from this context. The framework will be operationalized through more detailed action plans at the divisional and unit levels, clearly mapped to these high-level strategic directions.
The framework seeks to harmonize planning, management, and assessment processes over 18-24 month rolling time horizons, during which organizational efforts and investments would be reviewed and revised in an iterative fashion, on a quarterly or semi-annual basis. This review structure is conceived as a relatively light-weight, ongoing organizational conversation, facilitated within Management Committee, a group comprised of the Libraries’ executive team and middle managers. This structure provides a platform for broad discussion about changing user needs, emerging opportunities, and other evolving environmental factors through a series of structured processes for evaluating current activities, emerging opportunities, proposed initiatives and investments, and regular environmental scanning and stakeholder input at both the strategic and operational levels. Findings The implementation of the agile planning framework is ongoing. The authors will share findings of the parts of the process that will have been completed, which will include: (1) results and analyses from stakeholder assessments (focus groups and surveys) used to develop foundational mission, vision, values, and strategic directions documentation; (2) a visual representation of the agile planning framework and its supporting processes and time horizon model; (3) rubrics designed to provide periodic, lightweight environmental scanning and assess the alignment of current and proposed activities and investments with strategic intent; and (4) a discussion of the structural, cultural, and organizational development challenges confronted and gains experienced, while implementing the framework. Conclusions The motivating idea of the agile planning framework, following from the philosophy under pending agile software development, is to construct an architecture for strategic thinking and action that would obviate the need for the Libraries to drop everything every five years or so and consume itself for an extended period to build out a new, static strategic plan. If successful in creating an effective agile planning framework, the Libraries’ strategic intent and the operational work done to support it should evolve in a more organic fashion through meaningful, ongoing dialog about organizational priorities, informed by engaged interaction with users, key external stakeholders, and university-level planning efforts. If successful, the Libraries would be more adept in responding to emerging opportunities and recognizing when to move on from others because the mechanisms for detection and analysis are built into the framework and instantiated in organizational structures and processes. Originality and value Many academic libraries are exploring new approaches to strategic planning, ways to enhance organizational health, and manage change. The authors are unaware of an academic or research library that has attempted to design and implement a similar approach to strategic planning and its assessment. The agile planning framework provides an alternative to traditional “waterfall” approaches to strategic planning for libraries.
Biography
Damon E. Jaggars serves as Vice Provost and Director of University Libraries at The Ohio State University. He also serves on governing boards for the Digital Preservation Network, OhioLink, and the Big Ten Academic Alliance, as well as the editorial board for portal: Libraries and the Academy.
Tuesday, 1 August 2017 – Parallel 4: Public Libraries
Author(s)
Ms. Claire Creaser (LISU, Loughborough University)
Title
Putting it into practice II: Public library standards in Wales
Abstract
Purpose The purpose of this paper is to describe the outcomes and impact of the fifth quality framework for public libraries in Wales, 2014-2017. It follows on from a paper presented at the 11<sup>th</sup> Northumbria Conference describing the development of the framework (Creaser, Spezi, & Hepworth, 2015); this paper will describe the implementation of the standards and the impact this has had on public library services in a time of considerable change. It will conclude by looking towards the future for public library quality monitoring in Wales. Design, methodology or approach The fifth quality framework for public libraries in Wales, Libraries making a difference, represented a major departure from previous public library standards in the country by introducing a formal element of impact and outcome assessment alongside the more usual performance measures of inputs, outputs and satisfaction scores. It comprises 18 core entitlements and 16 quality indicators representing four broad areas of activity – customers and communities, access for all, learning for life, and leadership and development. The framework covers the three-year period from April 2014 to March 2017. All public library authorities in Wales are required to make an annual return to MALD, the Museums Archives and Libraries Division of the Welsh Government, reporting their results against the core entitlements and quality indicators, together with a series of narrative reports describing their contribution to government agenda, case studies of impact, and future direction. These returns are independently monitored and assessed, and a summary presented annually to the Welsh local authorities. Findings All 22 public library authorities in Wales have engaged with the quality framework. Performance against the standards has been mixed, with some authorities showing improvements over the period, while others have faced more challenging times. All returns have shown excellent examples of the impact of library services on communities and individuals, although quantifying such impacts has proved more elusive. Across Wales, there are a number of strengths, particularly in customer engagement and satisfaction, with an extensive range of activities and events offered, and 85% of users helped to achieve their goals. Users with special needs are well served, and free wi-fi is being rolled out, including in rural areas. The service operates with professional managers throughout, and there is a shared catalogue facilitating a prompt and efficient reservations scheme. A number of ongoing issues have also been identified – pressure on resources has impacted upon the traditional measures of provision and use, such as visits and loans, materials budgets and acquisitions, with a fragile staffing situation in some areas and falling opening hours. Library authorities are developing innovative strategies not only to maintain their service within the statutory framework but to enhance it with cooperative working and co-location. The involvement of local communities in support of their libraries, both within the statutory service and via transfer of a number of library service points to community organisations, has impacted upon what can be included in the returns, however. This has led to falls in some of the traditional indicators of use, although customer satisfaction remains high. Research or practical limitations or implications The paper presents a case study across all 22 library authorities in one country. The framework forms non-statutory guidance for local authorities in the operation of their library services. The issues addressed by the framework are universal to the provision of public library services, and as such could be applied in any jurisdiction. Some of the new indicators proved challenging to implement in some authorities and the ways in which such challenges were met will be described.
Conclusions The fifth quality framework for public libraries in Wales has continued to protect services in challenging times, through highlighting where services are most vulnerable, and bringing this to the attention of the Welsh Government and the relevant Minister. The body of evidence for the impact of library services on individuals and community groups, and the contribution of the service to both local and national agenda is growing, and appropriate use of this evidence in advocacy for the library service will further contribute to the service. As a consequence, the sixth quality framework has evolved from the fifth, retaining the emphasis on impacts and outcomes, while strengthening the guidelines for community supported libraries, giving standards for service delivery and provision which will allow those libraries which meet the standard to be kept within the statutory service and so included in the reporting. Originality and value This paper documents the process by which a new national framework for assessing the quality of public library services was implemented, in collaboration with national government, local authorities and the library services themselves, and describes the results of that assessment. References Creaser, C., Spezi, V. and Hepworth, M. (2015) Putting it into practice: the development of the Welsh Public Library Standards, Presented at 11th International Conference on Performance Measurement in Libraries and Information Services, July 20-22 2015, Edinburgh, UK. Welsh Government (2014). Libraries making a difference. The fifth quality framework of Welsh Public Library Standards 2014-2017. Retrieved from http://www.lboro.ac.uk/microsites/infosci/lisu/downloads/140425wpls5en.pdf.
Biography
Employed at LISU since 1994, Claire was appointed Director in 2007. Her main areas of interest are the use of statistical evidence for library management; analysis and interpretation of survey data; and scholarly communication. She is Chair of the BSI committee for Library and Publishing Statistics, and a Chartered Statistician.
Author(s)
Dr. Adeline Lee (National Library Board)
Title
Mapping the physical and digital reading habits of Singaporeans: findings from an integrated quantitative-qualitative study
Abstract
Purpose Global outlook on library visitorship and loans point to a trending down in physical visits and loans, whilst digital visits and loans are trending up. Singapore’s network of public libraries is not spared and experiences the same phenomenon of in the recent years. Although virtual visits have increased, it has compensated only in a small extent to an overall positive percentage growth in visitorship. The outlook for loans is dismayed, where the increase in e-book loans fails to compensate for the loss in physical loans. An inaugural nation-wide population study was launched in 2016 to comprehensively understand the state of reading in Singapore, reading motivation of and barriers to reading, lifestyle mapping, media technology habits, and information and knowledge seeking habits. The findings shed light on ways our libraries can transform, regain market share and attract Singaporeans to make use of our services. This paper shares the methodology, findings and implications. Design, methodology or approach The study was conducted in mid-2016 and employed quantitative and qualitative methodologies: a survey using a questionnaire and focus group discussions. The survey ascertained behaviours, in key aspects such as reading frequency, reading format and genre, time spent, reading motivation of and barriers to reading, lifestyle, media technology habits, and information and knowledge seeking habits. The door-to-door face-to-face survey interviewed 3,515 residents aged 20 years old and above. The sample was representative of the population by age, gender, ethnicity and dwelling type. Focus group discussions sought qualitative explanation of their behaviour, and included motivations and perceptions that drive their behaviour. 163 participants took part in 18 focus group discussions. An additional sample of respondents studied teenagers aged between 13 and 19 years old, however the focus of this paper is on adult Singaporeans. Findings The study found that the state of reading among adult Singaporeans is healthy, with eight in ten having read any material for leisure at least a few times a week. ‘Reading’ includes reading books, magazines and newspapers/news, in print or digital format, as well as articles posted on websites and social media. The majority spend at least thirty minutes reading, on a typical weekday or weekend. However, while almost seven in ten read news habitually (i.e., at least a few times a week), habitual reading of books is significantly and surprising low. Seven in ten have read at least one book in the past twelve months, but only one in five are habitual readers. Overall habitual reading of online articles, via websites or social media sites, was lower than expected, with two in five having done so. Nonetheless, this is higher for the 20 to 39 year olds, at three in five. In explaining their reading behaviour, focus group participants highlighted that reading news is a must, as this allows them to keep up with what is happening and has an impact on their work. It is also a form of social currency, where not knowing the latest would make them appear out of touch, and as a consequence, affects their social standing. Reading books is seen as an unnecessary activity, as they are already obtaining the latest information online. Books are associated with their past school days which they have outgrown. Other reasons include the lack of energy after work to read books and the inability to focus. In alignment with most who mainly read news, almost all of the respondents choose non-fiction topics due to its practical nature. In contrast, only six in ten read fiction topics. In terms of age analysis, fiction reading was the highest among the 20 to 29 year olds, but the proportion decreases with age. There are stark differences in age and gender. While young males read about health & fitness/sports, older males prefer Singapore-related information and politics. Young females tend to read about lifestyle topics, but older females tend towards cooking and Singapore-related information. The study also focuses on understanding book reading in physical and digital formats. The analysis determined that more than nine in ten book readers read physical books, but a lower proportion, i.e. four in ten read e-books, and an even lower proportion accesses audiobooks (less than one in ten). However, not unexpected, reading in the electronic format is higher for the 20-29 year olds. Tests of significance between independent variables such as age, gender, ethnicity, education, life cycle stage, dwelling type, lifestyle activities and information-seeking behaviour and the dependent variable reading habits were carried out. Significant results (p<0.05) was observed, in particular for age and life cycle stage.
Singaporeans’ attitudes towards reading, their motivations to read and barriers faced via eleven statements were asked, and factor analyses were carried out. Key difference between habitual and non-habitual book readers vis-à-vis their attitude towards reading was observed. Main barriers to reading books more often are that it is not as simulating as audio-visual content, and that people prefer to spend time on other activities. Again, this is more salient amongst the 20 to 29 year olds. Research or practical limitations or implications; Conclusions In 2016, NLB launched the National Reading Movement to promote a culture of reading, and support Singapore’s emphasis on life-long learning. The study contributes towards a quantitative baseline of the state of reading in Singapore. Key performance indicators and targets have been derived to strive towards sustaining reading among Singaporeans, be it through better offerings in library content, programmes and services. Subsequent runs of the study will help NLB to further nuance our campaign strategies to meet changing needs. This will in turn help our libraries to remain relevant. Originality and value This study was the first in-depth study to ascertain the state of reading amongst Singaporeans. Deliberate efforts were devoted to determine respondents’ behaviours related to reading, lifestyle, consumption, information seeking and media usage. The survey eschewed “opinion-centric” questions and, as such the findings have practical implications on future directions to transform our libraries into the ‘Library of the Future’. The study also provides fresh empirical findings on reading habits from an Asian context.
Biography
Adeline heads the Business Analytics, Knowledge & Risk Management Department of National Library Board, Singapore. The Department conducts library analytics, programme and outreach evaluation and insights of library patrons and non-patrons. Adeline holds a Ph.D. in media and communications, and specializes in the evaluation of public campaigns and community outreach.
Author(s)
Ms. Claudia Thompson (American Heritage Center, University of Wyoming)
Title
Fast access: Delivering finding aids to an impatient world
Abstract
Purpose In the inter-connected world, patrons expect immediate access to information of all kinds. But archivists and curators of manuscripts face a difficulty in providing instant access. Modern collections of personal papers can comprise hundreds of cubic feet of heterogeneous material. Even smaller collections can include papers, photographs, sound recordings, and electronic records, among other formats. And such collections differ from books in that each one is unique and no table of contents is included to assist the process of evaluation. In order to make personal papers useable, archivists, curators, and special collections librarians create finding aids. A finding aid may be anything from a catalog record to a calendar of individual items, but all require time to create. As a result many institutions have a backlog of undescribed collections that are invisible to researchers whose first (and often only) source of information is the internet.
The American Heritage Center is the University of Wyoming’s repository of manuscript collections, rare books, and university archives. The Center is supported by state taxes and by private donations and endowments. It is a public institution open to anyone who wishes to utilize its resources. No fees are charged for access. The Center’s ambitious collecting policy includes the history of Wyoming and the western United States, the American entertainment industry, 20th century mining and petroleum in many parts of the world, the environmental movement, aviation history, and journalism. The full-time staff, from clerical support to director, is usually about twenty.
In the mid-1970s the American Heritage Center began to lose control of its backlog. The problem was exacerbated in the 1990s when internet access created a demand for online finding aids, which had to be met by converting all legacy finding aids to electronic ones. By 2003, the Center had acquired 9490 individual collections comprising 87,050 cubic feet of material. Valiant effort on the part of the staff had converted 1890 finding aids to online format. Six thousand further collections had paper-based finding aids but no online presence. Sixteen hundred collections had no finding aid of any kind. Many of these backlogged collections were larger than the average, so although the number of undescribed collections was only 17% of the total, the overall volume was 34%. It was time for some drastic action.
Our goal was to reduce the time it took to create web-based finding aids for our archival and manuscript collections by experimenting with radical shifts in standard arrangement and description processes. We then had to assess the effectiveness of our experiments and integrate the successful new processes into our normal workflow. Design, methodology or approach We began by assessing the effectiveness of our creation process, considering the number of collections completed per year and the number of people doing the work. The latter proved to be an extremely variable statistic as positions fell vacant and were re-filled or part-time and volunteer labor was added to the mix. We then began to introduce innovations, including:
- collection level finding aid (catalog record) without lower level description;
- collection level finding aid (catalog record) linked to EAD (encoded archival description) without lower level description;
- legacy (not web-based) finding aid linked to EAD linked to catalog record.
Findings To increase speed of finding aid creation, we found it was better to create descriptions beginning with the general and moving to the particular rather than building from the particular to the general. We also found that it was desirable to bring groups of collections to an intermediate level of access together rather than moving each one individually from no access to full access. In this way we were able to carve out limited “projects” that could be assigned to less skilled or temporary workers. Research or practical limitations or implications There are some concerns and practical limitations that must be confronted in such a project. The researcher will find more collections described but not to as great a level of detail. The curator must be prepared to give up some intellectual control and admit to not knowing everything that is in the collection. Neither the researcher nor the curator may assume that the description is either perfect or complete. Conclusions When faced with a backlog, don’t try to do the same things faster, do different things. The problem of how to make archival collections accessible quickly without any increase in resources is not a simple one, and there is no one answer to it. The advancement of machine-searching is opening up possibilities, but it is necessary to abandon preconceptions to make best use of them. Originality and value The concept of breaking out of traditional arrangement and description patterns was broached by Mark Greene and Dennis Meissner in “More Product, Less Process: Revamping Traditional Archival Thinking,” (American Archivist, Fall/Winter 2005). Greene was director of the American Heritage Center when the article appeared. MPLP (as the concept became known) was officially adopted by the National Historical Publications and Records Commission, which is responsible for issuing large grants to assist American archival repositories. The article, however, was a license to experiment not a roadmap. Specific ideas vary according to the needs and resources of different repositories. The projects described here were conceived, under Greene’s direction, by his staff and continued after his retirement in 2015.
Biography
- Claudia Thompson joined the American Heritage Center, University of Wyoming, in 1984. She has managed the Arrangement & Description Department since 1997. Thompson co-authored “But You Promised: A Case Study of Deaccessioning at the American Heritage Center, University of Wyoming” in The American Archivist (Fall/Winter 2010).
Tuesday, 1 August 2017 – Parallel 4: Presenting & Using Assessment Data
Author(s)
Ms. Kay Grieves (University of Sunderland)
Title
Generating bespoke value and impact evidence to inform a thought-leadership approach to service engagement
Abstract
Purpose
Previous papers delivered at the 10th and 11th Conferences on Performance Measurement, shared our approach to developing a new service and performance model at Sunderland. That model and is now fully embedded within our Library and wider Student Journey Service where it underpins our increasingly evidence-based service culture, design and delivery.
In this paper we will share how our maturing model is enabling us to generate evidence and articulate the insights we draw from that evidence to inform and underpin our strategic service planning, resourcing and reporting. We will demonstrate how impactful articulation of these insights through data-visualization is enabling us to employ thought-leadership in our relationship management with stakeholders, by increasing their understanding of the benefit of engagement with our service offers and demonstrating value for money and the value and impact of our role in achieving institutional objectives.
Our paper will give an overview of the key techniques of our model and will then demonstrate its practical applications using the following examples:
- How our model is underpinning our approaches to thought leadership in relationship management by enabling us to effectively generate and articulate evidence to inform strategic faculty action plans.
- How our model has enabled us to develop a new graphical approach to annual reporting. By combining the variety of data-sets generated by the model we are able to articulate the outputs and impacts of cross-service holistic service offers and to clearly demonstrate how they fulfil institutional strategic objectives.
Design, methodology or approach
Our model is self-designed and continually evolving to meet current service need. We will demonstrate the key methodologies which influence our model:
- Strategic marketing theory: provided us with the framework to re-focus service design around a customer-centric approach to benefit-based service offers which subsequently enable us to generate outcome-focused evidence
- AMOSHEE Toolkit (AMOSSHE, 2011): this toolkit is designed to help Student Services professionals undertake evaluations of value and impact. It informed our ‘rounded narrative’ approach to evidence generation
- Our snapshot approach to evidence capture which enables us to focus on and evidence engagement with current strategic priorities
- Our campaign approach to facilitating conversation with our customers in order to capture these ‘rounded narratives’ of evidence
- Our progress in using innovative data-visualization technique to create tailored and impactful narratives and insights from our analysis which increase the impact of our data
- Our commitment to a relationship management approach and the importance of agile and targeted evidence in informing strategic action planning
Findings
We will discuss our key findings including:
- The importance of embedding our model at the heart of our service culture – both in terms of involving staff in data generation and of developing an evidence- based culture of service planning
- The benefit of meaningful data, analysis and insights in helping to inform and underpin strategic conversations and relationship management
- The transferability of our model across service settings
- The agility of a snapshot approach in enabling us to evidence and inform current strategic service priorities
- The impact of a ‘rounded narrative’ technique in articulating powerful human insights which demonstrate engagement, impact and value
- The importance of creative data-visualization techniques in communicating our insights for maximum impact with our customers and stakeholders.
References AMOSSHE (n.d.). Value & Impact online toolkit. Retrieved from http://www.amosshe.org.uk/value-and-impact-toolkit.
Biography
Kay Grieves in Service Engagement and Impact Manager within the Student Journey Service at the University of Sunderland. She is a member of the Relationship Management in HE Steering Group and has a professional interest in strategic marketing and library performance measurement.
Author(s)
Mrs. Selena Killick (The Open University)
Title
Exploiting customer relationship management analytics to improve the student experience
Abstract
Purpose
The Open University is the UK’s largest academic institution dedicated to distance learning, with over 173,000 students. Library Services within the University provide students and staff with access to predominantly electronic information resources; digital and information literacy skills training and a 24/7 virtual enquiry service. In 2004 the University introduced an institutional wide customer relationship management (CRM) system to record all student enquiries which was subsequently introduced to manage Library enquiries in 2008. This wealth of knowledge on our customers has provided the Library with an opportunity to exploit management information to improve our service. In 2016 a change of management saw a change in strategic direction for the enquiry service, with a shift to provide proactive support based on customer insight. In order to achieve this objective the data capture and reporting processes for the CRM required an overhaul. All enquiries are categorised by a fixed taxonomy of topic areas. Established initially in 2008 with a few incremental improvements, the taxonomy was overdue a review. The list was long, ambiguous and in some cases repetitious. Initial research with the enquiries team back in 2015 found inconsistent practices and understanding on what each topic area meant. Data usage focussed primarily on meeting the service level agreement of enquiries answered within two working days of receipt. Wider opportunities to use the customer insight for service improvements had not been established.
Design, methodology or approach The initial focus for the CRM analytics was to identify the service resource needs. As this could be achieved with the existing data without any system adjustments the team set about analysing the data to predict the staffing levels needed. The team focused on the daily volume of enquiries received via email, webchat, telephone and in person to identify appropriate staffing levels. Each communication channel has a different average enquiry handling time. Daily resource requirements were estimated for a whole year to identify the staff resource required to operate the service on any given day. This was completed by August 2016, the results of which have led to improved resource alignment and service level agreements being consistently maintained. Following this work the team turned their attentions in September 2016 to improving the data capture and reporting processes. The team has three overall key aims:
- Identify common enquiries throughout the annual student journey and develop timely proactive push-communications and training.
- Use enquiry analytics to pinpoint spikes in queries from specific courses; using that insight to work with faculty to improve the learning experience.
- Exploit the customer data to enhance the overall student experience by using the insight with teams across the Library Service to improve our service delivery.
The team initially conducted a data needs analysis with key stakeholders. This included enquiry librarians responsible for developing push communications; academic liaison librarians responsible for managing the faculty relationships and service owners across the library. Guidance from the University team with overall responsibility for the CRM was sought to identify if and how CRM system could be used to extract the data needed by the stakeholders. The project team have also worked with the wider enquiries team to improve the taxonomy of topics used to categorise enquiries. Following and initial review by the team of the enquiry data they worked with metadata specialists to develop a taxonomy of subject areas. As these are separated into area and sub area the project team also conducted a card sorting exercise with the whole enquiries team to instil a common understanding. Working with the systems team, data from the CRM has been included in the library data visualisation tool Kibana to enable patterns in enquiries to be identified quickly and easily. Trends in enquiries by topic area can be visualised over a time series. Enquiries received by specific course cohorts can be graphed out over time to identify spikes in activity. The number and nature of enquiries can be reported to service owners on a regular basis.
Findings Work is ongoing to take the data analysis into proactive support and is scheduled to be completed before the conference. This includes the exploitation of the data analysis to develop proactive support which can be pushed out to students via our social media and training materials to increase our one-to-many support. The data will also be used to support faculty liaison conversations to instil the need to improve learning materials through embedded library tuition or improvements to existing research activities. Monthly reporting to service owners is being established to drive service improvements. Conclusions
Customer relationship management systems in libraries can be powerful tools to aid enquiry handling, but the real benefits come from exploiting this data in multifaceted ways to improve the student experience. This paper will discuss how the CRM data at the Open University Library has been exploited to the benefit of students.
Biography
Selena Killick is the Library Services Manager responsible for Student Engagement at the Open University Library. Her remit includes leading the strategy, planning and evaluation of a consistent student experience within Library Services. She has presented, published, and provided consultancy services on using assessment methods to improve library services.
Author(s)
Ms. Tania Alekson (Capilano University)
Title
Storytime by the numbers: Humanizing library assessment through dashboards and data stories
Abstract Purpose
What’s going on in your library? Does everyone know how you’re doing, metrics-wise? My paper will discuss the benefits of creating interactive dashboards to inform library staff on the current state of your library’s ongoing assessment measures and engage library users on the value of the library. By translating static numbers into dynamic visualizations and tailoring the information to your audience, you can tell stories with data to anyone with a stake in your library. While some libraries have taken the steps to craft a comprehensive assessment overview of library value, as well as areas where improvements can be identified, the integration of these plans into the culture of the department can be difficult. Often, only one or, at most, a handful of librarians and staff are aware of assessment projects and their role across the library. Additionally, mechanisms to report this information out to library users are often underused and inadequate to the task of communicating library value in ways that our patrons find meaningful. “Data has to be presented in a way that is relate-able; it has to be humanized” (Yau, in Davis, 2009). This paper will demonstrate how a dashboard or data story, created with data visualization software available for free on the web, can make the assessment plan, projects and results visible to everyone. By creating dashboards with specific audiences in mind, your library’s data can be “humanized” and fulfil its potential for engagement and positive change. Design, methodology or approach Practical in orientation and rooted in the available literature on data visualization, this paper will use examples to illustrate the various types of visualizations and dashboards that can make library data come alive. Success stories from experience will be shared as well as pitfalls encountered. The focus of the paper will be tailoring visualizations to data, and dashboards to intended audiences, with the guiding purposes of better decision-making, more impactful improvements and closer relationships to stakeholders. Findings As Hilary Davis pointed out in her 2009 paper, “Not Just Another Pretty Picture,” “Good data visualization can drive home a point quickly and have lingering impact. Data visualizations can help you see something that you hadn’t noticed before.” Whether this means showing your acquisitions team at a glance where you are with budget spending, demonstrating to administration the overwhelming pressure on your student-booked study rooms or bringing to life for your patrons their patterns of use in your spaces, data visualizations and dashboards have diverse potential applications. While previous barriers to creating professional-looking visualizations and dashboards seemed insurmountable for all but the largest institutions, recent developments have made the necessary tools available and usable. With only a nascent understanding of data and visualization good practices, library professionals can begin to create impactful visualizations. With creativity and insight, these visualizations can become dashboards, stories and communications vehicles to interact with staff, administration and users. The expected outcome is higher staff and user engagement, as well as increased efficacy of assessment activity. Research or practical limitations or implications I am still in the early phases of introducing dashboards and data storytelling to myself and my library. In the six months between this proposal and the conference, I will be gathering more first-hand evidence and practical experience to augment my literature reviews and initial forays into this field. Conclusions Data is increasingly driving library operations and decision-making. However, data alone does not make improvements or communicate value – it takes human interaction with data to do this. By incorporating visualizations, dashboards and storytelling, and tailoring them to specific audiences, we can allow data to fulfil its potential power by revealing underlying truths and giving us the ability to create a better, more valued library. Originality and value While data visualization has been a topic of interest at conferences and in the literature for a little while now, there seems to be little support to take interested library professionals beyond one-off data visualizations and into the realm of dashboards and audience-targeted storytelling. This paper will attempt to build on current literature and my experience introducing these concepts into my own library over the past couple of years. While I have not conducted a survey of institutions, my discussions with other librarians have led me to the conclusion that very few libraries are creating dashboards to help their staff keep abreast of assessment goals and fewer are engaging their patrons in this manner. Given the advancements in usability of data visualization software, this would seem to derive from a combination of lack of awareness and inertia. This paper strives to overcome both of these factors to empower library professionals to make their data come alive. References Davis, H. (2009). Not Just Another Pretty Picture. In the Library with the Lead Pipe. Retrieved from http://www.inthelibrarywiththeleadpipe.org/2009/not-just-another-pretty-picture/.
Biography
Tania has maintained a strong focus on the user in her various roles since earning an MLIS in 2009, including Learning Commons Coordinator, Instruction Librarian and Student Experience Librarian.
Author(s)
Mrs. Elaine Sykes (Liverpool John Moores University),
Ms. Louise Makin (Liverpool John Moores University)
Title
Transforming data into insight: How marketing data can evaluate strategic and operational performance
Abstract
Purpose The focus of this paper is the establishment of a data driven marketing framework, to help evaluate performance and offer customer insight. The quest for demonstrable quality is an increasing topic on many library agendas. Quality is defined as ‘consistent conformance to customer’s expectations’ (Slack et al., 2010). Therefore, in order to measure quality, it is imperative to understand customer expectations, which is done through effective marketing. At Liverpool John Moores University (LJMU), the Library Services Management Team decided to create a bespoke marketing information system, to use its vast array of data to produce a regular marketing report. The aim of this presentation is to demonstrate performance levels, identify areas for development/investigation and to evaluate strategic initiatives for impact. Design, methodology or approach The methodology is based upon Smith and Raspin’s data to insight wheel (2008) and how best to use and organise data in order to develop customer insight. This considers how best to transform data to information, to create knowledge and insight in order to direct resources and planning to where it will have the most impact. A project team was formed of the Associate Director, the Team Leader for Business Control and two Library Services Managers to consider this more fully.
The LJMU approach took three main strands:
- Evaluation of Existing Position – An inventory of all performance measures (both qualitative and quantitative) was collected and classified accordingly. In addition, research interviews were carried out with several library managers to ascertain their perceptions of the existing data provision and how they used it to inform their decisions. The main findings from these interviews were that the department was lacking in combining its different service metrics in order to provide an overall picture of performance. In addition, that relative measures were, in general more useful than absolute figures. This meant considering data in relation to an appropriate frame of reference (e.g. past performance).
- Research – A focus group was held with the current Library Services Management Team, to ascertain which combination of the current performance measures would offer the best overall picture of current service levels. Prior to this focus group, the participants were forwarded copies of the latest NSS qualitative comments, and the list of existing performance measures. The former was to give insight into what aspects of existing service matter most to students. Participants were then requested to consider several aspects of service provision and any relevant data. This included services mentioned and not mentioned in the NSS comments, and data associated with each of these service offerings. Participants were then asked to identify those indicators which they believed to give the best overall picture of service.
- Design – Based on the findings of the focus group, the project team produced a draft report, using Power BI, a business intelligence tool, used for presenting data for easier interpretation. This combined those metrics identified as key by the focus group, to offer an overall picture of existing performance, and allow for connections between different service areas, in a more holistic way then has ever been attempted before.
Findings At the time of writing, this initiative is still in progress and therefore findings are limited. However, initial work has proved positive. A recent benchmarking exercise, using relative measures, has identified a couple of areas for service development, required as a priority.
Research or practical limitations or implications Although this particular initiative has mainly concentrated on quantitative data, it is important that qualitative data is not overlooked. Qualitative data is a necessary complement to any quantitative framework as it can often provide the ‘why’ to quantitative data’s ‘what’.
The framework is still very much in its infancy and therefore evaluation is still ongoing. There will be more evidence of its impact available by the time of the conference, however, it is not possible to give more detail at this point.
Conclusions This has been a very timely exercise for a number of reasons. Firstly, there has long been a desire by Library Managers to use data in a more systematic manner in order to assess service performance as a whole. In addition, it has provided a strong baseline by which to assess ongoing performance, including identifying potential strategic initiatives and evaluating their impact.
Finally, the University has recently requested that Library Services review its performance measures included in its monthly return to the Vice Chancellor’s office. This has offered a mechanism by which to ensure that the department really is providing the best possible service. It is anticipated that this is the first step in establishing a wider quality management framework which encompasses all aspects of library service, as well as a more data driven culture. Originality and value The work described in this paper is the culmination of using data to underpin a variety of business functions, including evaluation of strategy, operational performance and marketing. This paper would be of interest to those looking to assess overall library performance. It assists managers in their decision making, as well as identifying and evaluating strategic initiatives. Its methodology is widely replicable, and easy to implement, and therefore would have wide interest across different institutions.
References Slack, N., Chambers, S. and Johnston, R. (2010). Operations Management (6th Edn). Pearson: Harlow. Smith, B. and Raspin, P. (2008). Creating Market Insight: How Firms Create Value from Market Understanding. Wiley: Chichester.
Biography
Elaine Sykes is the Team Leader for Business Administration for Library Services at LJMU. A maths graduate, who is just completing an MBA, she has a strong interest in how data can be used to evaluate and improve services.
Louise Makin is a Library Services Manager at LJMU. A highly experienced librarian, Louise has departmental responsibility for marketing, and a strong professional interest in how marketing can be used to build excellent customer relationships.
Author(s)
Ms. Sue Thompson (California State University San Marcos)
Title
Building student support to facilitate change
Abstract
Purpose In fall 2016, California State University San Marcos (CSUSM) students voted to increase their own fees in order to help fund 24-hour study in the library. How did we get to the point that students were willing to spend their own money in support of the library? This presentation will describe assessment’s critical role in developing a strong relationship with our university’s students. Design, methodology or approach For over a decade, the campus-wide customer satisfaction survey showed students’ largest dissatisfaction with the library was its limited hours; but, little progress was made in solving this problem using conventional approaches. We changed our approach in 2013 to incorporate assessment evidence and strategic planning more effectively in decision making. In particular, we no longer saw assessment as a monolithic, linear process. We developed a strategy using multiple assessments and pilot studies combined with incremental improvements in an iterative process. However, we hadn’t anticipated how the new strategy would also enrich and deepen our relationship with our stakeholders — students in particular. Findings We have succeeded in increasing library hours by 30% in three incremental steps over the past three years. The campus-wide survey tracked our progress as, first, weekday hours and then weekend hours showed improved student satisfaction. However, the surprising side effect of our new approach was the increased involvement and buy-in from our students. The more we asked, the more students were interested in answering. We believe that this was largely due to making the library’s process to increase hours more visible to students, in part, by making changes shortly after asking for input and then communicating the changes in a variety of ways. The increased input from students made it clear that, while they were pleased hours were better, what they really wanted was 24-hour access – a goal that was still out of reach with the traditional funding available. At this point ‘buy-in’ from our students became literal. The university’s student government approached the library with a unique proposal to include funding for 24-hour library access in their proposed student fee increase. The student body showed its willingness to investment in the library by passing the fee increase in fall 2016. The library will begin 24-hour operation in fall 2017. Research or practical limitations or implications Our strategy of small, targeted assessments and pilot studies made it easier for students to see the results of their input and feel a sense of participation, especially if the library made concrete changes shortly after asking for student suggestions. We believe this strategy helped demonstrate to students the sincerity of our efforts to listen to their concerns. It also helped us deal with the ultimate practical limitation to improvements in user services — funding. By developing closer relationships with key players, such as students, we were able to find alternative sources of support to move our objectives forward. Conclusions
We found that attempting to use a large, ‘complete’ assessment instrument in planning changes was too slow, unwieldy and limited in its ability to solicit student input. The more we asked questions and showed some results, even if only partial, the more students showed interest in participating in library assessment, volunteering their opinions, and, ultimately, in supporting the library. Originality and value We weren’t expecting students’ feeling of ownership to increase to the extent that they would voluntarily help fund our final goal of 24-hour access. Our strategy showed the value of using multiple follow-up surveys and pilot surveys, which, while less complete that a comprehensive survey, achieved the important goal of creating ‘dialog’ with students. Making and advertising rapid, incremental improvements based on student data may have also made the library’s progress appear more visible and contributed to the development of a positive working relationship with our students.
Biography
Sue Thompson holds a Masters of Library and Information Management degree from the University of Denver. She created the new position of Evidence-based Assessment and Planning Librarian at CSUSM in 2013 to provide leadership in fostering a culture of assessment in the library and coordinating development of an effective assessment program.
Tuesday, 1 August 2017 – Parallel 6: Culture & Organizational Issues
Author(s)
Dr. Julie Miller (Butler University)
Title
Managing transitions: Using William Bridges’s Transition Model and a change style assessment instrument to inform strategies and measure progress in organizational change management
Abstract
Purpose
As academic libraries redefine their services and roles within higher education, library leaders are charged to implement transformative changes. The biggest leadership challenge in effecting change is the human element: helping employees to embrace and implement the changes necessary for transformation. This paper describes the change management process used by Butler University Libraries in 2013-14 to migrate to a cloud-based integrated library system that streamlined workflows and drove reorganization. Design, methodology or approach
The library leadership team relied upon two tools to develop strategies for change management and to track progress in employee perceptions of and participation in implementing change: the book Managing Transitions: Making the Most of Change by William Bridges and Susan Bridges and the Ryan Change Style Assessment instrument. Managing Transitions applies the Transition Model of psychological response to change to organizational change management. The model presents three phases of transition: “Letting Go,” the “Neutral Zone,” and “A New Beginning.” The book provides managers with concrete strategies to support employees through these phases. The Ryan Change Style Assessment developed by The Ryan Group, Inc., is one of several instruments available to determine how an individual responds to change. A self-scoring instrument, the Ryan Change Style Assessment identifies four change styles based on scales in four dimensions: preferred role during change (lead-follow, emotional expression (low-high), orientation (people-task), and openness to change (excited-cautious). The results place each respondent into one of four change styles: Initiator, Collaborator, Protector, or Questioner. Supporting documentation provides information the needs and contributions of each style during change. While planning for the system migration, the library leadership team read Managing Transitions and discussed each chapter at biweekly meetings. This reading provided a shared vocabulary for discussing change. Also during the planning phase, all library employees completed the Ryan Change Style Assessment. The results were compiled into a 2 x 2 matrix graphic. Two months before the new cloud-based system went live, we shared the results of the Change Style Assessment at an all-staff meeting and discussed the implications of our overall mix of change styles. Subsequently, the leadership team implemented a pre- and post- system migration exercise to determine employee readiness for and integration into their roles following system migration and reorganization. The exercise used a graphic “map” of the three phases of transition from Managing Transitions. Employees were asked to place a dot color-coded to change style on the map to indicate their current position in the three phases of transition relative to the system migration. The pre-migration map became an essential tool for the leadership team in developing strategies for communication and decision-making. The transition strategies used by the leadership team during the change implementation phase included intentional over-communication of all issues related to the system migration, for both internal and external communications; ceremonial events to acknowledge critical milestones; and increased employee engagement in redefining their roles and position responsibilities. Three months after the system migration, the mapping exercise was repeated to assess progress toward employee integration into their roles using the new system. Findings
In the pre-migration map of staff readiness for the migration showed that ten of 21 employees (47%) placed themselves on the border between the “Neutral Zone” and “A New Beginning”—they were ready for change. Not surprisingly, three of them identified as “Collaborators” and three identified as “Initiators,” the two change styles of the Ryan Assessment that are excited by change. Seven of the employees (33%) placed themselves within the “Neutral Zone.” All four change styles were represented in this group. Four employees (20%)—two Protectors and two Questioners, the styles cautious about change—placed themselves inside or on the border of “Letting Go.” This initial map demonstrated that most of the employees with change styles open to change were ready, while those cautious about change were less likely to feel ready for change. The leadership team used these findings to be more intentional in addressing the concerns of those with these change styles by communicating detailed information about migration tasks, workflow changes as information became available, and to reassure employees that these changes would not result in elimination of positions. The post-migration map showed that twenty of 26 (77%) of library employees positioned themselves in or on the border of the “New Beginning” phase, three (11.5%) positioned themselves within the “Neutral Zone,” and three (11.5%) positioned themselves in the “Letting Go” phase. All of those who placed themselves in the “Neutral Zone” or “Letting Go” phase identified as Questioners or Protectors.
Research or practical limitations or implications
The purpose of the pre- and post- system migration maps was to be able to visualize, generally, staff perceptions of their readiness for the system migration by change style. We therefore made the mapping process anonymous, and we did not attempt to draw conclusions about individuals’ progress toward “The New Beginning” phase. Also, the library experienced some staff changes during the period between the creation of the two maps depicting staff “location” relative to the phases of transition; some individuals who participated in the pre-migration mapping exercise did not participate in the post-migration exercise, and vice versa. Conclusions
Used in combination, the model for successful organizational change presented in Managing Transitions and the Ryan Change Style Assessment instrument provided the leadership team at Butler University Libraries with an effective process for minimizing disruption and discord during a transformative system migration. The outcomes we had anticipated include: improved communication through a common vocabulary for change management; better understanding of and ability to leverage individual and collective change styles to improve morale; a smooth migration facilitated by better communication and improved morale; and increased capacity for new library services. The outcomes we did not anticipate include: improved decision-making through balanced change-style composition on teams; institutional recognition of the library organization as a resource for change management; and revitalization of the library organization’s image as a forward-looking, vital resource.
Biography
Julie Miller has held leadership positions in academic libraries since 1998. She has published in library journals including College & Undergraduate Libraries, Technical Services Quarterly, and Education Libraries, and has served on the ACRL New Publications Advisory Board. She is currently dean of libraries at Butler University in Indianapolis, Indiana.
Author(s)
Prof. Lorene Flanders (University of West Georgia),
Dr. Janet Pilcher (Studer Group),
Mr. Elijah Scott (Florida Academic Library Services Cooperative)
Title
Building a responsive, transformational workplace culture utilising the Evidence-Based Leadership framework
Abstract
Purpose The workplace of choice is characterized by motivated employees and built by purposeful leader engagement with employees. This paper will discuss best practices for building cultures of excellence and accountability within library organisations utilizing the Evidence-Based Leadership framework. As part of the framework, leadership teams define and execute aligned systems and processes to maximise organisational and individual performance. The paper will offer perspectives from a leader newly appointed to head a geographically and organisationally diverse cooperative and from a long-term leader of a university library. They will describe their methods for seeking honest and challenging input by encouraging employees to openly as well as privately comment on employee engagement survey findings and discuss the importance of developing action plans for workplace improvements. They will be supported by an expert who leads a team that works with schools and institutions of higher education to apply the Evidence-Based Leadership framework to create cultures of excellence and accountability. In particular, the team has focused on applying tools and tactics that foster a culture of service with its colleagues and stakeholders. Leaders use improvement data to engage in a dialogue with their teams; they gather input and collectively develop improvement actions to create supportive and engaging workplace environments. Design, methodology or approach Studer Education, a division of Studer Group, is a leading service provider focused on improving education and health care outcomes in organisations throughout the world. Studer Education’s Evidence-Based Leadership model helps leaders achieve results by coaching around a continuous improvement framework centered on best practices to help institutions create cultures of excellence. The Florida Academic Library Services Cooperative conducted its first employee engagement survey in 2016 just prior to the arrival of its new Executive Director. This survey is especially important in this organisation which was recently created through a merger of four previous organisations with very different cultures. The Executive Director will discuss the importance of obtaining a baseline for culture and performance through a staff survey, analysis and discussion of results, and developing and implementing workplace changes through this process. The University of West Georgia has conducted annual employee engagement surveys since 2014, with response rates as high as 90%. The university received the 2015 Award for Innovation and Excellence in Leadership Development and Diversity from the American Association of State Colleges and Universities for Engage West, a campus initiative to assess employee engagement and develop accountable leaders supportive of shared governance and co-leadership. The survey solicits employees’ perceptions concerning engagement, leadership, mission and goals, communication, campus climate, pay and benefits, and work/life balance. Supervisors are required to roll out survey findings with employees and to develop action plans to improve the workplace in collaboration with their employees. The Dean of Libraries will discuss the survey rollout process and describe the evolution of methods used to glean honest employee feedback through open discussions, small groups, and methods of providing private input. She will describe how the library staff collaborates to develop action plans and how leaders periodically monitor plans through reality checks conducted through the year. Findings In opening dialogue with employees to transform workplace culture, leaders must explore the organisation’s lowest survey scores or greatest challenges, and work with employees to identify strategies to improve the workplace. They must be willing to engage in difficult conversations and encourage employees to overcome any hesitancy and fear in communicating their concerns and ideas. Leaders must also discern what employees find most positive about the workplace, celebrate successes, and identify ways to build upon successes. These processes allow leaders and employees to engage in ongoing dialogue and planning to create a nimble and responsive workplace culture supportive of transformational change, with assessment of activities and initiatives built into an annual cycle. Research or practical limitations or implications Though survey data provides a useful baseline for establishing a continuous quality improvement loop, library leaders can open dialogue with employees and take steps to improve the workplace absent a formal survey. The authors will describe simple and effective methods for encouraging employee input as a means of improving organisational culture if the library is unable to conduct an anonymous survey. Conclusions Opening dialogue with employees is vital to establishing and maintaining a positive workplace culture. Leaders willing to listen and respond to challenging input and take steps to transform the workplace establish trust and empower their employees through fostering a culture where risk taking is accepted and change is anticipated. As libraries evolve their service and collections models and re-envision their facilities, an atmosphere of openness to ideas is vital to responding to changes in libraries and higher education delivery. Originality and value Changes in collection management, service provision, and facilities continue to press libraries to evolve to support the needs of teachers and learners. In order to transform their organisations in ways that respond to ongoing and disruptive change, library leaders must systematically engage stakeholders in identifying challenges and finding direction. The authors will describe communication strategies based on transparent and open processes to encourage dialogue and input. These methods can be utilised with or without the results of a formal survey of employee engagement. The paper will offer perspectives from library leaders who are at different junctures in their careers with their organisations. They will be accompanied by a former professor of research and evaluation methods who currently works in leadership development and who trained the library leaders and their managerial colleagues in communication techniques and in utilising a framework developed to support workforce engagement and continuous improvement in business and educational environments. The paper will explore the challenges, successes, and evolution of processes the authors have witnessed in their organisations’ efforts to improve workplace culture.
Biography
Lorene Flanders has served as Dean of Libraries at the University of West Georgia in Carrollton, Georgia since 2005. The university has over 13,000 students. In April, 2017, she will assume the position of Executive Director of University Libraries at the University of South Alabama in Mobile, Alabama.
Dr. Janet Pilcher, Managing Director/Senior Leader and creator of the education division of Studer Group, has a Ph.D. in Measurement and Evaluation. She previously served as Dean of the College of Professional Studies and professor of research and evaluation methods at the University of West Florida in Pensacola, Florida.
Elijah Scott joined Florida Academic Library Services Cooperative in 2016 as Executive Director. FALSC operates the automated library system for Florida’s public colleges and universities. Its forty institutions and 156 libraries serve more than 1.3 million individuals. He was previously Dean of Libraries at Georgia Highlands College in Rome, Georgia.
Author(s)
Prof. Charles Lowry (University of Maryland, College Park)
Title
ClimateQUAL: Advancing organizational health, leadership and diversity in the service of libraries — a preliminary report
Abstract
Purpose
In many ways, the ClimateQUAL® survey protocol (originally Organizational Climate and Diversity Assessment–OCDA©) provides the ultimate management tool for effective organizational adaptation by employing deep assessment of a library’s staff opinions to plumb the dimensions of climate and organizational culture important for a healthy organization in a library setting. It tests critical attitudes around 26 empirically validated scales/dimensions the ClimateQUAL survey measures including work attitudes, diversity climate, leadership and several other dimensions of library climate. Respondents indicate the extent to which they agree to items belonging to each of the measures on a scale of 1 to 7. This paper is a preliminary report about the findings of the ClimateQUAL Project that began in 2000 at the University of Maryland. The ClimateQUAL experience now spans more than 15 years and has been applied in three countries. Much has been written about the experience in the 54 libraries, detailing how they used the protocol (see http://www.climatequal.org/pub). These case studies are of great value in understanding the impact of ClimateQUAL and its value to adopters. With one exception, this paper will not discuss any of these case-type experiences. Instead, it will provide the results of the psychometric research for the ClimateQUAL project, results based on approximately 12,000 staff observations.
Design, methodology or approach The academic libraries that have applied the ClimateQUAL protocol are of varying sizes though the majority were research libraries that were members of the Association of Research Libraries. In 2000 at the University of Maryland Libraries, we were seeking to improve the organizational climate, particularly focused on diversity issues, not thinking about a multi-institutional and, now, international research project. In time, we came to believe that other academic libraries could benefit from our learning. And as the protocol matured, we began to see that the resultant large data set gave us an opportunity for deeper learning, particularly given the IT platform that was developed to support it at ARL after 2008. To date over 54 libraries in the US, Canada and the UK have administered the protocol (7 of them at least twice), resulting in nearly 12,000 individual staff observations. The paper will describe the procedure for evaluating the structure and psychometric properties of each of the scales. The survey protocol provides feedback that is grounded in baseline normative data from the libraries that have already participated using robust scales that describe key dimensions of organizational culture and health. By using these normative scales and institutional results effectively, significant improvements can be achieved.
Findings The key (and original) findings of ClimateQUAL research add measurably to the psychometric research on organization for management and psychology. They include: Results demonstrated empirically what has been intuitively known for a long time. A healthy organization is better able to fulfill its service mission. In short, the research provides empirical evidence for the connection between these organizational climate dimensions and customer satisfaction in a library setting. The results break new ground by developing a theory of the “healthy organization.” Results also indicated that conflict that exists within an organization does have a direct negative impact on customers (the reverse is also true). In short, organizational climate (that is health) has a direct and perceived impact on the service experienced by library customers. This is validated by comparing the ClimateQUAL results on service quality with those from the many participating libraries’ that had LibQUAL+® results. Another important finding was that the climate for surface diversity improves the way an organization operates—this was the first time that this had been shown empirically. Similarly, findings included a significant correlation between valuing diversity and the extent to which customers say they can get information. The ClimateQUAL research shows that the most effective techniques for remediation are not top-down, but those that engage the entire staff. Nonetheless, as other research has shown, leadership matters and matters greatly. The paper will discuss the ways in which leadership engages the organization and how it motivates, enables and marshals employees to fulfill organizational objectives. It will point to leadership styles that have a positive impact on the various library climates, including the climate for service. As a preliminary report, the paper will touch on several key topics briefly:
- The evolution of ClimateQUAL from its inception through its incorporation as a basic offering of the Association of Research StatsQUAL® Program
- The healthy organization—properties of ClimateQUAL scales
- Leadership matters—The ClimateQUAL case
- Organizational climate and customer service—the ClimateQUAL and LibQUAL+ connection
- Improvement Strategies and organizational change using ClimateQUAL
- A look at how longitudinal change leads to healthy environments
- Differences and equity: a reflective analysis of ClimateQUAL demographics and organizational climate
- ClimateQual in the UK: applying the protocol in a different cultural setting
Conclusions
It is the hope of the whole ClimateQUAL research team that that this paper and forthcoming publications will advance understanding in the field about the importance of systematic efforts to strengthen the organizational health of libraries, not as an end in itself but because a library’s culture and climate are the foundation for the ultimate purpose of what libraries do—serve the information needs of their publics. Thus far, we have ample evidence from the 54 libraries that have used ClimateQUAL that this goal is strongly supported with such efforts.
Biography
Charles B. Lowry, Ph.D , director of five academic and research libraries and the Association of Research Libraries. He edited Library Administration and Management, served as a column editor for The Journal of Academic Librarianship, is a founding editor of portal: Libraries and the Academy.
Tuesday, 1 August 2017 – Parallel 6: Space
Author(s)
Ms. Heather Scalf (University of Texas Arlington)
Title
Getting to know you: Using mixed methods to guide an organizational redesign
Abstract
Purpose With the arrival of a new Dean of the Libraries, the University of Texas Arlington (UTA) Libraries leadership team looked at the future of academic libraries and realized that we were at a critical juncture. The time was ripe for us to consider how we might succeed in embracing the challenges presented by the growing needs and expectations of 21st century students, and the challenges that they faced moving into the workforce. As a critical first step in our strategic planning and reorganization process, the Libraries used a mixed methods approach to develop our understanding of our local environment and the broader environment of higher education in which we function. In 2012, the Dean of the Libraries invited all library staff to participate in one or more of five task forces created to develop this foundational understanding. Over nine months, more than 75% of the Libraries’ staff participated in these teams. The groups were wide-ranging in scope, from evaluating hiring practices locally and nationally, benchmarking UTA Libraries against current and aspirational peer institutions to identify strategic opportunities, and evaluating current and emerging best practices related to subject liaison work. Two other groups engaged in a broad range of ethnographic studies. Design, methodology or approach After her arrival in July of 2012, the Dean began an ambitious project of inquiry to learn more about the environment that surrounded the UTA Libraries, both locally and nationally. After several months of setting the table and challenging the organization’s preconceptions and encouraging all staff to think differently about the work that we do, she asked for staff volunteers to participate in one or more wide ranging task forces. The five task forces created used a variety of methods, combining quantitative and qualitative techniques and local and national data. The Benchmarking Task force conducted a broad scale environmental scan of other academic libraries to identify best practices through analysis of formal and informal publications, white papers and library web sites. The Liaison Program Redesign Task force used benchmarking interviews to explore the practices of other institutions, after conducting a thorough literature review designed to help focus our inquiries. The Population Studies task force focused on learning more about our population focus groups and a variety of anthropological techniques. Using six different qualitative techniques, including charettes, focus groups, interviews and photo and GPS mapping diaries we endeavored to answer the research question “What are the academic and research behaviors and attitudes of students and faculty, including the factors that influence them?” As an extension of this work, the Library Use Task force, known to us as “Where’s Waldo,” used visual floor and furnishings maps of the Libraries’ facilities to document and then analyze usage patterns of library spaces across time. Finally, a group was charged with revising the Libraries professional development and travel policy, with an eye for making it more broadly available to all staff and for increasing levels of support for professional involvement and presentations rather than simple event attendance. One of the key elements in the composition of the task forces was that the leadership of each group was determined by the group itself. Although members of the Libraries’ Management Team participated in various task forces, none led the efforts, either formally or informally. While there were many challenges to the work of the groups as the projects developed, the resulting shared leadership and broad participation helped to create a better foundation for the changes yet to come in the organization once the work was completed. Findings When the work of these groups was summarized and then evaluated to identify the broad themes, we discovered that the ability to have a space that could be customized for their use, in an environment that they could loosely control, was a paramount concern for students. While they preferred to work at home, the level of distraction was not conducive to completing their academic work. Instead, a “homey” space elsewhere, which was both comfortable and flexible, was sought out. With an average student age of 29, many of UTA’s students attend evening classes, and come to the Libraries to study and do academic work before or after classes. Availability of both food and drink were high priorities for students, and most commented that healthier choices, self-service options, and extended hours for food service were also desired. These observations informed decision making moving forward, including the eventual reorganization of the entire Libraries’ staff around a new mission of creation, connection, exploration and innovation. The findings have also been used in conjunction with new data from our card swipe entry system at the Central Library to make successful arguments for an expansion of service offerings at the Libraries, including a Smart Market food vendor, an Einstein’s coffee and bagel shop with extended hours and new partnerships across campus designed to facilitate student success.
Conclusions The results of the assessments provided validation of what many library staff and leaders already believed about student behavior, but having specific data to share with the University administration about facility and service usage, especially in the late night hours, was priceless. But the greater success was realized as an organization by the participation of so many staff in the assessment process. There were no experts, just people who cared about what the Libraries had to offer and were committed to find out more about our students’ needs and how we could help them be successful. Participants learned about assessment tools and techniques, and everyone had skin in the game when it came time to document and share the work. This collaboration was instrumental in forming new relationships and in communicating the findings of the study broadly across the Libraries and the university.
Biography
Heather has a BA in Biology from Converse College and an MLIS from the University of North Texas. She has been with the UT Arlington Libraries since 2005 and has served in various leadership roles within the organization, in both public services and assessment.
Author(s)
Ms. Maggie Faber (University of Washington)
Title
Sustainable space assessment: From data collection to visualization and decision-making
Abstract
Purpose
Libraries need a tool for understanding observational space count data that’s as flexible as the spaces they’re trying to assess. Tabular results and interactive maps can provide a detailed window into library space at one point in time, but what happens when changes are made to the area in response to the assessment results? This short talk draws on examples from libraries at the University of Washington to compare techniques for tracking and comparing changes over time.
Design, methodology or approach
At Foster Business Library, a focus on reconfiguring one particular space led to detailed interactive maps that could be manually compared, while at the Bothell Campus Library, a focus on changes to seating capacity and types of seating led to a dashboard that featured changes over time and projections into the future over more detailed results about the present configuration. Findings and Conclusions
Based on these two case studies, the talk will provide attendees with a model for setting up sustainable, continuous assessment that adapts to changes to the space at the data collection point and at the point of visualization and analysis. The presentation will also highlight ways to interpret and query results to reveal insights into space usage. Attendees will learn about options for effectively visualizing space data as the spaces themselves change over time.
Biography
Maggie Faber is the Data Visualization and Analysis Librarian at UW Libraries. She helps analyze and communicate assessment results and libraries-related data with interactive dashboards and visualizations. Recent presentations include a space assessment poster at the Library Assessment Conference and a poster on survey questions to demonstrate impact at ACRL.
Author(s)
Ms. Susan Beatty (University of Calgary)
Title
Learning in informal library spaces: The value for students
Abstract
Purpose The purpose of this study was to determine the relationship between the design of informal learning spaces in an academic library and student learning behaviours. As academic libraries improve their spaces and create informal learning spaces, it is essential to consider the nature of the learning behaviours of the students in the spaces. Informal spaces are the spaces in the library not normally used for classroom instruction. Through improved understanding of learner behaviours comes purposeful design of informal learning spaces in libraries. Design, methodology or approach This qualitative study is based on semi-structured interviews conducted with 21 students. Students were asked to discuss how, where and why they learn in the various informal spaces in the library. They also commented on various informal spaces in the library and discussed the features that related to learning. Findings 1. Space design elements Students were very forthcoming on the affordances that support their learning. The most frequently mentioned elements were:
-
- Sound and lighting – Environmental factors played strongly in their choice of space, with sound levels being the feature most commented on. While most preferred silent or quiet areas there were those students who preferred either low level ambient noise or noisy spaces.
- Distraction – Students judged a space as distraction free or having distractions, with either being desirable for various learners. Distractions could be people, traffic, reoccurring noise, smell, view, or other elements.
- Openness – Openness describes a space that is not isolated or confined. If it was too open, with too many distractions it would not appeal to some.
- Alone or together – Students commented on the value of having other learners nearby, to provide an opportunity to interact or to assist with learning, or to act as a motivator.
- Outlets – Proximity to outlets was a possible determinant, but not a deal breaker. If other elements were in place, the student would prefer their “best” space over others.
- Value of library learning spaces Students reported that they came to the library to accomplish their learning goals. The values (functionality, atmosphere, opportunity for space ownership and control, and academic symbol) that they ascribed to the library enabled them to be efficient and effective as academic learners. Research or practical limitations or implications
Limitations: This is a small study. Students self-selected based on a recruiting poster which simply called for students who learn in the library. While there was screening to capture as broad a demographic as possible, there was no attempt to recruit a representative sample. This was a study of informal learning spaces in one academic library. Further investigation of other students using other library spaces as well as other informal learning spaces across campus is needed.
Implications: This study identified various elements in space design that should be considered in combination when designing learning spaces. It is an exploratory study. Further research on the relationship between learning preferences/styles/behaviours and space is needed to increase understanding of how and why students use the library as a place for learning. In addition, there is value in talking to students about how they learn and conduct themselves in informal spaces. It informs facility decision making while validating the library as an essential part of the academy. Conclusions
A significant outcome of this study was uncovering the concept of comfort and its relationship to learning. Students commented on the need to be comfortable so that they could learn. Comfort to them meant physical, emotional and mental comfort that created a feeling of openness and readiness to learn. The elements identified in the study combined for each student in a specific “right way” so that each was ready to learn and to accomplish their goals. For the designer who is looking for hints the most likely best learning space for students is the space that looks like it offers the opportunity for them to be successful learners. They know it when they see it. The purpose and features of the space should be obvious. Make sure there is enough desk space, vary the table arrangement and size to allow for privacy or for space to be with others. Have the nature of the space be obvious – a place for quiet learning or a space for conversation. Transitional spaces should be in place and furnishings should be flexible. The appeal of various spatial elements is unique to each individual but the values underlying the choices are universal. It is not practical to make each space unique, the learner will do that, but it is desirable to talk to the learner to make sure you are doing it right. Originality and value
This study focused on student learning in informal spaces in the library. Throughout the study it became more apparent that not only had no one talked to the students about the way they learn in the library and what they think about the relationship of space to learning. It was also apparent that they had never really thought about it themselves. To them their choice of learning space was just natural. The results show that there is a need for further conversations and investigations into how students learn, where they learn and how those elements feed into designing informal learning spaces in libraries and elsewhere. As more academic institutions create informal learning spaces across campus they will need to understand the relationship between learning, space and learner in order to create purposeful, comfortable spaces for learning.
Biography
Susan Beatty is a Librarian at the Taylor Family Digital Library, University of Calgary. Her recent research is on the relationship between student learning behaviours and learning space design. Susan has presented at national and international conferences on various aspects of library leadership in learning.
Author(s)
Dr. Holt Zaugg (BYU),
Ms. Melissa Warr (BYU)
Title
Integrating a creativity, innovation, and design studio space within an academic library
Abstract
Purpose This presentation describes the efforts to set up a Creativity, Innovation, and Design (CID) studio within an academic library. The presentation will describe the reasons for creating a CID studio, assessment of the pilot study, and next steps. Design, methodology or approach The assessment used graduate students to conduct surveys, interviews, focus groups and observations of students and faculty to determine how well the CID fit into the library. Findings Initial findings indicate that the CID studio is a good fit within the library space as it supports, collaboration, discovery, and integration of library services. However, noise issues, equipment needs, and expansion of space are key future needs. Research or practical limitations or implications As libraries move from simple repositories of information to places of learning and collaboration, CID studio spaces provide an opportunity to integrate learning opportunities with library services.
Conclusions Through the first iteration, the CID has a unique and purposeful place within an academic library. It provides the opportunity for greater integration of library services. However, future iterations need to address key issues of space, equipment, and noise.
Biography
Holt Zaugg completed his PhD in Educational Inquiry, Measurement, and Evaluation from BYU. He is currently the Assessment Librarian at the Harold B. Lee Library BYU, where he designs, conducts, analyzes, and disseminates findings from a wide variety of evaluations including collaborations with other faculty and students on campus.
Author(s)
Ms. Marit Bull Enger (UiT The Arctic University of Norway)
Title
Space: Making the University Library the best place
Abstract
Purpose This text is a part of an ongoing study on how to improve the library space for the users at the University Library at UiT The Arctic University of Norway (UiT). UiT is the northernmost university in Norway, previously located only in the city of Tromsø. Since 2009, UiT has merged three times, the last time in January 2016. Today the University Library at UiT includes 11 libraries in five towns, spread over three counties. This project concerns UiT’s largest library the library of Culture and Social Science. Since March 2017, the ground floor in the Library has been going through a larger renovation, due to be finished in July. In the planning process, different departments and students were involved, to insure user input. During planning and preparing the remodeling, some issues came up. Such as how do the students use the Library, and how can we make the space more attractive for them? Can we make the library in to a Social arena on Campus? Maybe it is already, if so, we need data to corroborate this. We would like to investigate further to perhaps be more prepared in the event of future renovations of the library building. A study like this could be useful in day-to-day management and development of the library space. It will also be beneficial for other libraries at UiT. Design, methodology or approach; Findings
How do we gather the best data to enable us to create the best space possible for our users? In December and January, the University Library at UiT ran a user survey. The analysis of the data is still in progress but so far, the study of the survey shows us, not surprisingly, that in general our users are quite satisfied with the University Library. The main issues mentioned are air quality, group rooms and noise. This shows the necessity for better tools to map and assess user behaviour. At the Library of Culture and Social science, we are about to start “Tracking the Traffic” (TTT). This method systematically collects quantitative data about student behaviour in the library. The library staff will, several times a day, during a period, track what the students do at the library. Professor Tord Høivik at Oslo and Akershus University College of Applied Science developed the method. At the same time, we are going to carry out interviews with our users about their experience of the library. Conclusions Will the analysis of the survey, the TTT and the interviews provide the information we want? We hope that tracking the students and talking with them will give us a clearer picture of how they experience the library. This will in turn enable us to create a better workspace and environment for the students at UiT.
Biography
I am an art historian and work as a Librarian at The University Library, Library of Culture and Social Science, UiT the Arctic University of Norway.
Wednesday, 2 August 2017 – Parallel 7: Archives and National Libraries
Author(s)
Ms. Nancy Bell (Northumbria University),
Prof. Michael Moss (Northumbria University),
Dr. David Thomas (Northumbria University),
Prof. Julie Mcleod (Northumbria University)
Title
Developing an evidence based culture for documentary heritage collections
Abstract
Purpose This paper will present the main findings of a 12- month research project supported by The National Archives UK and Northumbria University’s I-School. The overarching aim of this study is to understand better the value of documentary heritage collections in an increasingly networked society where content is accessed and used by an increasingly diverse community of users. This transformation is blurring the boundaries between collections and user interfaces, and challenging the notion of a ‘collection’ as an intrinsically valuable physical asset, a place of learning and memory. To build resilience for a sustainable future, understanding how these collections are valued and the benefits they provide socially and economically is essential to building a critical evidence base. This paper will report the findings of our research broadly and will highlight existing value added services captured in a value chain model. Particular emphasis will be given to a proposed approach to building an evidence based culture in the sector, including but not limited to:
- an evaluation of a values took kit by sector professionals;
- ‘advocacy evidence’ fact sheets to support professionals in advocating on behalf of the sector;
- referencing critical research gaps, as a basis for an evidence research strategy for the documentary heritage sector.
Design, methodology or approach: The findings of this study build on existing methodological literature across disciplines, think-tank reports over the last 30 years. In addition, semi-structured interviews with professionals representing the documentary heritage domain were undertaken, and testing existing valuation models with user groups, and ‘advocacy packages’ with different user groups. Findings The key findings of this study underlined the conspicuous absence of documentary heritage collections from government funded evaluation studies undertaken over the last 30 years, a dearth of published evidence demonstrating the economic or social value of documentary heritage collections. The reasons for this are various. Archives and libraries have different missions from museums, different funders and the absence of critical data to support advocacy, to change practice or defend budgets. What strongly emerged from this study is a critical need to build an ‘evidence based culture’ in the archive and library sector, to develop skills and knowledge to effectively capture critical evidence to improve performance, and the need for research to develop nuanced methods that capture the value of these collections economically and socially. Originality and value
Extensive studies of the cultural heritage sector, including the movable and immovable heritage have been subject to significant evaluation over the last 30 years. Archives and libraries are noticeably absent and this must be redressed.
Biography
Nancy Bell is currently a Visiting Scholar at Northumbria University, I-School, and is charged with undertaking a 12-month project supported by The National Archives, UK, to understand the value of documentary heritage collections. Prior to taking up this role she was Head of Collection Care for The National Archives.
Author(s)
Dr. Roswitha Poll (University of Münster)
Title
Immediate influence or future benefit? The impact of national libraries
Abstract
Purpose The paper describes the actual and potential impact of national libraries: the various forms of influence, the target groups, and methods for assessing such impact. Design, methodology or approach In 2009, ISO published the Technical Report 28118, where for the first time specific performance indicators were described for national libraries. This Report is now in revision. But national libraries are also extremely interested in specific methods for demonstrating their relevance and impact. The new standard for assessing impact of libraries (ISO 16439) does not sufficiently consider the characteristics of national libraries. Therefore, in 2016 ISO has started the project of a standard combining both performance measurement and impact assessment for national libraries. In order to find adequate methods for impact assessment, it was necessary first to define the core functions of national libraries and to identify the main groups or institutions that will benefit from their activities. This was especially important, as national libraries, unlike most other libraries, have no defined clientele. Findings National libraries are unique institutions in their countries, and their tasks and activities are not easily comparable. Their influence and role depends very much on the social, political, legal and cultural environment. The emphasis on collections is of course stronger than in other libraries, but the focus is turning on services and users. The range of target populations is widening, especially by national libraries taking up educational services. In spite of the great variety in national libraries, it is possible to find methods for identifying the impact of national libraries on their different target groups and on society.
Research or practical limitations or implications There are not many publications available that deal with the influence of national libraries. Surveys are not yet as common as in academic or public libraries and if applied, have often not been published. Research has therefore mainly to rely on unpublished materials of national libraries. Conclusions The value of national libraries has usually been seen as consisting in the collection of the national documentary heritage. Looking at the various target groups and at the current functions of national libraries, a much broader range of benefits becomes visible. Application of impact assessment methods will help national libraries to prove as well their immediate as their future value. Originality and value The paper identifies impact of national libraries and adequate methods of impact assessment for this important library type. This first overview of a new topic will be of interest for all libraries with national or regional functions.
Biography
Dr. Roswitha Poll was chief librarian of Münster University Library from 1987 to 2004 and now works in projects dealing with evaluation of libraries and information systems. She is member of several ISO working groups (statistics, performance and impact assessment, quality of museums) and chairs the group for national libraries.
Author(s)
Ms. Wanda Dole (Archives & Records Management Committee, Episcopal Diocese of Chicago)
Title
Inventory and survey of archival collections in the Episcopal Diocese of Chicago: First steps in developing a plan and training materials
Abstract
Purpose This paper reports the results of a survey and inventory of the archival collections of parishes and agencies of the Episcopal Diocese of Chicago. The study replicates the 1975 survey conducted by Roderic B. Dibbert, then Historiographer of the Diocese, to identify the existence and location of valuable historical records, encourage improvement in the keeping of current records and enable the Historiographer to assist local churches and agencies in the creation and preservation of church records and increase of the heritage of the parish and Diocese as revealed in historic documents. The Archives of the Diocese holds materials about the Protestant Episcopal Church (now the Episcopal Church) in America’s beginnings in the Midwest, from the start of the 19th century. Organized in 1835 as the Episcopal Diocese of Illinois (and comprising the entire state of Illinois) and reorganized as the Diocese of Chicago in 1877, the Diocese established its Archives under the direction of Historiographer Samuel Chase in 1871. The collections reflect the settlement of the church and society in rural, urban and suburban settings. The Chicago collections for Chicago include documents from before and after the Chicago Fire and are especially strong on the Civil War. The Archives houses parish records for parishes and missions in Illinois since the 1830s. Materials include records of baptisms, confirmation, marriage, death; parochial reports; vestry minutes, correspondence; biographical material; historical material; photographs and architectural records. Like other dioceses and denominations, the Diocese of Chicago has seen a decline in membership and closing of parish churches. When Dibbert surveyed the Diocese in 1975 there were 150 parishes, 40 of which responded to his survey. The Diocese now contains 137 congregations of which 36 are missions. The aims of the 2017 replication of Dibbert’s survey and inventory are to: Establish a list of currently viable parishes and agencies, Identify the state of archival records, Identify individuals responsible for establishing and maintaining archival records, Identify parishes and agencies “in distress” and arrange for transfer of their records to the Diocesan Archives, Develop a simple workbook or guidelines for the Historiographer to share with parishes in need of training. Design, methodology or approach This paper replicates a 1975 survey devised to gather information on the parishes and agencies with archival records. The results of the 2017 survey are compared to those of the 1975 survey. Follow-up communications and visits to parishes are reported. The paper discusses the current state of church/religious archives in America and reviews the literature. Findings This is the first step in developing an action plan for the Historiographer and the Archives and Records Management Committee. Next steps include a training program to assist parishes in maintaining their archives. Preliminary findings indicate that there are parishes no longer capable of maintain their archival collections. A timetable and plan for transferring these collections to the Diocesan Archives will be developed. Research or practical limitations or implications This paper reports the data from one religious archive (special library). The results of the study of the Diocese of Chicago Archives may or may not be generalizable to all religious archives or special libraries. Originality and value The last study of the Diocesan archival collections was conducted in 1975. Since then there have been seven Historiographers. Clergy and congregations in the parishes have also experienced turn-over; many are not aware of the parish’s responsibility to establish and maintain historical records. In many ways, this is similar to the challenges that academic archives face in trying to establish and enforce records retention schedules for university departments and offices in flux. Preservation of historical records and providing access to information are universal core values for librarians and archivists. This paper provides a case study of one religious archives’ experience identifying collections historical records and training needs of those responsible for the collections. It provides guidance for other archives in developing plans and training materials.
Biography
Wanda Dole is Emeritus Dean of the Library, University of Arkansas at Little Rock and Professor. She held administrative positions at several academic research libraries and currently chairs the Archives and Records Management Committee of the Archives of the Episcopal Diocese of Chicago. She is an independent consultant and researcher.
Wednesday, 2 August 2017 – Parallel 7: From a different perspective: Research student presentations
Author(s)
Mr. Michael Rawls (Virginia Commonwealth University)
Title
Evaluation by contrast: Exploring the impact of libraries in both high- and low-productivity research environments
Abstract
Purpose This proposal is submitted under the “from a differing perspective” strand, as it represents a library-related portion of a broader PhD dissertation. The overall purpose of the dissertation is to understand how universities can best support the research productivity of their faculty. In approaching this topic, it is crucial to examine how investment in library resources and services factors into fostering a high-performance research environment. Focusing on the discipline of biomedical engineering (BME), this study analyzes how various library measures and other institutional factors correlate with the quantity and quality of scholarly output generated by BME programs at research-intensive US doctoral institutions. Using quantitative methods, these programs will be ranked in terms of their productive efficiency for the purpose of identifying a handful of programs at the upper and lower extremes of the rankings. These contrasting subsets of BME programs then form the basis of a comparative analysis – using both quantitative and qualitative methods – designed to illustrate which institutional factors most distinguish high productivity programs from their less effective peers, with a particular focus on the library characteristics that surround programs at each end of the spectrum. Design, methodology or approach Borrowing from the field of production theory in economics and, more specifically, the concept of total factor productivity, this methodology is designed to calculate how much high-quality scholarship a BME program ought to produce given it resources (e.g., level of grant funding, departmental faculty count, teaching loads, etc.). The true purpose is to identify which programs are actually producing more scholarship than should be expected, with the assumption that those who exceed expectations by the greatest margin have achieved the highest productive efficiency in the field. To begin, the number of peer reviewed articles produced by each BME program from 2013 to 2016 are aggregated and weighted by journal impact factor to serve as a dependent variable measuring the quantity and quality of each program’s scholarly output. Using OLS regression analysis, this number is then regressed against an array of programmatic and institutional inputs presumed to influence scholarly output, including a variety of library expenditure and staffing levels measures. The resulting regression equation describes the relationship between the inputs of the research process and the scholarly output for the field of biomedical engineering. Using this equation, it is possible to calculate the expected amount of scholarship that each BME program should have been able to produce given it inputs. As state above, the difference between this expected value and actual output forms the basis for establishing each program’s productive efficiency – with those who out produce expectations being considered high productivity and those producing below expectations being considered low productivity – enabling the comparative analysis. Findings The quantitative analysis for this study is currently in progress, but it is scheduled to be completed well in advance of the conference. The intent is to divide the presentation into three equal segments. The first segment will profile libraries associated with low productivity research environments. This will be contrasted with second segment that examines library characteristics associated with high productivity environments. The primary focus of these segments will be to discuss correlations between various library variables and scholarly productivity, as well as to define any cross-institutional trends that characterize libraries associated with either environment. The final segment will consist of the presentation of a qualitative framework designed to understand how researchers in both types of environments experience and interact with their libraries. Relying on group interviews with the researchers themselves, the assumption is that more can be learned by talking to researchers in these opposite and contrasting environments that could be learned by simply interviewing researchers in general. The qualitative portion of the dissertation will not be completed by the time of the conference, but qualitative framework has been designed and will be discussed in detail. Note: no identifiable information associated with any institution or program will be disclosed. Conclusions As mentioned above, the analysis is scheduled to be completed later this spring and conclusions will be developed accordingly. Originality and value At its most basic level, this study attempts demonstrate library value empirically by correlating library investment with scholarly productivity – although it remains to be seen whether the correlations will prove to be positive and statistically significant. Nonetheless, it is important to make this attempt simply because supporting scholarly research is so central a function of university libraries. The approach of studying both high- and low-productivity environments as a means of further illustrating the role the libraries can play in the advancing research is possibly unique. This, too, is intended to make manifest the value that a well-resourced library offers to its research community. The most fundamental reason for conducting this empirical analysis, however, is simply to set the stage for the qualitative investigation that will follow. Identifying high- and low-productivity BME programs will enable the collection of valuable testimonies, stories, and anecdotes from researchers who occupy drastically different environments about their interactions with their libraries. Although the qualitative study will unfortunately not be completed in time for the conference, it is hoped that sharing its qualitative framework will spur discussion about what and how we might learn by engaging our research community.
Biography
The author has worked for Virginia Commonwealth University for over ten years. In addition to contributing the field of library assessment, he is PhD candidate in public policy and administration concentrating in higher education administration.
Author(s)
Ms. Laetitia Louw (Aberystwyth University),
Prof. David Ellis (Aberystwyth University)
Title
Electronic information usage in a social sciences research organisation: A value assessment of library services and resources
Abstract
Purpose
An investigation was conducted to assess and evaluate the uptake of information resources subscribed to by the Library and Information Services (LIS) of the Human Sciences Research Council (HSRC), the national social sciences research institution of South Africa. Design, methodology or approach
Following a mixed methodology, this study illustrated the percentage uptake of resources through a quantitative analysis of the citations as reflected in the reference lists of the research output of the HSRC. It further investigated the information seeking behaviour of the researchers and authors of the research output, through a qualitative interview study, to enable an understanding of the information seeking processes of the researchers in the HSRC. It further endeavoured to advise a possible new strategy for information literacy training, as well as for the role of the Liaison Librarians of the organisation [1]. Findings
The results of the citation analysis revealed a substantial uptake of the electronic journals, but a relatively low uptake of the books (including electronic books), which was consistent with the character of social science researchers as depicted by previous research. The results of the qualitative information seeking behaviour investigation revealed a high level of proficiency in the use of electronic resources, irrespective of rank (or age) in the organisation. The study found that participants in the interview study, who used electronic tools and resources such as bibliographic management tools and electronic databases, have received some form of information literacy training. Barriers in the use of LIS subscribed resources were predominantly due to the fact that some researchers, with affiliations to other institutions such as Universities, preferred the bigger platforms offered by such institutions. Research or practical limitations or implications
A challenge experienced during the pilot phase of the quantitative analysis was that certain types of publications proved to be unsuitable for the study, due to the fact that they did not consistently contain References. These research outputs, therefore, had to be excluded from the population before sampling could commence. The small group of 11 researchers who were sampled for the qualitative investigation, presented another challenge for this study: age-related differences in information seeking characteristics couldn’t be generalised to the full body of the population. The Author could therefore, report on the result of the individual opinions of the participants as presented in the interviews only, but no generalisation in this respect was possible. Conclusions
This investigation proved a well-represented uptake of HSRC LIS resources as reflected in the reference lists of the organisation’s research output. An investment in the HSRC LIS presents value for money, as outlined in the result of the quantitative citation analysis, where uptake of the HSRC LIS journal collection proved to be 82.34%. The availability of HSRC LIS resources, therefore, contributed to the success of the HSRC in securing grant funding from the South African National Department of Science and Technology. Usage of HSRC LIS services and resources were demonstrated by the proved uptake of these resources in the research output collection. This was substantiated by the result of the qualitative interview study, confirming usage of the resources and services offered by the HSRC LIS. The information seeking characteristics of the researchers who were interviewed were consistent with the studies of previous research presented in the literature review. The information seeking behavior study also presented solid evidence of the use of the HSRC LIS Information Consultancy and Interlibrary Loan services and provided valuable ideas to develop the Information Literacy programme of the HSRC LIS in a more effective way. Without the expert knowledge by Library Professionals of electronic resources and the content thereof, knowledge of most effective business models to operate these systems on, knowledge about Interlibrary loan schemes worldwide, services, resources and tools to enable researchers to work optimally and function efficiently, a research output which adheres to all performance indicators successfully will not be achieved. [1.] The terms “Liaison Librarian” (or for purposes of the HSRC LIS) “Information Consultant” refers to what is commonly known in the LIS industry as Subject Librarian, Information Specialist or Reference Librarian.
Biography Laetitia Louw (MSc MLIS Aberystwyth University, UK) is a department head and Research support Librarian in the Human Sciences Research Council in SA. She deems regular impact assessment and service evaluation to be the key towards long term sustainability of Library and Information services in a continuously changing research environment.
Author(s)
Ms. Kristiina Tuomikoski (University of Oxford)
Title
Does the gravity model hold at micro-distances? Investigation into University of Oxford students’ library choices
Abstract
Purpose
The aim of this research is to find out the effects of distance on consumer choice. The analysis will use a unique dataset on university student visits to Bodleian libraries. Even though not always immediately obvious, distance is a key economic variable. For example understanding how consumers react to distance is vital for the location choice of businesses. In the economics literature distance is often examined in gravity modelling, which is generally used to describe trade flows as a relationship between sizes and distances of countries. This relationship is surprisingly stable across studies and suggests a distance elasticity of about -0.9. However, it is not clear whether the association holds at very short distances, as most of the gravity studies focus on country level data. Analysing visitor flows from Oxford colleges to Bodleian libraries allows estimating a gravity relationship at micro level avoiding any potential aggregation bias and will help determining the importance of distance for consumer choice of local amenities.
Design, methodology or approach This study will use standard econometric techniques for analysis. Fixed effects estimation will be used to model the visitor gravity between colleges and libraries. Further, a probit model will be used to investigate individual students’ choice of libraries. Preliminary findings suggest a negative, statistically significant effect of distance on library visits.
Conclusions This research has the opportunity to provide more insight on economic interactions at short distances. It will also describe students’ library usage patterns that can be of interest to a wide academic community.
Biography
Kristiina is a University of Oxford student finalising her MPhil Economics degree this June. The research on Bodleian library visits constitutes her master’s thesis. Her research interests lie in international trade and spatial economics. Before Oxford she completed a BSc Economics degree at the University of Sussex.
Author(s)
Mr. Faiz Alotaibi (Manchester Metropolitan University),
Prof. Jennifer Rowley (Manchester Metropolitan University),
Dr. Frances Johnson (Manchester Metropolitan University)
Title
Reviewing the factors affecting the use of university digital resources
Abstract
Purpose
In recent years, universities have increasingly provided access to information resources in digital form, which offers remote, fast and flexible access with good quality search facilities (Thong et al., 2002). However, despite the significant investment in digital access to resources made by university libraries (Hong et al., 2002; Awwad & Al-Majali, 2015), there is evidence that such resources are under-utilised (Awwad and Al-Majali, 2015; Park et al., 2009) and further, that it varies according to user groups (Kim, 2010). In the last fifteen years there has been a growing body of research that examines the factors affecting user acceptance and use of e-libraries and a variety of types of digital libraries, conducted in various countries, including Tiawan, USA, Jordan, Hong Kong, Turkey and Saudi Arabia. This paper’s aim is to analyse a set of prior studies with a view to identifying: the theoretical stances adopted; and, the factors included in research models relating to e-library acceptance and use, as a basis for proposing a research model for a PhD study into the factors affecting the use of e-libraries in Saudi Arabia, by Masters students. Design, methodology or approach The articles to be analysed were identified through a thorough search of prior literature in refereed academic journals in information and computer science on the acceptance and use of e-libraries and digital libraries. Only studies adopting a quantitative approach and providing a research model were included. Findings Previous researchers adopt two distinct, but related theoretical frameworks to understand the acceptance of e-libraries: the Technology Acceptance Model (TAM) (Chen, et al., 2016; Park et al., 2009; Thong et al., 2002) and the Unified Theory of Acceptance and Use of Technology (UTAUT) (Al-Gahtani et al., 2007; Awwad & Al-Majali, 2015; Tosuntas et al., 2015) or UTAUT2 (Huang and Kao, 2015). A typical TAM model has one of ‘intention to use’ (Park et al., 2009; Thong et al., 2002), ‘attitude towards using’ or ‘usage behaviour’ (both in Chen et al., 2016) as the dependent variable. Leading into one or more of these dependent variables are ‘perceived ease of use’ and ‘perceived usefulness’, which in turn have influencing variables that differ between studies. Thong et al. (2002) have three clusters of influencing variables: Interface characteristics (terminology, screen design, navigation), Organizational context (relevance system accessibility, system visibility), and Individual differences (computer self-efficacy, computer experience, domain knowledge). Park et al. (2009) use some of the same variables, but without clustering them (experience in computer use, domain knowledge, visibility, accessibility, relevance), coupled with English literacy, interest in publishing and library assistance. Fang et al. (2016) develop a model that integrates TAM with the IS Success Model (ISSM), introducing variables such as service quality and user satisfaction. ‘Behaviour intention’ and ‘Use Behaviour’ are also the dependent variables in UTUAT. Independent variables are: performance expectancy (related to ‘usefulness’), effort expectancy (related to ‘ease of use’), social influence and facilitating conditions. All of these are included in all of the UTAUT studies reported here. The UTAUT model also incorporates the moderating effect of demographic factors on the relationship between the key factors in the underlying model, including gender, age, experience and voluntariness. Researchers tailor these to their research. For example, Awwad & Al-Majali (2015) use gender, age, experience, education level and academic discipline, whilst Huang and Kao (2015) use habit and price value. Huang and Kao also add ‘hedonic motivation’. Conclusions The proposed research model will be discussed in the paper. This model will be used to design a research study whose objectives are to identify the factors that influence Masters students’ acceptance and usage of e-library services at King Saud University in Saudi Arabia, as a basis for formulating recommendations supporting enhanced use of e-library services by Masters students. The two phase mixed methods study will include a survey to profile the situation, and interviews to gather deeper insights. References Al-Gahtani, S.S., Hubona, G.S., and Wang, J. (2007) Information technology (IT) in Saudi Arabia: culture and acceptance and use of IT. Information & Management, 44, 681-691.
Chen, J.-F, Cheng-Wan, J.-F. C., and Huang,K.Y.-M. (2016) Integrating ISSM into TAM to enhance digital library services: a case study of the Taiwan Digital Meta-Library. The Electronic Library, 34 (1), 58-73.
Choi, S. J. N. (2016) Understanding users continuance intention to use online library resources based on an extended expectation-confirmation model. The Electronic Library, 34 (40), 554-571.
Hong, W., Thong, J.Y.L., Wong W.M, and Tam, K.Y. (2002) Determinants of user acceptance of digital libraries: an empirical examination of individual differences and systems characteristics. Journal of Management Information Systems, 18 (3), 97-124.
Huang, C.-Y and Kao, Y.-S. (2015) UTAUT2 based predictions of factors influencing the technology acceptance of phablets by DNP. Mathematical Problems in Engineering, Article 603747.
Kim, Y.-M. (2010) The adoption of university library website resources: a multi-group analysis. Journal of the American Society and Information Science and Technology, 61 (5), 978-993.
Park, N., Roman, R., Lee, S., and Chung, J.E. (2009) User acceptance of a digital library systems in developing countries: an application of the Technology Acceptance Model. International Journal of Information Management, 29, 196-209.
Sulieman, M, Sawsan, A., and Al-Majali, M. (2015) Electronic library services acceptance and use: an empirical validation of unified theory of acceptance and use of technology. The Electronic Library, 33 (6), 1100-1120.
Thong, J.Y.L., Hong, W., and Tam, K.-Y. (2002) Understanding user acceptance of digital libraries: what are the roles of interface characteristics, organizational context and individual differences? International Journal of Human-Computer Studies, 57, 215-242.
Tosuntas, S.B., Karadag, E., and Orhan, S. (2015) The factors affecting acceptance and use of interactive whiteboard within the scope of FATIH project: a structural equation model based on the Unified Theory of acceptance and use of technology. Computers & Education, 81, 169-178.
Biography
Faiz is a PhD student at Manchester Metropolitan University. His research centres on the factors affecting the use of e-resources in university libraries.
Author(s)
Mr. Leo Appleton (Edinburgh Napier University)
Title
Using a longitudinal focus group methodology to measure the value and impact of public libraries
Abstract
Purpose
There is a general acceptance that public libraries contribute to ‘community’, and at least have the potential to have a very positive impact on civil society. (Varheim, 2007). This ‘impact’ is longitudinal, or ongoing, in that library users can realise the impact and value of libraries throughout their lives. Brophy suggests that “libraries are at the heart of social systems; they exist to serve the needs of people, to help them live, learn and develop and to act as part of the social glue which holds communities together” (Brophy, 2006, p. 3). However, whilst libraries perform this multi-functional role, reaching out to citizens and spanning across different types of community provision, the value of libraries, in times of austerity, appears to be under increased scrutiny by the local authorities running them. The purpose of this paper is to report on a PhD research project currently being carried out, which seeks to investigate and demonstrate the impact and value of the public library service on UK citizenship development. The paper will focus on the specific longitudinal qualitative methods used in order to ‘measure’ this impact and value.
The measurement of library performance to demonstrate value and impact has become a significant sub-discipline of library services management. Understanding library users, their demands and expectations is essential for identifying success criteria and impact indicators (Hernon & Altman, 2010, p.10). Similarly, social impact is frequently associated with public library services and the impact that they have on their communities and constituents (Kerslake & Kinnel, 1997, p.12). With this discussion in mind, two main research questions form the basis of enquiry for the project:
- To what extent is an individual’s position advantaged or disadvantaged as a result of using public libraries?
- What is the impact of using a public library service on individual and community citizenship?
Design, methodology or approach One way to address these questions is to talk to library users and ask them about how their library usage benefits them and has an impact on their development as active citizens. Library users at eight different public library locations across the UK took part and initial focus groups, with up to ten participants in each, were convened and carried out at each location during 2015-16 as the first phase of the empirical study. The sample included a mix of rural county councils, city councils and metropolitan borough councils. Participants discussed and shared their experiences of using public libraries and reflected upon how their library usage had affected them (understanding, knowledge, participation, lifestyle, citizenship, etc.) Having access to multiple participants (57 across the eight focus groups) has allowed for the research questions to be addressed, within an accessible and understandable conversation, allowing participants to freely discuss their experience of library usage. Studies suggest that longitudinal studies are effective ways of observing and evidencing social development (Lewis & McNaughton Nicholls, 2013, 61-62) and a fundamental part of this methodology is its longitudinal approach. Because of the nature of the themes being discussed, it is desirable to reconvene each focus group, with the same participants at set intervals during the period of empirical research. This enables participants to reflect back on their most recent library usage and to discuss its impact and value in a current and personal context. Reconvening focus groups in which the participants are familiar with each other, also allows for a deeper and more open discussion, which in turn enables deeper and richer data to be obtained. Therefore, a second round of focus groups (phase two) was completed during 2016 in which participants were asked to discuss and reflect specifically on their personal development and involvement in their communities during the previous eight months (the period of time in between focus groups), and whether any of this had been facilitated through their library use. A third and final round of focus groups, at the same locations, and with the same participants, is scheduled for Summer 2017. Findings Whilst the paper will focus largely on the methodology, there will also be an opportunity to discuss the findings of the first two rounds of focus groups. The main trends and themes identified so far include:
- A focus on the epistemic function of libraries;
- Print monographs being perceived of the main vehicle for the dissemination of knowledge and information;
- Community ownership of public libraries;
- Embedded digital citizenship within public library provision;
- Inclusive and accessible libraries allowing for capital to be generated and exchanged.
Research or practical limitations or implications There are limits in that the empirical study has only accessed eight public library authorities, which in effect is simply a sample with regard to UK active public library use. Conclusions This paper will discuss the advantages and limitations of a longitudinal focus group methodology in public library performance measurement research. The paper will also bring together the findings and analysis of the longitudinal focus groups and will discuss some of the ways in which public libraries demonstrate value and impact on the development of individual and community citizenship.
Originality and value This methodology is not widely used and the empirical study is now far enough on that the validity of the method and of the initial findings can be discussed. The paper and discussion will be of value to anyone with an interest in the use of longitudinal focus groups, as a qualitative method for measuring and demonstrating library performance. References Brophy, P. (2006). Measuring library performance: principles and techniques. London: Facet Publishing.
Hernon, P. and Altman, E. (2010). Assessing service quality: satisfying the expectations of library customers. Chicago: American Library Association.
Kerslake, E. and Kinnel, M. (1997). The social impact of public libraries: a literature review. London: British Library.
Lewis, J. and McNaughton Nicholls, C. (2014) “Design Issues” in J. Ritchie, J. Lewis, C. McNaughton Nicholls, and R. Ormston (Eds.) Qualitative Research in Practice: a Guide for Social Science Students and Researchers. Los Angeles, Sage.
Varheim, A. (2007) Social capital and public libraries: the need for research. Library and Information Science Research, 29, pp. 416-428.
Biography
Leo is also a doctoral candidate at Edinburgh Napier University, where he is a researcher at the Centre for Social Informatics and is part way through a longitudinal research study into the impact of public libraries on citizenship development in the UK.
Wednesday, 2 August 2017 – Parallel 8: Innovative Methods II
Author(s)
Dr. Andrew Asher (Indiana University),
Dr. Donna Lanclos (University of North Carolina, Charlotte)
Title
Mapping student days: Measuring library value through exploratory multi-sited ethnography
Abstract
Purpose Research on students’ educational experiences demonstrates the importance of a holistic understanding of the complexity of students’ lives to developing library programs, services, and resources that effectively address undergraduate needs. Investigating the local expression of student “taskscapes,” or the ensembles of interrelated social activities that take place across space and time, provides critical information about students’ lived experience. The “A Day in the Life Project” sought to understand these taskscapes at eight university campuses across the United States through a multi-sited ethnographic investigation of students everyday activities. By examining these taskscapes, this project was concerned not only with how students conduct work in the library, but also the library’s relationship with other spaces and places of academic work and the broader experiences of student life. While previous studies have examined similar issues at single universities, this study was unique because it was conceived and designed specifically to enable cross-institutional comparison and analysis of data from a cross-section of universities that were chosen to represent different types of higher education institutions and the diversity of the US student body. Design, methodology or approach The “A Day in the Life Project” developed an innovative mixed-methods approach to data collection that combined text message surveys delivered via students’ mobile telephones, geolocation, and qualitative ethnographic interviews. Participating students were asked to chart the course of their movements during an academic day by responding to periodic text messages with the places they visited and brief description of their purpose for going there or the activity they were participating in. The research team then created a map of each student’s day that was used to guide an interview about the student’s daily tasks and activities, the spaces and locations in which the student conducted academic research and day-to-day work, and the student’s overall educational experience. These interviews enabled the research team to build a grounded perspective about the variety of pressures and motivations that inform the choices students make about when, where, and how they structure and organize their academic lives and balance academic activities with professional, familial and personal needs.
Findings Both the mapping data and the qualitative interviews revealed strong patterns in students’ spatial experiences among the universities. These patterns indicated that a university’s location and setting had a strong effect on students’ educational experiences and practices, with three distinct groups emerging among the eight institutions: residential campuses, non-residential campuses in semi-urban locations, and non-residential campuses in highly urban locations. Nevertheless, students from all eight universities reported very similar distributions of both educational and non-educational activities. These results suggest that the tasks of student life are quite similar among students at all types of universities, but where and how these tasks get accomplished and the qualitative experience of these tasks vary significantly, and are profoundly affected by external spatial constraints and social obligations. These patterns also indicated the importance of developing library service models that meet student needs in ways that fit within these broader experiences and contexts. Conclusions Assessment approaches in higher education spaces such as libraries can tend to come from a perspective that frames students only through their identity as students. However, students are never just students, and our performance evaluation approaches need methods and analyses that make this clear and comprehensible. The holistic approach of the “A Day in the Life Project” helped us see the whole person and students’ multiple expressions of identity as they negotiated roles such as as a friend, employee, daughter, grandson, parent, sister, or cousin, in addition to student. All students, but especially commuter students at non-residential institutions, were frequently negotiating and navigating these identities throughout their day. The complexity of these identities meant students were constantly layering tasks: they were studying on the way to work or on the way to pick up their little sister from school. They were completing an assignment as they helped their child do her homework at the kitchen table. They were posting to a discussion on the learning management system while working at their job. They were responding to a text message from their child’s daycare while in class. Understanding the complexities and realities of these taskscapes are critical to understanding the needs and priorities of our students.
Originality and value The “A Day in the Life Project” makes an argument for the importance of open-ended assessment of student experiences and for bringing more holistic pictures of student life into conversations in libraries and higher education about supporting and facilitating student success. For this reason, this study purposefully chose to take an exploratory approach that did not focus on one part of the university, such as the library, but rather the webs of interrelated places and activities that comprise students everyday educational experiences and are both internal and external to their institutions. Moreover, because of its unique design, this study provides a rare opportunity for direct comparison of these experiences across multiple types of universities, allowing us to draw conclusions and make recommendations that might be broadly applied by institutional policy-makers. To this end, this paper will be particularly concerned with discussing next steps, and will address how we move from the results of this type of project to concrete intentions and actions.
Biography
Andrew Asher is an anthropologist and the Assessment Librarian at Indiana University, where he leads the libraries’ qualitative and quantitative assessment programs and conducts research on the information practices of students and faculty. Asher specializes in utilizing ethnography in libraries, and is the co-editor of College Libraries and Student Culture.
Donna is an anthropologist working to inform and change policy in higher education, in particular in and around libraries, learning spaces, and teaching and learning practices. She blogs about this and other work at donnalanclos.com, and you can also find her on Twitter @DonnaLanclos.
Author(s)
Mr. Michael Williams (Bodleian Libraries, University of Oxford)
Title
Using data to strategically deploy staffing resources
Abstract
Purpose Our environmentally controlled library warehouse stores 10 million collection items and its team of staff provides a delivery service to our students and researchers. Statistics were showing that the demand for the service was much higher than forecast at the design stage and, as a result, the operationally intensive environment had bottlenecks and backlogs which were affecting the service level agreements. It was clear that the staffing levels were inadequate to meet demand so we needed to capture data to enable evidence-based decision making to restructure and supplement staffing. Design, methodology or approach Each of the activities undertaken by staff was observed and timed over extended periods which resulted in detailed measurements for each elemental task. For batch activities, items were counted to provide averages. Based on known demand for services we were able to extrapolate these measurements to model the demand on services and, therefore, the staffing requirement for a whole year. Holiday and sickness was taken into account enabling us to demonstrate the number of staff required across the range of activities. Alternatives methods considered tracking the life cycle of requested items to gain an understanding of the time taken to complete a full cycle of the process to deliver an item to a reader and return it to the warehouse. With so many variables and the difficulty of obtaining data for items that had left the warehouse, this idea was not pursued. Findings We were able to provide evidence to show that the levels of staffing were inadequate and two further full-time equivalent (FTE) staff were required. The data also highlighted specific areas that required higher and lower levels of resourcing than were currently provided. Research or practical limitations or implications The evidence was used to make a convincing case to obtain additional resources to recruit two FTE staff. This was deployed as one full-time staff member and two part-timers as the evidence demonstrated that there was a specific time of the day (evenings) when activity intensity increased and, therefore, two additional people were required to supplement existing staffing at that time. In areas identified as requiring lower levels of resourcing, we were able to redeploy staff from low intensity tasks to higher intensity activities. Conclusions Operational backlogs and bottlenecks were removed and service levels agreements were met by restructuring staffing and supplementing it with additional resources. Unexpected benefits were subsequently observed: the team has benefited from variety in their work and the data provide convincing evidence of the team’s value which has improved motivation. The data has also supported performance management.
Originality and value The research was motivated by operational need in an environment where meeting service level agreements is a key performance indicator. The methods can be applied to many library activities and is especially applicable to repetitive or high intensity tasks. It is equally valuable in library operations where there is a perception that staffing levels are inadequate but it is not clear how additional staffing should be deployed.
Biography
I am responsible for the £26M Book Storage Facility at Swindon, with space to store 153 miles of books and currently holding 8.9 million items. I am interested in getting value for money and improving the impact of our services through evidence-based decision making.
Author(s)
Ms. Diane Green (University of Sydney)
Title
Using Lean Six Sigma to focus the Process Improvement agenda for the University of Sydney Library
Abstract
Purpose
The University of Sydney Library commenced implementing a significant organisational change plan in late 2014, and through that, a journey in process improvement. The motivation behind the process improvement program was a mechanism to bring about change in the culture of the organisation to one which embraced accountability and continuous improvement – one which put the client at the centre of the service proposition – learning to ask the question “what does the client want?” The thinking was rooted in customer-centric Design Thinking principles, and used Lean to provide a common language and methodology for process improvements to meet those client needs. Common with all service providers, the Library needs to justify its ongoing value and relevance, and to compete for limited funds. In the University environment where there is high demand for infrastructure, research and teaching resources, operating budget increases are rare. Therefore, where the Library requires additional resources, this is achieved through generating efficiencies and redeploying savings, rather than additional allocation of funds from central finance. A formal Library Process Improvement Strategy and Blueprint was developed. Implementation of Lean Six Sigma was a central pillar of this strategy. From the outset the Library worked with KPMG Consulting, who provided Lean training for 93 staff and support for 3 pilot and 12 Green Belt projects. Design, methodology or approach
Six Sigma Yellow Belt training was initially provided to 66 staff. Yellow Belt provides the fundamentals and is designed to provide a basic working knowledge of both Six Sigma and Lean methodologies. Through training and support, attendees developed the skills and confidence to be a valuable key contributor to a business improvement project team. Yellow Belt training was delivered through a 2 day course and exam. All 66 staff successfully attained their Certificate of Lean Competency. Following the training in fundamentals, 3 pilot projects were then undertaken to test the approach. The three pilot projects undertaken were:
- Returns to Shelves: Leading to significant reduction in the time taken from return of print items to shelving – the service proposition for customers was improved.
- Inter Library Loans: Leading to implementation of processes to ensure consistency in search methods, improved knowledge of databases and improved understanding of hints and tips, thereby decreasing rework and improving productivity and service to customers.
- Reading Lists: Leading to changed processes to reduce the variation of Reading List requests and ensuring they were received with enough time before semester start dates to balance the workload for teams processing them, thereby improving quality and reducing staff stress.Once the success of the Lean approach was proven, the next phase was the roll out Green Belt training to 27 staff, which included delivery of Lean process improvement projects. The cohort of 27 was selected from across all units and levels of staff. They were trained concurrently which had the benefit of providing peer support, but the disadvantage of 27 staff taking on additional load along with day-to-day work. The program was intensive, incorporating an initial training course of 2 days, then 5 months later a 3 day course, two exams and a formal presentation of work. The program was tough, and of the 27 staff, 19 received accreditation, 6 received certification and 2 received participation awards.
Findings The 12 projects undertaken were:
- Client Engagement & Referrals: Leading to a standardised process to hand over patrons requests from Site to Academic Liaison Librarians, with a monitoring system to track and measure from end to end, thereby providing both enhanced client service and accountability.
- Customer Complaints & Feedback: Leading to the development of a standardised easy to use process to track Library feedback, which delivers a consistent client experience Library wide, and provides timely resolution of issues – an improved customer experience.
- Document Delivery: Leading to establishment of standardised processes for both incoming and outgoing bonus plus deliveries, thereby reducing delivery time from 6 days to the customer expectation of 4 days – an improved customer experience.
- Facilities Management: Leading to improved processes for reporting and monitoring issues, and thereby reduced time to fix faults and better presentation and functioning of the Library spaces – an improved customer experience.
- Library Teaching Evaluation: Leading to a standardised process for the management of research skills assistance requests and improved allocation of Academic Liaison Librarian support – customers received more timely and appropriate support.
- Fines Enquiries: Leading to shortened average time to resolve fine appeals, an outcome that empowered staff and was more in line with customer expectations.
- Missing Books: Leading to a standard and consistent procedure across all locations in managing missing item requests, thereby better meeting client expectations.
- New Repository Services Model: Leading to streamlined repository processes and better integration through an overarching enterprise level system, allowing requests to be attended to more efficiently.
- Orchestral Scores Reading Lists: Leading to process changes so that musical scores could be sourced in a more timely fashion then tracked for early return after performances.
- Rare Book Requests: Leading to implementation of a consistent and prompt service, with low client waiting times.
- Serials Shelf Ready Workflow: Leading to an 80% reduction in accessioning time, making resources available for customers sooner.
- SUP Production Workflow: Leading to reduced delay in the release of eBooks after print books, thereby better meeting client expectations.
The Library continues to use Lean for process improvement initiatives and encourages the Lean Community of Practice Group which is championed by the Innovation & Planning Team from within Central Services. The source of future projects is from within the teams themselves. Green Belt Staff members are invited to lead initiatives, and Yellow Belt staff members invited to participate. Staff who have not been formally trained are welcomed and mentored in the methodology. Conclusions Lean provided a common language and organisational framework which facilitated objective review and improvement of a number of legacy processes through engagement by the staff owners as opposed to direction by Library Management. Lean proved a successful change agent.
Biography
Diane is a financial professional who leads the Central Services Division of the University of Sydney Library. Central Services provides the enabling functions supporting three client facing divisions. She is new the University sector, joining the University of Sydney in 2012 after a 30 year career in Banking & Finance.
Author(s)
Prof. Sebastian Mundt (Stuttgart Media University)
Title
Enhancing library user feedback though activating survey methods
Abstract
Purpose Web-based questionnaires are used abundantly in the public. As a consequence, target audiences are increasingly tired of participating in “yet another survey”. Besides other factors, this ‘over-surveying’ leads to often marginal response rates which limit the relevance and usefulness of surveys for management and strategic planning. Personal survey modes and other ‘active’ recruiting methods, on the other hand, have rarely been employed as they usually require more resources and specific knowledge and competences. Their usefulness to activate audiences, however, is generally unquestioned. Design, methodology or approach To investigate their suitability for an academic library environment, a ‘paper & pencil’ classroom survey and an in-person tablet questionnaire were compared with a standard web-based questionnaire in a student library survey at Stuttgart Media University. Findings The use of classroom and in-person survey modes led to a much higher degree of engagement and interaction from students than the self-administered web survey. In comparison, the ‘active’ survey modes showed a dramatically better overall response rate, a better representation of low-interest users and partly more detail in open-ended comments.
Conclusions Both classroom survey and in-person survey methods demonstrated their usefulness for building and developing the relationship with users; they also had a strong positive impact on data quality. Originality and value The paper will introduce the methods used and will discuss their costs and benefits as well as their application to other library scenarios and target groups. The results are intended to stimulate the discussion about survey data quality and introduce alternative methods to activate structured library customer feedback.
Biography
Sebastian Mundt is Professor for Media Management and Information Services and chairs the Research Center for Digital Media Transition at Stuttgart Media University. His research interests focus on survey methodology and user tracking of online media. His professional practice includes positions as department head and project manager in academic libraries.
Author(s)
Dr. Holly Rick (University of Phoenix),
Ms. PJ Purchase (University of Phoenix)
Title
Making a motion for time: Assessment of virtual librarian productivity
Abstract
Purpose As libraries build or enhance their assessment efforts around their library services, leaders must look not overlook their productivity of the employees as they deliver library services. Productivity for any employee can be contingent on many factors such as motivation, training, jobs security, and conditions of the working environment. From review of the assessment of services outcomes, the user statistics noted a decrease in the use of the reference services. This identified an opportunity to understand what the virtual librarians were completing during their scheduled shifts. Design, methodology or approach Using a modified time motion study methodology as identified by Frederick Taylor, the University library completed a review of the virtual reference staff reviewing the time, frequency and duration of interaction with patrons. Findings and Conclusions The analysis identified the virtual reference staff were only working with patrons approximately 2 hours out of a 6 hour shift. This has allowed the library to review the strengths of the librarians, assigning them additional duties to support other library service functions. The virtual librarians were cross- trained on different aspects of the library, including building library guides and assisting with the virtual document delivery system.
Biography
Holly Rick is a librarian and business faculty. She has taught at the master’s level and doctoral level. She has a bachelors in business administration-finance, MBA, MLIS and PhD in Organizational Management-Leadership. She has been presenting at conferences on topics such as library support, professional development, and assessment.
Author(s)
Dr. Martha Kyrillidou (QualityMetrics, LLC)
Title
Communicating library value through ‘Grants to States’ planning and evaluation
Abstract
Purpose The purpose of this paper is to describe major findings surfacing from the evaluation and planning activities of State Library Administrative Agencies (SLAAs) through the ‘Grants to States’ program administered in the USA. The Library Services and Construction Act (LSTA) is one of the major funding programs managed by the Institute of Museum and Library Services (IMLS) and the Grants to States Program is one of its largest programs. The LSTA (US Code 20) requires that every five years an independent evaluator reviews the strategic plans and objectives achieved over the last five year period and provides an assessment of the impact federal funds have on state supported library activities. Through the evaluation work done across more than 20 states in 2016-2017, we will highlight some of the major threats and opportunities libraries face when communicating their value and developing future plans. Design, methodology or approach The evaluation studies that took place assessing 2013-2017 strategic planning activities for SLAAs deployed a mixed methods design, balancing qualitative and quantitative methods, observation, in person and virtual approaches. The evaluators brought in depth library assessment experience in the evaluation process. Findings We found that a number of critical focal areas are posing major opportunities for libraries in the future and are accompanied by threats as well. Areas of important focus for library development include: Lifelong learning, literacy, information access, capacity building, and civic engagement. Noteworthy examples of exemplary projects in these areas will be highlighted as well as what the evaluators see as some major threats that need to be considered carefully by library leadership in future years. Research or practical limitations or implications Evaluation studies have inherent limitations in them as they balanced practical considerations with aspirational methodological demands. This study is limited in generalizability across 20 different SLAAs in the USA. The concepts explored, the opportunities and the threats library leadership needs to examine have global and generalizable dimensions across different countries and cultural contexts. Conclusions Lifelong learning, literacy, information access, library capacity building, and civic engagement are fundamental directions for libraries to consider and grow in future years. Identifying and sharing best practices is more critical across library and other cultural organization sectors than ever before. Originality and value Despite the rich evaluative research studies taking place at each state in the USA, there has been little effort to synthesize these studies across different state boundaries in the past. This proposal is unique in that we rarely have opportunities to scale evaluation research across geographic boundaries and identify opportunities and threats, best practices and thematic focal areas for charting future directions. The Grants to States planning and evaluation cycle is taking place once every five years and most library evaluators are not aware of the opportunity far less of the possibility of scaling the approach across collaborators, researchers, and library leaders.
Biography
Martha Kyrillidou is the Principal of QualityMetrics, LLC, a private research company specializing in libraries. Martha worked at the Association of Research Libraries leading a premiere program in library assessment. She has worked with hundreds of libraries and her website is available at http://marthakyrillidou.com.
Authors
Rachel Bury and Helen Jamieson (Edge Hill University)
Title
You get few answers from a survey: what you get is insights and more questions!
Abstract
Purpose: The purpose of the paper is to outline and share Edge Hill University (Learning Services) journey in relation to collecting, using and interpreting a wide range of information and data to inform service developments in terms of the delivery of TEL (Technology Enhanced Learning). A survey can be a blunt instrument and can actually produce more questions than answers which is where other methods have been used to triangulate data and comments and feedback.
The Student eLearning Survey was introduced in December 2008 as a means of exploring the learner perspective of TEL at Edge Hill University. The survey is an important tool used to identify and examine trends and system performance where there are direct comparisons to past surveys. The intention is to improve the student experience through a better understanding of what makes a good experience of TEL, and more importantly, what does not. It also provides an indicator of the extent to which the institutional VLE, Learning Edge and embedded library and information resources are integral to the learning experience of Edge Hill students.
Approach: In the paper we will outline our approach which includes the very rich, longitudinal data and comments we receive via the survey but also how we triangulate using user experience tools and also system management data.
The survey is a rich source of intelligence and insight into students’ use of technology in support of their learning; their views, likes, dislikes and preferences on tutor use of the virtual learning environment and associated technologies (including library) are captured, analysed and considered. Feedback on institutional systems and personal ownership and use of devices is also collected to form part of picture, and as time passes the survey’s value has grown incrementally.
The survey is one part of the portfolio of evidence gathered by Learning Services, and the along with the user experience it absorbs a considerable amount of staff time and effort to administer, run and analyse the responses and UX recordings, but the effort is more than rewarded by the intelligence gained.
Findings: Now in its 7th iteration, it is possible to draw a correlation between action taken in response to issues raised in earlier surveys, and the usability testing (2 periods of testing have been undertaken) and noticeable improvements in satisfaction today. Strategically, the survey responses have formed a valuable evidence base, which have prompted a number of follow-on actions. These have ranged from business cases for additional investment, conversations with individual departments on their use of technology to support student engagement, and changes to practice, for example, how the Library supports students who bring their own devices.
Research: We will outline how we have attempted to triangulate the survey data against other sources e.g. system data, reported issues and usability data. We will describe how we have used this information to inform our decision making in terms of improvements and change to the VLE and library support for mobile apps for learning. In July 2016 as a result of the 6th eLearning survey, a number of high level changes were made to the VLE interface and in particular the library resources and support embedded therein and these were further tested with questions in the 7th survey. User experience testing of the new interface for the library ‘tab’ informed by the survey’s findings will be undertaken in March 2017 and will be presented at the July conference.
Conclusions: We will share our experiences of using a rich set of data and usability observations with the audience and how this approach to performance measurement and impact has allowed us to make a number of high level improvements to the universities VLE including access to library resources via the library tab. The now bi-annual survey, clearly provides signposts to both where we may want to investigate further or to highlight experiences which are having impact.
Value of the proposal: We feel that this paper will be useful for attendees who are thinking about using usability tools to triangulate other qualitative and quantitative data they may have.
Rachel Bury is working in a role within Learning Services which manages Quality, Communication and Marketing. The quality aspect of the role involves use of KPIs and liaison with the University Academic Quality Unit regarding the validation of new programmes, re-validations, and collaborative provision.
Wednesday, 2 August 2017 – Parallel 8: LibQUAL+ and Beyond
Author(s)
Dr. Martha Kyrillidou (QualityMetrics, LLC),
Dr. Prathiba Natesan (University of North Texas),
Dr. Colleen Cook (McGill University)
Title
Longitudinal invariance of LibQUAL+®: How has LibQUAL+® changed over time?
Abstract
Purpose Library researchers are interested in understanding how their library services are keeping up with the rapid advances in technology and newer methods of obtaining information. Accordingly, the commonly used LibQUAL+® instrument, which is used to measure library service quality, may need to be updated to reflect these developments. The purpose of this paper is to study the longitudinal invariance of LibQUAL+® for data obtained from all libraries that use LibQUAL+® since 2013. This study is a more thorough and large-scale replication of a study that considered invariance over a period of three years at the library of the University of North Texas. The results of our study will inform how consistent the dimensions of affect of service, information control, and library as a place are over time. Design, methodology or approach The study will analyze data from 1,327 number of libraries and 2.3 million participants across 29 countries. After screening the data for univariate and multivariate normality, a series of structural equation models will be fit to the data. These will include: (a) a confirmatory factor analysis model based on the three factor structure established in the literature, (b) a multitrait multimethod model (MTMM) with time as the method factors and the three factor structure as the trait factors, and (c) longitudinal invariance models. Findings All models will be fit in LISREL software. Model fit will be evaluated using chi-squared values and fit indices such as comparative fit index (CFI), root mean square error of approximation (RMSEA), and standardized root mean squared residual (SRMR). Research or practical limitations or implications MTMM is a novel use of testing the validity of LibQUAL+® across time because it can partial out method effects (i.e. methods (time) of data collection). This adds to evaluating validity of the instrument, which cannot be done in a cross-sectional study. Longitudinal invariance model will inform us how (if at all) these factors have changed over time. Conclusions We expect factor structure of the LibQUAL+ model to be replicated across time. However, the loadings may differ which would indicate that the users’ perceptions of affect of service, information control, and library as a place have changed across time. This will indicate directions for future research and extension of LibQUAL+®. Originality and value Very few researchers have access to complete LibQUAL+ data across institutions and time. We use the latest methodological developments in instrument validation to examine the invariance of LibQUAL+® across time. To the best of our knowledge, no study in library science has examined how to partial out method effects. This can inform researchers if their method of data collection has an impact on the responses. LibQUAL+® has benefitted tremendously by the interest of highly skilled academic researchers in the past and continues to be critically analyzed through interdisciplinary collaborations between library administrators and researchers with expertise in both libraries and methods.
References Cook, C., Heath, F. and Thompson, B. (2003). LibQUAL+™ from the UK Perspective, 5th Northumbria International Conference Proceedings, Durham, UK, July 2003. Cook, C. and Thompson, B. (2001). Psychometric Properties of Scores from the Web-based LibQUAL+™ Study of Perceptions of Library Service Quality, Library Trends, 49, 585-604. Fagan, J. (2014). The dimensions of library service quality: A confirmatory factor analysis of the LibQUAL+ model, Library & Information Science Research, 36(1), 36-48. Kieftenbeld, V. and Natesan, P. (2013). Examining the measurement and structural invariance of LibQUAL+® across user groups, Library & Information Science Research, 35(2), 143-150. Kyrillidou, M., Olshen, T., Heath, F., Bonnelly, C. and Côte, J. (2003). Cross-Cultural Implementation of LibQUAL+™: the French Language Experience, 5th Northumbria International Conference Proceedings, Durham, UK, July 2003, 193-99. Lane, F., Anderson, B., Ponce, H. and Natesan, P. (2012). Factorial Invariance of LibQUAL+ as a Measure of Library Service Quality Over Time, Library & Information Science Research, 34(1), 22-30. Natesan, P. and Aerts, X. (2016). Can library users distinguish between minimum, perceived, and desired levels of service quality? Validating LibQUAL+ using multitrait multimethod analysis, Library & Information Science Research, 38(1), 30-38. Nitecki, D.A. (1996). Changing the Concept and Measure of Service Quality in Academic Libraries, The Journal of Academic Librarianship, 22, 181-90. Parasuraman, A., Berry, L. and Zeithaml, V. (1991). Refinement and Reassessment of the SERVQUAL Scale, Journal of Retailing, 67, 420-50. Rehman, S.U., Kyrillidou, M. and Hameed, I. (2014). Reliability and Validity of a Modified LibQUAL+ Survey in Pakistan: an Urdu Language Experience, Malaysian Journal of Library & Information Science, 19(2), 83-102. Thompson, B., Cook, C. and Kyrillidou, M. (2006). How Can You Evaluate the Integrity of Your Library Assessment Data: Intercontinental LibQUAL+® Analysis Used as Concrete Heuristic Examples, Proceedings of the Library Assessment Conference: Building Effective, Sustainable, and Practical Assessment, Charlottesville, VA, August 4-6, 2006. Yeager, A. and Kyrillidou, M. (2015). Design Thinking and LibQUAL+: The landscape of changing user needs and expectations of faculty and undergraduate students in ARL libraries – Trends 2003-2014, Poster, Canadian Library Assessment Workshop, Toronto, Canada, October 16-18, 2013, and Southeastern Library Assessment Conference, Atlanta, Georgia, November 16-17, 2015.
Biography
Martha Kyrillidou is the Principal of QualityMetrics, LLC, a private research company specializing in libraries. Martha worked at the Association of Research Libraries leading a premiere program in library assessment. She has worked with hundreds of libraries and her website is available at http://marthakyrillidou.com.
Author(s)
Ms. Ciara McCaffrey (University of Limerick)
Title
Ten years of transformational change, one step at a time: A LibQUAL study
Abstract
Purpose This study considers a continuous change programme that has spanned a decade at the University of Limerick in Ireland, the impact of which can be measured through LibQUAL data. The LibQUAL survey is a cornerstone of the Library’s continuous improvement programme and is a key enabler of evidence-based decision making. The survey has been run five times from 2007 to 2017. Following each survey, a quality action plan is put in place, the impact of which is measured in the following survey. The presentation will describe the experience of the University of Limerick in its regular approach to continuous, systematic assessment and will identify some trends and patterns that have emerged in the data over the decade. An overview of some of the trends in SCONUL data for the same period will be presented. The purpose of the study is to explore how continuous, methodical assessment can lead to transformational change in academic libraries.
Design, methodology or approach The University of Limerick Library LibQUAL survey data forms the basis of this study, together with comparative SCONUL data for the period 2007-2016. Findings In the five iterations of the survey, the UL LibQUAL scores have consistently risen, indicating that the improvement strategy has been successful and that readers’ perceptions of the quality of library services have greatly improved. Furthermore, the impact of targeted efforts or long-term investment of resources directed at a specific issue were visible in the data. Research or practical limitations or implications The experience of the University of Limerick may be useful for other libraries who are using LibQUAL or considering using LibQUAL as part of their assessment activities. For those that have a number of years of LibQUAL data, this study may provide some ideas on how that data can tell a compelling story.
Conclusions Longitudinal analysis that tracks a library’s commitment to continuous improvement can be a powerful tool to demonstrate value to university management and library users.
Biography
Ciara McCaffrey is Deputy Librarian at the University of Limerick where she coordinates all cross-library management processes, namely HR, finance, quality, assessment, communications and library-wide projects. She is committed to performance assessment and evidence-based librarianship and is an advocate of librarians engaging in primary research, academic writing and publishing.
Author(s)
Dr. Jackie Belanger (University of Washington),
Ms. Maggie Faber (University of Washington),
Mr. Steve Hiller (University of Washington)
Title
After the survey: An integrated strategic approach to action
Abstract
Purpose Surveys have long been a staple of library performance measurement and user needs assessment. There is a substantial body of literature related to the design, development, implementation and analysis of survey results. However, a continuing concern lies in the challenge of moving from analysis and communication of results to actions that lead to improvements for users. This paper will illustrate methods that can provide a more integrated and strategic approach to using quantitative and qualitative survey data to improve library services. These approaches include an emphasis on communicating results more effectively to library staff; taking a more rapid and agile approach to follow-up assessments through methods such as design thinking; aligning survey development and results with strategic planning; and using survey results to underpin organizational change processes. In discussing these methods, the authors aim to provide attendees with concrete strategies they can employ in their own efforts to use survey data for library improvement. Design, methodology or approach The paper will review literature on issues involved in moving survey results to action and identify common barriers as well as best practices from existing studies. The paper will also highlight the importance of key steps in the survey design phase that are critical to the effective use of results. These include clearly articulating survey goals and engaging both staff and user input during survey design in order to develop a survey that elicits actionable results. We will use as a case study the University of Washington (UW) Libraries 2016 Triennial Survey. This is the eighth iteration of the Triennial Survey, which began in 1992. As a locally-designed and implemented survey, changes are made for each survey iteration in order to gather the most relevant and timely information for our library. Questions focus on user needs and experiences, especially as they relate to library and institutional mission and goals. Separate surveys are conducted for undergraduate students, graduate and professional students and faculty. The authors have extensive survey experience and have employed a variety of methods to make use of survey results, including several new ones for the 2016 survey. Findings The findings are highlighted by examples from four areas. Targeted and Multifaceted Communication of Survey Results to Libraries Staff: One key finding from our approach was the importance of communicating results in a variety of formats for different Libraries audiences, including topic-specific reports (e.g., on scholarly communication, teaching & learning) and interactive Tableau dashboards. While the topic-specific reports provided stakeholders with a focused, highly detailed window into the results that mattered most to them, the dashboards enabled staff to drill down into both qualitative and quantitative results and identify patterns across groups. In making both the qualitative and quantitative results more accessible via targeted reports and dashboards, we found that staff were more empowered to act on these results within their own units or working areas. Rapid follow-up assessments: A second key finding was the necessity of a more nimble and strategic approach to following up on survey results using other assessment methods such as design thinking. Like many large-scale user surveys, the UW Libraries Triennial Survey sometimes yields results that are indicative of important user needs and patterns, but may not point directly to concrete actions for improvement. Using these high-level survey results to inform focused follow-up assessments enabled us to move more rapidly from survey results to actionable steps for service improvement. The assessment team, working closely with key Libraries stakeholders, identified selected areas from the survey results that warranted further investigation. These areas (including citation management tool support for graduate students) are now being explored in more depth using approaches such as design thinking, which enables libraries to take a more agile and focused approach to specific service improvements. Alignment with Strategic Planning: Survey results play an important part in evaluating and informing the UW Libraries strategic priorities and directions. Visibly aligning 2016 survey questions with our strategic priorities facilitated the process of gathering information needed to move forward in such areas as scholarly communication, data management, resource discovery, research support, teaching and learning, and digital scholarship. Survey results and follow-up discussions were also used in the initial phase of creating the next iteration of our strategic plan. Organizational Change/Institutional Alignment: An additional finding was that survey results can be used strategically to help in initiating organizational change. For example, survey results were used as part of a mixed-method approach for an organizational review for our Research and Learning Services unit to ensure that services were configured in the most effective and efficient way to address user needs. Survey results were also highlighted in our annual budget narrative to the University and were effective in demonstrating the need for increased collections support, funding personnel transitions, and in showing how the Libraries was aligned to institutional priorities. Research or practical limitations or implications While the case study will focus on the example of a large U.S. research library, the approaches outlined in this paper can be used across a range of libraries. Conclusions A multifaceted and integrated approach to survey design, analysis, and communication will help turn survey results into action. Utilizing new methods of communication to facilitate understanding and create engagement with results, employing strategies such as design thinking to follow-up on survey data, and closely aligning survey results with strategic planning, budgeting, and organizational change initiatives can all assist libraries in producing meaningful changes for their user communities. Originality and value Surveys are widely used by libraries to acquire user input, evaluate library services, and understand user needs and priorities. While a substantial amount of data is collected through surveys, the application of survey results is mixed, especially in terms of advancing larger library and institutional strategic priorities. This paper provides a overview of the issues involved in turning results into action and uses the long-established UW Libraries Triennial Survey as a case study in applying a strategic approach that focuses on improving services that are aligned with institutional and community needs.
Biography
Jackie Belanger is the Assessment Librarian at UW Libraries. Recent publications include articles on rubric assessment of student learning outcomes (published in portal: Libraries and the Academy), collaborative faculty-librarian assessment of information literacy outcomes (in Library Review), and assessment management systems (ACRL Conference Proceedings and the Journal of Academic Librarianship).
Maggie Faber is the Data Visualization and Analysis Librarian at UW Libraries. She helps analyze and communicate assessment results and libraries-related data with interactive dashboards and visualizations. Recent presentations include a space assessment poster at the Library Assessment Conference and a poster on survey questions to demonstrate impact at ACRL.
Steve Hiller is Director of Assessment & Planning at UW Libraries. In this position, he leads the Libraries strategic planning process, assessment program, and institutional and national reporting efforts. He is co-founder and co-chair of the Library Assessment Conference, and the recipient of the 2016 David B. Thorud Leadership Award.
Wednesday, 2 August 2017 – Parallel 9: Collections
Author(s)
Dr. Shanna Jaggars (The Ohio State University),
Dr. Amanda Folk (The Ohio State University),
Mr. David Mullins (The Ohio State University)
Title
Understanding students’ satisfaction with Open Educational Resources as course materials
Abstract
Purpose In order to ameliorate rising textbook costs, The Ohio State University Libraries and colleagues in the Office of Distance Education and E-Learning have launched a program to offer grants that support faculty in replacing their traditional textbooks with high-quality digital open educational resources (OER) or library materials. Under the first cohort, faculty adopted OER for 12 courses in Fall 2016, and a second cohort is working on a similar number this year. The projects in the first two cohorts are projected to save students nearly $1.3 million annually. In this paper, we introduce a new survey instrument, which assesses the quality of OER designed under our program. Existing surveys of OER quality and satisfaction rely on a few holistic satisfaction items, which are not designed to identify specific areas for improvement; or include only items on general textbook quality, which are not attentive to the unique qualities of OER. In this paper, we will present the new survey instrument and its results, and discuss how the survey can be usefully applied to other OER programs’ improvement efforts. Design, methodology or approach We distributed an online survey to students enrolled in course sections that adopted OER in Fall 2016, with N = 683 responses across 12 courses representing a wide array of disciplines (e.g., Art Education, Economics, Engineering, Geography, Math, and Psychology). The survey included 21 Likert-scale (1 to 5) items divided into three sets (Set A, B, and C). Sets A and B asked students to compare the OER with traditional printed materials used in their other courses, and reflected commonly-cited areas of student satisfaction and dissatisfaction with the quality of OER (based on open-ended student feedback gathered from the previous literature), supplemented with items adapted from a published scale that measures general textbook quality. Set A focused on the quality of the OER itself (e.g., its search capabilities, visual quality, writing quality) and Set B focused on how the OER affected the student’s learning experience in the course (e.g., the student’s ability to access materials when needed, shut out distractions while studying, review and remember the material).
Set C dealt with the instructor’s implementation of the OER material, based on the literature regarding instructor behaviors that predict whether students actually read a course textbook. For example, students reported the extent to which the instructor required quizzes or discussions on the OER materials. In addition to the 21 Likert scale items, a final overall satisfaction item (similar to one used in previous studies) asked students to imagine a future course and to indicate whether they would prefer to take a section that used a traditional printed text, or digital materials similar to the one in their current course. In addition to the student survey, we surveyed the 12 courses’ instructors with parallel items for Sets A and C, to get their perspective on OER quality and instructor implementation. The instructors’ survey also included an additional set of eight Likert-scale items on implementation, focusing on how the adoption of OER changed their course design (e.g., breadth of coverage, depth of coverage, proportion of class time spent in different activities). Overall student responses are shared in the “Findings” section below. For this conference paper, we will also conduct additional analyses to establish the reliability and validity of the items. First, for Sets A and C, we will correlate student and instructor responses to understand how well students and their instructor agree on OER quality issues and the extent to which instructors required students to read/watch the OER. Second, using exploratory factor analysis on the student sample, we will investigate how the 21 Likert items can be collapsed into a smaller set of subscales, and will explore the internal reliability of each subscale. Third, using the smaller set of subscales, we will investigate which items are most predictive of the final overall satisfaction item. We will also investigate whether the instructor implementation items (measured at the course level) moderate the relationship between OER quality and satisfaction. For example, we might expect to find the strongest relationship between quality and satisfaction in courses where the instructor requires students to read the textbook. Based on these analyses, we will create a final recommended survey instrument, along with guidelines for its use in program assessment. Findings Overall, students’ responses were very positive. For example, 63% felt OER content was more relevant than a typical printed textbook (while only 5% felt it was less relevant), and 66% felt digital OER was easier to access (while only 7% felt access was more difficult). However, we have flagged a few areas for general improvement (e.g., 18% felt the quality of study aids was worse; 27% had more difficulty shutting out distractions while studying, and 17% had more difficulty taking useful notes on the material), as well as areas for improvement within specific courses. In general, about half of the students reported that they were consistently required to read/watch the OER materials. As noted above, for this conference paper, we will conduct additional analyses to establish the reliability and validity of the survey instrument. Originality and value Library administrators currently have few tools available to assess the quality of OER they help faculty create. Student end-of-course evaluations rarely include items related to the quality of course materials, and currently-published surveys of OER quality are not designed for program assessment and improvement purposes. Our new survey is grounded in previous research on students’ areas of satisfaction and dissatisfaction with OER, and will be more useful in terms of demonstrating the pedagogical value of an OER, as well as diagnosing the specific aspects of an OER that need improvement. We suspect that our inclusion of items on instructor implementation are also necessary if administrators wish to predict student usage, satisfaction, or academic outcomes from text quality. Based on further explorations in terms of reliability and validity, in this conference paper we will present a final recommended version of the survey for others to adopt in their own organizations.
Biography
Dr. Jaggars is the Director of Student Success Research for the Office of Distance Education and E-Learning at The Ohio State University, where she leads both qualitative and quantitative research projects focused on instructional improvement initiatives, academic support programs, and patterns of student academic progression.
Author(s)
Ms. Lorraine Estelle (Project COUNTER)
Title
COUNTER: Release 5 of the Code of Practice
Abstract
Purpose
COUNTER published its first Code of Practice in 2003 and has since been improving this international standard to meet the evolving needs of its library, publisher and vendor membership. Release 5 of the COUNTER Code of Practice published in the summer of 2017. The challenge for Release 5 has been to find the balance between addressing changing needs and reducing the complexity of the Code of Practice to ensure that all publishers and content providers are able to achieve compliance. The major themes of Release 5 are consistency, clarity, simplification, flexibility and continuous maintenance. This paper presents Release 5 and provides an opportunity for questions and discussion.
Biography
Lorraine Estelle was CEO of Jisc Collections for 13 years, leading the UK academic library consortium. As director of COUNTER, she is focused on improving the Code of Practice for the recording and reporting of online usage statistics. She is also co-editor of Insights: the UKSG journal.
Author(s)
Dr. David Wells (Curtin University Library)
Title
Did You Find It? The Curtin materials availability survey
Abstract
Purpose
Libraries are always concerned to obtain data from clients about the effectiveness of their collections, discovery systems and information literacy programs in order to inform decisions and planning of services. In the past, Curtin University Library has used the methodology developed in the late 1990s by the Council of Australian University Librarians (CAUL) to survey clients about their ability to locate the specific physical items that they are looking for. As libraries have moved in recent years increasingly towards the provision of information resources in electronic format and increasingly have clients who are not physically present in the library, the original CAUL methodology is no longer adequate. The present project, which builds on a pilot undertaken in 2013, transfers the concept of the materials availability survey to the electronic environment. Participants were asked to comment on whether they found the electronic items they were looking for and, if they did not, what they consider to be the reasons for this failure. The survey results were then analysed by library staff to identify patterns to help us understand use of information resources and the library catalogue by clients, identify possible improvements to catalogue data, catalogue design, library collection development and information literacy design, and test and refine the survey methodology to maximise its value as part of the library’s ongoing quality program. Design, methodology or approach
Clients were randomly invited to participate in the survey through a web browser window that appeared when a catalogue search was initiated from the library’s home page. Participants were asked to record a specific electronic item they are looking for, whether they found it, and reasons for not finding it, if applicable. Survey data was collected using Qualtrics software and responses forwarded immediately to library staff, who endeavoured to reproduce “failed” searches, and confirm whether the clients’ analysis was correct. Survey results were coded and tabulated for further analysis. Findings
The survey was run during the week of 27 March 2017. Results will be available for discussion by the date of the conference. Research or practical limitations or implications
The expectation is that the electronic materials availability survey methodology developed through this project can be used routinely as a quality assurance mechanism to identify areas for improvement, and to provide a performance scorecard measure. Running the full-scale survey in 2017 will allow us to identify any technical or methodological issues that will need to be resolved before the survey can be adopted as a regular instrument. Already in preparing for the 2017 survey we have needed to make modifications in order to simplify the invitation and survey delivery process to ensure that it operates effectively with the full range of web browsers and that from the participant point of view completing the survey is not too far removed from the process of searching for the library item required. We also decided to limit the survey to electronic items rather than also including print, partly to make the survey experience simpler for clients, and partly to reflect the declining importance of print for the library’s collection. Conclusions
Analysis of the survey outcomes is currently under way. We expect that the survey will provide valuable data for understanding client behaviour and informing improvements to library services. Originality and value
If successful, it should be possible to apply the methodology of the Curtin electronic materials availability survey to other libraries as well, allowing not only for internal quality control, but also for benchmarking across multiple institutions.
Biography
David Wells is Manager, Collections, at Curtin University Library, with responsibility for acquisitions, cataloguing and strategic management of the Library’s collections, and for the Library Catalogue. He has published a number of articles about discovery system design and patterns of use in the electronic library.
Author(s)
Ms. Ruth Elder (University of York)
Title
Applying a collections categorisation framework: A pilot project at the University of York
Abstract
Purpose This is a case study showing how the University of York designed a framework to identify, categorise and “tag” library print book collections based on analysis of their characteristics, subject focus, uniqueness and use level. The framework provides a formal structure through which to understand the collections, to inform and prioritise local library actions and – potentially – to support wider collaborative collection management discussions and decisions. The Higher Education (HE) Library Collection Management community is currently exploring the potential of collaborative working across shared print retention scenarios. The building of collaborative collections is dependent on commitments agreed between trusted partners. The agreements need to ensure the retention of “last available” or rare or unique items of value to the institution and/or the wider community. This information needs to be readily available in an accessible and searchable format through the “categorisation” or “tagging” of items. Findings The collections review process identified stock considered of particular value to York, appropriate for permanent retention, and potentially a priority for preservation and promotion. A process was successfully developed for tagging items at both an item and bibliographic level. Matrices relating to level of stock use, period since acquisition and rarity were developed for three alternative categorisations, (potentially informing future decisions around location, and level of access and review period for stock.) The outcome of applying the categorisation framework reflected a research intensive post graduate focused library containing some of York’s particular collection strengths, with a “long tail” of material with low, infrequent use. Research or practical limitations or implications The practical implications of this pilot have the potential to be useful in collaborative schemes at local and national level. Conclusions The pilot project led to the development of an action plan for the branch library, and it informs subsequent discussions regarding the roll out the framework across the main University library collections. The work also informs ongoing discussions with regional partners in regard to identifying a shared lexicon and understanding of collections categorisation and its input to collaborative collection management developments. Originality and value The case study will be of interest to libraries in exploring new ways to approach collection management issues, and in contributing to discussions around collaborative collections communities.
Biography
Ruth Elder is an experienced practitioner with a thorough awareness of current collections management (CM) discussions on the regional, national and international scene, who regularly contributes to regional and national collaborative projects.
Author(s)
Mr. Ng Chee Yong (National University of Singapore),
Mr. Eng Aun Cheng (National University of Singapore)
Title
Using dashboard and automated scripts to improve end to end processes, from ordering to cataloguing
Abstract Purpose The National University of Singapore (NUS) Libraries comprises eight libraries. The acquisitions and cataloguing functions are centralised in the Technical Services Unit. The Ordering and Cataloguing Teams processes more than 10,000 orders and titles every year. Most of these publications are published in the United Kingdom and the United States. Over the years, the teams had attempted to measure the performance of the acquisition operations. However, the procedures to generate the performance reports were tedious and data extracted was not current. Nonetheless, these reports showed that the ordering process for books from the United Kingdom and the United States took 6-8 weeks and the cataloguing process, another one week. In total, the end-to-end process cycle time took 7-9 weeks (49-63 days). Only 75% of the books ordered were received and catalogued within 50 days, against the Key Performance Indicator (KPI) requirement of 80%. It was therefore necessary to improve both processes to reduce the cycle time. Design, methodology or approach The Ordering and Cataloguing Teams utilised database query and visualisation tools to measure the performance of the operations and improve the processes. Two developments aided the teams. Firstly, NUS Libraries upgraded its Integrated Library System (ILS) to Innovative Interfaces Sierra in 2013. It features Sierra Database Navigator (SierraDNA), a facility for querying the ILS externally using Structured Query Language (SQL). Secondly, NUS Libraries had purchased a visualisation and dashboarding tool providing for multiple database connectors and scheduled query, through an earlier project. The combination enabled the teams to construct dynamic queries and export ordering and cataloguing data from the ILS on an hourly basis. With data visualisation, the teams are able to identify bottlenecks, make improvements and monitor the situation. For example, we can now drill down using the orders turnaround report and retrieve the list of orders that did not meet the KPI, so that we can arrange for alternative. In addition to visualising data for potential delays, the teams reviewed processes to eliminate unnecessary steps and or improve processes further to reduce the processing time. One such improvement was the automatic assignment of Library of Congress (LC) call numbers. The cataloguers identified a pattern for certain copy-catalogued titles where automatic assignment of LC call numbers is possible. Findings Before July 2016, only half the titles ordered were received and catalogued within 50 days. With the dashboard, the Ordering Team can now easily generate the list of outstanding orders for more than 21 days and check the order status. The team can better decide whether the vendors are able to fulfil the titles in time, and take remedial actions where necessary. In addition, other reports allow the Ordering Team to check created orders before these were sent to vendors, ensuring that the orders were sent right the first time. The Cataloguing Team automated the construction of Library of Congress (LC) call numbers using Microsoft Excel. A scripting tool, AutoHotKey, was used to identify such titles, populate the list in Excel, generate the call numbers and insert the call numbers into the records. The improvements resulted in 89.4% of orders received and catalogued within 50 days. 99.1% of received titles were catalogued within 1 working day, well beyond the KPI of 90% within 2 working days. Productivity and output quality improved; 74 man-hours was saved every month and error rates was reduced. The processes were well controlled and the results are sustainable. Research or practical limitations or implications There were challenges encountered during the improvement efforts. Firstly, the SQL queries are complex and executions take a few minutes to complete. In addition, the dashboard uses a custom SQL-like language, which imposes a substantial learning curve. Secondly, the Ordering Team had to learn the new claims techniques. The team struggled to clear the initial long outstanding orders to bring the ordering process under control. Thirdly, the Cataloguing Team had to be trained to operate the AutoHotKey scripts. The scripts need to be run in stages and in a particular sequence; successful execution of one stage depends on the previous stage. The Cataloguing staff needs to be alert and correct the errors so that they would not be compounded as the scripts progress.
Conclusions NUS Libraries utilised a combination of ILS features and the dashboarding tool to produce a dashboard to better manage the book acquisition process and take action to meet the KPI. AutoHotkey Scripts and Microsoft Excel were used to automate the call number assignment for a segment of copy-catalogued titles and manage to exceed the target. These improvements to the end-to-end process enhances the user experience, shortens the wait time and provides for a more consistent service delivery experience to the patrons. Originality and value This is a useful case study for the Technical Services team of other libraries if they are to consider implementing visual monitoring of their acquisition process, and for libraries which need to assign call numbers according to in-house practices.
Biography
Mr. Ng Chee Yong is a Senior Librarian at the Central Library, National University of Singapore. He is with the Technical Services team, handling book acquisitions. Prior to this, he was doing projects in Library Information Technology Services team including working on the project to build an organisational dashboard.
Mr. Cheng Eng Aun is Senior Librarian at the National University of Singapore Libraries. He leads the centralized Acquisitions operations of print and online materials for all NUS libraries. He is also responsible for analysing usage statistics and generating the statistical tables of the Library annual report.
Wednesday, 2 August 2017 – Parallel 9: Information Literacy and Liaison
Author(s)
Ms. Maggie Faber (University of Washington)
Title
Choose Your Own Adventure-style information literacy assessment
Abstract
Purpose Flipped classrooms, gamification, and multimodality are familiar concepts in education. Choose Your Own Adventure-style instruction brings together and builds on these ideas by allowing students to direct their own path through more traditional learning materials. Developing and assessing nonlinear content poses some significant challenges to librarians and instructors, but also significant opportunities to reinforce learning objectives, model behavior, and increase student engagement. This method also provides robust and naturalistic insight into student research behaviors and decision-making far beyond the pre- and post-testing common to many information literacy practices. Design, methodology or approach This paper presents an innovative method for assessing information literacy skills that moves beyond the need for quizzes and feedback surveys. Prioritizing flexible, nonlinear content and building the tutorial in a platform that natively provides usage information gives instructors and librarians the ability to see the choices students make and the rationale for their decisions. A branching narrative structure gives tutorials the flexibility to adapt to student choices, providing more information to students who request it, or skipping past sections a student is familiar with. By analyzing usage data, librarians can see the stumbling blocks, skipped content, and “best of” choices students make as the tutorial progresses, and identify unused content that could indicate a need for restructuring the material or changing the emphasis of their instruction. By sharing key insights from the first year of Choose Your Own Adventure-style information literacy tutorials at the University of Washington and the University of British Columbia, attendees will gain an understanding of how these tutorials work, when they might be an appropriate method, and the challenges and opportunities of working with information literacy content in a nonlinear format.
Findings While this method has some limitations, key findings for this method are centered around three major themes. Naturalistic insight into student behavior and decision-making: One key finding was the value of seeing student choices at every stage of the process. When large numbers of students selected one information source over another, expressed confidence or uncertainty about a skill, or self-reported the rationale for their decisions, this gives librarians a better understanding of student research habits, and an opportunity to tweak the tutorial and shift emphasis in response to student behaviors and feedback.
Responsive content that gives feedback to librarians at the point of need: Students had opportunities to express interest in new tools or explore familiar ones more in-depth. While some students indicated that the information was valuable and chose continue exploring, others indicated that the volume of information was getting overwhelming. The flexible paths allow those students to skip past the optional content to the end, reducing information overload, and gave librarians a sense of where this overload happened, allowing them to restructure the distribution of information or provide more static content, such as a handout, that could be absorbed later. Low-impact assessment that increases student engagement: Because the tutorials were built in a survey platform, no additional assessment was needed for students to provide feedback or test their learning, and instructors and librarians were able to see how and if engagement and satisfaction shifted throughout the module. Students who engaged in the content were willing and excited to explore non-mandatory paths and expressed a high amount of interest in the Choose Your Own Adventure-style format. Of the 180 completed responses, 77% expressed a high rate of engagement—selecting choices that indicated they “loved” the module—and 42% expressed a desire for additional tutorials in this format. The biggest disengagement point in the asynchronous module was immediately after the first question. Combined with the high positive feedback at the end, these results indicate that students who dislike this style are able to self-select out of the content—and that the majority of these students recognize their dislike quickly before investing too much time. Allowing students the opportunity to opt out and select more traditional, linear content allows them to exert control over their learning process and engage in ways that make the most sense to them. Research or practical limitations or implications
Developing and analyzing nonlinear tutorials can be a challenge. Restricting the available choices makes the content easier to work with, but provides less robust insight into student behavior. There are a few tools that provide an interface for drafting interactive branching narratives, but importing this content into a survey platform can be an intricate process. And while this method can generate rich data, any data extracted from a nonlinear platform will reflect the complexity of the content, require familiarity with the underlying narrative to analyze effectively. If a “branch and bottleneck” structure is used, with different choices eventually leading to a central node seen by everyone, the data would have significant gaps from any unchosen paths and selections, and the choices as well as the end points would be important information to analyze. Without an established framework of analysis or template for visualization, librarians may rely on limited results from a few key questions instead of synthesizing complex usage patterns into a coherent narrative. Finally, a forced-path branching narrative will never be as natural as observational research methods, but this gamified assessment strikes a balance between the resource intensive observational methods and rudimentary pre- and post-testing methods.
Conclusions; Originality and value In the 1980s, Choose Your Own Adventure books were experiments in storytelling that put the power to direct the narrative into the hands of the reader. This style of content can be adapted for instructional purposes, allowing students to direct their own learning and flexibly encounter more traditional material. Choose Your Own Adventure-style tutorials offer a playful and engaging method for assessing information literacy beyond pre- and post-testing. The method has a low impact on users, but gives plentiful insight into student research habits, decision-making, and engagement with the tutorial content. Developing and analyzing this content can be resource intensive and challenging, but in contexts where such an approach makes sense, this method can revitalize more traditional instruction and provide ongoing, sustainable feedback for information literacy programs.
Biography
Maggie Faber is the Data Visualization and Analysis Librarian at UW Libraries. She helps analyze and communicate assessment results and libraries-related data with interactive dashboards and visualizations. Recent presentations include a space assessment poster at the Library Assessment Conference and a poster on survey questions to demonstrate impact at ACRL.
Author(s)
Mr. Daniel Mack (University of Maryland Libraries),
Dr. Gary White (University of Maryland Libraries),
Ms. Yelena Luckert (University of Maryland Libraries)
Title
Performance measures for liaison librarians: Documenting impact to communicate value
Abstract
Purpose This presentation will discuss the creation of performance measures and implementation of an assessment program for liaison librarians serving as subject specialists in academic and research institutions. Using the University of Maryland Libraries as a case study, the presentation will discuss practical measures for the creation and employment of performance measures to communicate the value of the library to a variety of constituencies, including administrators, faculty, students, and other users. Two of the authors will discuss performance rubrics outlined in their book “Assessing Liaison Librarians: Documenting Impact for Positive Change” published by the Association of College & Research Libraries. The presenters will discuss practical experience of the implementation. Design, methodology or approach This program discusses measures to document the impact of these librarians’ work, and to communicate their value in six areas:
- research services
- collection development
- instruction and learning
- scholarly communications/digital scholarship and data management
- outreach and engagement
Three panelists will address this topic by focusing on three areas: institutional setting and environment, policy creation, and program implementation. In the area of institutional setting and environment, the presenter will discuss the university, research center or other, wider organizational setting of the library. This will focus on the wider organization’s mission and goals, who serve as its primary stakeholders, and how the library can support the organization. The emphasis will be on identifying both metrics and qualitative indicators for assessing liaison librarians’ work, and how these performance measures can communicate impact to the larger organization. The presenter will outline the genesis of the work in establishing and developing a liaison assessment program for an academic library. In the area of policy creation, the presenter will examine the need for policies to support evaluation and assessment of liaison librarians’ work. This will focus on how to identify the need for new policies as well as revision of existing policies. The presenter will highlight the need for involving both front line personnel and administrators in creating policies to support documentation of impact, and will also address issues such as budgeting and technology support. In the area of program implementation, the presenter will discuss the mechanics of implementing a comprehensive program to assess liaison librarians’ work, how to incorporate both quantitative and qualitative assessment measures and other performance measures into work plans and reviews, and how to communicate value to the library’s various constituencies. This discussion will also include creating a professional development program to support subject liaisons’ assessment activities. Findings Attendees will learn specific techniques and rubrics that they can implement in their own libraries to better assess, document and communicate their liaison activities. They will also learn how to develop new services and initiatives to address the changing higher education environment and to better demonstrate the value of liaisons within their own institutions. Attendees will also learn how to address conflict, resistance and other barriers to the successful implementation of a liaison assessment program. Research or practical limitations or implications In time of rapid and fundamental changes in academic and library landscapes, libraries are increasingly engaged in assessment activities to monitor and measure their impact. A successful liaison program is critical to meeting library and institutional goals, to demonstrate how librarians are contributing to the broader institutional mission, and how libraries are providing value in their support of research, teaching and learning. Many institutions of higher learning re-deploy their librarians to meet their strategic goals. However, how do we know what is successful and what is not, and where do we need to put our energy and resources? Proper assessment tools will help build and develop a meaningful liaison program, specific to any given institution. Conclusions With a greater emphasis on assessment and demonstrating impact and outcomes in higher education, the assessment of liaison work provides a great deal of insight to library administrators, which they can use in allocating resources and presenting a positive vision of the library to campus administration. The assessment is also a very powerful tool for liaison librarians themselves, in seeing the impact of the work they are doing, how they connect to their campus environment, and where they need improvement and growth. As the nature of liaison work continues to evolve, it is imperative that liaison librarians adapt their work and develop appropriate assessment tools to demonstrate the positive impact of their activities. Originality and value Many academic libraries are embracing liaison systems as a way to provide better services and engage deeper with the campus community. Many think that this is the future of librarianship, and perhaps even survival of libraries. However, very few institutions have been able to develop and implement a full-scale assessment program which guides these efforts. This presentation will bring all elements of developing and implementing a liaison assessment program in one place at one time, providing members of the audience with valuable ideas and tools that they can use in their own settings.
Biography
Daniel Mack is Associate Dean, Collection Strategies and Services, at the University of Maryland Libraries in College Park, where he provides leadership in policy creation and implementation, strategic planning, and assessment for library collections. Mack’s research interests include assessment of academic library services and collections and interdisciplinary in the academy.
Gary White is Associate Dean, Public Services, at the University of Maryland Libraries, where he provides leadership in policy creation and implementation, strategic planning, and assessment for all types of services to library users. Dr. White’s research interests include assessment of academic library services and human resources.
Yelena Luckert is Director, Research & Learning at the University of Maryland Libraries, where he provides leadership in policy creation and implementation, strategic planning, and assessment for liaison librarians program, reference, and instructional services. Luckert’s research interests include assessment of academic library services, international librarianship, Russian and Jewish area studies.
Author(s)
Mr. Greg Careaga (University of California, Santa Cruz),
Ms. Deborah A. Murphy (University of California, Santa Cruz),
Mr. Frank Gravier (University of California, Santa Cruz),
Mr. Kenneth Lyons (University of California, Santa Cruz),
Ms. Laura Meriwether (University of California, Santa Cruz)
Title
Measuring information literacy outcomes: Process as value-added
Abstract
Purpose The University of California (UC), Santa Cruz University Library partnered with the campus Writing Program and Office of Institutional Research, Assessment, and Policy Studies to participate in the 2015/16 Association of College and Research Libraries (ACRL) Assessment in Action Program in order to assess the effectiveness of an online library research tutorial for conveying relevant Information Literacy skills to Composition 2 students completing a research assignment. This case study will give an overview of the project outcomes and describe the library’s strategy for using project data and design artifacts to support campus goals for defining Information Literacy learning outcomes across the curriculum. Design, methodology or approach The project team negotiated a common understanding of Information Literacy outcomes that mapped to ACRL Information Literacy Competency Standards for Higher Education, Composition 2 research skills learning outcomes, and—with varying degrees of precision—to online tutorial elements. The team used this common frame of reference to develop an assignment-based Information Literacy rubric that measured seven competencies across three domains. The rubric was informed by the Association of American Colleges and Universities Information Literacy VALUE Rubric. During fall term 2015, 115 Composition 2 students completed a locally developed online tutorial for the database Academic Search Complete. They subsequently completed research process coversheets as part of their culminating research papers. Instructors provided these students’ assignment bibliographies. Project librarians applied the rubric to the coversheets and bibliographies. Findings Students demonstrated proficiency at articulating their information needs and in deliberately selecting the types of resources that satisfied their assignment requirements. They were less adept at competencies surrounding translating information needs into effective searching strategies and managing sources. Project data proved useful for identifying areas where students needed additional support. They were less so for identifying deficiencies in specific tutorial elements. The work of defining Information Literacy competencies in the context of a national standard while respecting the traditions and reporting requirements of campus Information Literacy stakeholders has proven to be at least as valuable as the project data. In parallel with this project, Writing Program faculty revised their program learning outcomes. Former “research skills” outcomes for Composition 2 were recast as Information Literacy outcomes based on ACRL competencies. Information Literacy outcomes were added to the Composition 1 course for the first time. The common frame, rubric, and scoring exercise serve as a model for articulating Information Literacy outcomes that are measurable and support transfer of learning across the curriculum. Research or practical limitations or implications The ACRL Framework for Information Literacy in Higher Education superseded the ACRL Information Literacy Competencies shortly after we completed our primary data gathering. We did not attempt to redefine our negotiated understanding of Information Literacy outcomes or any of our rubric elements before we scored our data. Conclusions: Librarians at UC Santa Cruz are not members of the faculty and the Library has long struggled to gain a foothold with faculty efforts to describe or measure Information Literacy outcomes. Our collaboration with the Writing Program and with the Office of Institutional Research, Assessment, and Policy Studies has burnished our bona fides and given us a model to build measurable Information Literacy outcomes that fit the needs of the curriculum.
Originality and value: This case study may be of interest to librarians looking to rationalize various stakeholders’ interests in Information Literacy around a recognized standard. It may also be of interest to those who wish to develop or apply a competency-based rubric to specific course outcomes.
Biography
Greg is Head of Assessment and Planning and the Undergraduate Experience Team at the UC Santa Cruz University Library where he has worked for sixteen years. Prior to that, he was worked at California Lutheran University and Wayne State University in Detroit, Michigan.
Author(s)
Ms. Giovanna Badia (McGill University)
Title
What’s in a rubric? Meaningful scoring of information literacy instruction
Abstract
Purpose Educational assessments help in teaching and learning by identifying students’ acquired concepts and knowledge gaps. An assessment activity does not have to be solely the end-product of a class. It can be part of the learning process, especially if an authentic scenario is used in the assessment. Authentic assessment of information literacy (IL) instruction evaluates students’ abilities to research their genuine questions using the skills they have been taught. While authentic assessment is beneficial for testing whether students can employ learned concepts to situations outside the classroom, thus also encouraging students’ creative and critical thinking, it is more labour-intensive and challenging to evaluate responses since there is no single correct answer. Many studies about the assessment of IL instruction have reported the use of specific rubrics, i.e., grading criteria, to evaluate students’ responses. Rubrics can be divided into two general categories: 1) holistic, where one score is assigned for evaluating the successful completion of a task; and 2) analytic, where a grade is subdivided into discrete components needed to complete the task and each component is given its own score for achievement.
Design, methodology or approach
This paper will investigate which scoring method (employing holistic versus analytic rubrics) is the most effective for evaluating students’ search strategies in terms of providing constructive feedback to students and informing the librarian’s future teaching practices. These scoring methods were applied to the search strategies of over 100 engineering students who received in-class IL instruction, and were compared using multiple criteria such as the time required to apply each rubric, the reproducibility of results, and the information gleaned from the final grades.
Findings and Conclusions
The findings from this study will be useful for teaching librarians seeking to evaluate responses to open-ended assessment questions.
Biography
Giovanna Badia is an engineering and physical sciences librarian at McGill University, whose responsibilities include answering reference questions, providing instructional services, and collection development. Her research interests include information retrieval and information literacy instruction. She serves on the Executive Board of the SLA Engineering Division.
Author(s)
Ms. Denise Pan (University of Washington),
Prof. Margaret Bruehl (University of Colorado Denver),
Prof. Jefferson Knight (University of Colorado Denver),
Prof. Marino Resendiz (University of Colorado Denver)
Title
Enduring exposure: Methodology from Tracking Information Literacy in Science Students (TILISS)
Abstract
Purpose Does early exposure to scientific literature produce more information literate students? Can core subject instruction in critical thinking and practical analytical problem solving lead to long-term student success? The proposed presentation will focus on the methodology from a four-year study on teaching information and scientific literacy in undergraduate chemistry courses and briefly describe some initial findings. Design, methodology or approach The authors developed the Tracking Information Literacy in Science Students (TILISS) project to investigate the long-term impact of explicit instruction in scientific literature. This study builds on the momentum achieved from student learning outcomes assessment conducted at the University of Colorado Denver (Pan, Ferrer-Vinent, and Bruehl, 2014). In the present second phase of the project, the authors continue their interdisciplinary collaboration with the purpose of assessing the longevity of student impacts. Specifically, faculty teaching courses spanning the chemistry curriculum partner with a librarian to investigate the value of reading and citing library online journals on student learning outcomes over their undergraduate education. Our project began following students at the start of their chemistry studies, specifically students enrolled in Honors General Chemistry (HGC) I and II Laboratory courses. Each semester, students were given assignments to introduce them to the scientific literature with a special emphasis on building their information literacy. Later, some of those students enrolled in 3rd or 4th year chemistry courses, such as Biochemistry (BC) I and II. Similar to the HGC students, these students were also given a project that required them to search the scientific literature for concepts, record their research process, write a paper to communicate their understanding, and cite sources used in a bibliography. This study analyzes the journal articles viewed and cited by 3rd and 4th year chemistry students, relative to the grades received, and whether the students previously participated in the HGC course and assignments. Initial results shed light on how students develop information literacy skills over repeated exposure and usage, and suggest that early instruction in use of library resources impacts academic performance over four years. Findings This case study includes a comparative population analysis. Initial results support the hypothesis that students who took HGC performed better on a BC research assignment than those who did not. Positive trends are observed between prior experience and outcomes from 3rd or 4th year chemistry students who completed BC in Academic Year (AY) 2015-16 and were previously enrolled in HGC during AY 2012-13 or 2013-14. The correlation is stronger for the first semester BC assignment than for the subsequent second semester BC assignment, indicating that both early (HGC) and repeated literature instruction are retained and contribute to positive learning outcomes. Additional work includes the examination of the number of resources read versus those ultimately cited in the final assignment and the quality of these resources as judged by journal reputation. Research or practical limitations or implications As a case study, the results are not necessarily generalizable. The research protocol, however, could be replicated by other librarians, instructors, and institutions. The authors share their methodology and initial findings that contribute to the growing body of literature in support of explicitly articulating and assessing student learning outcomes. Moreover, the connections between the introduction of scientific literature to chemistry undergraduates and ultimate long-term student success is demonstrated, which will have implications on collection development practices. For example, instead of conceptualizing pedagogical library materials as discrete and separate from research resources, the interconnectedness of the higher education environment encourages more complex consideration and analysis. Conclusions Research into the value of library resources in the classroom has provided a useful framework for collaboration between librarians and science instructors. The authors have collected and analyzed data on instructional background, journal usage, and student performance, and have used this data to assess levels of achieving student learning outcomes and enhancing their success. Through this work, the authors are pursuing new interdisciplinary avenues for the use of library collections and assessing the impact of teaching information and scientific literacy to science students throughout the chemistry curriculum, elemental-to advanced courses. Originality and value The literature on information literacy typically focuses on library services, such as reference and instruction. Integrating the value of library collections with student learning assessments is an emerging area. The TILISS project is original in its interdisciplinary focus on comparing information and scientific literacy, use of library online journals, and learning outcomes for chemistry students over four years, three instructors, and one librarian. References Pan, D., Ferrer-Vinent, I., and Bruehl, M. (2014). Library value in the classroom: Measuring student learning outcomes from instruction and collections, The Journal of Academic Librarianship, 40 (3-4), 332-338. Retrieved from https://doi.org/10.1016/j.acalib.2014.04.011.
Biography
As the Associate Dean for Collections and Content, Denise Pan oversees the areas of Acquisitions, Cataloging, Collection Development, and Preservation. Her research contributes to librarianship through publications and presentations focused on defining and demonstrating the impact of library resources in research and teaching.
LibPMC 