Questions of Usage vs. Impact
Providing valuable insights to enrich the information profession is an understatement in light of the experience I had at the 13th International Conference on Performance Measurement in Libraries. It was a wide scope of activities which comprised of learning, meeting people from different cultures, sight-seeing, networking and fun which impacted me positively in many ways.
It was my first time to visit United Kingdom and I had the honour of visiting Aberystwyth University, Wales the venue of the conference. The weather was warm and friendly and the University is beautiful, located on a hill and providing beautiful scenery as one looks down to the sea. Fferm Penglais, the hostel where I was staying provided comfort and privacy. It was a short distance from the University and I could walk sometimes as I enjoyed seeing the beautiful landscape. The people in the neighbourhood were also very friendly.
I thank the conference organizers for giving attention to details, a factor that contributed to its success. Through keynote addresses, workshops and paper presentations the conference provided a rich combination of learning experiences. There was reliable shuttle-bus service from the hostel to the conference venue and Wi-Fi. The time allocated to different activities was strictly observed. All sessions were educative and at times I was tempted to be in two places at the same time! Goodies like hand-held folding fans, bags, water bottles, pens and chocolates made the conference more enjoyable, not to mention the exquisite Gala dinner, scrumptious daily meals and a visit to the Old College near the seafront.
Dr. Frankie Wilson, Chair of the conference and Head of Assessment at the Bodleian Libraries, University of Oxford, gave the opening remarks and spoke to the conference theme Data and analytics: changing the information landscape. She acknowledged the travel award winners and I was one of them. It was such a privilege for me to travel from Kenya to meet with other International information practitioners and researchers in the United Kingdom. What’s more is that I learned some bit of Welsh for starters! Bore da- Good morning, Croeso- Welcome and Llyrgell- Library among other terms.
The terms usage, value or impact are usually confused in discussions of performance measurement. You will often hear librarians saying, ‘ We count the number of patrons who use our library’; ‘So many accessed our e-resources platform’; ‘We have taught ten Information literacy classes this trimester’, when referring to creation of impact and so on. Actually, most managers insist on numbers as a measure of how effective an institution or service is. The question that comes to my mind is; of the 100 patrons we served or 50 who accessed our e-resources, how many did we actually have a positive impact on? During the good, the bad and the ugly session, Karin de Jager paved way for this argument and it was pointed out that measurement is not the same as impact! Impact can only be measured if there is a difference in the experience of the user before and after a service is offered. The discussions established that these are the misconceptions that many people have about impact. This resonated throughout the conference with the keynote speakers, workshops and in the paper presentations, reaffirming my thoughts before I came to the conference. I am taken aback to issues of monitoring and evaluation and I liken these two to usage and impact respectively. The aforementioned examples would be measures of usage and not impact.
The Keynote addresses shed light on the issue of value and impact of digital collections, Open Science and data librarianship and how these will influence the future of information professionals. In the first keynote address, Prof. Simon Tanner, Director of Digital Humanities, King’s College London in his ingenious presentation about ‘The Balanced Value Impact Model: understanding the impact and value of digital in libraries’ illustrated how the impact of digital resources can be measured to be of benefit to users. The BVIM v.2 emphasizes value lenses, strategic perspectives and impact. I particularly loved the way in which he provoked the audience to state the values they attach to digital collections through a voting exercise. The values included Community, Education, Existence/Prestige, Inheritance/ Legacy and Utility. This opened my mind on how people think so differently!
In the second keynote address, Prof. Dr. Konrad Förstner, who works at the Information Centre of Life Sciences, Cologne-Germany skilfully presented on ‘Data librarianship: Preparing librarians for their future tasks.’ His presentation demonstrated and sensitized the audience on the importance of librarians understanding issues of handling data. He indicated that in Germany, plans are underway to train librarians in data librarianship. He further unfolded Open Science especially on how in many ways it is influencing the information sector. A good example is in publishing where he pointed out that open access might soon become the default mode of publishing. He further pointed out that research librarians have a major task to preserving research output and providing access to it through open access means.
The workshops I attended were amazing giving me practical skills that I will apply at work. Starr Hoffman, Director Planning and Assessment, University of Nevada, Las Vegas Libraries, USA presented on creating an outcome-based assessment plan and inventory. Lessons learned were that as one is preparing the assessment plan, one must factor in the organizational values such as, increasing efficiency, focusing on user needs, quality and continual improvement, demonstrating impact and value, ensuring data and performance transparency, support of decision making and issues of Return on Investment. Sara Goek and Emily Plagman presented on Project Outlook. I learned that it has an inbuilt online designed toolkit to carry out surveys. As the name suggests, libraries can easily measure patron outcomes to understand how beneficial their services and programs are to them (ideally a measure of impact). It is a free and open source software and I managed to register for an account with them.
I got valuable insights regarding different methods used in measuring performance by various institutions. Mrs. Selena Killick’s presentation titled, ‘But what about us? Developing an inclusive approach to library insight particularly interested me. She is the Associate Director at The Open University Library and shared on how the library cooperates with a Library student research panel (that incorporates all user groups) to provide evidence based performance measurement and assessment in informing decision making. David Marshall’s presentation on ‘The use of remote, digital contextual enquiry through the mobile app ‘dScout’ in the Futurelib ‘Student Learning Journey’ project at the University of Cambridge shed light on how using the dScout app, the library is able to ask questions such as, How are students studying, How do they feel when doing so, How do they reflect on their academic and information literacy skills and practices and so on, enabling an effective measure of impact.
Library strategic plans sometimes do not align to key performance indicators that speak to the University wide strategic plan as was demonstrated by Mahlangu in his framework for evaluating effectiveness of university libraries. Important Key Performance Indicators include student enrolments, student retention, student success, student learning, student engagement, faculty research outputs and faculty teaching. Mahlangu’s findings collaborates Ellie Kohler’s and Connie Stovall’s in their paper, ‘On Creating, Recalibrating, and collecting assessment metrics for Strategic Initiatives and Annual Reporting: A case study at Virginia Tech.’ as well as Cecile’s paper on ‘European Academic Libraries KPI: how comparison helps decision making’ in the fact that emphasis is laid on relating Key Performance Indicators to the Institution’s strategic plan and focus being on improving student success and research performance among others. Ira Foltin’s and Roswitha Poll’s paper on National Library Statistics: how to get reliable data exposed important elements to consider in recording library statistics.
Library spaces was another important aspect of measurement that was discussed. Dr. Frankie Wilson’s presentation revealed important findings. A library study space is different from any other study space in the sense that it is not just about the silent studious atmosphere but by a culture or perception of those using it. Group learning can only be achieved in a library set up. Martha Kyrillidou’s project on small and rural libraries on using a design and implementation strategy given by OCLC to support community engagements, established that transformations in active learning can happen even for small libraries in rural areas.
I would go on and on in explaining about my experience but the truth of the matter is that I cannot put it all on paper. The long and short of the presentations is that in matters pertaining to performance measurement, there is more than meets the eyes in numbers which is the true measure of impact. Information professionals need to look beyond the numbers and establish proper metrics that provide insights on value or impact of their services.
My special thanks go out to the conference organizers and sponsors who enabled me to travel all the way from Africa to meet with International participants in this prestigious conference. I am truly humbled and honoured because of the extent to which the committee stretched itself to meet my travel requirements for me to attend the conference! I’m proud of you and I pray for the success of all future conferences as the knowledge acquired will transform many institutions. I plan to disseminate the knowledge and I look forward to a time when such a conference will be hosted in Africa. Diolch!