HILJ CPD reading Volume 35 No 3 – Developing a generic tool to routinely measure the impact of health libraries

Welcome to the second experimental online reading group aimed at encouraging discussion of interesting articles in HILJ.  The first attempt took place around Volume 35 No 2 on CILIP Social Link (link may require CILIP login and may not take you to the right place).  Unfortunately we found SocialLink did not really offer quite what was needed so future editions will rove across any ones blog that cares to host.

I raised the possibility of having a regular discussion on articles from HILJ at HLG2018 having muttered about it for some time and as others expressed an interest (in particular Lisa Burscheidt, Morag Clarkson, Catherine Mclaren and Tom Roper) here we are.

As an HLG Member you should have access to HILJ via the link below https://archive.cilip.org.uk/health-libraries-group/health-information-libraries-journal/access-health-information-libraries-journal-hilj though many have it in a Wiley bundle and that maybe easier! The article this time is OpenAccess anyway so should be straightforward.

The idea is that an article will be selected from each issue to be discussed. The group have picked an article but there might be a vote in future or we may carry on picking a favourite by some other means (perhaps the host blogger gets to choose). The intention is to select articles with practical applications. We will offer some questions as prompts but the discussion can go where interest takes it.

The article selected this time is:

Developing a generic tool to routinely measure the impact of health libraries

Stephen Ayre, Alison Brettle, Dominic Gilroy, Douglas Knock, Rebecca Mitchelmore, Sophie Pattison, Susan Smith, Jenny Turner

Pages: 227-245 | First Published: 18 July 2018

Abstract
Background

Health libraries contribute to many activities of a health care organisation. Impact assessment needs to capture that range of contributions.

Objectives

To develop and pilot a generic impact questionnaire that: (1) could be used routinely across all English NHS libraries; (2) built on previous impact surveys; and (3) was reliable and robust.

Methods

This collaborative project involved: (1) literature search; (2) analysis of current best practice and baseline survey of use of current tools and requirements; (3) drafting and piloting the questionnaire; and (4) analysis of the results, revision and plans for roll out.

Findings

The framework selected was the International Standard Methods And Procedures For Assessing The Impact Of Libraries (ISO 16439). The baseline survey (n = 136 library managers) showed that existing tools were not used, and impact assessment was variable. The generic questionnaire developed used a Critical Incident Technique. Analysis of the findings (n = 214 health staff and students), plus comparisons with previous impact studies indicated that the questionnaire should capture the impact for all types of health libraries.

Conclusions

The collaborative project successfully piloted a generic impact questionnaire that, subject to further validation, should apply to many types of health library and information services.


I picked this article as this has been a hot topic for some time now.  I expect many of us will have experience and views on the generic impact questionnaire so there should be useful discussion.  I have not read the article before selecting it!

Starter Questions –
What? What do you think of this article / the generic impact questionnaire / etc?
So what? Does this change your view of the tool?  What changes might we want to see with the tool?
Now what? Are you going to do anything with it?

The next edition of the HILJ CPD Reading experiment (name suggestions welcome! #HILJClub perhaps?) will appear when volume 35 no 4 appears and be hosted by Lisa Burscheidt over at That Black Book.

Look forward to the discussion!  The comments box is further down in this template than I realised so do scroll down to reach it!

Advertisements

Journal Clubbing – Understanding Academics: a UX ethnographic research project at the University of York

Summer edition of my team journal club this time we read

Blake, M. and V. Gallimore (2018). “Understanding academics: a UX ethnographic research project at the University of York.” New Review of Academic Librarianship: 1-25.

I picked this article as I was interested both in the methods used and the potential findings.  I lead on UX work and Uni of York consistently do interesting things in the liaison and engagement sphere so this was an easy one to select.

Overall it left us wanting more.  The methodology section was very light and did not address a number of questions that would have been useful to support the article.  There is no detail of the recruitment strategy or of the semi structured interview schedule for example.  We wanted to know more about the cognitive maps and it would have been great to see a bit of these. While we know that new contacts were made we do not know to what extent the data were gathered from already friendly faces.  They would at least have been sufficiently well disposed towards the library to engage in an extended interview exercise.

The description of academic lives was felt interesting but not surprising. We did wonder if there was a nervousness in writing something that would be published and visible to the interviewees given the emphasis on relationship building.  Generally we wanted to push further into the questions.  A colleague had recently attended a “secret life of an academic” talk where a number of important topics were discussed that have not surfaced in the article at all.  We wondered about the absence of research data management from a library perspective.

Finally we wanted to know more about the changes that had resulted.  This was clearly a major undertaking and the need to see impact from this was felt imperative.  The section on how the data was used to generate user requirement for a change of Reading List software would have been brilliant to read – how did they do this? what difference did it make versus what was known already? etc.

Having said all this – it was a good article for our journal club prompting lots of discussion.  We had useful thoughts on what we might want to know from academics and how we might ask them.  And colleagues at York definitely have expertise and experience we would value!

 

 

On reading – Frugal Innovation

When I started in libraries a couple of years ago (ahem) I thought we were pretty stretched budget wise.  It turns out I started in a period of relative plenty for NHS libraries with a relative flood of cash coming into the system.  Despite that we were always trying to see how to make the money go further while dealing with the expanding possibilities from all things tech.

And here we are now – pressure on the money we have, not a lot of money in prospect, ever growing demands and possibilities.  So what to do?

I saw a positive review of “Frugal Innovation – how to do better with less” (2016) Radjou, N. & Prabhu, J. so thought it might be worth a read (you can check out the brief version and watch the TED Talk if you like).

The book is heavily based in the corporate world but is useful for all that.  There are a wealth of case studies included which help illustrate successes (but not much talk of failures that I recall).  The authors identify six principles for frugal innovation that I will consider in a library context.

Principle 1 – engage and iterate

I think there is a lot of potential for libraries here.  A big part is how we can become more agile.  Smaller NHS libraries have a real advantage here. A small team working in a manageable sized organisation can rapidly take an idea from light bulb moment to launch (provided it doesn’t cost too much money!).  I loved working like this and always enjoy seeing people doing things like this in the Sally Hernando awards.  My experience is that this kind of agility is definitely harder to drive in the much larger services found in a university environment.

We are increasingly active in seeking (and gaining) engagement with the users of our services. This can only be to the benefit of the service.  We are not our users and the more we can understand their motivations, needs and world the stronger the chance of us innovating towards them.  UX work is making great progress with understanding the library in the life of the user.

Principle 2 – flex your assets

A lot of the examples in this chapter are from manufacturing.  Sharing resources is a path libraries have long pursued and there may be yet more mileage to go.  We could consider also how we can improve our supply chain. Big academic services still buy a lot of stuff so there must be potential to work better with suppliers to direct this.

In many cases libraries have a significant amount of prime space and there have been some good initiatives aimed at bringing related activities into that space to drive better uptake of evidence resources and services.  How might we package our services differently to bring them closer to the people who need them?

Principle 3 – develop sustainable solutions

Improving our sustainability can have positive impacts.  For manufacturers they can turn waste products into other products or find others who need to buy them.  There are markets for our used books (though most things are only fit for pulp when we have finished with them).  Encouraging reduced use of plastic can have an impact on our waste bills.  Sustainability is also a big driver for many of our staff and can create real engagement around thinking about how we might encourage reduced use of paper for example.

Principle 4 – shape customer behaviour

There are some interesting ideas here about gamification and visualisation.  These cna be used to encourage positive behaviours.  Currently peoples data is quite locked up in our systems – could we provide ways for people to understand the pattern of research they carried out with our tools?  Another area where we might find interesting lessons is around the use of social pressure.  With pressure on study space at peak times we might explore ways to encourage good citizenship.

Principle 5 – co create value with prosumers

This should be a great one for libraries as once you strip away the awful prosumers term you are talking about building relationships with engaged library users to create champions.  NICE Evidence champions have been an effective means to support peer to peer teaching and building expertise in the user base has the potential to greatly extend our reach.  Perhaps there is the scope to bring more people into our teams in loose ways?  Can we power up the PDNs or rope in the pharmacists?

Principle 6 – make innovative friends

We could all do with more friends.  It would be great to see a more systematic approach to using the good stuff in the Sally Hernando Awards.  We should also look at working our way into the Academy of Fab Stuff (I know some are already there) as this will put us in touch with people who are moving things forward and looking to improve.  Working through networks is the norm for NHS libraries in particular and I would argue that this is the base for hyper collaboration options.  It would be good to see us getting more interesting people involved with improving library interfaces.

All in all this was an interesting book.  The business orientation is a bit of a barrier but the ideas shine through.  Time to do more with less again!

 

 

 

Journal Clubbing – Principles and practice in impact assessment for academic libraries

A change of tack at the team Journal Club with a paper on impact assessment

Christine Urquhart, (2018) “Principles and practice in impact assessment for academic libraries”, Information and Learning Science, Vol. 119 Issue: 1/2, pp.121-134, https://doi.org/10.1108/ILS-06-2017-0053

It was a different paper in that it was a literature review so was rather more general than some of the articles we have been using.

We found the idea of reciprocal value propositions worth exploring.  What happens when these go wrong?  There is a danger in both sides being willing but not always able to deliver on what they might wish for.  Opportunities for using value co-creation could also be imagined.  The idea of student reviews on the value of particular reading list choices could potentially lead to some tricky conversations but would only be providing a formal recognition of discussions that already take place amongst the students.

In many areas the literature points to the importance of strategic alignment with our organisations wider goals.  This is not revolutionary but worth trying to do well.

Discussion on the time involved in qualitative data work along with more generally on gathering and managing feedback should help us in future to more carefully frame what we are trying to achieve.  GDPR should also drive care over data collection and retention. Just because data might be available does not mean it is practically or ethically desirable to use it.

I found the referenced paper Mengel, E. and Lewis, V. (2012), “Collaborative assessment”, Library Management, Vol. 33 Nos 6/7, pp. 357364[Link] on developing a set of measures for a balanced score card particularly interesting given the difficulty of this task and passed it to relevant colleagues.

Overall we found a lot to discuss in the paper but I am not sure it worked as well as some of the research papers we have used previously.  Too often we were left with too little information without going on to read the underlying papers.

 

On reading – Libraries and Key Performance Indicators (2017) Appleton

One of the fun things I did last year was contribute a case study based on my work with the KfH Metrics Task and Finish Group to the book “Libraries and key performance indicators: a framework for practitioners” by Leo Appleton.

Cover of book

I was really pleased to have the opportunity to share our work in this way and to get my name in print!

Prompted by reading a review of the book (in the December issue of the HLG Newsletter) and by an upcoming workshop I am preparing for health librarians in the North I thought I would have a read myself.

It is a compact book at 150 or so pages including references.  I think brevity has a lot to recommend it in a practical text and this could be dipped into or read completely fairly quickly.  It covers a lot of ground in a short time including a useful review of past efforts at performance management in library services and the influence of current trends around user experience approaches.  There are a number of examples from different library sectors which is useful for widening the perspective.

There were areas where I would differ – for example around the amount of confidence that can be placed in the various statistical return series.  Changes are coming to the long standing NHS statistics return reflecting careful consideration of how useful a number of these measures are in practice – particularly given likely variation in collection.

The chapters on methods provide good overviews with references to follow up. The librarian tendency to count anything that moves has been exacerbated by the opportunities offered by digital resources to do this and the book is good on tempering this enthusiasm.  I would perhaps have liked more on how to manage a regular flow of qualitative data in such a way as to support KPIs.  A contribution to a bundle of performance indicators across a single KPI perhaps?

Terminology is a bit of a muddle and I found myself confused at times about what was being referred to.  A definition of a KPI is provided but merits clearer flagging.  While there were a lot of excellent warnings about potential pitfalls and dead ends I wonder if more could be done to highlight the positive ways forward?  The various case studies were useful in providing some idea of how people have been able to advance with this work.

It was a relief to read my case study in context and I think it makes a useful contribution to the book. The principles advanced in the NHS Metrics work are widely applicable and certainly supported by the wider research presented in the book.

Having declared my bias up front – I think this is a useful book and I hope people will read it!

Journal clubbing – impact of physically embedded librarianship on academic departments

After something of a gap it was good to have a return of the Journal Club at work.  The article this time was

The Impact of Physically Embedded Librarianship on Academic Departments – Erin O’TooleRebecca BarhamJo Monahan 2016 Portal: Libraries and the Academy

This was interesting for the team as a way to consider how librarians might best approach closer working with faculties and in particular whether physical collocation is important.

The article examines the impact of a shift to three liaisons being based more with their faculty following changes to the delivery of enquiry services within the library.

There is a big emphasis on counting different routes to interactions.  The picture from these figures is unconvincing.  There are a number of variables that can be controlled for.  There is little consideration of any change in the type, quality or depth of the enquiries.  This would be more useful to know – a fall in enquiries could be a positive thing if more useful enquiries are replacing them.

Given the focus on quantitative data it was also disappointing to not have any examination of data around their use of Libguides.

Generally the study would have been more interesting by including qualitative elements. There is a brief mention of chats with faculty and it would be these interactions that are interesting.

So a helpful paper from prompting discussion but not one where you can draw much that is transferable.

 

On reading – an end to the Update updates?

I read quite a lot. I suspect this is not an unusual statement for people working in library and information roles.  Keeping up with the constant flood of things I want to read, happen to read and feel I should read is a challenge.  This blog has recorded the evidence of me working through one of my back logs and I have been tackling slightly smaller piles of a whole raft of other regular publications.  Generally things are starting to look okay with me reading about stuff that actually happened recently.  I haven’t really got back up on my blog reading – perhaps a next project.

Anyway – one conclusion from all the Update updates is that while I value some of the things I read I suspect that my ramblings are of less value (even to me).  Accordingly future Update updates will be perhaps be about a single item that particularly strikes the imagination.

I was pleased in the Dec 2016 / Jan 2017 issue to see a snippet about the publication of the Principles for Metrics report that was produced by a task and finish group I chaired for the Knowledge for Healthcare programme.  Work continues on creating ways to engage people and build their understanding of metrics (see previous posts on this). A new version of the quality metrics template is under development to help with this.

On a personal note I was glad to read an appreciation of Julian Lendon who died last year.  I never knew him but the Solihull Central Library he oversaw the planning and building of was a formative place for me.  This video preview for a talk there on Beatrice Cadbury gave me a chance for a little look around and is a nice example of how libraries make a difference.