#HLG2016 Metrics made practical

As promised here are my slides from #HLG216 in Scarborough.

A few thoughts follow to build on the content in the slides for those not around at the time / to make them a bit more helpful generally.

As my talk progressed I became very aware of how the metrics work linked closely to my day to day. We operate in a functional structure and I am part of a team responsible for Partnership & Liaison. This translates for me into wanting to have lots of meaningful conversations with people.  Metrics are a means to that end. I didn’t mention it during my talk but this year I produced Annual Action Plan style reports (a la York) for the NHS Trusts I work with. These proved much more engaging than my old annual report. They featured a small number of carefully picked metrics with explanations of what I thought they might mean.

The presentation builds through the various models / methods we considered as we researched the use of metrics.  It was good to tap into experience in the room of why our measures can be unconvincing, hard to share and obscure. I was really pleased to find by chance (picked up from the weeding trolley at work) a 1990 text by some of the greats “Quality assurance in libraries: the healthcare sector” which strongly affirmed the areas people were focusing on and some of the approaches under way. Not much changes in the end.

We ended up with four overarching principles for good metrics

Meaningful – the core of this is that the metric must be something people (other than you) care about. It should be aligned to organisational objectives and be readily understood by stakeholders.  You need to talk them about it! An extension of this is to remember that framing metrics as a target should be approached with caution. There is the potential to set targets for the sake of it that will lack meaning for stakeholders. Tell them how you are performing and then discuss whether this is more or less than they need. Metrics need to be kept under regular review to reflect changing priorities and remain meaningful.

Actionable – for a metric to be useful for us it needs to be in an area we can influence. If we cannot make it move one way or the other it then we don’t want to be held accountable for it. A good metric will drive changes of behaviour and service development.  We also need to remember that the metric is only an indicator and we need to carry out appropriate research to back up what we suspect the figures are telling us.

No numbers without stories – no stories without numbers

Reproducible – this principle contains quite a lot.  It starts from the position that tracking a metric is a piece of research. Accordingly we need to be transparent about our methods and we need to be so before we collect the data. We should use the best data available to us.  Replication implies that we should get consistent results if two people examine the same thing at the same time.  Finally we also want our metric data collection to not be excessively burdensome. If it takes two solid months to crunch the data then it probably isn’t reproducible (or you would really have to get an awful lot from it).

Comparable – finally we want metrics that allow us to see change over time.  Often we will need to recognise that this can only be internally. We may be able to look to benchmark externally but we should be realistic. Even if we are transparent it will remain difficult to establish consistent data and there are frequently influencing factors that we may or may not be aware of. For example – what is the impact of being in a Trust three times bigger? or with three sites? or thirty? what kind of staffing model is in place? how is the service funded and delivered?

All this is a fair bit to keep in mind so the Metrics task and finish group prepared a Quality Metrics Template. This is designed to support people in creating, documenting and sharing their metrics.  The slides include a worked example and others were distributed in the room for the final group work section were people had a chance to start drafting some out or just discuss the principles.

In discussion the potential was seen to use completed templates as the basis of a process of refinement seeking best of breed metrics around particular questions.  Hopefully a tool will be available to collect them in the first instance and then an approriate group might be assembled. There was some concern that metrics might be imposed but this strikes me as unlikely.  The diversity of services and the needs of local stakeholders mean that one size will definitely not fit all.  There was discussion of the NHS national statistics return and the importance of considering these in the light of the principles.

I hope people will find the principles and template useful. It was great to talk to such an enthusiastic audience. More conversations please!

Revalidation – going for the threepeat

Spookily almost a year to the day since I last submitted I have once again completed the documents to revalidate my MCLIP.

I hope the current discussions at CILIP will finally see the launch of the online register of practitioners the long overdue public face of revalidation.

It has been another packed year professionally with a host of new conferences, visits, LibUX, Metrics and more. Plenty to reflect on! If anyone wants to talk revalidation do give me a shout – happy to talk you through it.

This years mandatory revalidation celebratory tune from MJ Hibbett and the Validators is – Things will be different when I’m in charge from the Album “This is not a library”

April 2016 Cilip Update (plus October 2015)

Items of note in the new issue and the back issue…

April starts with the usual news.  A small item in the news section about Sci hub is hopefully a signal for some deeper analysis of this.  Given the universal preference for no logins and simplicity in access this is one of the big stories in scholarly publishing.  The creator of the site has a very different definition of Open Access to the one librarians would recognise but it feels like this could lead to a more rapid transition to a more legally realistic Open model.

The opposite page links nicely with Sandra Ward discussing the need for skills around risk, security and data privacy.  Good luck to the people who have given Sci hub their network logins and think they will only be used to access journal articles.

The article on the value of trained library and information professionals is encouraging. The full report will warrant reading.

I enjoyed reading about work to bring student art into the library space in Wimbledon. Not sure how we could support something of this kind with our students. Adding visual and physical learning aids would be a start.

Now back to October 2015 with a health focus apparent…

News wise this feels like an issue where CILIP HQ is starting to show the response to member pressure for a more actively advocating institute. The initial response to the AGM resolution to oppose amateurisation was underwhelming but it feels like My Library By Right and other work is now moving things in a better direction.

An item on faculty / librarian collaboration looks very relevant to my interests – I had missed this one at the time and it sounds like a potential team journal club conversation starter.

The cover article on Knowledge for Healthcare is a good run through. The one year updates are now circulating and a lot has been achieved. My own involvement in the Metrics work has been stretching both in terms of building knowledge on the topic but also in leading a distributed team. I wish I had pushed for a face to face meeting sooner as this really improved our subsequent work together.

More health with an informative article from RCSEng on their collection review. I am lucky to sit as an external on their joint Museums, Archives, Library and Surgical Information Services committee and this article really filled out the story on work I had heard about there. The close working relationship presented here is a great example of how heritage collections can underpin modern practice. I highly recommend the recently opened temporary display at the Hunterian on vaccination which includes letters from Edward Jenner, film, moulages from the Gordon Museum and historic antivaxxer postcards.

It was useful to think about what happens to our social media accounts on our death. It some ways a companion piece to the talk by Brian Kelly at Internet Librarian 2013 ago on what happens to your digitally speaking when you leave an employer. I definitely have some work to do to be ready for either of these eventualities.

Update March 2016 (and November 2015)

A new issue of Update means a new back issue pulled screaming from the vaults.  First up November 2015…

I like the Karen McFarlane definition of KM as ‘good IM practices alongside knowledge sharing behaviours’. It helps get past the nagging voice of Tom Wilson in my head whenever I see KM.  I am looking forward to seeing what the NHS Knowledge for Health group working in this area comes up with.

A note of a warning from the Local Government Ombudsman that outsourcing does not remove responsibility for quality of service is interesting in context of a recent article on council insourcing as contracted out services prove inflexible and expensive.  The plans for Gym Libraries in Lambeth is a good example of what can come from this.  This looks like a straightforward reduction in service and commercial land grab.

The puff on two library supply companies merging is seriously lacking in any critical examination of whether this is in fact a good thing for libraries.  There is rather a lot of churnalism style content press release cut and paste in this issue.

Great to read David McMenemy as a much needed counter weight to the “news”. It was really helpful to have something place developments in the profession in the context of developments in politics and society.  In a recent consideration of a team goal I drew up a statement that reflected values around empowerment of people to advance the cause of human knowledge.  This was felt to be hard for people to relate to their day to day work but this article reassures me that we should be grounding our practice in the long term and those things that are true.

An article on reinventing public libraries was interesting on two counts.  The “community driven” future vision sounds to me (as a non public librarian) to be what I would have expected to be the model already (I have had some experience of the work of commonfutures that they mention as it happened in my local library).  I was also interested in the 6 future services they identify.  Open content has real potential – moving beyond the current limited access available via Finch.  The role I would envisage would be in supporting research and access. I was also interested in the combination of MOOCs and widening participation.  This struck me as something with real possibility.  Working with students we can see how important the library is a place that is warm, safe comfortable and equipped for study. There must be a place for more collaboration between education and the public library network.  An obvious example is that my local library is no longer open in the evenings when it would be most useful for students from the local college (as well as those who work away in the day time).

I have to declare an interest in the next article on the Quality and Impact work stream of Knowledge for Healthcare. I contributed as Chair of a task and finish group looking at metrics.  We have prepared a report on principles for good metrics which will hopefully surface soon. As part of the research I prepared a poster for the LHL NHS/HE conference in the autumn. Annoyingly the group members are listed incorrectly – the main folk involved have been Dorothy Curtis, Lorna Wilson, Tracy Pratchett and Richard Parker.  It has been a real challenge working with a group spread across the country. It was striking how much more effective we were after having had a face to face meeting – something I would certainly try to have sooner in future such projects.

Back to the present – March 2016 Update…

The launch of the Cilip Online register of practitioners is announced as set for the 14th of March – not before time. I had an enlightening conversation with a colleague who I would have expected to be all in favour but who had voted against revalidation.  Sadly I think it reflects a need for much greater communication and engagement around the new revalidation process.  The traffic on jiscmail lists on this topic was depressing.

The Yale MeSH Analyzer looks like it could be a nifty tool.

An article on life as an outsourced library chimed with what I just been reading from the November issue.  The experience shared does not seem to me to offer any great advantage from being outsourced other than reducing MoD headcount. Providing a service to other organisations myself under SLA I know how much trickier it is to work with an organisation when you do not fully belong to them – I suspect this is less of an issue for the case discussed but I doubt it helps.

Work on developing data science courses connected to recent involvement I have had with information governance around data and high performance computing. There are major management problems to anticipate and a big role for metadata managers.

The Knowledge for Healthcare update this time is on the major work around Resource Discovery.

The cover article is on IP crime.  The focus seems to be mostly film / music with a side order of small fry ebook theft.  Missing is the storm around Sci-hub which is taking the unlicenced sharing of journal articles to a whole new level. I hope there will be something on this shortly as it raises enormous questions for a swathe of work we do.

Finally – a listicle from Matt Holland – I think a lot of what he says this time could be applicable to my own role where I do not often work at home but do not have a “home” at work either.  I like the idea of “not being strange” I would extend this to say “have some boundaries” – when work follows you home it can be hard to turn off.

Hello 2016 (goodbye 2015)

The new year is going to see a shift in the balance of my role. Over the past couple of years I have been delivering a fairly steady stream of sessions on literature search and basic critical appraisal. The time has now come for me to hand the bulk of this work over.  I will be sorry in some ways as I enjoy delivering training, get positive feedback and it is useful for meeting people. I won’t miss the admin and look forward to being able to focus more on the engagement part of my role.

In terms of last years plans it has been a mixed bag.

I did revalidate OK (and am very disappointed that the latest attempt to require this failed to pass). I completed the first year of my AKC which was a relief given the very different nature of this course and the need to complete a hand written exam. I also enjoyed running my workshops at UKSG.

Less successful was clearing the CILIP Update backlog – I ended 2015 even further behind than I started it. I singularly failed to blog about an article from each issue of HILJ. I got next to nowhere with the PKSB.

I do not plan to review all of 2015 here – there are plenty of posts to look back over to give a flavour of it and I will doubtless reflect more when I complete my 2015 revalidation.

I am going to have another stab at clearing the Update pile. I will do HILJ if I get to it. I want to have a look at the CILIP CPD system generally and the PKSB remains a target. There will be work to do on the Metrics group I am chairing and I suspect I will end up talking about that at HLG.

I am looking forward to the year ahead a great deal.

From the RSM to the RCN

Royal Society of Medicine https://www.flickr.com/photos/pressreleasefinder/5389444315/

I took a trip with a colleague to the Royal Society of Medicine Library recently. I was interested in the range of services they provide and the kinds of spaces on offer.  The library is spacious with substantial areas dedicated to the extensive collections.  Paper journals are being phased out and this will release space for other purposes. Special collection type materials were presented in a heritage centre.  There are regular temporary exhibitions.

Generally there was a quiet atmosphere that felt generated by the users of the space. It was notable how much space each user had – very much one chair per desk.  Some areas felt like they were due a refresh and it was notable that power sockets were an issue for one user who was struggling to lift a hatch in the floor.

Training sessions were longer than the ones we tend to offer. Potentially this is more acceptable as people are already taking time out to go to the Library where courses closer to work may face more pressure to get back to the ward.

A series of study carrels looked well used and a small flexible group room at the end of this area was nicely presented. Green glass wall mounted panels allowed for note taking and would have good colour contrast.

An area outside the library had some really funky furniture and it would be nice to see some of this move into the Library as they dispose of the journal display furnishings.

Walking back to the tube we dropped into the Royal College of Nursing Library and Heritage Centre.  This was a huge contrast with the RSM Library having been the subject of a major redevelopment in the past few years.  The picture on the front page of their website gives a good idea of the feel.  There is a cafe area open to the public and a very well presented selection of nursing heritage related items.

I liked the fabric sound baffles but suspect that there must be noise issues due to the mezzanine design.  There were a  number of nicely parceled off spaces for small group work and a flexible training space much like the one at the University of Greenwich.  A digital room booking display for this was very clear and allowed immediate on the spot booking which was a nice touch.

Generally there were lots of different working environments in a fairly tight space.  There was a good amount of space required for the book stock.  Shelving was white as were the walls and most of the display type units.  This made things very clean and bright.  Excellent use was made of graphics to make it less overwhelmingly white – there were full colour blow ups of various items from the archives that were attractive and set a good organisation specific feel.  Signage was notably clear.

I loved the finish and feel at the RCN though I suspect if I was going to pick one to study at myself I would likely go to the RSM.

RSM image by PressReleaseFinder (C) Creative Commons

 

A trip to the pharmacy

Some time ago now I visited the Library at the UCL School of Pharmacy.

After an informative talk we went for the usual poke around.  Access is via a rather narrow space which holds a lot of kit.  There are currently several print systems in operation – a not uncommon complication but unlikely to be helpful for staff or users.  A new self issue system was being put in operation.  The space was rather odd but did serve as a useful noise barrier.

The Library itself is fairly compact and combines white walls with glass accents and red furniture for a bit of colour.

UCL School of Pharmacy Library desks

A range of study spaces are available with a small amount of group study space.  Frosted glass dividers allow light to pass while offering privacy. An area for group work was divided off using large wooden pUCL School of Pharmacy Library group areaanels that were attractive but would be tricky to shift I suspect.

 

 

 

 

 

UCL School of Pharmacy Library display case

 

Some use was made of special collections type materials to give it a more pharmaceutical feel.

UCL School of Pharmacy Refectory

But I could not help wishing they had been able to have the space now used as a refectory that visitors to other libraries of the period (eg Senate House, NIMR at Mill Hill) will recognise as quite clearly designed for library use!

A Wellcome trip

L0023184 The Food Value of the Banana. Boston, 1928

I had the pleasure of visiting the Wellcome Library recently (trip organised by CILIP ARLG L&SE).

While I have visited the revamped Collection spaces on a number of occasions the last time I was in the Library was some moons ago to hear Robert Kiley talk about the Internet and other such new fangled things.

The space very much matches the transformation downstairs.  I loved the shelf end bays that use images from the collection linked to the materials in that area.  There was good use of furniture to create different spaces though the book stock does dominate a lot of areas.

The Reading Room was an unexpected change and one I look forward to returning to.  The mixture of books, exhibits, art, interactives and seating was great.

We had a visit to the scanning mines of the UK Medical History Library project. They are going great guns in digitising large numbers of books including some from my employers special collections. A fascinating resource is being built (though the most viewed items are unsurprising).  I had not seen a foot pedal operated scanner in action so it was good to get an idea of how these work.

We also popped into the more cautious digitisation work where all manner of skills are in place photographing the large collection of early books.

A rich collection and one that I will heartily recommend to all those were it might be of use.

Image Wellcome Images L0023184 The Food Value of the Banana. Boston, 1928

Update catch up February 2015

You know the score by now…

A big piece on the Sieghart report the recommendations of which look pretty thin to me.  Not impressed latterly watching how the wifi actions are unfolding with overt commercial advertising presence and a bidding process rather than a universal offer.

The same news section carries details of the disaster at the Library of Birmingham which appears to be being gutted and propped up partially with more advertising cash.

Ask folks about the library and they tell you it is fine – but they don’t know any different. Paraphrasing R David Lankes.

A good bye article from Annie Mauger.  CILIP came on under her leadership though it still seems to be some way off winning back a lot of hearts and minds sadly. I hope she is enjoying her new role.

Phil Bradley tells me I should join Facebook for the good of my professional activity.  This is a tough one as I have steered firmly clear thus far and would prefer to keep it that way.  Not shy of social media but do I want another place?

I was interested to read about the continuing work around the Knowledge and Information Management as a recognised profession across the civil service.  These must be very tough times in those services and seeing a positive approach based around professionalism and skills is heartening.

Not an early Latin printed books expert but enjoyed the article on overcoming challenges around digitisation of these texts.

Ben Showers article linked to his book on Library Analytics and Metrics is very timely as I am now chairing a Knowledge for Healthcare task and finish group on this very topic.  His three areas:

  1. Measure what really matters
  2. Don’t collect it, or measure it, if you’re not going to act on it
  3. Make as much data available as possible

Are bang on really (and chime with other interesting things I have been reading and will blog about later).  I really like the idea that our data should be like dandelions – finding niches where ever they can rather.

Update catchup December 2014 / January 2015

Social media traffic this week from people pondering cancelling their membership of CILIP accompanied by statements that they had just popped another Update unopened on the pile.  I think this series of blog posts has shown that at least for this member there are always things of interest.
Who knows what will be useful in the future?  Not reading something is not going to help that is for sure.  Anyway that said, despite the double issue, this Update is not awash with articles that interested me.
There is a huge amount of news – almost half the issue.
CILIP Electionwatch was launched.  I was pleased to see this attempt to raise the profile of libraries as an issue at the general election.  Sadly the early days of the new government promise to be as destructive to library services as the equivalent period of the previous one. Hopefully the new CEO will bring continued focus on what are existential issues for many services.
The Library A to Z launch is also reported. Having backed this campaign I was disappointed that CILIP did not do so.  There were other organisations involved and I can see no reason why they were not.
There are some nice pictures of the new Uni of Greenwich Stockwell Street Library – since visited and discussed on this blog!
The book review of “The One-Shot Library Instruction Survival Guide” makes this sound an interesting read – one to recommend for the professional collection at work.
Spotted my name in the list of revalidating MCLIPs which was a nice surprise. I am waiting on this years efforts.
I liked the idea of the 23 librarians campaign to raise public awareness of the diversity of roles that librarians carry out (and their skills and impact).
Finally the article on student workers at Teeside Uni library was very relevant to developing areas at work. They used the Peer Assisted Study Sessions model which is doubtless old news to many but not something I recall meeting before.  It is great to see something building on what works.  Having looked at research literature on student roles around academic libraries there is a lot of reinventing and then not evaluating the wheel going on.  The article has what appears sound practical advice. One to share.
PS. Included in the envelope was the CILIP annual report with quote from when I submitted my first revalidation – had forgotten about being asked!