Sep 132011

This post will take around 231 seconds to read.

Update, December 2011: the final report from UKOLN’s evidence, impact and metrics work has now been published. This includes a summary of blog posts and a framework for metrics.

Hybrid events, aimed equally at both face to face and remote participation, can be challenging. With the need to balance several audiences the event architecture can become quite complex.

The UKOLN workshop on metrics and social web services, which I attended remotely on 11 July, looked specifically at quantitative evidence of impact for social media within higher education. All a bit meta! Clearly a popular topic, the event attracted over 30 f2f participants, and +/- 20 remote (watching the stream and/or engaging in discussions using #ukolneim). Not bad given that the infrastructure was set up on the morning of the workshop.

The event webpage gives full details of the extensive amplication. Livestream was used for video, meaning that the focus was solely on the speaker – and while a talking head is nice to have, the real action was elsewhere, in the slides and backchannel. As juggling three windows is a bit of a challenge for the later sessions I dropped Livestream and kept up by switching between Slideshare and Tweetchat. CoverItLive was also deployed for remote participants’ use during the break-out sessions…while the Livestream chat window was solely used to report issues with the video this meant a total of three places to leave comments.

One of my perennials with streaming, but probably not something widely offered, is the need for a clear subtitle with the name of the speaker and/or title of the preso somewhere on the screen. Remote attendees often dip in and out of sessions and it can be hard to get up to speed with the programme. Being in a different time zone only adds to the confusion! Similarly, event webpage layout needs careful consideration – the programme needs to be at the top, at least while the event is live, with links to any abstracts.

Kirsty Pitkin amplified the event and afterwardd reflected on the need for guidance at hybrid events:

Amplification…is really about using multiple channels…to get a message out to the widest possible audience across time and space. A hybrid event…normally presents the material in a more co-ordinated way to create a crafted event experience…The more hybridised an event becomes, the more centralised the focus becomes, making attendance as a remote participant more about going to a fixed location at a fixed time and receiving a tailored experience…there needs to be some kind of co-ordinated dashboard or page where remote participants can see everything they need in one place.

There are many approaches to event amplification, but in order to move firmly beyond the experimental phase and achieve take-up wíth less forgiving audiences it may be advisable to curtail the number of channels used. How much flexibility do participants really want? When does amplification become volume rather than giving context? No doubt UKOLN’s own evidence of impact has given them insights into what works best, is sustainable, ROI etc.

The official event report and the Storify summary give access to videos and slides from the presentations. Three of my reflections:

  • data wranglingRanjit Sandhu described using Web analytics to model predictions about applications (similar to sales leads) and calculate cost savings of online prospectuses vs print. Interestingly, he commented that the problem in generating applications is not traffic, but rather the path from faculty page to admin/application pages. He also showed a wonderful institutional dashboard or two (see slides). But is the technology dictating what we deliver? Rather than big numbers = good we should be trying to understand recommendations and model reputation.
  • problems with numbers/metrics – a measure is fine as a measure, but once it becomes a target, it ceases to be as useful (Hirst; Goodhart’s law). See also Campbell’s law (targets game the metrics). Summary statistics can lose information and the story in the data. Now numbers are not my strongest suit, but this all reminds me of long lists of links from an event presented as evidence of how successful it was.
  • hashtag communitiesone thing numbers cannot do is demonstrate relationships. Analysing followers on Twitter may give you an idea of how the public view you based on your connections – it’s possible to map how you are socially defined in terms of discipline/specialisms. ‘Birds of a Feather flock together’ and ‘Likeminds attract’ ?

A key takeaway was that raw data may not tell you that much – use pictures and comparisons to tell stories and help put the numbers into context.

 Leave a Reply



You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>