Wednesday, 8 October 2014

Three steps to developing a successful app

This comment is based on studies of healthcare apps, and some recent conversations I've had, but I'm guessing it applies more widely:
  1. Consider what the 'added value' of the app is intended to be. 'Because it's the 21st century' and 'Because everyone uses apps' are not added value. What are the benefits to the user of doing something with an app rather than either doing it some other way or not doing it at all? Make sure there is clear added value.
  2. Consider how people will fit app use into their lives. Is it meant to be used when a particular trigger event happens (e.g., the user is planning to go for a run, or isn't feeling well today), or regularly (after every meal, first thing every morning, or whatever)? How will people remember to use the app when intended? Make sure the app fits people's lives and that any reminders or messages it delivers are timely.
  3. What will people's motivations for using the app be? Are there immediate intrinsic rewards or longer term benefits? Will these be apparent to users? Does there need to be an extrinsic reward, such as competing against others or gaining 'points' of some kind, or might this be counter-productive? Are there de-motivators (such as poor usability)? Make sure the app taps in to people's motivations and doesn't put obstacles in the way of people realising the envisaged rewards.
A fourth important point is to recognise that every person is different: different lifestyle, different motivations, different on many other dimensions. So there almost certainly isn't a "one size fits all" app that everyone will love and engage with. But good and appropriate design will work for at least some people.

Tuesday, 12 August 2014

What's the value of human factors?

I was recently in a meeting where a technologist was reporting on a situation where a team had developed a new technology to support work in a clinical setting, but when they tried to deploy it they discovered that people didn't work in the way they had expected. From a user-centred design perspective, this is Rookie Error Number 1: so basic that it is amazing that it still happens so frequently.

It was the next statement that caught me slightly off-guard: that they should have consulted the clinicians ahead of time. Yes, of course they should. But actually, that's probably not enough either. Because clinicians (like all of us) have limited understanding of, or ability to articulate, their own work. The speaker's implicit assumption was clearly that we are all fully capable of describing our work to others in the detail that they would need to be able to design to support that work. That leaves me, as a Human Factors specialist without a job!

Which of course led me to reflect on what my role in such a situation is. What is it that I, or any other HCI / Human Factors specialist brings to the situation that the clinician or the technologist doesn't? In the case of the technologist (or at least, of the technologists who feature in this story), it's a limited understanding of what we're able to articulate, or even know, about our own activities: an assumption that if you ask someone a direct question they will give you a full, frank and accurate answer. In the case of the clinician, like anyone, it's a partial understanding of their own work practices and of how to describe them to a third party.

Many moons ago, Gordon Rugg and I wrote about this in the context of eliciting people's implicit knowledge of their own work, and I'm going to paraphrase some of our own writing:
In terms of understanding what people can articulate about their work, there are three main kinds of knowledge:
  • explicit knowledge (knowledge which is readily available to introspection, and that people can readily articulate);
  • semi-tacit knowledge (knowledge that can be accessed by some techniques but not by others); and
  • tacit knowledge (knowledge which is not accessible to introspection via any elicitation technique). 
There are various types of semi-tacit knowledge, including taken-for-granted knowledge (that is so familiar to the speaker that they will not think to mention it); front and back versions (the official version of what should happen and a more realistic account of what actually happens); and knowledge that depends on recognition rather than recall
Tacit knowledge is subdivided into compiled skills (which have become so habitualised as to be inaccessible to introspection) and implicit learning (knowledge that is tacit throughout, never having been articulated).
So there I have at least a partial answer to my existential angst: that as a Human Factors specialist, I have knowledge of how to elicit semi-tacit knowledge about people's work – and specifically about that work for the purposes of designing technology to support it. So both the technologist and the clinician have a reason to get me involved. Whew!

Friday, 16 May 2014

Let's be pragmatic: one approach to qualitative data analysis

Today, Hanna, one of my MSc students, has been asking interesting questions about doing a qualitative data analysis. Not the theory (there's plenty about that), but the basic practicalities.

I often point people at the Braun & Clarke (2006) paper on thematic analysis: it’s certainly a very good place to start. The Charmaz book on Grounded Theory (GT) is also a great resource about coding and analysis, even if you’re not doing a full GT. And I've written about Semi-Structured Qualitative Studies. For smallish projects (e.g. up to 20 hours of transcripts), computer-based tools such as  Atlas ti, nVivo and Dodoose tend to force the analyst to focus on the tool and on details rather than on themes.

I personally like improvised tools such as coloured pens and lots of notebooks, and/or simple Word files where I can do a first pass of approximate coding (either using the annotation feature or simply in a multi-column table). At that stage, I don’t worry about consistency of codes: I’m just trying to see what’s in the data: what seem to be the common patterns and themes, what are the surprises that might be worth looking at in more detail.

I then do a second pass through all the data looking systematically for the themes that seem most interesting / promising for analysis. At this stage, I usually copy-and-paste relevant chunks of text into a separate document organised according to the themes, without worrying about connections between the themes (just annotating each chunk with which participant it came from so that I don’t completely lose the context for each quotation).

Step 3 is to build a narrative within each of the themes; at this point, I will often realise that there’s other data that also relates to the theme that I hadn’t noticed on the previous passes, so the themes and the narrative get adapted. This requires looking through the data repeatedly, to spot omissions. While doing this, it's really important to look for contradictory evidence, which is generally an indication that the story isn't right: that there are nuances that haven't been captured. Such contradictions force a review of the themes. They may also highlight a need to gather more data to resolve ambiguities.

The fourth step is to develop a meta-narrative that links the themes together into an overall story. At this point, some themes will get ditched; maybe I’ll realise that there’s another theme in the data that should be part of this bigger narrative, so I go back to stage 2, or even stage 1.  Repeat until done!

At some point, you relate the themes to the literature. In some cases, the literature review (or a theory) will have guided all the data gathering and analysis. In other cases, you get to stage 4, realise that someone has already written exactly that paper, utter a few expletives, and review what alternative narratives there might be in your data that are equally well founded but more novel. Usually, it’s somewhere between these extremes.

This sounds ad-hoc, but done properly it’s both exploratory and systematic, and doesn’t have to be constrained by the features of a particular tool.

Tuesday, 8 April 2014

A mutual failure of discovery: DIB and DiCoT

Today, I have been doing literature searching for a paper on Distributed Cognition (DCog). By following a chain of references, I happened upon a paper on Determining Information Flow Breakdown (DIB). DIB is a method for applying the theory of DCog in a semi-structured way in complex settings. The example the authors use in the paper comes from healthcare.

The authors state that "distributed cognition is a theoretical approach without an accepted analytical method; there is no single 'correct way' of using it. [...] the DIB method is a practical application of the theory." At the time that work was published (2007), there were at least two other published methods for applying DCog: the Resources Model (2000) and DiCoT (Distributed Cognition for Teamwork; 2006). The developers of DIB were clearly unaware of this previous work. Conversely, it has taken me seven years from when the DIB paper was published to become aware of it and my team have been working on DCog in healthcare for most of that time. How could that happen?

I can think of several answers involving parallel universes, different literatures, too many different journals to keep track of, the fragility of search terms, needles in haystacks. You take your pick.

Whatever the answer actually is (and it's probably something to do with a needle in another universe), it's close to being anti-serendipity: a connection that is obvious and should have been expected. We clearly have some way to go in developing information discovery tools that work well.

Saturday, 5 April 2014

Never mind the research, feel the governance

In the past 5 days, I have received and responded to:
  • 16 emails from people in the university, the REC and the hospital about one NHS ethics application that required a two-word change to one information sheet after it had already been approved by both the university and the REC - but the hospital spotted a minor problem and now it has to go around the whole cycle again, which is likely to take several weeks at least.
  • 6 emails about who exactly should sign one of the forms in a second ethics application (someone in the university or the hospital).
  • 12 emails about the set of documents (I lost count of what's needed past 20 items) needed for a third application.
I dread to think what the invisible costs of all these communications and actions are, when scaled up to all the people involved in the process (and my part is a small one because I delegate most of the work to others), and to all the ethics applications that are going on in parallel.

I thought I was getting to grips with the ethics system for the NHS; I had even thought that it was getting simpler, clearer and more rational over time. But recent experiences show otherwise. This is partly because we're working with a wider range of hospitals than previously, and every one seems to have its own local procedures and requirements. Some people are wonderful and really helpful; others seem to consider it to be their job to find every possible weakness and block progress. I have wondered at times whether this is because we are not NHS employees (or indeed even trained clinicians). But it seems not: clinical colleagues report similar problems; in fact, they've put a cost on the delays that they have experienced through the ethical clearance process. Those costs run into hundreds of thousands of pounds. We don't do research to waste money like this, but to improve the quality and safety of patient care.

Today, there's an article in the Guardian about the under-resourcing of the health service and the impact this is having on patient care. Maybe I'm naive, but if the inefficiencies that we find in the process of gaining permission to conduct a research study in the NHS are replicated in all other aspects of health service delivery, it's no wonder they feel under-resourced.

Tuesday, 1 April 2014

Looking for the keys under the lamp post? Are we addressing the right problems?

Recently, I received an impassioned email from a colleague: "you want to improve the usability of the bloody bloody infusion pump I am connected to? give it castors and a centre of gravity so I can take it to the toilet and to get a cup of coffee with ease". Along with photos to illustrate the point.

He's completely right: these are (or should be) important design considerations. People still want to live their lives and have independence as far as possible, and that's surely in the interests of staff as well as patients and their visitors.

In this particular case, better design solutions have been proposed and developed. But I've never seen one of these in use. I've seen plenty of other improvised solutions such as the bed-bound patient being wheeled from one ward to another with a nurse walking alongside holding up the bag of fluid while the pump is balanced on the bed with the patient.

Why don't hospitals invest in better solutions? I don't know. Presumably because the problem is invisible to the people who make purchasing decisions, because staff and patients are accustomed to making do with the available equipment, and because better equipment costs more but has minimal direct effect on patient outcomes.

An implication of the original message is that in CHI+MED we're addressing the wrong problem: that in doing research on interaction design we're missing the in-your-face problem that the IV pole is so poorly designed. That we're like the drunk looking for the keys under the lamp post because that's where the light is, when in fact the keys got dropped somewhere else. Others who claim that the main problem in patient safety is infection control are making the same point: we're focusing our attention in the wrong place.

I wish there were only one problem to solve – one key to be found, under the lamp post or elsewhere. But that's not the case. In fact, in healthcare there are so many lost keys that they can be sought and found all over the place. Excuse me while I go and look for some more...

Thursday, 27 March 2014

Mind the gap: the gulfs between idealised and real practice

I've given several talks and written short articles about the gap between idealised and real practice in the use of medical devices. But to date I've blurred the distinctions between concerns from a development perspective and those from a procurement and use perspective.

Developers have to make assumptions about how their devices will be used, and to design and test (and build safety cases, etc.) on that basis. Their obligation (and challenge) is to make the assumptions as accurate as possible for their target market segment. And to make the assumptions as explicit as possible, particularly for subsequent purchasing and use. This is easier said than done: I write as someone who signed an agreement on Tuesday to do a pile of work on our car, most of which was required but part of which was not; how the unnecessary work got onto the job sheet I do not know, but because I'd signed for it, I had to pay for it. Ouch! If I can accidentally sign for a little bit of unnecessary work on the car, how much easier is it for a purchasing officer to sign off for unnecessary features, or slightly inappropriate features, on a medical device? [Rhetorical question.]

Developers have to work for generic market segments, whether those are defined by the technological infrastructure within which the device sits, the contexts and purposes for which the device will be used, the level of training of its users, or all of the above. One device probably can't address all needs, however desirable 'consistency' might be.

In contrast, a device in use has to fit a particular infrastructure, context, purpose, user capability... So there are many knowns where previously there were unknowns. And maybe the device fits well, and maybe it doesn't. And if it doesn't, then something needs to change. Maybe it was the wrong device (and needs to be replaced or modified); maybe it's the infrastructure or context that needs to be changed; maybe the users need to be trained differently / better.

When there are gaps (i.e., when technology doesn't fit properly), people find workarounds. We are so ingenious! Some of the workarounds are mostly positive (such as appropriating a tool to do something it wasn't designed for, but for which it serves perfectly well); some introduce real vulnerabilities into the system (by violating safety features to achieve a short-term goal). When gaps aren't even recognised, we can't even think about them or how to design to bridge them. We need to be alert to the gaps between design and use.