Keynote to appear at First Asia Pacific Conference on Human-Computer Interaction.
Harold Thimbleby
Middlesex University
LONDON, N11 2NQ, GB
Email: harold@mdx.ac.uk
Abstract
The conventions of drama present the planned as spontaneous, stimulating the imagination of greater interaction potential than there is. This paper argues for a distinction between design for demonstration and design for interaction. The distinction is needed on the Internet, which supports the greatest range of discourse -- spontaneous to planned -- and therefore wide scope for confusing dramatic presentation for effective interaction.Keywords
Design, discourse, drama, human-computer interaction, hypertext, scenarios
HCI -- human computer interaction -- is about design, which is planned interaction: it is concerned with the effective planned 'writing' of a design to be later consumed as 'conversation.' HCI, viewed like this, is a transformation from one discourse style to another. Yet so is drama; arguably, this leads to confusion when an interactive system is demonstrated. Is there more to the demonstration than is presented, or is the demonstration all there is? Drama encourages the audience to believe there is more than there is on the surface; in contrast, HCI requires capabilities beyond the demonstration. This potential interaction must be carefully planned, to be substantial rather than drawn out of the user's imagination. Although good HCI can be demonstrated, a good demonstration does not imply good HCI.
Despite the obviousness of this, it seems many demonstrations of prototypes are converted to products too speedily. Understandably, managers, after viewing an impressive performance, may tighten production schedules. Users are left with a void between their expectations (after all, the performance exercised their imagination!) and what they can actually do.
HCI is about interaction, and interaction can be characterised as moving around in a conceptual network of possibilities -- just like hypermedia. We come full circle with the World Wide Web as hypermedia, blurring the distinction between users and designers. All are communicators. Despite the scope for interaction breakdown on an unprecedented scale, the Web certainly enables humans to engage in new styles of discourse. As HCI professionals we must try to ensure that users are empowered to explore this new medium, their relation to it, and their roles in it, without interaction breakdowns escalating beyond control.
"Writing, when properly managed, (as you may be sure I think mine is) is but a different name for conversation." Tristram Shandy, Book I.
If we ignore broadcast (newspapers, radio, TV) the next major innovation in communication was the telephone. The telephone did not change anything; it just allowed people talking to each other to be further apart. In fact, many countries passed laws to emphasise the distinction between spoken (spontaneous, transient) and written (formal, permanent) language. In England, for example, it is normally illegal to record a telephone conversation. In other words, the law -- that is, society -- wishes to distinguish between the transient spoken form of communication and the written record. There is something different about the use of recorded (usually written) language that changes its status. A tape recording of a private conversation on a telephone is an invasion of privacy. The law recognises this; or, put another way, society finds the conventions important. People like to place clear boundaries between different forms of communication.
Special forms of communication were developed for special purposes. For example, drama was developed to present the intimate as public, to present the spontaneous as repeatable, to engage the audience in issues they could literally walk away from at the end of a performance. An actor speaking personal thoughts aloud uses a convention to communicate to the audience; whereas a person walking along a road in a public space and talking aloud is easily thought to be mad. Indeed, talking to yourself is the first sign of madness -- unless you are on stage, or using a mobile phone (in both cases you are talking to other people).
Notably, the broadcast medium of television naturally adopted the conventions of drama (though, earlier, it was with some effort that cinema separated from stage [1]). Formal material on television is rare, and is usually taken especially seriously -- with few exceptions, it is only really successful when it is edutainment or politically motivated investigative journalism.
In the last few years the Internet introduced the first truly new form of human discourse for five thousand years. Email, just one style of interaction on the net, is a mixture of spontaneous and planned written communication. So-called 'flames' are instant thoughts captured and reinterpreted as formal written accusations. The World Wide Web allows individuals to create home pages, 'speaking aloud' about themselves to the whole world. In real life, one is mad to speak to nobody. Yet on the net, it is legitimate to speak to nobody now, because the spontaneity is recorded and can be listened to later, perhaps by millions around the world. It may be a coincidence that many personal home pages are crazy, or maybe people import dramatic conventions of self-expression into the new medium. Nevertheless there are some things in the Web that are serious, and they try to be taken at face value, not as drama that can be taken or left.
The net is a plastic medium that merges spontaneous, recorded, broadcast and personal discourse. I can type about as fast as I can speak; is, then, my email spoken or written? Sometimes it is one, sometimes the other. The recipients of my email may read it as spontaneous spoken or as formal written. They can do new things with my email that cannot be done with conversation: they can forward it to others -- it can be treated as a recorded object; they can save it and reply months later, reinstating the historical conversation to the present. To substitute for the conversational manoeuvres of speech, new written conventions are employed: such as ':-)' to label humour; turn-taking by '>' quoting (i.e., making the history of the conversation explicit); and identity by digital signatures (cf. Zimmermann, 1995).
With email I can send a simple yes or no, or I can send an entire book. Who would appear on television just to say no? Who would read a book over a telephone? The flexibility is exactly why email is so compelling.
The distinctions we are making are important. We give some examples:
Source | Destination | Exceptions | |
Spoken | Spontaneous. Private. Composed in order, no revocation, except by more speech. Free. Always new and free of history. Requires a present audience. The speaker represents themselves. | Chosen by speaker and hearer. Transient. Interactive. Involved, though hard to reflect without making the reflection part of the conversation. Cannot be reused. Can only be consumed in given order. Free. | Drama is rehearsed spontaneity, and is often presented to a large, non-interactive audience. Recorded speech is not interactive, can be reused, and is typically copyrighted. |
Written | Considered. Can be edited before commitment to communicate. Formal. Expensive. Steeped in history, often explicitly building on other written sources. Done without a present audience. The writer can represent many characters. | Chosen by readers. Not interactive. Can be saved and reused (though there are copyright conventions). Can be consumed in any order. Priced. Easy to reflect and argue. Destroying written information is highly symbolic. | Transcripts are written recordings of speech -- they look chaotic (Chapanis, 1981). Speeches and plays are written but intended to be used as spoken. (Hypertext is discussed in the body of the paper.) |
Net | Any of the above. Typing is writing at the speed of speech. | Any of the above, though destroying electronic information has little significance. | None. |
Though separation in time is important, spatial distance is not; indeed telephone technology permits both spoken discourse and written, using faxes, both as if speaker and listener were adjacent. The hiding of distance may encourage some users to under-estimate cultural diversity: engaged in interacting with like-minded people around the world, they may think that this select group of individuals is more representative that it is. Certainly the notion of 'local' neighbourhood is widened.
When we look at the discourse on the Internet there is surprising flexibility. Social conventions have not been established. English law -- even if we suppose this to represent the UK's distilled social conventions -- seems quite happy to hide behind the excuse of not understanding the 'new' technology. More positively, the fluidity is something we should exploit so that new forms of discourse can develop that go beyond current socio-legal traditions, and which may stretch our neurological dispositions on which social habits have been founded [2].
Newsgroups (e.g., the Usenet) are at once written and shared, and at the same time, the lively spoken views of like-minded people. Someone coming into a newsgroup culture can be subject to strong forces to conform. They are directed to FAQs to be initiated into the group's customs and shared history. Someone who does not share the group's views might well see the FAQs etc. as having an identification role like myths.
The table makes clear that drama breaks down normal discourse conventions: the dramatic context (usually clearly flagged, with stage, masks, etc.) allows -- sometimes contentiously -- a spontaneous or private communication to become public. There is even the distinguished profession of theatre critic; a role that would not be tolerated intruding into normal conversation! Recording has a shorter cultural history, and only two uses seem permitted: one is that the recording is a recording of a dramatic performance (including music), the other is the use of the recording for a formal or legal purpose. Most people feel betrayed if their spoken communications are recorded in the wrong context.
These new styles of discourse are not just of specialist interest. The net is the largest collaboration of humans the world has ever seen. Its power for good is phenomenal. Unfortunately, especially when building bridges between cultures -- spanning different conventions of discourse -- the net provides opportunities for misunderstanding, expression of anger, intimidation or destructiveness on a scale and speed never before imagined.
It is our duty, then, as HCI professionals, to ensure that the technology of the net itself does not contribute to misunderstanding. It is our duty to understand the transformation that happens between minds, as computers broadcast, record, and exchange ideas. To a large extent, what people do is their own responsibility, but if misunderstanding is increased between people by the lack of an undo function (for example), that would be something worth understanding and planning for, or designing to avoid.
Whereas drama converts intimacy into theatre, and indeed gains some attraction by doing so, HCI gains its attraction by converting static plans into effective interaction. A computer program is an object that the user brings to life, and to just that sort of life planned for it by its designer. Although drama may convert a script into a living experience, the users of the experience are the audience: they are disconnected from the 'lives' of the characters. In the terms of Winograd and Flores (1986), the audience of a drama is out of control of their thrownness; and their thrownness is easy to confuse with potential readiness-to-hand that is imagined to be the case in an actual interaction. Drama is unlike HCI, where the user is involved in the act, and has a personal commitment to the outcome. (Notwithstanding some HCI is about better drama; see also Carroll's classic paper (1970) on the analogies between entertainment and work.)
HCI, as the field that concerns itself with communication from designer to user, recapitulates the field of discourse we saw in all human communication, and on the Internet in particular.
Everything is hypertext. At once this is the strength of the idea, and its limitation. No longer does writing require any planned structure before it can be released to its users, for users are now supposed to make what they will of it. Many writers can engage in the text in a way that is both 'structured' and more flexible than any other style of written text. The texts can be linked together with no respect for any conventions (such as story or lexical order, required for most encyclopaedias). So even the nature of writing and co-authoring is transformed.
Though the World Wide Web must be one of the HCI design successes [3], hiding the complexities of using the net and making it extremely easy to use, it does not hide the complexity of information. The issue of getting lost in hyperspace has changed: the Web has become a place where surfing means treating the medium as spoken discourse, rather than written discourse with a plan that one could get lost in. The new systemic issues must be clearly distinguished from conventional HCI, which seems, to my mind, to be excessively concerned with simple low level details (e.g., Kellogg & Richards, 1995) that in any other technology would be dismissed as outrageous and feeble engineering.
As HCI professionals we must assume that complexity imposed on users is our responsibility. Yet the designers' problem of creating usable material is harder than the readers'. A reader has only one course of action, namely the one that he or she takes. The designer has to cater for more than one user on one occasion. Each choice that any user is given at least doubles the size of the design problem. After only ten alternatives for one user (or one alternative each for ten users) the design space is a thousand times larger. Ten choices is trivial and provides little scope for interaction: a more realistic interactive system would allow for thousands of choices, representing an astronomically large design space. If writing books is difficult -- certainly, not everyone is a successful book author -- then writing good hypertext documents is very much harder.
It is useful to introduce new terms to be clear. Interaction is what a user does. Interaction potential is what the designer has to plan; the design allows for many potential interactions, but a user only experiences one, namely 'the' interaction. Even over a period of time, the sequence of user interactions explores the potential, but is still a single interaction, just longer. Thus we may say that Laurel's Computers as Theatre (1991) emphasised interaction, not interaction potential [4].
Users do not experience the designer's problems, because they are in the flow of their own experience. The alternatives they did not follow -- which the designer should have planned for them just in case they did -- are hidden from them. Not just hidden: any interaction is simpler than any interaction potential, the designer's work appears simpler than it is. Thus people think the design of hypertext is very much easier than it really is. This lack of reciprocity between user and designer is important: on the World Wide Web, users are designers, but the lack of reciprocity remains.
There are several consequences.
First, 'everyone' thinks design is easy. The result is that designers are under enormous pressure from marketing, management, and everyone else, to deliver complex products faster than is possible consistent with doing a good design. Conversely, designers become disassociated from users since their job is not easy; hence the relevance of phenomenological views like Ehn (1988), Laurel (1991), Suchman (1987), and Winograd and Flores (1986).
Secondly, good design can be faked. If we know before-hand which choices a user will make, then the system can be constructed implementing only those choices. It is more like a film than a computer program. (Indeed there is an established professional interest in emphasising the presentation aspects of the medium.) The problem is that a film is ideal for a system demonstration: and will all too easily give management or marketing the idea that the product is far closer to market than it really is. So, again, presentation (that is, drama) can masquerade as interaction potential. Scenarios instantiate interaction potential as short takes from 'spoken' interaction discourse: running the risk of confusing realism for generality [5] -- but see below.
The dramatic confusion of presentation (encouraging people to imagine interaction potential) with actual content leads to the ascendancy of superficial fashion in interactive systems: most computer 'solutions' are chosen because they are attractive and fashionable rather than effective. Design professionals competent at presentation are adept at exploiting users' imagination to fill in the interaction with potential -- potential that may not be there.
So: the world fills up with poor hypertext. Users then become less demanding. It then seems easier still to create hypertext. Standards plummet. And the gulf widens between what users do and what theories designers have to use in their work. HCI seems even less comprehensible.
As a specific example of poor interactive device design, consider the Sony KV-M1421U type TV with its remote control, the RM-694. The figure shows two statecharts (Harel, 1987) specifying how the user potentially interacts with each device. We do not need to understand statecharts to see that the devices are very different; even the corresponding buttons do different things. Some features can be done on the TV alone, some can be done on the remote control alone, some features available on both are done differently on each device. It is not clear that the complexity is justified. Although there is only one application, namely the TV, the user has two user interfaces to understand, with their own rules; this, in turn, requires a user manual of double the thickness. Moreover if the user becomes skilled with one device then their skill is of little use with the other. Pity the user who loses their remote control!
Television | Remote control |
![]() | ![]() |
Statechart showing all features available using television's control panel. | Statechart showing only buttons that correspond to those on the television; the remote control provides additional features not shown here, such as teletext. |
Unfortunately the Web is bigger. If Sony have trouble (I'm not sure they realised it) with such a trivial design, how much worse will world wide hypertext become? Ironic that Thompson (1961) writes, "people like being sold rubbish and enjoy being deceived by advertising [É] they continue because the consumer accepts his disappointment philosophically or puts it down to his own misjudgement." Let us hope that discourse on the net can yet be worthwhile, not another medium swamped by the lowest common denominator.
One reason the TV is awkward is a few buttons do many things. Visible physical space has been traded for hidden interaction potential. Instead, at the other extreme, the TV might have been designed to have 400 buttons each doing exactly one thing. Such visible physical complexity would have been ridiculous.
The point is knowing it is ridiculous is easier when a design is physical. Hence we can make better HCI judgements in objectified media. Statecharts make the TV interaction issues clear because they are a suitable object medium for planning interaction potential. Interestingly, the advance of formalism historically has been attributed to the writing down of the arts, particularly 'theatres for the mind' (Yates, 1966).
Yet the Web does not conceal the fundamental problems of complexity, nor of a new medium finding its niche. Design on the web characterises all HCI design: the designer's job is far harder than anyone -- even the designer! -- can imagine. Users think so highly of demonstrations that they demand production systems before designers can honestly create them with the appropriate quality.
On the net, no longer are user and designer different, with designers out-numbered by users -- for all are designers, all are users [7]. We can contemplate that only 0.0000001% of these designer/users are aware of any HCI principles. The new discourse of the net (MUD, Web, IRC, etc.) makes new design issues manifest.
At least we saw it coming.
What can we contribute? A start will be to be aware of these issues, to consider, discriminate, and be selective in our choices for the future. As the First Asia Pacific Conference on Human Computer Interaction proceeds, ask yourselves where each presentation stands -- and where it is going.
Don't be impressed with demonstrations, passively imagining the interaction potential, get involved and interact to explore their actual potential. Specifically, when you see a demonstration, ask: is the interaction potential in your imagination (i.e., it is good drama) or is it actual (i.e., it is a good system)? What is the reusable (recorded, written) thing it contributing to the world -- and how are we going to take advantage of it?
Theatre is fun, but let us criticise it where ever it pretends to be our future relying on our imagination to do the creative work that should have been done by designers for the users.
A. Chapanis (1981), "Interactive human communication: Some lessons learned from laboratory experiments," in B. Shackel, ed., Man-Computer Interaction: Human Factors Aspects of Computers and People, NATO ASI Series E(44), pp65-114.
J. M. Carroll (1970), "The adventure of getting to know a computer," IEEE Computer, 15(11), pp49-58.
J. M. Carroll (1993), "Creating a design science of human-computer interaction," Interacting with Computers, 5(1), pp3-12.
D. Crystal (1987), The Cambridge Encyclopedia of Language, Cambridge University Press.
P. Ehn (1988), Work-oriented Design of Computer Artifacts, Almqvist & Wiksell International.
D. Harel (1987), "Statecharts: A visual formalism for complex systems," Science of Computer Programming, 8(3), pp231-274.
J. Johnson (1996), "The information superhighway: A worst-case scenario," Communications of the ACM, 39(2), pp15-17.
D. Thompson (1965), ed., Discrimination and Popular Culture, Penguin.
B. Laurel (1991), Computers as Theatre, Addison-Wesley.
W. A. Kellogg & J. T. Richards (1995), "The human factors of information on the Internet," in J. Nielsen, ed., Advances in Human-Computer Interaction, 5, pp1-36, Ablex.
L. A. Suchman (1987), Plans and Situated Actions, Cambridge University Press.
H. W. Thimbleby (1990), User Interface Design, Addison-Wesley.
H. W. Thimbleby (1991), "Can Humans Think?" The Ergonomics Society Lecture 1991, Ergonomics, 34(10), pp1269-1287.
S. Turkle (1995), Life on the Screen: Identity in the Age of the Internet, Weidenfeld & Nicolson.
F. A. Yates (1966), The Art of Memory, Pimlico.
T. Winograd & F. Flores (1986), Understanding Computers and Cognition, Ablex.
P. R. Zimmermann (1995), The Official PGP User's Guide, MIT Press.