Human Computer Interaction

HCI

 

 

HCI = psychology + engineering

 

History

  1. Started with machines, interact using electronics/switches/etc.

  2. Algebraic languages (e.g. Fortran) on paper tape

  3. Punch cards – keep a box of these, interaction like paper record-keeping

  4. Command lines – using teletype terminal

  •  

    • as user types the characters are printed so user can see input

    • computer or person at other end of the link doesn't see the input until end of input

  1. Line editor

  •  

    • user can see one line at a time, and edit it or do commands to it

    • commands like “P12 S/Z/X P13 D13”

    • evolved into being English-like (e.g. Unix)

  1. WYSIWYG – use of “glass teletypes” allowed full screen editors and reduced paper us

  2. Modeless interaction

  •  

    • interaction with modes: e.g. VI (does different things whether in command or edit mode)

    • instead a given keystroke has the same effect in any context within the interface

  1. Menus (recognition rather than recall)

  2. Pointing devices

  •  

    • light pens, then after much development the mouse

  1. Graphical display

  •  

    • previous glass teletypes only allowed characters

    • these terminals often switched between teletype and graphic mode (did not draw command text onto screen in graphical way)

  1. Icons and windows

 

History shows us that much HCI theoretical approaches depended on developments in the technology.

 

Important attributes of direct manipulation

  • an object of interest to the user should be continuously visible in the form of a graphical representation

  • operations on objects should involve physical actions instead of commands

  • actions should be rapid and make incremental changes which are reversible

  • the effect of actions should be immediately visible

  • there should be a modest set of commands doing everything a novice might need, but are expandable to gain access to more functionality as the user develops

 

Style guidelines

  • users' experience of the usability of an OS is generally through the software of third parties

  • style guidelines are published by an OS to advise developers how to make usable applications

  • however most guidance is concerned with making applications look part of a family

  • sometimes the corporate branding overshadows usability

 

 

Psychological user models

  • need: came originally from WWII where hardware performance started to be limited by user ability

    • started describing users as black boxes so can use engineering closed loop control and information transfer

 

 

 

 

 

 

 

 

 

 

 

 

 

  • models of visual input:

    • recall Part IB Graphics

      • rods and cones

      • characteristics of vision described using colour spaces (RGB, etc.)

      • but eyes also adapt to brightness, local contrast and impose quantisation

    • Marr's theory of vision

      • works on the assumption that constructing an internal model of the world is a prerequisite for carrying out any visual task

      • proposal of a series of black boxes

  1.  
    1.  

      1. first level is the retinal image, and is processed to find boundaries of relatively uniform regions to get

      2. the primal sketch, whose structure is analysed to get

      3. the 2½D sketch (z ordering, rather than z co-ordinate)

  •  
    •  
      •  

        • sufficient for when we have stacked windows rather than complete 3D interface

        • resolved into

  1.  
    1.  

      1. 3D model

  •  

    • Gestalt laws of perception

      • proximity: elements close together tend to organise in units

      • similarity: objects that look alike tend to be grouped together

      • good continuation: we see lines as being continuous if they do not bend sharply

      • closure: we prefer to see regular shapes, inferring occlusion (closer object masking the further) to do so

    • visual tasks

      • depth perception: is not just done by binocular stereo vision, but by other monocular depth clues (which are what we use on a monitor to give 'depth')

      • face recognition:

        • processing faces in a scene happens much faster than processing other things

        • can also perform high precision tasks such as gaze direction with great accuracy

      • visual search

        • visual search made quicker if there is some factor which makes an object “pop-ou”, such as

          • colour

          • orientation

          • brightness

        • Hick's law (time to find one item among n similar items, discounting pop-out effects):

  • models of physical output

    • reaching action

      • high speed approach

      • slower homing phase

      • whilst hand is moving forms the appropriate shape for grasping

    • people make successive strikes more quickly with fingers from alternate hands

    • pointing at something using a mouse

      • defined empirically to give Fitts' Law

A = Amplitude (distance of pointer to target)

W = width of the target

  • models of memory

    • seven +/– two rule

      • observation by Miller

      • people can recall up to 7 +/– 2 objects in verbal short term memory

      • applies to number of steps to do an operation

    • connectionist theories

      • store long term memories in association with things already known

      • can improve learning and retrieval by rich associations

        • UI should mimic familiar situations/applications

    • visual short term memory

      • independent of verbal short term memory

      • can exploit with mnemonic techniques associating images with items to be remembered

  • models of problem solving

    • Generalised Problem Solver

      • GPS operates in a search space characterised by possible intermediate states between some initial state and a goal state

      • problem solving consisted of finding a series of operations that would eventually reach the goal state

      • recursively apply the heuristics:

        • select an intermediate goal that will reduce the difference between the current state and the desired state

        • if there is no operation to achieve that goal directly decompose it into sub-goals

      • the difficulty of the problem is indicated by the depth of the sub-goal tree

        • if the tree is too deep the user will forget what to do next to finish the problem

  • system models for HCI

    • Model Human Processor

      • assume task can be broken down into perceptual, cognitive and motion events

        • perceptual: [50 ~ 200] ms

        • cognitive: [25 ~ 170] ms

        • motion: [30 ~ 100] ms

    • Keystroke Level Model (see later)

    • GOMS (Goals, Operators, Methods and Selection)

      • methods that are used to achieve goals

      • methods are sequential lists of

        • operators that the user performs and

        • (sub)Goals that must be achieved

      • selection rules are invoked to determine which method to choose, depending on the context, if there is more than one Method which may be employed to achieve a Goal

        • they are generally if ... then ... else statements in natural language

      • provides quantitative methods for the

        • process of goal hierarchy decomposition

        • working memory

        • learning process in acquiring new methods

      • implies that the requirements of the user can be predicted exactly from the nature of the software application, and design a UI which takes an optimum path through the goal hierarchy

        • but behaviour of early AI simulations like GMS bears little resemblance to the behaviour of real humans in complex, real interfaces

      • KLM is just a restricted version of GOMS (restricted set of operations and no selection permitted)

  • unfortunately empirical methods can generally only analyse simple and uninteresting actions

 

Mental models

  • describe the structure of the mental representation that people use for everyday reasoning and problem solving

  • e.g. electricity being like a fluid in pipes

    • model breaks down when e.g. switch left on with nothing plugged in; electricity is leaking out?

    • doesn't do people harm in this case

  • basic claim: if you know the users' beliefs about a system you can predict their behaviour

    • the user infers the result of their actions in a mental simulation, before doing it

    • where the model is incomplete, inference will usually rely on analogy to other devices the user already knows

  • the user is working simultaneously in two problem state spaces

    • one space describes the actual structure of the users' goals (which may not explicitly recognise the computer at all)

    • the other space describes the understanding of the device state space

    • state spaces yoked together (moves in one space can only be accomplished by equivalent moves in the other)

  • it is difficult to make a predictive computational simulation

    • have to study behaviour and try to work out users' reasons

  • difference between user model and designer model

    • designed creates UI that gives unnecessary information about the design structure (CL colouring)

 

 

User oriented design methods: prototyping

  • particularly useful within the waterfall development's portrayal of the UI at the specification phase

  • previously client has not understood requirements for user functionality

    • makes sense: client had no mental model to work from

  • rapid prototyping: construct functional user interface for discussion using rapid prototyping tools

  • iterative refinement: using spiral model + rapid prototyping instead of waterfall

  • incremental prototyping: the tools also meet the engineering requirements of the final system

  • deep prototyping: one aspect of the system functionality is fully implemented before developing the rest of the interface

  • investigation of multiple prototypes requires low cost techniques to produce them

  • low fidelity prototyping: glue & paper and Wizard of Oz technique

 

Evaluation techniques

  • controlled experiments

    • series of observations (measurements made when user using experimental interface)

      • e.g. “how long did it take for Fred to finish task A?”

    • analyse results statistically (one measurement doesn't help much)

    • HCI experiment involves one or more treatments that modify the UI

    • the normal distributions of performance of the “good” UI vs. the “bad” UI will overlap

      • compare effects of the treatments by looking at the overlap

      • statistical significance test (e.g. the t-test)

        • what is the probability the observed difference is due to random variation?

        • normally insists p < 0.05, but can insist p < 0.01

    • sources of variation in the measurements (tend to combine to random variation within the normal distribution, by the Central Limit theorem)

      • variation in the task performance

      • the effect of the treatment

      • difference between subjects (e.g. IQ)

      • different stimuli for each task

      • distractions during the trial (sneezing etc.)

      • motivation of the subjects

      • accidental intervention by the experimenter

    • validity of the result:

      • does the observed effect generalise?

      • could it be replicated?

      • does some psychological theory explain it?

  • other empirical techniques

    • hypothesis testing can hide a lot of useful information (feedback from users of a protoype)

    • surveys

      • closed questions

        • yes/no

        • choice on the Likert scale (1 to 5 rank of how much agree with a statement)

      • open question

        • needs some methodical coding technique to structure the result

    • questionnaires

      • particular type of survey

      • generally used to gather responses from a larger sample

    • think aloud studies

      • user asked to carry out task whilst talking as continuously as possible

    • bad techniques

      • simple subjective reports

      • proposing a usability hypothesis and not testing it “using more colours should increase usability”

      • introspection of the developer about their performance is irrelevant to most users

 

 

Task oriented analysis

  • observation and task analysis

    • structured interviews

      • conduct interviews to discover the requirements of system users

      • have to be planned to be effective

      • use because user needs can be overlooked in system requirements meetings (at start of the project)

    • observational studies

      • less intrusive way of capturing data about users' tasks

      • video the user at work

      • can analyse

        • time spent on different subtasks

        • common transitions between different subtasks

        • interruptions of tasks, etc.

      • if the task ranges over locations, have to follow the user

        • might need to use ethnographic techniques

        • could have user keep a diary with precise timings, etc.

    • ethnographic field studies

      • investigator has to interact directly with the subject

      • have to take care to gain complete and objective information

      • attempt to observe subjects in a range of contexts over a substantial period of time

      • can help understand technology and products in completely new ways

        • can contribute to requirements capture

    • personas

      • product designers don't want to spent a year on field work

      • they need to gain perspective of a user, outside their own technical mindset

      • write fictionalised description of user to help understand what type of person the user is

    • field tests: e.g. “follow me home” programme by Intuit Inc.

  • use case design

    • software design: ultimately concerned with the needs of users

    • UML use case analysis

      • behaviour of the system is described from the point of view of abstract actors

      • use case is narrative of some specific interaction of a specific actor with the system

  • cognitive dimensions of notations (see later)

  • research trends

    • computer supported cooperative work

      • traditional research of user in front of computer neglects the environment

      • users generally have to collaborate with other users

        • how can UI support this collaboration?

      • Research, for example

        • shared networked whiteboards

        • social networking sites

        • second life

    • information appliances/ubiquitous computing

      • what new modes of interaction can we have with computers in toasters, desks, TVs...

      • systems which locate and identify a user

      • tangible user interfaces: physical objects are used to interact with the computer (captured by video camera, RFID tag, etc.)

 

 

  •  

    • new interaction devices

      • pen input: text entry or gesture languages

      • extended physical interaction: foot operated devices or gaze tracking

      • interaction in 3D

        • still very hard

        • standard hardware:

          • head trackers

          • 3D mice

          • data glove

        • 3D display can provide either of

          • immersive virtual reality

          • augmented reality: data displays are superimposed on the user's view of the real world

      • speech commands

    • end user programming

      • in the past most computer users did some programming

      • these days people do things that resemble programming (end user programming):

        • define the behaviour of agents

        • scripting

        • programming by demonstration (system watches user actions and defines a macro)

      • macros in desktop applications are not widely used

      • specifications/debugging in spreadsheets etc. (non-professional programming)

        • many spreadsheets with bugs with safety/business critical consequences

      • languages for teaching programming principles (to CompScis, schoolchildren, preschoolers)

 

Heuristic Evaluation

  • usability evaluated by a panel of experts

  • each expert works from a list of usability heuristics (similar to principles of direct manipulation)

    • individual heuristics unimportant: essence of technique is applying them to make a systematic evaluation

    • panel of experts intended to bring some objectivity to subjective interpretations of heuristics

  • procedure:

    • have multiple evaluators (preferably from different academic backgrounds)

    • each evaluates the interface on their own, perhaps using a scenario

    • go through the interface at least twice, look at compliance to the heuristics

  • sample heuristics

    • visibility of system status: keep users informed by appropriate feedback in reasonable time

    • match between system and the real world:

      • speak user's (not system) language

      • follow real world conventions

    • user control and freedom: undo, redo and emergency exit

    • consistency and standards: platform conventions, etc.

    • error prevention

    • recognition rather than recall:

      • user should not have to remember information from one part of the dialogue to another

      • instructions should be visible or easily retrievable

    • flexibility and efficiency of use

      • accelerators for expert users

      • allow users to tailor frequent actions

    • aesthetic and minimalist design

 

  •  

    • help user recognise, diagnose and recover from errors:

      • precisely indicate the problem

      • constructively suggest a solution

    • help and documentation

      • easy to search

      • focused on user's task

  • applicability

    • most popular technique

    • simple and cheap

    • provides little opportunity to address deeper system issues

    • does not systematically generate solutions to problems

 

Keystroke Level Model

  • provides quantitative information about usability

  • developed from model human processor

  • assumes the user is an expert user performing a routine task

  • procedure

    • decompose the total task into unit operations

    • using established times, add estimate times for each of the operations to get task completion time

  • basic components

    • K: time to press a key: 0.28s for average typist

    • H: time to home position on mouse or keyboard: 0.40s

    • P: time to point with mouse: Fitts' law

    • D: time to draw using a mouse

    • R: time the system takes to respond

    • M: time to mentally prepare for an action: 1.35s

  • mental preparation

    • every operation much be preceded by mental preparation

    • no mental preparation is needed between two unit operations that form a chunk

    • rules:

      • insert Ms in front of all Ks that are not part of a string (of numbers of letters)

      • insert Ms in front of all Ps that select commands

      • if the operator after an M is fully anticipated by the operator before it, delete the M

      • if a string of Ks belong to a cognitive unit (e.g. command name) delete the Ms between them

      • if there are two Ks which are both terminators (e.g. return key) delete the M between them

      • when a particular command string is always always followed by a terminator delete the M between them (consider the terminator a part of the command)

  • applicability

    • useful for analysing situations where there is a limited UI used in a repetitive way

    • only useful for comparative estimates – the absolute accuracy varies widely from user to user

    • needs to be confirmed by empirical measurements

    • chunking rules

      • are ambiguous

      • only apply to command based systems

 

Cognitive walkthrough

  • assesses usability of the system where the user

    • is not an expert

    • may be attempting an entirely new task

  • behaviour model, the model describes how a notional user

    • sets a goal to be accomplished with the system

      • e.g. “check the spelling of this document”

    • searches the interface for currently available actions

      • availability may be observable as presence of menu items, buttons, available command line inputs, etc.

    • selects the action that seems likely to make progress towards the goal

    • performs the selected action and evaluates the system's feedback for evidence that progress is being made toward the current goal

  • procedure

    • manual simulation of the user iteratively carrying out the stages of the behaviour model

    • evaluators need

      • general description of the type of user and their relevant knowledge

      • description of one or more representative tasks for the evaluation

      • for each task a list of correct actions to perform to complete the task

    • evaluators are

      • interface designer

      • group of peers

      • a scribe and a facilitator

    • at each step moving through the UI

      • examine the interface

      • tell a story about why user would choose an action

      • evaluate story

        • consider what user's current goal would be

        • evaluate accessibility of the correct control

        • evaluate the quality of the match between the control's label and the goal

        • evaluate the feedback provided to the user after the action

  • applicability

    • applicable to WIMP/direct manipulation interfaces

    • assumes evaluators are knowledgeable designers who can evaluate using relevant theories from cognitive psychology

    • more structured than heuristic evaluation

      • less likely to suffer from subjectivity

 

Cognitive dimensions of notations

  • broad-brush approach (vs. “death by detail” in KLM)

  • every user interface is a compromise: there is no perfect UI

    • even if perfect for one type of user is not for another (e.g. novice vs. expert)

    • trade-offs!

  • describes the system as an information artefact: something for processing, storing and communicating information

  • each information artefact provides one or more notations in which the information being manipulated is encoded

    • environment used to manipulate the notation is also important for usability

  • cognitive dimensions used as a vocabulary for design discussion

    • can discuss the trade-offs that result from design decisions

 

  • cognitive dimensions

    • premature commitment:

      • can you do a task in any order you like?

      • does the system force you to think ahead and make certain decisions first?

        • which things?

        • what problems can that cause?

    • hidden dependencies

      • if changes of one part can affect another part (of a system) are the dependencies visible?

      • how can it get worse when making a large description?

      • do some actions cause the dependencies to get frozen?

    • secondary notation

      • can you make notes/express information that is not recognised as part of the notation

      • e.g. annotating a piece of paper

    • viscosity

      • resistance to change

      • are there any changes which are particularly difficult to make?

      • “houses are viscous”

    • visibility

    • closeness of mapping

      • closeness of representation to the domain

      • which parts seem particularly strange way of describing things?

    • consistency

      • similar semantics are expressed in similar syntactic forms

      • are there places where things ought to be similar but the notation makes them different?

    • diffuseness

      • in the notation can you say things briefly or long-windedly?

    • error-proneness

      • are some kinds of mistakes particularly common or easy to make?

    • hard mental operations

      • what kind of things require the most mental effort with the notation?

    • progressive evaluation

      • how easy is it to stop in the middle of creating a notation and check work so far?

      • can you do it at any point?

    • provisionality

      • degree of commitment to actions or marks

      • can you sketch things out (e.g. if don't know what to do next)?

      • which parts of the notation helps you do this?

    • Role expressiveness

      • can the purpose of a component be easily inferred?

      • are there some parts which are hard to interpret/you don't know what mean?

    • abstraction

      • can you define new facilities or terms within the notation, so you can extend it to describe new things or express your ideas more clearly/succinctly?

      • does the system insist that you start by defining new terms before you can do anything else?

 

 

  • sub-devices

    • complex systems can have several specialised notations, which can be separate from the rest of the system

      • e.g. post-it note on a computer screen

    • types of sub-device

      • helper device: e.g. note of number back of an envelope; complete system is telephone system + the paper notes

      • redefinition device: change main notation, e.g. defining keyboard shortcut

    • generally have their own notations

  • notational activities: ways the user can interact with the notation

    • search

      • notation doesn't change

      • sections the user sees will vary

      • visibility and hidden dependencies can be important factors

    • incrementation

      • adding further information to a notation without altering the structure

      • since the structure does not change then viscosity is not very important

    • modification

      • changing existing notational structure

      • possibly without adding new content

    • transcription

      • copying content from one structure or notation to another notation

    • exploratory design

      • combining incrementation and modification

      • e.g. “hacking”

      • viscosity can make this much harder

        • hence for hacking may be dynamically typed or use type inference

        • loosely typed languages more likely to suffer from hidden dependencies (trade off with viscosity)

  • procedure

    • identify the main notation of the system, describing

      • the medium in which the marks of the notation are expressed

      • the environment in which it is manipulated

    • identify sub-devices

      • describe the notation for each sub-device

    • consider each notation in terms of the list of dimensions

      • identify where system characteristics are inappropriate to user activity

      • e.g. viscosity in exploratory design

    • where problems have been identified consider design manoeuvres to adjust that dimension

  • applicability

    • can be more useful as early design tool than evaluation technique

    • provides specific guidance as to what needs to change and likely consequences of the changes

    • can be used to analyse complex software which isn't otherwise analysable

    • useful for relative needs of different types of users