ARIADNE Project on Digital Libraries · Publications
A slightly revised version appears in Proceedings of the Second ACM International Conference on Digital Libraries (DL'97), (Eds.) Allen, R.B. & Rasmussen, E., Philadelphia, PA, 221-228.
This paper describes the use of ethnomethodologically-informed ethnography as a means of informing the requirements elicitation, design, development and evaluation of digital libraries. We present the case for the contribution of such studies to the development of digital library technology to support the practices of information-searching. We illustrate this by a particular study of the help desk at a university library, examining the implications it has for designing appropriate functionality for a digital library. This requires us to address the problems of using ethnographic data in systems design.
'In defining the role of a digital library it is essential to incorporate the concept of proactive intermediation ... so that the digital library is not limited to passive warehousing of navigable information.' [7]
We believe that in order to be effective, in order to achieve organisational usability [15], digital libraries must take account of the social aspects of information seeking and support the processes that occur in social information seeking [27,28]. Such a contention is surely far from controversial, but it raises the question as to how this support might be designed into such systems. We argue that the provision of such support can best be brought into the development cycle through the employment of a particular method - namely ethnomethodologically-informed ethnography - as part of the requirements elicitation process.
We claim that such ethnographic study can provide systems designers with an insight into the practice of seeking information in collaboration with a member of library staff as seen from the point of view of parties to that action. This gives a better understanding of the potential users of the system; it provides rich, detailed descriptions of activity, making use of categories that are those of the social actors themselves as they undertake their ordinary activities and make sense of the activities of others.
We illustrate our claim by examples of data from studies of a library help desk. Despite the very detailed account of the organisation of individuals' information-seeking practices, the approach makes plain the recurrent organisation of such activities as a resource for the design of a range of digital library facilities to support the fundamentals of searching. Thus we claim that although this particular study focuses on a help desk, it carries implications not just for a help desk of a digital library, but for other forms of collaboration including informal help giving between library users.
We make the case for a revision of Brewer's 'proactive intermediation', to see the production of queries and candidate solutions as emerging in and through the mundane interactions of library staff and users as they make use of the library's OPAC system.
The requirements elicitation process is characterised [9] as one
"which is variably conceived, and much debated, but it is that aspect of the design process which is concerned with achieving an understanding and characterisation of the domain of application. In general terms its concern is with identifying the functions that the system should deliver, how these may be displayed to users, what parameters of the human-computer equation should be satisfied, and so on."
In the field of CSCW ethnomethodologically-informed ethnography has achieved some prominence as a contributor to the design of distributed and shared systems. Ethnography is one of the oldest methods in the social research armoury, and recent efforts to incorporate it into the system design process has much to do with the realisation, mainly among system designers, that the success of design depends, though in complex ways, upon the social context into which systems are placed. The more traditional and often cognitively based methods of requirements elicitation were seen as inadequate, or in need of supplementing by methods better designed to bring out the socially-organised character of work settings. It was also argued that such methods needed to be more attuned to gathering relevant data in 'real world' environments; that is, settings in which systems were likely to be used rather than in laboratories or other artificial environments remote from contexts of actual system use.
One reason for the recognition of such a need was a number of high-profile 'failures' of systems, where, despite operating unproblematically in software terms, the lack of fit with the system's application domain led to rejection by users and ultimately expensive overhaul or even scrapping of the system. [21]. We hope that by employing ethnography in the design of digital libraries, designers can avoid such failures in this domain. Note that here we include designs that take into account information science theories of for example searching behaviour but still fail to provide support for the actual practices that constitutes information-seeking.
Within the CSCW field there now exists a canon of such field research, including, amongst others, studies of the work of cooperative work in financial services [23] and manufacturing engineering [14].
There is a rich tradition within Library and Information Science of concern for understanding and supporting the user's needs (e.g. [11, 16]). Although we looked at brief help-giving activities, there are similarities with the formal reference interview (e.g. [10]).
Taylor's [26] classic paper covers many of the issues that we regard as important, particularly the key problem of finding out what another person - an inquirer, user, information seeker etc. - wants within the constraints of or rather, in terms of the library system or catalogue's organisation.
The problem however, is that we cannot see in Taylor's account how that problem is recurrently solved by intermediaries in situated real time; i.e. the actual 'lived work' of solving the problem is hidden from view or glossed in the printed material. This is a criticism we apply to other work. Although in some cases this may be merely a constraint of space (some actually did observations of the work in progress, others rely on post hoc reports) it reveals a problem for systems designers that the ethnographic data is big and messy and often needs to be explained. Thus obtaining the desired information from a published paper is less than ideal compared to discussing it with the ethnographers involved.
There are a growing number of studies that use ethnographic methods for studying aspects of library use, including some focusing on digital libraries [5][20]. Nardi and O'Day [17] use ethnography to investigate the requirements for intelligent agents by better understanding what human agents do.
Several LIS researchers refers to 'ethnographic-type' (e.g. [16 p80, 18]) methods to describe a range of user-oriented qualitative methods. It is in order to contrast with such methods that we use the term 'ethnomethodologically-informed ethnography' and elaborate in this paper what we mean by that. Within CSCW the term has, somewhat confusingly, been elided to simply ethnography.
Fischer and Reeves [12] studied a quite different context; customers and sales agents in a hardware store. Like our study, they try to make clear just how the sense-making activity occurs. We find it intriguing that they found very similar patterns of activity to those which we observed in a library. This increases our confidence that the activities observed are likely to obtain in a wide variety of contexts including novel modes of use in digital libraries. The authors claim, and we concur, that the results of their study can be applied to guide research in a different context; cooperative problem solving. The results can also, more modestly, be used if necessary to inform digitally mediated help giving, such as an Internet-based mail order store.
Studies of software support hotlines [22,19] indicate that there are again interesting similarities with activity at a library help desk, despite both the different domain and the fact that in the former the interaction is remote: by telephone.
Ethnomethodology is a determinedly unconstructive enterprise; it rejects explanation, as construed in constructive analysis and model building and instead places methodological emphasis on the rigorous description of the ways in which situated action is produced everyday. As other researchers in CSCW note [24] it is here that the ethnomethodologically-informed approach exacts considerable analytic purchase.
The approach helps in identifying the subtle and often unremarked cooperative aspects of work, the small scale constellations of assistance and deployment of local knowledge that enable the work (in this case the browsing) to be accomplished: that is, the method's focus is on the interdependence of work activities and those who are party to them, rather than viewing, for example online browsing, as a single discrete task.
In describing the everyday recurrent practices productive of situated action we wish to suggest tentatively that ethnomethodologically-informed ethnographies provide system designers with certain sensitising concerns through revealing the domain conditions a system is going to have to satisfy if it is to function successfully. In the case of our studies in the library, we wanted to draw out an understanding of the kinds of skills and tacit knowledges that are brought into play by staff and library users in their acts of collaborative browsing. Along with Hughes et al. [13] we contend that even if one's design undertaking is one intended to transform the application domain (regardless of the merits or problems of such a course of action) it remains crucial that designers take into account just how the domain is currently organised rather than relying upon a gloss of how it is in theory:
"... even though design may be concerned with developing a completely new system understanding the context, the people, the skills they possess, what kind of work redesign may be involved, and more are all important matters for designers to reflect upon."
So - in the case of the library study - despite the fact that system design for digital libraries may be intended to produce search facilities that do not involve face-to-face, or even person-to-person collaboration, it is our contention that if they are to support information-seeking effectively there is much to be gleaned from developing an understanding of just how collaboration in the library is used to produce the practices in which such seeking is embodied.
It is a common misconception that ethnomethodologically informed ethnography concentrates analytically solely upon the individual at the expense of attention to social structures or other formulations of higher order organisation of activity [25]. In its attention to the recurrent nature of activities in a library, ethnomethodologically-informed ethnography draws attention to the practices that are involved in information-seeking time and time again as part of the general organisation of work in and use of the library, rather than simply highlighting activities as idiosyncratic. Thus our general argument is that systems informed by such ethnographic detail should seek to support work practices as they are recurrently accomplished rather than being designed to support idealised versions of 'the search process', idealisations which are likely to have only a weak relationship to the practicalities of situated work.
We are not implying that we have uncovered the 'silver bullet' solution to the design requirements issue. Instead we present our work in the spirit of exploration of possibilities and the presentation of examples from work-in-progress.
The Service Desk in a certain UK University Library is organised into two sections:
Staff are members of other work specific teams (e.g. registration, reservations, cataloguing etc.) to which they return in between Service Desk work. They work in both sections on a rota basis, but the Service Desk is not organised in terms of staff's other specific work team competencies. Training is informal and hands-on, consisting of the trainee shadowing an accomplished member of staff.
Staff see their job as primarily one of 'helping' or 'giving assistance' to users to the extent to which they characterise 'a lot of the work' as consisting in 'finding out what people want', as 'getting details out of people', as 'trying to find what they're looking for', or more generally, as 'filtering' work.
The desk itself comprises six computers providing for 'household management' (specifically regarding borrower details) and access to the library's in-house OPAC (On-line Public Access Catalogue); hard-copy catalogues and reference documents, organisational forms and ledgers, manuals, notices regarding the whereabouts of various catalogue items, various 'frequently asked for' documents (maps, telephone directories etc.), items to be collected or reshelved, service items (e.g. photocopy cards), a bulletin board for displaying staff whereabouts and a bell for summoning assistance in particularly busy periods.
We now offer a detailed account of the practices involved in generating an understanding of what the user is searching for - an understanding that must be 'worked up' in the talk of library staff and users - and the provision of some form of acceptable candidate solution, that again must be 'worked up' and expressed in library-relevant terms. It is just these practical specifics to which design must attend if it is to make a serious attempt at providing digital library support for the search process.
In delineating an information query users typically initially give a vague description of their information requirements:
User: its erm .. its . like information . information about er . these particular products and services ... market intelligence and leisure intelligence etcetera etcetera'
Besides staff asking 'what the problem is', the provision of a vague description is the first action in the concerted practice of intermediated sense-making and thus of solving information requirement problems.
Vague descriptions must be categorised or made intelligible in terms of the catalogue's organisation as this highly edited interaction illustrates:
Staff: is er . is it a serial?
User : no . its not a journal
User: basically its a reference book . and it tells you about particular market products and services and what to look for
The problem we need to understand in order to address the research or design question is: just how are vague descriptions made intelligible within terms of the catalogue's organisation? We found that one way staff and users achieved this was by employing organisational artefacts:
Staff: what have you got there. is it something you've got written down?
User: yeah . em I'm trying to find out about this (shows staff a list and points to a titled item on it) this part here
Staff: (looking at list) it sounds more like figures and graphs and things
User: yeah
Not all users are in possession of lists and our observations suggest that lists, whether hand written notes derived from previous information searches or task lists, are inadequate for purposes of specific categorisation - i.e. for actively establishing the precise nature of information requirements - in that they provide only partial details of information requirements. The product of list use however, is the establishment of information requirement parameters or boundaries through the establishment of preliminary information requirement categories (IRCs): e.g. 'it sounds more like figures and graphs and things'. In establishing preliminary IRCs and requirement boundaries, list use provides for the next problem solving action. In other words - and here we simplify for the sake of brevity - artefacts such as reading lists are frequently used in formulating preliminary IRCs, which are in turn used as tools in the ensuing interaction that 'works up' potential categories of candidate solution:
User: so . I got that one .. what about these ward lock guides .. what . (points to list) . what would I put in there
Staff: er
User: er . the guide books . like travel guide books
Staff: are they more like a tourist guide
User: it would be like a tourist guide . early tourist
guide nineteen hundred and . nineteen . yeah probably early nineteen
er . twentieth century
User: I've been told
Staff: but
User: to look in second hand book shops y' know . for them . y' know
Staff: yeah I mean all of these (pointing to items on list) where they're showing y' volume number
User: yeah
Staff: although it doesn't say journals . you would expect to find that . at option six . that journal serials title
In this edited segment of talk, the user and intermediary simultaneously orientate to and employ the list to provide and elicit categorisable descriptions of the information requirement: e.g. travel guide books, early tourist guides, journal, serials. The use of lists trades specifically on:
_ users' personal knowledge of information requirements
_ the details of the user's prior information searches leading up to the current enquiry
_ staff's knowledge of catalogue contents organisation
By list employment, staff and users use this knowledge to actively establish preliminary IRCs which bound and provide for the next action in intermediated problem solving practice within library systems.
Establishing specific information requirement categories is the next action within intermediated sense-making practice in library systems. This part of the practice consists of formulating specific IRCs:
S: (looking at list ) it sounds more like figures and graphs and things
U: yeah
S: aren't they .. um . we'll see what we get just looking under 'title' (initiates OPAC search ) cos that's (inaudible) (turns screen towards user ) there's a few . options you can use really on the computer . you've got keyword search . you've got subject search
U: yeah
S: and once you find a relevant class mark area for the subject
U: yeah
S: y' know . then you can look on the shelves to see if its available . er ... what have we got (browsing display - approx 11 seconds )
S looks at user then at screen, makes an inaudible comment
U looks at screen, makes an inaudible comment
Both browse retrieved title display on screen in silence - approx 6 seconds
S: it could be that its worth looking around that (points at item on retrieval list) .. oh that's a video . that's not very helpful . really .. its an ancient one as well (inaudible) erm .. (inaudible) class mark A . it could be . er (types in new search commands)
Both browse display making inaudible comments
S: its more to do with science
U: um
S: ooh . hey look ... right ...... um that's putting you more in the physics area I think . I think if you don't find it in science what could be worth you looking at is . er . having a word with the subject librarian
U: yeah
S: there are a lot of maps that give . er ... I don't know what you're looking for
Taping interrupted - approx 30 seconds - staff and user browsing a new retrieval list)
I think we'll send you to the librarian . cos with me browsing like that .. the subject is quite specific
While not directly solving the user's information requirement in this case, the above segment of talk makes visible the work of specific IRC formulation. Having established a preliminary IRC - "it sounds more like figures and graphs and things" - staff initiates an OPAC Title Search as provided by the list. Staff then turns the OPAC screen towards the user - a common feature in Service Desk interactions - and browses the retrieval list with the user. In browsing a series of OPAC retrieval lists - three in this case - staff and user concertedly formulate increasingly more 'specific' IRC's: in and through browsing the catalogue within the boundaries of the preliminary IRC, "figures and graphs and things" is worked up into something "more to do with science" and then something "more in the physics area": a description of the information requirement that in situ is specific and provides for the next problem solving action: referral of the user to a subject librarian and further specific IRC work.
Lists and the catalogue are formulating artefacts specifically employed as information requirement elicitation, and thereby categorisation, devices. All IRC work is incremental - achieved not only through the employment of such artefacts, but also through natural language; notably the use of interjective particles: 'ooho', 'um' 'ahh' etc. or, more formally, 'yes', 'no' or 'perhaps' etc. In cases where it is not possible to establish specific IRCs, sense-making work is brought to an end. Users are usually referred to general areas of the catalogue as established through preliminary or last formulated specific IRC in order to gather further information.
The OPAC system was being used in a manner for which it had not been designed. In the abstract this is an entirely unremarkable observation. What is of interest is the specifics of this additional use - as a resource in the ongoing collaborative formulation of enquiries and the production of candidate solutions. This use was improvised, with staff and users ordering their interactions around gestures to the OPAC terminal as they worked up increasingly specific information retrieval categories. Such recurrent practices clearly raise interesting issues for design that will be addressed in the next section of this paper. It is worth noting here that it is precisely these kind of practices which display the considerable tacit skills involved in the practices of information-seeking, an understanding of which goes a considerable way to reformulating the notion of designing support for information-seeking in a digital library context.
"Look, I don't care about all this ethno babble, just tell me what to build!"
"All you ever do is tell me stories"
"Why is this useful? What can I do with it?"
"OK. One more time. Are you saying that.."
Quotes (from a Computer Scientist) in project meetings
We must acknowledge the problems with making use of ethnographic data in systems design. Although the CSCW researchers at Lancaster have considerable experience of this activity over a range of projects, the quotations above illustrate some of the problems. Computing and ethnography have two quite different world views. They use different language and, worse, the same language to mean quite different things. The terms "process" and "model" spring to mind as yielding major arguments. The achievement of a common understanding of what has been found and what developers can make use of is a time consuming process of continual explanation and clarifying questioning.
The growing experience of the use of ethnographic methods has led to a lively debate[13]:
"For many software engineers ethnography seems far too unsystematic a method, its results presented in a discursive form, design options are not clearly stated and do not attend sufficiently to engineering needs."
Those involved need to strike a balance between the production of rich ethnographic descriptions and the necessity for abstraction and schematisation underpinning the computing discipline.
A problem with ethnographic information, from the perspective of a computer scientist, is that it is non-judgmental: it tells you what is, not how it ought to be. By contrast, from the perspective of the ethnographer this is seen as a strength of the method not as a problem! To tell you how it 'should be' immediately negates the role of describing the situation from the social actor's point of view, and thus risks disembedding the design system from social context.
However the information can carry warnings about potential implementations on the lines of: 'This observed activity works successfully by this mechanism. Will your new system support the same or an analogous way of addressing the problem?' Clearly it does not tell you much if you believe the problem being solved by the activity will cease to exist in the new system.
Ethnography, even though it focuses on the particular, can provide information that can be abstracted. In the case of this study, we believe that it reveals the importance of help giving, particularly in uncovering initially vague and hard to specify information needs. Even though in the study, the clarifying dialogue was between users and librarians, we claim that it carries implications for supporting many different kinds of dialogue in digital libraries.
Note that the librarians were able to offer help even in cases where they did not have substantial expertise in the subject domain. As such, we can use this information as a starting point for studies of informal help-giving between users as well as for designing systems for supporting expert-user help-giving. Furthermore, we would claim that even when working alone, information searchers would need to go through equivalent processes of refinement to those made visible because they occurred in a dialogue. The similar data from other contexts [12,22] supports this claim for generality.
Although our studies are preliminary, ethnographic data does reveal a set of issues that can be used to inform the design of digital libraries:
Collaboration is a significant way in which some users achieve their goals. Clearly there is a bias in this study: it was focusing on a place where collaboration does occur. Nevertheless it is salutary for systems developers to remember that users will need help and their first resort is not necessarily a help system, a manual, or having taken a course of instruction but rather people to whom they have access [1]. It would make sense to design systems that take account of this, rather than ignoring it. With technological support a greater variety of forms of help giving become possible, including but not limited to those currently existing in physical libraries. By knowing the mechanisms by which help is done now, we can inform our designs for functionality to support this.
The context of use, a university library with a well designed OPAC, nevertheless led to considerable problems for some people. These were mostly not to do with the interface of the system but rather that of determining how their information needs could be met by using the functions available by the system. The extracts illustrate the difficulty caused by the initial vagueness of some information needs. It seems unlikely that the conceptually more unfamiliar digital library, no matter how much more sophisticated the search facilities it provides, will cause any fewer problems. We need to recognise that we must design systems for many different kinds of users including 'perpetual novices'[6].
The library staff on the help desk were not necessarily experts. They have a range of skills (and share the work amongst themselves - not discussed in this paper) and frequently manage to help even though they lack domain knowledge that would have led to a more rapid solution of the problem. This bodes well for peer support, provided that the library system supports help giving.
Where collaboration occurs, a computer system and most especially its interface can serve as a conversational resource. In the study, the librarians at the help desk would often twist the terminal round so that the enquirer could see it. This was not just on occasions to show how to do a particular kind of search, but also where it was the result of a search that served as a focus of discussions. Both Librarian and user may point at items on the screen to support their mutual clarifications about what the conversation was actually about.
Note that this is quite a different way of talking about interfaces to support collaboration to that usually discussed in CSCW. In the study we are not talking about a computer interface that allows people to collaborate remotely through it. Rather we have an interface to a system that is being used both in the way in which it was intended, to query a database and return results, but also as to help people to determine what the search need was and how to go about finding it, by discussing the changing items on the screen.
Context is extremely important in help giving. Librarians are aware of this and almost snatch at any piece of paper that an enquirer may be holding (such as a booklist, handwritten reference or task description) that may help provide context. Contextual information allows the giving of more effective and especially more efficient help. In the absence of appropriate context, participants in a dialogue may persist in talking at cross purposes, or fail to identify the underlying need.
Information searching is more complex than just deciding what you want, working out the right query to compose and then typing that in. This is not news to LIS people (e.g. [2][20]), and clearly this awareness should inform digital libraries systems design. Designers should allow for the idea that 'finding something' consists of a whole number of searches. A search activity may extend over a long time and be interrupted, then resumed. Searching for information is a fluid process: using the system affects the goals of the search process.
We can begin to consider how these issues can actually be used in systems design:
Firstly, we believe the method is generally useful as a means of informing systems designers of the ways in which their systems will be used. Ethnographic data can be used to tell illustrative stories that can serve as scenarios of use [8]. This can tackle a maybe excessively simplistic view of what the need is, by use of carefully selected examples of activities. These can be contrasted with the perhaps simplistic abstractions of how people 'ought' to do it.
Secondly there are some implications for developing functionality and interfaces that support different kinds of help giving, both co-located and remote, synchronous and asynchronous. The importance of context in help giving implies that a system that supported the providing of context would be more useful and usable. We can expect that where help giving is remote and even asynchronous, the importance of context is even greater. One way of providing context is for the system to record a search history and to provide a visualisation of this history. We have developed a system that provides such a feature [27] though we wish to stress that it is merely one example of providing such functionality. The ethnographic data does not tell us that this is the right way, just that there ought to be some sort of way.
Finally, the findings raise issues for research agendas for investigating the development of quite new interfaces and functionalities to meet the needs identified in the detailed study of a particular context of use, but to be applied in a different context (say remote help giving) [12,17]. Given the acknowledgement that searching can be a prolonged activity, some of which can be undertaken alone, some in conjunction with others, we can wish for better ways of preserving this information over time and place (the user may well move around during this process). In trying to uncover what the user wants, it helps to have a richer context. It would be useful to have ways of informally representing what the user wanted to do as well as what was actually done. A representation of goals that would need to allow for their rapid change has done, is doing, wants to do. It may be also useful to have a representation of common processes and plans [3], again acknowledging that they will frequently be abandoned or mutated. These representations of common processes could be used not only to determine what someone has done so far and why, but also to provide explanations of things they might want to try next.
The activity of intermediated information seeking is resolutely embedded in the social activities of talking in the library, making use of lists and the OPAC. We suggest that formalising this practical process of 'seeking' into 'querying a database' can, through the disembedding of library work from its social context, have negative implications for the ability of the digital library to provide support for the kinds of collaboration identified in our fieldwork.
We feel that there is a danger in designing digital libraries that are dependent on a particular, abstract and solitary model of information-seeking; rather than being grounded in empirical observations of the practical accomplishment of organisational activities. We need to provide support for actual behaviour rather than abstract models of that behaviour. As such this paper is as much a methodological recommendation - in support of ethnomethodologically-informed ethnographic methods (despite their problems) - as an introduction to some of the systems design principles which emerged from the study.
The authors would like to thank Mark Rouncefield for general comments and John Hughes and Lou Armour for their methodological insights into categorisation work. This work was funded by the British Library Research and Innovation Centre.
[1] Barry, C. The Digital Library: the needs of our users. Paper presented at the International Summer School on the Digital Library (Tilburg University, The Netherlands, 5th August 1996).
[2] Bates, M.J. The design of browsing and berrypicking techniques for the online search interface. Online Review 13, 5 (1989) pp. 407-24.
[3] Bates, M.J. Where should the person stop and the information search interface start? Information Processing & Management 26, 5 (1990) pp. 575-91.
[4] Benson, D and Hughes, J.A. - The Perspective of Ethnomethodology, Longman, London, 1983.
[5] Bishop, A.P., Star, S.L., Neumann, L., Ignacio, E., Sandusky, R.J. and Schatz, B. Building a university digital library: the need for a user-centered approach. Paper presented at Higher Education and the NII: From Vision to Reality Conference (Monterey, CA, September 26-29, 1995).
[6] Borgman, C. Why are online catalogs still hard to use? Journal of the American Society for Information Science 47, 7 (1996) pp. 493-503.
[7] Brewer, A., Ding, W., Hahn, K. and Komlodi, A. The Role of Intermediation Services In Emerging Digital Libraries', Proceedings of the 1st International Conference on Digital Libraries, ACM, New York, 1996, pp 29-35.
[8] Carroll, J.M. How to avoid designing digital libraries: a scenario-based approach. SIGOIS Bulletin 16, 2 (1995) pp. 5-7.
[9] COMIC Deliverable 2.2, Field Studies and CSCW, Lancaster University, October, 1994.
[10] Dervin, B. and Dewdney, P. Neutral questioning: a new approach to the reference interview. Reference Quarterly 25, 4 (1986) pp. 506-13.
[11] Dervin, B. and Nilan, M.S. Information needs and uses. Annual Review of Information Science and Technology 21, (1986) pp. 3-33.
[12] Fischer, G. and Reeves, B. Beyond intelligent interfaces: exploring, analyzing, and creating success models of cooperative problem solving. Applied Intelligence 1, 4 (1992) pp. 311-32.
[13] Hughes, J., King, V., Rodden, T. and Andersen, H. - 'Moving Out of the Control Room: ethnography in system design", Proceedings of CSCW'94, 1994, pp. 429-40.
[14] Hughes, J., Kristoffersen, S., O'Brien, J. and Rouncefield, M. - 'When Mavis met IRIS: ending the love affair with organisational memory' Proceedings of IRIS 19, 1996. pp. 767-88.
[15] Kling, R. and Elliot, M. Digital library design
for organizational usability. SIGOIS Bulletin 15,
2 (1994) pp. 59-70.
[16] Kuhlthau, C.C. Seeking Meaning: a process approach to library and information services. Ablex, Norwood, NJ, 1993.
[17] Nardi, B.A. and O'Day, V. Intelligent agents: what we learned at the library. Libri 46, 2 (1996) pp. 59-88.
[18] O'Day, V.L. and Jeffries, R. Orienteering in an information landscape: how information seekers get from here to there. In Proceedings of INTERCHI'93 (Amsterdam, The Netherlands, 1993) ACM, New York, pp. 438-45.
[19] Orlikowski, W.J. Evolving with Notes: Organizational Change around Groupware Technology. MIT CCS Technical Report 186, 1995.
[20] Paepcke, A. Information needs in technical work settings and their implications for the design of computer tools. CSCW 5, 1 (1996) pp. 63-92.
[21] Page, D., Williams, P. and Boyd, D. (1993), Report of the Inquiry into the London Ambulance Service, Communications Directorate,South West Thames Regional Health Authority, London, Feb. 1993.
[22] Pentland, B.T. Organizing moves in software support hot lines. Administrative Science Quarterly 37, 4 (1992) pp. 527-48.
[23] Rouncefield, M. An Ethnography of Cooperative Work and Technology in the Financial Services, PhD Thesis, Lancaster University 1996.
[24] Shapiro, D. The limits of ethnography: combining social sciences for CSCW. In Proceedings of CSCW'94, 1994, pp. 417-28.
[25] Sharrock, W. and Watson, R. 'Autonomy Among Social Theories; the incarnation of social structures', in Fielding, N. (ed) Actions and Structure, London, Sage, 1988.
[26] Taylor, R.S. Question-negotiation and information
seeking in libraries. College & Research Libraries
29, 3 (1968) pp. 178-94.
[27] Twidale, M.B. and Nichols, D.M. Collaborative
browsing and visualisation of the search process. Aslib Proceedings
48, 7-8 (1996) pp. 177-82.
[28] Twidale, M.B., Nichols, D.M. and Paice, C.D.
Browsing is a collaborative process. Information Processing
& Management (to appear).
http://www.comp.lancs.ac.uk/computing/research/cseg/projects/ariadne/docs/talking.html
ariadne@comp.lancs.ac.uk