Introduction to Part II: Studies of Interaction Analysis
Looking back on the preceding historical documents—little modified from several years ago when the reported investigations were just winding down—from the perspective of 2004 as= I compile this book, the contributions of the individual studies to the overa= ll inquiry becomes more clear. I will reflect from this hindsight briefly here= as a way of motivating part II.
Study 1, TCA. This effort always seemed ahead of its time. It failed to attract continued funding because while the funding source approved of its goals, the reviewe= rs were worried how the quality of the curriculum content and its indexing cou= ld be guaranteed. Similarly, although our contacts at Apple Corporation liked = the concept, they could not see how such software would help their quarterly financial bottom line in the near future. In the intervening decade, the NSF has spent millions on a major national science digital library initiative (NSDL) and Apple tried to build its own educational object economy (EOE) on= line repository of curricular artifacts. Today, the Math Forum, one of NSDL’s most successful digital libraries, consi= sts primarily of content submitted by users, demonstrating the power of communi= ty knowledge construction. Large international efforts have gone into trying to define and negotiate standard metadata ontologies (e.g., Dublin Core); they remain incomplete, inadequate and superficial bec= ause systems of interpretive categories cannot be legislated, once and for all, = from on high; they must evolve with usage and understanding within specific communities. Ten years of technological advance, the pervasive growth of the Web and the establishment of digital libraries have changed the way = TCA would work today. But they have not eliminated the basic needs that = TCA was designed to address. Publishers and teams with NSF funding have meanwhi= le produced integrated curricula along constructivist principles, but still-isolated teachers have benefited little from the collaboration potent= ial of the Internet. Some of the ideas of TCA have yet to be tried by groups of teachers and to be evolved in response to their usage. In general, the idea of virtual communities of teachers sharing best practices has not really taken off yet, despite various efforts like Tapped-In, Merlot or ENC.org. Perhaps it is no coincidence that I am now working with the Math Forum and my TCA collaborator, Tamara Sumner, is working with DLESE, another major education= al digital library project, trying to increase their utility for constructivist learning.
Study 2, <= span style=3D'mso-bookmark:_Toc39394818'>Essence. The use of LSA (latent semantic analysis)—explored in this study̵= 2;is still being pushed as both a panacea for automating student evaluation and = as a model of human semantic understanding. This book probl= ematizes the assumptions underlying LSA by discussing many means by which people understand language in non-algorithmic ways: through their life experience situated in specific activities within a meaningful physical, cultural and historical world; by means of the collaborative negotiation of shared meani= ng in small groups and communities; through personal, professional and cultural perspectives; and thanks to subtle processes of contextualized interpretati= on. The success of LSA in State the Essence is due to the fine tuning that took place as the algorithms, coefficients a= nd mechanisms were co-adapted with the research effort, the teacher presentati= ons and the student expectations or behaviors. This suggests that it is possibl= e to take advantage of this technology, but only in certain carefully designed applications and through extensive trial and adjustment. The = Essen= ce software was originally conceived as a way of evaluating the product of individual work, as LSA is usually applied. However, the study suggests also using such software to stimulate collaborative interactions and to provide feedback and motivation to small groups. This is an approach that deserves further thoughtful exploration in the context of CSCL.=
Study 3, <= span style=3D'mso-bookmark:_Toc39394818'>CREW<= /b>. The problem of group formation is one of the first issues I face each term = when starting a course organized around small-group projects. Supporting the self-organization of students into effective work groups is a function that= is particularly needed in virtual communities, but has not been extensively researched. This has been identified as an important area for exploration in the Virtual Math Teams (VMT) project that is now underway at Drexel, but is mostly beyond the scope of the present book (see chapters 17 and 21). The approach reported in the CREW study is quite different. It attempts to aid administrators in the formation of astronaut crew groupings by giving them feedback on the probability that gi= ven individuals will perform well psychologically under given mission condition= s. It develops a temporal model of individual factors based on case study data under analogous conditions, using a combination of adapted AI and statistics methods. Pushing this approach to its limits, it shows the enormous requirements such a system has for high quality data across the whole range= of interest. Given that little relevant data currently exists and the difficul= ty of evaluating its status, as well as the practical and political barriers to collecting much more data, it seems doubtful that this sort of approach can succeed in many realistic situations. Without adequate data, such AI methods are empty promises: garbage in / garbage out. In addition, the expectation = that relevant data can easily be collected in an explicit and context-free manner was shattered with the failure of the expert system craze.
Study 4, <= span style=3D'mso-bookmark:_Toc39394818'>Hermes. This system suffered the problem of all DODEs. = The effort required to configure a system for a particular application domain is enormous. One must be an expert in that domain, understand the detailed work-flow, spot the functions that can usefully be supported and seed the system with vast amounts of domain knowledge. All this is necessary before anyone would even consider trying out the system. In order to support the w= ork of a group of domain experts, the system must be complex and sophisticated, combining advanced features as well as all the basics. To develop such a sy= stem requires the combined talents of software developers and domain experts, al= ong with a budget on the order of a million dollars—just to produce a sys= tem that can be tested by a small community of friendly users. There are engineering and medical applications where such an effort might be financia= lly practical, but it does not seem to be a workable approach in the grossly under-funded education arena.
Study 5, <= span style=3D'mso-bookmark:_Toc39394818'>CIE. As the studies of part I progress up through this one, there is a growing awareness that the important information is not just domain-specific and gradually evolving, but is specific to each community of practice and is constantly changing and being re-negotiated. The studies from this point on= try to respond to this finding. The idea of supporting communities of practice = with specially tailored computational media seems plausible. The issue then is o= ne of attracting a whole community to a particular platform and getting them in the habit of using that system for their regular community participation.
Study 6, <= span style=3D'mso-bookmark:_Toc39394818'>WebGuide. This study confronts the crisis of adoption. Designers can go to great leng= ths to design systems to provide wonderful tools, but people—even students studying the design and use of such tools—resist using them. <= /span>WebGu= ide provided a sophisticated system of inter-connected per= spectival views on an asynchronous discourse. The intention was to support on-going knowledge building. But it was under-used and at best served to exchange personal opinions. It failed to merge ideas of different people together in= to effective group cognition.
Study 7, <= span style=3D'mso-bookmark:_Toc39394818'>Synergeia. This system added many features, based on a review of what typical CSCL and CSCW systems offered. In particular, it supported group negotiation as well= as perspectives and the definition of groups, providing a structure for the interaction of ideas at the individual, small-group and community levels. As necessary or useful as such features are or could be, the proliferation of features is not sufficient to overcome the barriers to adoption confronted = by all collaboration systems. To design more effective media (the goal of part= I), we will need better models of computer-mediated collaboration, clearer conceptions of group negotiation, and detailed studies of small-group interactions (as is illustratively undertaken in part II).
Study 8, BSCL<= /b>. The <= span style=3D'mso-bookmark:_Toc39394818'>BSCL study carefully conceptualized negotiation based on current understandings of collaboration. The failure of the <= span style=3D'mso-bookmark:_Toc39394818'>BSCL negotiation mechanism to be used as intended showed the need for more detailed analysis of how people actually collaborate and negotiate in normal life. Parts II and III will therefore empirically and theoretically investigate how knowledge is actually constru= cted and negotiated in small-group interactions.
How should one evaluate the success of = these studies? Each provided a valuable learning experience in the design of groupware. Some never got much past the conceptual design phase—perha= ps producing a detailed scenario, a set of interface designs or a limited work= ing prototype—while others have survived in one form or another. <= /span>
TCA ended before ever being tried by teach=
ers,
but its designers are now deeply involved with major digital library projec=
ts
that carry on much of that vision. Several versions of Essen=
ce
were tested in classrooms for two years, eventually demonstrating statistic=
ally
significant improvement in learning outcomes in controlled experiments repo=
rted
elsewhere. The technology refined in Essen=
ce
is now used in scoring various national tests. =
Essen=
ce
itself is now being used in dozens of schools in the state of
Versions of WebGu=
ide
were used in classes as reported in the study, although the final implement=
ed
version was never actually deployed. Its perspectives concept reappeared in=
Synergeia
and <=
span
style=3D'mso-bookmark:_Toc39394818'>BSCL, simplified and integrated with
negotiation support. Another version of personal, group and class perspecti=
ves
on threaded discussion appeared in the Polar=
is
system from the
As other researchers have also discover= ed, threaded discussion and chat systems, as their names suggest, are generally used for the relatively superficial exchange of opinions rather than deep, interactive knowledge building. To the extent that the systems presented he= re were designed to support group cognition (knowledge building, situated interpretation, intertwining of perspectives, knowledge negotiation, etc.), this shows how far we still have to go. The fostering of group cognition is= a socio-technical problem, which is not automatically solved by offering cert= ain functionality in a technical system (Kling & Courtright, 2004). It will require designing whole activity systems or shared worlds around such systems, base= d on a detailed analysis and understanding of collaboration.
Two releases of Synergeia were fielded in European schools as reported. Extensive surveying of teache= rs and students who used it showed that they liked it and had no fundamental criticisms of it. However, this form of evaluation provided little guidance= for further software development. In particular, it is not clear that the negotiation mechanism was even used in the schools. I used BSCL<= /span> in two of my Drexel courses on human-computer interaction and had my studen= ts design extensions to the negotiation mechanism. In this process, I experien= ced again how hard it is to adopt the use of that mechanism in an effective man= ner. The <= span style=3D'mso-bookmark:_Toc39394818'>BSCL version of the software has now been integrated as an option of the popular BSCW<= /span> collaboration system and is being used in many European classrooms.<= /span>
Groupware is hard to assess. To see how= it really supports groups, one must have groups use it under relatively natura= listic conditions and for a long-enough time to become comfortable with it. But th= is requires not only building a sufficiently complete and robust prototype for group usage, but also finding an appropriate group of users, training them = in its use and involving them in a meaningful application of the software that properly exercises the functionality of interest.
Getting a group to use software is not = easy, even once all the preconditions have been established. I found this repeate= dly when using groupware prototypes in my classes. Users resist. Information science majors who are interested in the design of innovative software, stu= dents whose grades depend on entering comments by using the software, and participants in courses whose activities have been designed around the use = of the software, are all reluctant to use the software, and they constantly lo= ok for more familiar alternatives: meetings, conference calls, email, instant messaging, etc. Adoption becomes th= e issue. It dominates over all the technical issues of groupware design. As o= ne experiences the studies of this part, it becomes clearer and clearer that t= he problem is to design socio-technical systems, where the technological produ= ct is simply an artifact to mediate the important, complex and poorly understo= od processes of group collaboration. That brings us to the need to increase our understanding of the social-systems aspects of groupware design through analysis and theory of small-group interaction.=
The preceding eight groupware design st= udies thus supply a sense of the potentials, the issues and the challenges inhere= nt in the design of collaboration technology. They provide a shared experience= to motivate and lead into the parts that follow. Like all case studies, the experiences they offer are limited by their specifics; they should not be looked to for conclusions concerning the effectiveness of their innovations. They may, however, be legitimate and worthwhile explorations of what is possible through investigation of what actually happened under unique and irreproducible conditions. Taken in this sense, the experiences of part I furnish useful occasions for the situated interpretation of what it means t= oday to support collaborative knowledge building and group cognition.
In developing the studies of part II, t= wo analytic perspectives played a major role: socio-cultural psychology and co= mmunication analysis. I actively pursued an understanding of them in order to resolve s= ome of the mysteries that arose in my earlier software studies.
Socio-Cultural Psychology. Vygotsky’s thinking had an immediate catalytic effect on me when I first read his Mind in Society (1930/1978). I was excited by = his deep and original appropriation of Hegel and Marx, and by his materialist theory of mind. I was intrigued not so much by what he actually explored in= his experiments and what is generally interpreted as a psychology still centere= d on the individual mind, but by the vision he sketched, often between the lines= , of a truly socially-constructed mind, whose consciousness is derivative of the culture in which it was constructed. While my reading of Vygotsky is explicated more in part III, his emphasis on the role of artifacts in mediated cognition is already central to part II. In particular, these stud= ies pursue the question of how people come to understand the meaning or afforda= nces of artifacts and what implications this has for the design of groupware conceptualized as a mediating artifact.
Communication Analysis. In my search to understand perspectives and negotiation, I tu= rned to communication analysis. This choice was obviously also compatible with <= span class=3DSpellE>Vygotsky’s emphasis on language and interaction. Colleagues, methods and ideas from the discipline of communication made possible the analyses of this part, particularly chapter 12. The most relev= ant work for me was that of ethnomethodology (Garfinkel, 1967) and conversation analysis (Sacks, 1992). In general, I thi= nk that interactionist theories of communication h= ave led the way in understanding the philosophical and methodological issues th= at are essential for developing a theoretical framework, empirical analysis and software support design practice for collaboration.
These five essays try to analyze the na= ture of small-group interaction as it actually occurs. They:
· propose a model of small-group knowledge building. (chapter 9)
· critique prevalent CSCL methodologies for systematically ignoring the group interactions. (chapter 10) =
· suggest new approaches that focus on the gro= up discourse. (chapter 11)
· conduct micro-analyses of small-group interaction, detailing a group decision-making negotiation and looking at i= ts cognitive ramifications for group understanding and activity. (chapters 12 and 13)
Chapter 9, a model. This study started= as a tentative working paper when it first occurred to me that we needed to have some kind of graphical representation of the important process called knowledge building <= /span>(Scardamalia & Bereiter, 1996) that I had started= to refer to constantly, but that seemed only vaguely defined. Gradually, as I circulated the paper for comment, shortcomings of the model became apparent, both in terms of its representation and its content. Nevertheless, the diag= ram has endured with only minor modifications and continues to prove useful. Th= is paper—with its graphical model—has always been one of my most popular and suggestive writings because it starts to articulate what goes i= nto collaborative knowledge building.
Chapter 10, rediscovery. When asked to look at a couple of representative CSCL papers, I began to question the adequacy of available analytic methods. In particular, I bumped into the old lamppost problem: people tend to search w= here it is easiest to see and measure things, even if the important things lie elsewhere. It struck me that some of the most essential phenomena of computer-mediated collaboration were being systematically eliminated by the very methodological procedures that were recommended for rigorously analyzi= ng them.
Chapter 11, contributions. At the CSCL ‘02 conference, which I viewed as an occasion for injecting a more theoretical perspective into the field, I proposed a set of four notions th= at could contribute to a deeper understanding of collaboration: knowledge building, group and personal perspectives, mediation by artifacts, and conversation analysis. By pulling together these four themes, the paper effected a transition from the design issues of knowledge-building support, perspectives mechanisms and software artifacts to the micro-analysis of col= laborative interactions.
Chapter 12, a moment. This study looks closely at a transcript from an intense half-minute interaction among five students involved in an activity with SimRocket, a computer simulation of model rockets. A thick description of this hard-to= -interpret discussion shows how the small group constructed group knowledge, which each participant came to share. The phenomenon of group cognition appeared here, where the indexical, elliptical and projective character of the utterances showed that their meaning only existed at the small-group unit of analysis,= not as something attributable to individual cognition. This study provides a pivotal point for the book. Its transcript is repeatedly referred to in the subsequent chapters. In fact, much of the theory presented in part III is derived from this 30 second episode, illustrating how much can be learned f= rom detailed reflection on a brief case study.
Chapter 13, references. The preceding study’s analysis is expanded here to dissect the nature of the group cognition that took place around the SimRocket artifact. It is argued that, before the collaborative moment, the group cou= ld not see the structure of the SimRocket list of rocket characteristics, but that through their interaction they lea= rned to see the new kind of structure and taught each other to see it. This group conceptual change allowed the group to repair the breakdown in relational references of their utterances to the artifact. This incident provides a key case study for the theoretical reflections on group cognition in part III.