Glossary · References · Reading This Document
5.2 Case Studies of
Research Webs
There have been four attempts to develop Research Webs, and while none
were successful, much knowledge was gathered. The design of Research
Webs was largely theory driven, but practical experience has been gained
from the attempts to develop RWs. Much of the knowledge was applied
to the tools, especially DocReview, but other knowledge gained has come
from understanding human nature, especially the failure to participate.
While perhaps a few minor problems with RWs may be laid at the feet of technology, by far and away most barriers are related to human behavior: psychological, sociological, organizational, and cultural. The research questions we examine are designed to uncover behavior patterns and the reasons that may contribute to causing those patterns. Discussion of counterproductive behavior leads us to some suggestions to modify that behavior. To the extent that such causes may be remediated by technology, the tools applied to the RW will be augmented or changed.
5.2.1 History of Research Web Technology 
Technological aspects of Research Webs have co-evolved with the World
Wide Web. Beginning from the initial release of the Mosaic web
browser in 1993, the potential of the WWW was clear. The technology
was at that time blooming on a monthly basis and, as time allowed, the new
capabilities were incorporated into the Research Webs.
The prototype RW was the Migration RW, originally a simple hypertext installation with but one image, a Dorothea Lange photograph of an Okie family on the road to California during the Great Depression. The first new feature incorporated was the inclusion of an organizing model, a diagram of the migration process from information gathering to settlement. The diagram, a petri net (Peterson, 1977), was image mapped so the user could click on an object in the diagram, a node or link, and be transferred to a page that described the object. This RW was essentially a technical proof of concept site, and included crude bibliographic and glossary links, and a commenting capability limited to global comments on each web page. No research team was assembled.
The first RW, the Chromium VI RW, was the first opportunity to engage a team of scholars. By that time DocReview had been developed, allowing participants to make annotations on small chunks of text from the web page. DocReview was used to annotate documents and meeting minutes. In the later stages of that RW, JavaScript was employed to provide the ability to pop up small auxiliary windows on the screen. If the user wanted to view a DocReview, bibliographic information, a glossary definition, footnote, or a sidebar, those features would be displayed in small windows without overwriting the main document window.
In the final two RWs, the Soil Crust RW and the Earthquake Disruption RW, the ability to annotate bibliographic entries and glossary entries was added, bringing the RW technological environment to its current state.
5.2.2 Research Questions
There have been several research efforts that used the collaborative
tools of the Research Web environment. Four of these efforts utilized the
fully developed concept of the research web, while others used only parts
of the concept or the incompletely developed concept. The case study of
the Research Webs will address several research questions.
The research questions and associated propositions are:
1. What was the focus ( issue domain) of the RW?
2. What were the geographic distribution effects on the RW?
3. How many people were invited to participate in the Research Web?
4. What incentive(s) did each of the participants have to participate?
While several additional questions were considered, only these could be addressed properly. The questions were posed after the active lives of the Research Webs. Several could not be answered due to lack of data. Others would have needed questionnaires for proper evaluation. Several of these currently unaddressable questions are presented in the Future Research section.
5.2.3 Design of Data Collection System
The data collected on the research webs consists primarily of copies
of the web sites, meeting minutes, and correspondence between the RW's
scientific coordinator and the
facilitator. Web sites include data
not only in web pages, but also data in the form of annotations in
DocReviews, Annotated HyperBibliographies, and Annotated
HyperGlossaries. DocReview builds a log file, which contains all
transaction activity: creation, annotation, reading and archiving.
There were two hosting servers that went off-line during this research,
but the four Research Webs described below were captured before
destruction, or were recovered from the server host after the server
software failed.
5.2.4 The Research Webs
In this section we will describe four cases, a prototype and three
Research Webs, and using those cases and events and circumstances in
associated enterprises will discuss the research questions (above
§5.2.2). Each RW had as its issue domain a topic that was subjected
to scholarly research. All the cases were hosted on web servers at the
University of Washington and were facilitated by the author.
5.2.4.1 Migration Prototype Research Web
This Research Web was initiated as a prototype and test bed for the
Research Web concept to demonstrate the power of the WWW to facilitate
research. The topic was internal migration viewed from a behavioral
standpoint. The site capitalized on the work done in my Master's Thesis,
a section called "A Model of the Migration
Process"
(Hendricksen, 1994). The
importance of this site is related to the testing of the technology later
applied to Research Webs and to the realization that the Research webs
were a social organization driven by social and personal goals, not
technology.
5.2.4.2 CREAT and The Chromium VI Research Web
The Collaborative Risk Evaluation Approaches Team (CREAT) was an
attempt to build an interdisciplinary team to investigate a small set of
problems centered on the cleanup at the Hanford Nuclear Reservation.
It was staffed with members who were supported by the Consortium for Risk
Evaluation with Stakeholder Participation (CRESP) project at the
University of Washington. CREAT was the first user of many of the
tools now used in Research Webs. A Research Web was initiated to
investigate one issue that concerned CREAT, hexavalent chromium
contamination. It was hoped that the Chromium VI RW would serve as a
template for describing several other contamination issues.
5.2.4.2.1 Mission and RW Topic
The mission of the Collaborative Risk Evaluation Approaches Team (CREAT) is to provide information about hazards and risks to human health, ecological health, economic health, and socio-cultural health within and around DOE sites. CREAT is developing an easily accessible, computer-based tool for collecting, cross-indexing, displaying, and comparing the components influencing risk. The tool will enable interested parties to understand what is at risk and how and why it is at risk. In addition, it will provide an analytical means for comparing risks within one site as well as across sites within the DOE complex. Finally, it will bring the results of research by members of CRESP to the attention of stakeholders, Tribes, DOE and other interested parties and will provide a forum for discussion of important issues in the area of risk evaluation and hazard reduction.--- CREAT mission statement produced by the team, 1996
It was decided to open a Research Web on the topic of environmental impact and the remediation of hexavalent chromium contamination. The mission statement of CREAT's RW was, "Initially, the focus of this research web is to examine one specific hazard: hexavalent chromium found in the 100 Area [near the plutonium production reactors] along the Columbia River on the Hanford Site. The principal risk posed by this hazard is as a stressor to the salmon stocks that spawn in the Hanford Reach of the Columbia River. There are some risks to human health, and these aspects will be investigated as well. The research web will work with absolute risks not relative risks. By using this approach we hope to develop a conceptual and informational framework that can be applied to other hazards."
5.2.4.2.2 Organization
CREAT was always an ad hoc voluntary organization. It was never
funded on any budget, but was a management-approved activity. Its
existence was justified by two possibilities: that it might produce
publishable research; and that it would produce explanations, "fact
sheets," of detailed issues surrounding some of the health
hazard problems from the contamination at the Hanford site.
The nominal leader of CREAT was a member of the CRESP administration and the administrative leader was a scholar funded as a member of a CRESP task group. There were six contributors of Research Web Essays, and twelve team members contributed annotations. An additional four team members participated in face-to-face meetings, but not in the RW. The team operated as a collaboration, with all documentsopen to annotation from all members.
5.2.4.2.3 Focus
CREAT emerged as a focus group within the Health Hazards
Identification Group of CRESP. The group was founded in September of
1996 as the Health Hazard Identification Focus Group (HIF). Briefly
this group was identified by its nature and methods as the
"inter-disciplinary collaborative risk evaluation and analysis
group" (ICREA). The group took the name CREAT on October 22,
1996.
The founding challenge for the CREAT group was to produce short essays on ten questions on hexavalent chromium contamination. Three members of the ICREA GROUP proposed the questions. At my urging, the CREAT group enthusiastically agreed to participate in a Research Web. The ten questions were placed on the RW's web site on December 6, 1996 and were refined by members of CREAT and other members of CRESP. The ten questions were then answered by short essays by team members, and were put into the RW web site for viewing and for critical annotation using DocReview.
The ten questions were:
What is Chromium?
How did Chromium become a contaminant?
Why is Chromium a "contaminant of concern"?
Where is the contamination and how much is there?
How do the levels of Chromium compare with regulatory standards?
How is the Chromium concentration measured?
What is the quality of the Chromium data?
How have Chromium concentrations changed over time?
What is presently being done to mitigate or control the Chromium hazard?
Have new methods of controlling the Chromium hazard been suggested?
In addition to essays on the ten questions, seven other essays were contributed to the site. One team member created four ecologically oriented essays dealing with bioremediation of hexavalent chromium and the effects of hexavalent chromium on salmon, invertebrates, and aquatic plants. I introduced three additional documents in order to introduce some measure of process modeling into the site. These documents were essays on the Environmental Remediation Disposal Facility (ERDF), the pump-and-treat decontamination processes practiced at Hanford, and a process model of the chromium contamination processes ( See Figure XII below). The process model was designed as an organizing model to tie together several of the other essays.
CREAT's work was extended to a parallel study of tritium contamination, but this work never reached the Research Web stage. The efforts of the CREAT team members were directed by management toward a grander project called the Risk Information Tool (RIT) that slowly became moribund due to a number of issues: lack of resources, competition between CRESP management units for control of public information, lack of participation from most CRESP task groups, and a diversion of effort due to a management mandate for concentration on scholarly publications. Unfortunately for the Research Web, RIT was designed for a different audience than the chromium RW, so there was an incompatible conflict in goals. While RWs could provide information to the RIT, it was RIT that had a wider audience, and because of that, RIT received the attention of CREAT.
The Chromium VI RW was generally considered to be a good idea, but was destroyed by incorporation into RIT, an enterprise that languished. Its life, from inception to inactivity, was about ten months.
5.2.4.2.4 Case Study Data
Meeting minutes were recorded and archived, most in DocReview.
In addition to CREAT minutes, minutes of the Data Characterization,
Analysis and Statistics (DCAS) task group of CRESP recorded some CREAT
activity, since several DCAS members were CREAT members. Personal
e-mail archives also contain discussions of issues relating to CREAT.
5.2.4.2.5 The Web Site
There were seventeen essays on the site, the ten questions and seven
others. Members contributed four essays on ecological topics, and
the facilitator contributed three more to provide an organizing model (see
Figure XII below) for the RW.
5.2.4.2.6
Conscription Devices
The seventeen essays were all DocReviewed and most received
annotations. The process model was not image mapped, and because of
that only general comments about the process could be made. There
was no Annotated HyperBibliography or Annotated HyperGlossary, as that
software was not yet in existence.
5.2.4.2.7 Participation Profile
Most members of CREAT, especially the content providers, were well
engaged and participated in DocReviews. Several members of the team
were frank advocates. Requests and reminders to review documents
were issued freely, but consciously avoided importunity. Four
members of the group never participated online, but did attend
meetings. One of these was openly resistant to the Internet, and
found hours of time to attend meetings but never the minutes to
participate online. Participation by the team would likely have
continued, but a motivating stream of new content (conscription devices)
never developed.
5.2.4.3 Soil Crusts RW
The Microbiotic Soil Crusts RW was begun in late 1996 with the
intention of creating a Research Web specializing in the study of crusts
of lichen, mosses and cyanobacteria that form on some soil surfaces of
semi-arid lands. The topic was a natural extension of the scientific
coordinator's research interests, and would also provide a collaborative
environment for the study and cataloging of soil crusts on the
Hanford Nuclear Reservation in Washington State.
5.2.4.3.1 Mission and Topic
This Research Web was founded to coordinate the efforts of eight
researchers that were studying microbiotic soil crusts in the northern
steppe ecotone. The work that brought them together was a survey of soil
crust lichens on the Hanford Nuclear Reservation. There was a need
to obtain this data in order to determine the feasibility of using these
micro-communities for indicators to measure disturbance of the ecological
habitat on the Reservation.
5.2.4.3.2 Organization
This enterprise was funded indirectly by CRESP and Pacific Northwest
National Laboratory (PNNL), and by a small grant from The Nature
Conservancy. CRESP and PNNL have ongoing research interests at
Hanford. The Nature Conservancy was a contributor and interested
party by virtue of its long research association with the
Fitzner-Eberhardt Arid Lands Ecology (ALE) Reserve. Team members
were from the University of Washington (2), Washington State University,
The Nature Conservancy, and Pacific Northwest National Laboratory (2).
5.2.4.3.3 Focus
The RW was begun in late 1996 on the heels of the successes of the
CREAT Research Web. The scientific coordinator was one of the
energetic collaborators in the CREAT Chromium RW. There were high
hopes for this RW as it included a set of collaborators with an authoring
project, funding, an enthusiastic scientific coordinator interested in
long-term research, and an eager facilitator.
A paper was to be produced by the team as the result of field work performed by seven members of the team. The principal author was somewhat reluctant to share early drafts of the manuscript with the team at large, even by hardcopy; but eventually a draft of the paper was DocReviewed on the RW. Another paper was planned, on field sampling techniques, by the scientific coordinator, but never reached first draft stage.
The Research Web slowly became moribund in 1998 as its only effective conscription object, the scholarly paper (Link et.al 1999), neared completion. The scientific coordinator and facilitator attempted to keep the RW alive in hopes of attracting interest among the very small and specialized soil crust community. Closing the server computer in 2001 terminated all activity.
5.2.4.3.4 Case Study Data
The complete web site and 344 e-mail messages related to the RW have
been archived.
5.2.4.3.5 The Web Site
The web site contained a full complement of tools: a home page,
essays, an interactive page to discuss research interests, an interactive
page designed to discuss plans for Autumn 1997 field work, Annotated
HyperBibliography with author and title indices, species list, a Lexicon
installation, Annotated HyperGlossary, a photo album, indices to
DocReviews (both active and archived), both public and team partitions,
and an
authoring team. Minutes of
team meetings were mounted on the web site and were DocReviewed.
The facilitator, with the help of the scientific coordinator, planned and prototyped a new tool, the Species List. This software was designed not only for cataloging each of the lichen species found, but also to provide an online annotation capability so specialists could insert "micro-essays" on a species. These "micro-essays" called Specialist Views in the species list were designed to allow the appending of interdisciplinary knowledge to the species list. For instance, specialists in range management or fire ecology might have notes on species important to their work. This tool was an example of creation of a new tool suited to the distinctive needs of the team's research.
5.2.4.3.6
Conscription Devices
Conscription devices installed to draw out the knowledge of the team
members included a Lexicon with many entries intended to assemble entries
for the Annotated HyperGlossary, an Annotated HyperBibliography with 78
entries, two essays, a species list prototype, 7 DocReviews including a
draft of a professional paper, and an interactive research interest
page. There was no organizing model presented.
5.2.4.3.7 Participation Profile
The scientific coordinator promoted the RW actively. The
facilitator worked closely with the
scientific coordinator, but engaged with the team only at the request of
members, making no independent contacts. The collaborators were well
distributed among four organizations in Washington State. The
distance from Seattle-Tacoma to Richland was such that face-to-face
meetings would require considerable travel time by road or air. At
best, a meeting would take one very long day of travel. It was
assumed that this distribution would encourage on-line participation.
Two invited members never participated in the RW, and one of those, a senior researcher who would have been of great value to the team, could not be convinced of the utility of the WWW, and absolutely refused to interact with materials not in hardcopy. The attitude of this person to the technology was so extreme that they were stereotypical. Attempts to recruit new members were made in 1997, but little interest beyond polite replies was encountered. Participation by members other than the scientific coordinator ceased after the first draft of the paper was DocReviewed.
The RW was presented at the 1998 convention of the Society for Ecological Restoration ("Can soil crusts act as indicators of the biological condition of the shrub-steppe? Using the world wide web to foster scientific collaboration."). It was received with interest and a few people expressed a desire to participate in such an enterprise, but nothing developed.
5.2.4.4 Earthquake Disaster Mitigation
In the Autumn of 1998 work was begun on a RW to support the US-Japan
Cooperative Research on Urban Earthquake Disaster Mitigation. This
project was based on the experiences of researchers with two major
earthquakes in the early 1990's: the Northridge Earthquake in Southern
California and the Great Hanshin Earthquake in KobeJapan. Damage to
regional transportation infrastuctures was profound in both these
disasters.
5.2.4.4.1 Mission and Topic
This WWW site supports an interdisciplinary team of scholars studying the impact of catastrophic earthquakes on urban transportation systems. This team is distributed around the Pacific Rim in Japan and the United States. The team's goals are to develop both a broad synthesis of the impact of large earthquakes on transportation systems and several more specialized studies. The knowledge produced by the specialized studies is expected to support the broad synthesis; and the synthesis is expected to illuminate the specialized studies and to produce new insights and hypotheses.--- from "Site Design and Research Support" web page
5.2.4.4.2 Organization
The team was composed of twelve scholars well dispersed on the Pacific
Rim: one in Seattle, five in Los Angeles, and six in Japan. The
Japanese scholars were all affiliated with the Disaster Prevention
Research Institute (DPRI) of Kyoto University. The American scholars
were not organized into a formal association. The scientific
coordinator was a colleague of the author at the University of
Washington.
5.2.4.4.3 Process
My colleagues were always on the lookout for research enterprises that
might become Research Webs. Tim Nyerges, my committee chair, told
Stephanie Chang about the concept and how it had worked in practice.
She contacted me in September of 1998, and the decision to go ahead with a
Research Web was made in the following month. At a team meeting in
December, Dr. Chang presented the Research Web concept and it was accepted
with enthusiasm.
Content in the form of meeting minutes, CVs, and professional papers were added over the year of 1999, but participation was minimal. In March of 2000, a public partition was added, but that did not spur any further participation. At the time of writing, the RW is moribund.
5.2.4.4.4 Case Study Data
The complete web site is available and 163 e-mail messages related to
the RW have been archived.
5.2.4.4.5 The Web Site
There were 40 web pages in the RW web site. The site was
partitioned into two partitions, a public partition with 14 pages and a
passworded team partition with 26 pages plus DocReviews.
In the team partition, the web site contained several infrastructural pages: a team page, listing all members and their affiliations, with links to the five members who have home pages; a Mail Room page that allowed the user to send e-mail to any team member; a What's New page that allowed the user to obtain a list of activity in any or all DocReviews; a meeting schedule page; a page describing the web site and the support available for the researchers; a page with links to DocReviewed professional papers; an index to all DocReviews; a page providing links to transportation system data in Seattle, the region of interest to the scientific coordinator; and links to project archives.
5.2.4.4.6
Conscription Devices
Two published papers were mounted on the site in DocReview format so
the team members could annotate the documents. No annotations were
made. Eleven documents were DocReviewed and collectively gathered
only seven annotations, all by the scientific coordinator and one
colleague. There was no organizing model, Annotated HyperBibliography or
Annotated HyperGlossary developed for this RW.
5.2.4.4.7 Participation Profile
Of the twelve researchers, only two participated in the Research
Web. The scientific coordinator promoted the RW at appropriate
times, avoiding overt advocacy. The facilitator engaged only at the
request of members, making no independent contacts.
There was a lack of conscription devices to attract participation. Working papers were not shared except within the authoring teams. Two papers were presented for DocReview, not as working papers, but as finished or submitted documents. One computer model was produced, Walter Svekla's master's thesis, but was not incorporated into the Research Web.
5.2.5 Case Study Analysis
In this section, the data gathered from experiences with the four
Research Webs are analyzed. Several research questions are
introduced and interpreted, and the propositions arising from the research
questions are discussed.
5.2.5.1 Research Question 1:
What was the focus (issue domain) of the RW?
Framework for Analysis:
The issue domain of the RW can be sharply focused or very
diffuse. Even a sharply focused issue domain may be, however, far
too large in scope for a RW team. An expansive issue domain may be
so sparsely populated by content that the units are not
interdependent. Independent units will not encourage mutual
collaboration by their authoring teams. In other words, they will
not be contributing to a unified whole. There is also the
possibility that the scope of the RW is too narrow for a long-term
collaborative enterprise. Such is the case where the collaboration
is focused on a single scholarly paper with no interest in either
enlarging the scope of the RW to include closely related topics, or
elaborating the details of the objects or processes of the RW; in other
words, by building neither a supermodel nor submodels.
Propositions:
I) A diffuse focus for the RW will likely result in little participation.
An organizing model that provides a central point of interest and a set of potential work objects. The function of work objects is not only to produce tangible results and publishable documents, but also to conscript the members of the team into active participation. The principal conscription devices are models of the issue domain. Other conscription devices, such as RW essays, bibliographic information, glossaries, discussion forums, and document reviews, encourage participation; but such participation is generally not central to the research but rather to discrete initiatives within the research effort.
Focus and scope are closely related. Focus is the issue domain at the core of the RW. Scope is the outer boundary of reasonable extensibility of the RW. A narrow focus is necessary to provide interdependency of the topics of the issue domain. If the authoring teams address unrelated topics, then it is likely that the research team will self-segregate into subteams. The volume of communications between the subteams will be minimal. Indeed, the division of effort may cause the subteams to drop below critical mass. An overly narrow scope will not provide sufficient research opportunities for a research team that is large enough to provide critical mass.
Chromium VI Findings
The issue domain of the RW was the universe of causes, processes,
damage, hazards, and remediation efforts associated with Chromium VI
chemical contamination from the plutonium production reactors along the
Columbia River in the Hanford Nuclear Reservation. A process flow model
provided the organizing model. This model related many of the physical
objects and processes defining the behavior and character of the
contamination. Each element of the organizing model provided ample
scope for expansion of the topic by description and theory building.
Many of these elements provided topics for potential research papers
including the application of geology, natural history, engineering,
ecology, and human health to the issue domain. Indeed, plans for
research papers were begun on bioremediation and the effects of
contamination on salmon spawning; and descriptive pages, such as one that
described the Environmental Restoration Disposal Site, a repository for
contaminated soil, were added to support the model. The life of this RW
was not sufficiently long to determine if the organizing model would have
been effective.
This was the first RW and as such all its aspects were not well understood by the participants. The organizing model was not criticized and little attention was given to contribution of essays to flesh it out. What was well understood was that ten questions needed to be answered in short essays. Those ten essays occupied the attention of many of the members. Two members did contribute essays beyond the ten questions. This RW had proper scope, focus and a coherent organizing model. Its ultimate failure was not due to shortcomings in scale or focus, but rather to changes in the mission of the research team.
Migration Findings
The issue domain of the prototype RW is migration by household units
focused on the behavioral processes by which such migration comes about.
This issue domain is far too broad for a RW as proposed in this
dissertation. Its organizing model is quite adequate for the
description of process and ultimately for theory building. Very likely,
even the next lower level of abstraction is such that it will also be of
interest only to scholars with broad synthetic theory building
interests. Only at even lower levels still less abstract will issue
domains be found that are properly focused for Research Webs. In support
of the contention that such very high levels of abstraction preferentially
attract theory building is the observation that three well-established
scholars contributed (Tobler, Davis, Amrhein). Despite a personal
appeal to a graduate seminar on migration, little interest was shown and
no substantive contributions were made.
Soil Crusts Findings
The issue domain of the Soil Crust RW was the nature of the soil
crusts in the northern shrub steppe biome. Soil crusts are mats of
lichens, mosses, and cyanobacteria that form on undisturbed soil surfaces
in semi-arid lands. The scientific coordinator and the facilitator agreed
that the topic was sufficiently specialized to provide a proper scope for
a RW. Given those assumptions, work was started immediately on a
comprehensive bibliography and glossary of terms. There was no
organizing model, though there were several bases for organizing
models. The ecology of soil crusts is characterized by mutualism,
symbiosis and perhaps parasitism. How an ecosystem dominated by only
four classes of organisms: cyanobacteria, lichens (fungus and algae), and
mosses interacts as a general system could serve as an organizing
model. Microclimates and geographical models of the ranges of
species are other potential organizing models.
The project's immediate goal was to provide a forum for the cataloguing of the soil crusts on the Hanford reservation, with the intent to expand the scope of studies in the future. Once the principal work object, a research paper cataloging the lichens, was finished participation dried up. Several conscription devices were available, but did not attract participation from anyone except the scientific coordinator. The fact that this properly scoped RW went moribund is a demonstration of the importance of rewards. When the reward of authorship had been spent, there was no further participation. If there had been an organizing model, a plan for a series of research articles, and scholars to perform the research, the RW might have survived.
Earthquake Disruption Findings
The issue domain of the Earthquake Disruption RW was the impact of
large earthquakes on urban transportation systems. The grant proposal that
described the work clearly indicated that the work to be done was a
reconnaissance of the field designed to support scholars who would
hopefully describe the field and generate hypotheses. The diffuse
goals of the work precluded any organizing model, though the mandated
synthesis of the field will perhaps provide one.
With no organizing model and a working mandate that encouraged independent, though related research, there were few common work objects. Furthermore, all rewards were clearly related to production of research papers. Potentially unifying conscription objects such as glossaries and bibliographies were not initiated. This RW had no research focus, thus little need for research collaboration outside the authoring teams.
Discussion of Findings:
A
research web needs to be properly
scoped to be successful. The scope of the RW is determined in part
by topic and in part by size. The number of members required
to perform the research determines the size or scale of the RW. The
permanent members of the research team must be sufficiently interested in
the entire issue domain to contribute
criticism to all documents. Clearly
there needs to be a critical mass of dedicated researchers; beyond that
there is support staff needed, including at least a scientific coordinator
and a facilitator. The topic, or issue domain, must cover a set of
clearly interrelated elements that are sufficiently specialized. The
specialization should be such that each element is either a good topic for
a single research paper or a topic that can encompass a small family of
very closely related research papers.
Scope was clearly an issue in the lack of success with the Migration RW. It was so vast that only philosophers could work at that level. While the organizing model was very interesting, the elements of that model were not sufficiently specialized to produce either research papers or small families of research papers.
A lack of interrelation is indicated by difficulty in developing an organizing model. The Earthquake Disruption RW showed this difficulty. Likely the work undertaken in the founding grant will act as a research reconnaissance and will synthesize organizing models for the issue domain. The Soil Crust RW also had no organizing model because the RW was organized around a single research paper. Had the team been interested in developing the RW around biological or ecological system models rather than a cataloging of species present in a given area, it may have survived.
Both proper scope and focus were demonstrated in the Chromium VI RW. Lack of success can be attributed to a failure of leadership. While management initially approved the RW, it failed to recruit specialists to contribute research essays (and eventually papers). The RW ended when management redirected the efforts of the research team into a project focused on providing information rather than supporting exploratory research.
II) Confederations (groups with different focuses) under a single RW will fail.
The CRESP project that supported two of these cases was formed as a large-scale collaborative project joining several disciplinary specialty groups with a single support group dedicated to gathering data, providing statistical services and supporting a Geographical Information System. While CRESP was far too large for a single RW, it could easily have provided an umbrella organization to support several loosely interrelated RWs. Instead it devolved into a successful confederation of independent authoring groups that generated many professional papers, but little collaborative work toward its original mandate. Failure to provide an organizing model resulted in pathologies that ended all collaboration. Not only did each disciplinary group stay within their specialty, but also the existence of two cooperating Universities locked into an inferior/superior structure took its toll. The research team was fragmented both by discipline and institution. These same pathologies can destroy RWs.
Chromium VI Findings
The Chromium VI RW had only a single group of collaborators (CREAT),
but their efforts were fragmented in several directions. First, the
members of CREAT were participating under a matrix management agreement,
essentially on part-time loan from several task groups of the CRESP
project. While this regime was interdisciplinary, and thus positive
from a collaborative sense, there was competition in agenda setting.
The research direction could remain interdisciplinary, or could veer off
into a specialty area such as ecology, toxicology or human health
hazards. Specialists could satisfy personal, disciplinary and task
group goals by writing papers that dealt with more narrowly defined
professional issues. And secondly there was a mandate in effect that
had defined one of the CREAT goals as developing a set of "fact
sheets" (the ten questions) for each of several contaminants.
Several of the team members withdrew after these fact sheets were
done. The efforts of CREAT were also split into the study of two
contaminants, chromium VI and tritium.
Soil Crusts Findings
The Soil Crust RW team had only two authoring teams, one large effort
to produce a paper on lichens on the Hanford Reservation, and the other a
solo effort on sampling techniques in field work. The purpose of the
sampling techniques paper was to discuss methodology for field studies of
soil crusts. Unfortunately, the field sampling methods had already
been selected, so there was little interdependency between these efforts.
While there was no competition between the authoring teams, the only
active participation from the Eastern Washington state members was
directed to the lichen paper. Only members from Western Washington
participated in the sampling techniques paper, but also contributed to the
lichen paper.
Earthquake Disruption Findings
This RW had a confederation organization. By mandate, the
research team was encouraged to investigate independently. A set of
investigators in Southern California had collaborated on papers many times
before and intende to continue that team effort with new papers. The
Seattle investigator was isolated from the California team, though she had
worked with them in the past. She produced a paper that was
geographically focused on her locality. The Japanese team was
unified by membership in (DRPI), but was split into small authoring teams
of scholars in close proximity.
Discussion of Findings:
If the RW is organized as a confederation of authoring teams with
little interdependency of topics, then the authoring teams will be
naturally isolated. This isolation is a product of attentional
economics: there is no reward in paying attention to work not related to
your own efforts, and your attention will naturally be given to the paper
your team is producing. This isolation can be exacerbated by geographic
concentration, as the relative lack of communication barriers favors
working closely with neighbors, especially people one has worked with
before.
The Earthquake Disruption RW exhibited several isolating tendencies: geographic clustering, existence of previously existing authoring teams, little interdependency between topics, and perhaps language preferences. The mandate from the granting agency specified that independent research be pursued by geographically dispersed authoring teams. None of these factors contributes positively to a RW collaboration. Faced with these difficulties, this RW degenerated from an attempt at collaboration to a file-sharing web site and finally simply vanished.
This finding demonstrates the importance of an organizing model that shows how objects and processes relate to the topic of each research paper. A well-defined issue domain will support an organizing model that will show how (or if) the constituent topics are related. If there is no mandate for synthesis and collaboration backed by effective leadership, then authoring teams will tend to isolate themselves. The academic reward system is such that research papers are essentially the only professional reward, so the leadership of the RW must provide incentives for participation beyond the writing of research papers.
5.2.5.2 Research Question 2:
What were the geographic distribution effects on the RW?
Framework for Analysis:
Geographic distribution effects include not only the physical dispersion of the team, but the existence of socially bound clusters of members and isolated members of otherwise concentrated teams. Another isolating geographic influence is native language: people do prefer to work in their native tongue rather than in other languages, especially since a research paper can always be translated as a single stand-alone document. Other geographic effects include the temporal dispersion of the team: for vast distances may separate instance team members, but still be in the same or nearby time zones. Separation by several time zones makes synchronous communication problematical.
Propositions:
I) RWs with strong concentrations of people who can easily
communicate in person will fail.
This proposition was suggested by the media competition theory. This theory suggests that the most accessible communication modes will be preferred to those requiring more effort. Thus synchronous communication, especially face-to-face communication, will be preferred to asynchronous communication. This preference will naturally lead to a tendency for interacting with colleagues close at hand, socially and intellectually isolating remotely located colleagues (schraefel et.al. 2000, §2.2). Furthermore, most face-to-face and telephone communication goes unrecorded and hence unavailable to members both remote and local.
Chromium VI Findings
The Chromium VI RW team was all located in the same city. Most
of the team members were not, however located in the same office
suites. The team members met frequently, and communicated by phone
and informally. No indication of ill effects due to proximity was
noted. There were no isolated members, since all members were drawn
from a pool of people who were working on a single large grant project
(CRESP). Participation in DocReviews of essays and minutes was active and
successful. E-mail apparently was preferred to telephoning due to
competing schedule demands. Messages were frequently shared by forwarding
and multiple addressing.
Soil Crusts Findings
This RW had two members who worked at the same laboratory, and were
near the lead author of an authoring team. These people were also
close to the location of the field study that was the basis for the
research paper. The scientific coordinator was isolated from these
people, and could meet with them infrequently or individually by
long-distance telephone. The
scientific coordinator did actively participate in
biannual fieldwork with that local group. There was a noticeable
social strain in this RW, perhaps due in part to the communication
problems.
Earthquake Disruption Findings
The Earthquake Disruption RW had three centers of activity with teams
in Japan and Southern California and an isolated member, the scientific
coordinator, in Seattle. Within the Japanese team there were four
members from Kyoto, and two from other Universities. Four of the
five members from Southern California were from the University of Southern
California (USC). The USC members formed a particularly tight group,
having authored over thirty research papers jointly in some combination or
other. The RW team then consisted of two four-strong centers and four
isolated members. The extent to which this configuration contributed
to the failure of the RW is unknown.
Discussion of Findings:
In the Earthquake Disruption RW most collaboration was done within the
authoring groups, each isolated in its own geographic region. It is
expected that authoring teams will concentrate their attentions on
documents of their own. Leadership and training materials will have
to remind them that all members have a responsibility to contribute to the
refinement of content contributed by others.
The Soil Crust RW had a similar lack of conscription devices, though the scientific coordinator did attempt to draw members into collaboration by several weak conscription devices: an Annotated HyperBibliography, a Lexicon designed to build the Annotated HyperGlossary, and an attempt to build a Species List. The only effective conscription device was a research paper that was DocReviewed in an advanced draft. Team members expressed an interest in collaboration on the WWW, but few work objects were offered.
In order to detect problems of this nature in any RW, both isolated members and collocated groups will have to exist in the RW. This condition was not present in the Chromium VI RW. Any conclusions drawn on the limited experiences herein are conjectural and will need to be investigated as a natural experiment in the future when a RW with a widely distributed team with clusters emerges.
II) RWs with widely distributed members who live in different time zones are more likely to succeed than RWs with concentrated membership.
People who live in time zones far removed from their collaborators cannot engage in synchronous activities such as teleconferences without disrupting their daily activity cycle. This fact is likely to make a very dispersed team more inclined to accept asynchronous communication, and with that the environment of the RW.
RW Findings
The Soil Crust and Chromium VI RWs were all based within a single time
zone. The Earthquake Disruption RW was a transpacific enterprise that had
teams separated by 8 time zones. Recognition of this fact was
perhaps a factor that caused the team to accept the concept of the
RW. Unfortunately, the lack of conscription devices made
participation in the RW rather pointless.
Discussion of Findings:
There is little empirical evidence to support or reject this
proposition. If there is an effect, it is likely to be weak when
compared to strong influences like the presence of strong leadership,
member commitment, and an abundance of conscription devices.
5.2.5.3 Research Question 3:
How many people were invited to participate in the Research Web?
Framework for Analysis:
There are very fuzzy upper and lower limits to the size of effective
research teams. Team size is effectively left to chance when the RW
concept is adapted to existing teams rather than building the team to suit
the concept for application to a specific issue domain. Here we are left
with the problem of comparing the teams that were assigned by circumstance
to the teams that might have been designed for the RW.
Propositions:
I) The critical mass theory holds for research webs.
Critical mass is a function of the size and organization of the research team. There are upper and lower limits to the size of an effective RW team. The upper limit is reached when there are so many scholars studying of the issue domain that scholarship is exhausted, the field becomes known territory. The lower limit, critical mass, is reached by having enough active and interdependent conscription devices to hold the interests of the entire team. In a RW, critical mass is necessary to insure a reliable flow of new content, essays, e-mail, annotation, and research paper drafts. The presence of a large body of content open to annotation (conscription devices such as models, essays, bibliographies, and glossaries, etc.) is a good base, but new content is necessary to prevent the collaboration from going stale.
Chromium VI Findings
The Chromium VI RW directly invited 18 people to participate and made
the
URL for the RW available to the entire CRESP
team, perhaps forty people. This RW failed to recruit a full range of
scholars in fields that could contribute to the understanding of the issue
domain. Remediation is an important part of the study of
environmental contamination. Even though there was a well-funded
contingent of environmental engineering scholars available in a CRESP task
group, they did not respond to requests to join the effort; leadership was
not able to persuade them to do so.
This team was very close to having critical mass. The research team was well represented in several appropriate disciplines: ecology, human health hazards, geography, statistics, and risk management. This team was comfortable with the RW technology, contributed content when requested to do so, and was not reluctant to criticize content through annotation. While the lack of participation from engineering was damaging, it may have been overcome in time, since several team members were capable of contributing essays on the topic of remediation techniques.
Soil Crusts Findings
The Soil Crust RW invited four members of the research paper authoring
team, two people from a granting agency, two co-workers of the scientific
coordinator, and two outside scholars. Despite a thorough briefing about
the concept of Research Webs from the scientific coordinator, this small
team seldom showed any inclination to participate in activities other than
the authoring of a single paper. The lack of interest in any
systemic studies of soil crusts ensured that there would be little
likelihood of attracting new members. There was no
interdisciplinary work and thus the group was not likely to attract new
members. Critical mass was not approached.
This RW was started with the understanding that a recruiting effort would be required to attract critical mass. These new recruits would hopefully be drawn from scholars interested in systematic studies that would complement the existing team's interest in taxonomy and local inventories. The scientific coordinator attempted to attract these scholars but had no success, though a few scholars expressed interest. Perhaps organizing models need to be present in order for new members to understand how their work can benefit from the RW. With no funding or colleagues with plans to produce papers, recruiting was a rather hopeless task.
Earthquake Disruption Findings
The Earthquake Disruption RW invited all twelve scholars funded by the
founding research grant. Critical mass was not reached in this RW for
several reasons, principally a failure of adoption of the RW as a medium
of communication. The failure of adoption was in turn triggered by a
lack of conscription devices. In other words, contributions were not
solicited effectively.
The character of this RW was such that interactivity was subordinated to independent research. Lack of interdependence leads to a paucity of reciprocal communication. It is difficult to see how this RW on similar, if not competing research.
Discussion of
Findings:
Critical mass implies the presence of not only a sufficient number of
participating scholars, but also an organization that will support
collaboration. The organizational character of the RW must create a
generous number of interdependent interests
(Markus 1987, 503), and the conveners must have set out the terms of
team membership in order to reduce the possibility of free riding and
non-participation
(Marwell and Oliver 1993, 36).
The scope of the issue domain is the key to developing interdependent units that can engage the interests of the team members. The Soil Crust RW had a scope that was perhaps proper, but did not attract enough members. There was no organizational model set up to outline potential interdependent research units within the issue domain. Without a model there was no basis to attract specialists. The Earthquake Disruption RW had no stated organizing model, and the research units were not interdependent, but rather similar. The Chromium VI Research Web had an organizing model, several interdependent research units and, very likely, enough scholars to engage those research topics. On this basis, it seems that the only RW that approached critical mass, and thus a chance to succeed, was the Chromium VI RW. It was the largest RW, and still too small.
5.2.5.4 Research Question 4:
What incentive(s) did each of the participants have to participate?
Framework for Analysis:
This research question goes straight to the heart of the reward
system. For lead authors and the members of their authoring teams,
clearly the incentive is to have research published. This is the
well-understood academic reward system in operation. For the
critics, there is the reward of showing one's peers that you do understand
the issue and can contribute. Criticism is the personal expression
of the mandated skepticism of science. It is also obvious that
critics can be invited to participate in authoring teams if their
observations are acute and well expressed.
Conveners and scientific coordinators are likely to equate success of the RW to administrative as well as scientific accomplishments. The rewards at this level are likely to be career related milestones that transcend authorship, though their close involvement assure that authorship is almost automatic. Career milestones include being selected as Principal Investigator, awarded an endowed chair, and perhaps leadership positions such as Laboratory Director.
Collaborators, those that make modest contributions to infrastructure such as model building, glossaries and bibliographies, as well as criticism, can aspire to eventual inclusion in authoring teams. Graduate students and staff assistants can earn their bread in collaboration, and can also be rewarded for their efforts by acknowledged contribution in research, a mechanism called legitimate peripheral participation (Lave and Wenger, 1991). Facilitators may be staff or collaborators. Their incentives may be process related, contributing to the collaboration process; and/or topic related, contributing to building knowledge in the issue domain.
Propositions:
I) In order to be successful the RW must provide rewards beyond authorship.
There is considerable overhead in a RW. The costs of the knowledge-building efforts directed to the understanding of the entire issue domain must be borne by the team. Researchers focused on the writing of a single research report cannot justify these costs. There must be some additional rewards to encourage the team to invest in model-building and collaborative criticism.
Chromium VI Findings
The Chromium VI RW though short-lived, exhibited the presence of
authoring incentives. The seventeen essays provided the authors the
opportunity to exercise scholarship with essays that could have provided a
bioremediation of hexavalent chromium, and on the measurement of chromium
contamination could have been extended into research papers.
The need to develop some rather simple research essays provided opportunities for participation without great effort. The facilitator, a graduate student, was highly motivated by the expression of interest among team members. Several graduate students and to contribute, but also showing interest.
Soil Crusts Findings
The existence of a research paper as a conscription device certainly
rewarded all the scholars on the authoring team. The scientific
coordinator saw the possibility of creating not only a successful RW with
multiple products, but also a career-enhancing position as host of a site
that might attract new members throughout the discipline. Her
research interests, soil ecology, included soil crusts. That the
soil crusts were being studied as funded research associated with her
position made the fit perfect. One of the stakeholders, a grant
provider, was an active participant in some DocReviews. This
participation was clearly offered in a collaborative spirit since he was
essentially office-bound by his position.
The RW provided the opportunity for two graduates student to join the research process. One of the members was a doctoral candidate studying lichens, her dissertation topic. The facilitator not only practiced the running of the RW's site and assisting the scientific coordinator in her efforts to make the effort a success, but was also carried as a co-presenter of a paper at a conference. The Soil Crust RW provided an opportunity for the facilitator to polish some of the tools and to produce a new tool specially suited to the issue domain.
Experience with the Chromium VI RW caused one of the contributors to start the Soil Crust RW, perhaps as a career-enhancing strategy. Recently, in a career move, this person accepted an environmental consulting position. She attributes her experience with the RWs as a major factor in obtaining the position.
Earthquake Disruption Findings
rs>The Earthquake Disruption RW offered few rewards other than those
offered by the founding grant. The mounting of research papers in
the public partition of the web site gave the papers wider circulation.
Discussion of Findings:
Rewards emanating from the RWs were few. Other than authorship on one
research paper there were no obvious rewards save the pleasure of active
participation in a collaborative enterprise. Exposure to computer-aided
collaboration has been reported to be positively viewed by potential
employers. There was not a great deal of effort put into development of
rewards, most effort was directed toward development of the web site and
intellectual content. The short lives of the RWs did not allow
development of any management philosophy.
5.2.6 Conclusions
These case studies provide explanations for failures, examples of
successes, and suggestions for correcting pathologies and capitalizing on
successes. In most cases the RW concept was applied to preexisting teams
with either inappropriate or ill-defined issue domains. The establishment
of goals should precede the determination of the research team’s
composition. In a RW, the principal goal of a research team is always the
understanding of an issue domain. Secondary goals, such as publications,
will be produced as a byproduct of the search for understanding.
There are two major problems in defining the issue domain: defining a scope that is large enough to develop critical mass, and defining a scope that is small enough to ensure that the majority of authoring topics will be interdependent. The prototype, a study of migration behavior, had a scope that was far too large. The Earthquake Disruption RW had a scope that was probably appropriate, but the team members were set on parallel tracks rather than interdependent tracks. The Soil Crust RW had an issue domain but never developed an organizing model. If one had to express an issue domain, the only statement would be: anything about soil crusts, but especially those factors that contribute to our work object, a single research paper. Once the research paper reached an advanced draft, the team had no remaining goals. The Chromium VI RW had a well-defined issue domain and proper scope. It failed due to withdrawal of management support.
Critical mass was approached only in the Chromium VI RW. The issue domain was defined with sufficient accuracy to determine where the team needed to be supplemented. Had the RW lasted a few more weeks, the needed researchers would likely have been recruited. The other RWs had small teams, but suffered from pathologies in addition to simple lack of critical mass. Based on these studies and suggestions from the literature, it seems likely that the critical mass for a RW may be as much as a couple dozen researchers.
Proposition 1: There are only two stable states of interactive
medium usage in a community: all or nothing. Either usage will
spread to all members of a community (universal access will be achieved)
or no one will use the medium (for communications internal to the
community), either because no one started using it or because usage fell
off in the absence of reciprocity.
In her discussion of this proposition, Markus depended on a small set of natural experiments, since it was difficult to find documented evidence of participation in interactive communities at that pre-WWW time. Since then the Internet has provided ample evidence in the form of listservers and discussion forums. In our case studies, none of the communities reached a positive participative equilibrium, providing evidence that if there is a threshold (critical mass), then groups of less than two dozen or so are below it. I find no difficulty in visualizing such a threshold since people very quickly abandon an enterprise when it is failing to thrive.
Participation is a necessary attribute for success. Only one RW had adequate participation: the Chromium VI RW. Why? This team was socially integrated; most people knew each other from team meetings on several levels. The team was technologically well served and were adept users. There was an abundant supply of work object, opportunities to participate. There were 43 web pages on the web site, and most of those were available for annotation. Several members were authors of RW essays. It appears that both critical mass and frequent introduction of new content are necessary to generate adequate participation.
Geographical and time zone distribution effects were not seen in these cases. Some obvious problems centering on team dynamics dealing with cliques and isolated members remain to be investigated. Distribution did not appear to alter the technology requirements of the teams investigated.
Financial support appears to contribute to success. In a world full of interests competing for attention, money provides a simple metric for selection. Members of the successful Chromium VI RW were all supported to some degree by grant money from the CRESP project. CRESP personnel in the Soil Crust RW were weakly supported as a "management approved activity,” but some members had to scramble for support or approval. All members of the Earthquake Disruption RW were supported by grants. The unsuccessful prototype RW was a purely volunteer effort.
Leadership is another quality that must be present. Though all scientific coordinators and the facilitator were enthusiastic, clear and unambiguous continuing support from the team's senior scientists was evident only in the case of the Chromium RW.
Management's role in project failure is sufficient but not necessary.--- Charlie Hendricksen, 1983
5.3 Discussion
A synthetic work like this dissertation must eventually turn to
evaluation of its products. Is the concept of Research Webs likely to
survive a test in the real world? Under what circumstances is the RW
effective and when not? Is the RW compatible with the research culture
and academic institutions? Is the high overhead of the RW justified by
more productivity and/or higher quality?
5.3.1 The Optimal Environment for a Research Web
The fundamental assumption underlying this work is that there is a
great need for methods to support large-scale long-term research. If that
is true, then we can discuss what kinds of research might benefit from the
concept of Research Webs; and what kinds of research will not benefit
(
see §5.3.2). Where does the RW fit into the
existing types of research? The question of “critical mass” has arisen so
often in this work that we need to discuss that issue. What disciplines
are suitable for employment of Research Webs?
The Nature of the RW’s Research
Theory-building research is the logical home for the RW. Its models
are the expression of theory, according to the tripartite models of
realism
(Aronson, Harré and Way 1995)
and conform well to the tripartite research methodology proposed in the
VNS
(Brinberg and McGrath 1985). Problem solving does not involve theory building,
but evaluates proposed solutions based on existing theories, assumptions,
myths, or rules.
Action research can
employ the RW as a theory-building activity that operates in parallel to
the design of action to solve a problem and the evaluation of the
implemented action
(McKay and Marshall 2001) (
see §2.2.4.6.5). More appropriately, it
might be better to look at action research as a technique to be employed
to investigate portions of the issue domain. Actions applied to problems
are field experiments analogous to experimental scenarios submitted to
simulation models.
Size
Small-scale research simply cannot afford the high overhead of the
RW. The RW needs economies of scale to justify modeling, bibliographic
research, glossary building and the construction of an elaborate web
site. This fact effectively eliminates the RW as an organizing method for
solo and small group research.
Extraordinarily expansive issue domains, such as migration and poverty cannot employ the RW as an organization for the entire issue domain. The focus of such issues is simply too broad to be parsed into research tasks that are interdependent. Lack of interdependency reduces the potential for collaboration. Note, however, that such very large issues might contain smaller constituent issue domains that are quite suitable for treatment by the use of RWs. There are signs of this sort of organization in the MacArthur Foundation’s work, where their Research Networks are all tied into on overarching objective to improve the human condition and community development (MacArthur Foundation 2000).
The proper size for a RW’s research team is an open question. In the limited body of experience, it does seem that there are definite lower limits, and that those limits are significantly higher then is usually seen in social science research. The upper limits are likely to be established more by the extent of the issue domain. The scope of the issue domain is established on the high side by the need for maintenance of interdependency, and on the low side by the presence of an adequate body of related and attractive research topics ( see §3.1.1).
Geographical dispersal
The initial reasons for investigating the RW came from the
circumstances that academic social science is beset with. As discussed
above, isolation of specialists caused by economic necessity ensures that
a critical mass of scholars can only be found by reaching beyond local
sources for collaborators (
see §1.1). After the investigation was
entered there were mechanisms found beyond geographical dispersal that
make the RW and attractive organization for research. One of the most
interesting findings was that, with modern technology, because of
scheduling incompatibilities, even colleagues within shouting distance
were likely to be contacted via e-mail. Once the medium of communication
is asynchronous, dispersal becomes a less serious problem.
A corollary of physical dispersal is cultural difference. If the research team becomes intercultural, a number of negative factors come into play: language skills, power structures, and workday asynchrony. The pathologies introduced by all these factors are remedied to some extent in the RW. Written language skills are higher than spoken skills, power structures are blunted, and workday asynchrony becomes a minor inconvenience. The positive virtues of a culturally diverse team include a multiplicity of viewpoints.
One problem that remains uncertain is the effect of media competition. Whenever local groups, even pairs, collaborate, there is a strong tendency to revert to habits of speech and the building local tacit knowledge, thus neglecting the documentation and distribution of new knowledge. This unfortunate property may make such groups less effective collaborators in the RW. This mechanism may also increase the isolation of remotely located individuals.
Discipline
Physical sciences such as molecular biology have embraced general
systems theory and from that have built some quite elaborate models of
metabolism pathways
(Ideker et.al. 2001). The Institute for Systems Biology (ISB) in has
integrated collaboration, modeling and thorough exploration of each
element in the models. While the ISB has the transfer of knowledge to
mankind as its goal, the presence of extremely large potential profits in
industrial research may stifle free exchange of information.
Researchers in the humanities have done some collaborative work that makes good use of the current technological environment (Ruhleder 1995). The issue domains investigated are often very elaborate catalogs of works of art or literature (Eaves 1997), (Landow 2002). The need for theory building in these issue domains seems slight.
The social sciences seem to be a natural home for the RW. Theory building is necessary to account for the behavior of human behavior. The need to accommodate several disciplines simultaneously is a characteristic of large-scale social science research. The social sciences routinely have to view their interests from several points of view and inferential methods. The inexact nature of the objects, processes and measuring techniques makes critical thinking not only likely, but also essential. Criticism is a major process in content development in the RW.
5.3.2 The
Research Web
Compared to Conventional Research Teams
The question of the value of Research Webs must be answered here. To
efforts, both long-term and large-scale. One of the efforts is assumed to
be a conventional research team, perhaps a university institute, the other
a team using the Research Web concept and tools.
Evaluation factors that remain similar for both
teams:
Salaries and support requirements for scholars, administrators,
research assistants, and a copy editor should be identical. The technical
support environment (hardware) should be identical. In the academic
environment WWW servers are part of the university infrastructure.
Software costs are only very slightly higher for the RW, and any higher
cost is miniscule compared to salaries. Travel costs may be higher in an
RW because it is more likely to be dispersed; but, if the conventional
team is dispersed as well, then its travel expenses will likely be greater
than the RW team’s expenses, because the RW has much better communication
channels and therefore less need to travel.
Factors involving personnel cost:
Facilitator
The active presence of a facilitator is absolutely essential.
Every member of the team must know that there is someone there to take on
all the mundane tasks that the content providers are best relieved
of. As Marwell and Oliver say about collective
action
(Marwell and Oliver 1993),
"The most important determinants of collective action in our models
are the interest and resource (or contribution size) levels relative to
the cost of contributing." The duties of the facilitator are
simply to assume the technical workload introduced by the necessity to
support the web site, models, and document formatting. The facilitator
reduces the cost of contribution to the team every time one of the many
gritty technical chores is taken off a contributor's plate.
From a management point of view the facilitator is not only a cost, but also a bottleneck in some aspects of the information flow 1 . In the RW, most communication is not mediated or facilitated, but some tools are designed under the assumption of intermediation. Access to those tools is by password, and should any team member care to operate the software, then the password can be shared. Experience shows, however, that most members are uncomfortable with the need to learn software and also are reluctant to spend time on such tasks.
Modeler
The services of a modeler may be required, probably on a part-time
basis, especially if a simulation model is being produced. While the
researchers will certainly become comfortable with the models from
examples, there are nuances of data modeling that require more specialized
knowledge. For the descriptive, auxiliary, and explanatory models, any
moderately skilled programmer/technician should possess the necessary
skills. The facilitator or a research assistant may assume these duties.
In a mature RW, the services of a more highly trained modeler will be of
great value. A skilled modeler can bring a rigor and intellectual clarity
to the process that will increase the value of the models.
Training for Collaboration
It should not be assumed that members and staff have any experience
with collaboration in large groups. As discussed in the preceding
chapters, the experiences of most researchers are likely to have been more
competitive than cooperative and limited to occasional small-scale shared
writing (
see 2.2.5.4). Training
that introduces the team to the RW concept and the value and costs of
collaboration must be undertaken early in the life of the RW and should be
repeated as new members come on board. The cost of this training must be
balanced against the value of acquired skills. I think it is likely that
a couple of hours of training will be an unquestioned bargain.
Web Site content creation and maintenance
This expense is unique to the RW, although the conventional research
team might have a "brochure" web site to publicize its work. A
brochure site is equivalent to a token public partition (
see §3.5.1) of the RW. The RW's team working
area (
see §3.5.2) and guest
partition (
see §3.5.1) have no
equivalent in the conventional research effort. Many of the RW's software
tools have been built with the intent of reducing the work required to
create and maintain content. Much of the content of the RW's web site is
actually collected and installed automatically by software. The costs and
benefits of this content are discussed below under the "Artifacts
Produced" headings.
Collaborative review of documents
In the Research Web team members are expected to review documents as
the team produces them (
see §2.2.4.6.7). This review process
informs the reader and allows the reader to contribute insights that the
authoring team may have overlooked, especially links to other research,
not to mention the occasional correction of error or the offer of a
reference to support a point. A trained collaborator will come to
understand the value of critical reviews when his or her work comes under
review. In conventional research, the obligation to review the work of
those outside a single authoring team is likely to be minimal.
Artifacts Produced
Models
Models in conventional research are, if present, usually sentential or
tacit and deal with only the topic of a single research paper. A good
model provides a description that is far more complete than the usual
abridged and partially tacit narrative description found in conventional
reports.
"A fairly common strategy in group research is to conduct studies to explore some interesting phenomenon or technique. … Each of these has produced a 'minitheory' of the phenomenon under study, and some of these are ingenuous and useful. However for the most part, these efforts do not attempt to tie into a larger theory and therefore remain isolated findings."--- Poole 1990 (Poole 1990, 239)
In the RW, the "minitheories" are expressed, connected, annotated and thoroughly documented. More importantly, they are strongly connected to other minitheories in the RW’s issue domain. The question that needs to be answered is, "Is the modeling effort worth the expense?" We discuss below each of the several models that exist in a mature RW.
The Descriptive Model
In conventional research, the objects and processes that apply to the
topic of a research paper, usually an isolated phenomenon, are described
by reference and narrative description in the research report. No attempt
is made to describe attributes of the objects that are not essential to
the report. These ignored objects include superclass objects (embedding
objects) and objects that might have an unknown or presumed infinitesimal
effect on the phenomenon.
Process models concern themselves only with elements
essential to the argument of the paper.
In the RW, every object within the issue domain that is mentioned in any document is described in the Descriptive Model ( see §2.1.2). Every attribute of objects that suggest themselves to the members is listed; and attributes found of interest are elaborated in appropriate detail including operationalization methods. Every object's attributes are open to annotation by the research team. Since several authoring teams share the same models, attributes that are of no interest to one team may be essential to another. Furthermore, some attributes may constrain others.
Processes models are descriptions of observed behavior of the objects. Like obscure attributes of objects, processes may have "side effects" that are of little interest to the observers of the phenomenon being investigated for a single paper. In a RW, one team's side effects may be another team’s phenomenon of interest.
The Auxiliary Model
In conventional research, the
auxiliary model is generally restricted to narrative
description of the operationalization of variables used in experiments and
analysis of data. In the RW, all attributes of objects used in
experiments, data analysis, or the simulation model are formally
operationalized. The investigation is recorded by adding an extension to
the
Descriptive Model. The
extension will include measurement criteria for the attribute: data type
of value, precision, range, and a description (
see Table II §3.2.4.2.1). In the auxiliary model
several options for operationalization may be offered. Choosing the
appropriate operationalization is the duty of theoreticians and
experimenters. The auxiliary model will be open to critical annotation by
the team, just as any document in the RW. The operationalization of the
attribute is a directive to the programmer managing the simulation
model.
The Explanatory Model
Conventional research expresses its
explanatory model as a narrative discussion leading to
the development of hypotheses. The hypotheses are the basis for
experimentation or argumentation. In the RW, the explanatory model is
likely to be somewhat more formal, but incorporates all elements of the
explanatory model presented in conventional research. The hypotheses
investigated in conventional research are related primarily to the topic
of the single research report. The great difference is in the scope of
the model. The explanatory model may be abridged for inclusion in
research reports. In the RW, the team contains many authoring teams, each
intimately familiar with the phenomenon under investigation in their own
paper. The RW's explanatory model (
see §2.1.3) will show how each
phenomenon is related to another. Examination of the relationships is
likely to modify the conceptualizations of the phenomena. Certainly
additional hypotheses will be suggested.
The Simulation Model
Simulation models are infrequently produced in conventional research
in social sciences. Their value is unquestioned in the physical and
natural sciences. The reasons for the rarity of simulation in the social
sciences are many and varied, but they center on modeling’s frequent
failure to produce reliable results. The RW can mitigate many of the
likely elements of failure. First, the objects and their
operationalization are better described and thoroughly discussed; second,
the process models of the explanatory model that form the basis for the
dynamics of the simulation are more accurate than the isolated
"minitheories" of conventional research; and finally, the entire
team will be able to "exercise" the model and thus expose
potential problems. An argument could be made that high quality
simulations might open the phenomena of any issue domain to experimental
social science, just as has happened in some domains in economics.
The Context Model
The context model is the most abstract of all the proposed models. It
describes the scope of the issue domain (
see §3.1.1). In conventional research the
contextual model is likely to be expressed only in the charter of the
sponsoring organization. This expression is likely to be carefully
crafted for political and organizational purposes. The political aspects
mentioned are designed to demonstrate the moral grounding of the
sponsoring organization. The description is likely to be purposely vague
in order not to alienate too many sources of grants.
The context model in the RW has definite purposes. Precision in establishment of boundaries is important for two reasons: if the issue domain is too small,then the issue domain may become unproductive -- all potential research topics may be used up; if the domain is too large, then the authoring teams may not be interdependent --- the research team will be fragmented to the extent that critical mass is not maintained. The representation of the context model is 1986)0204231222, (Wellman 1994) and concept mapping. The model must be able to establish not only the necessarily fuzzy boundaries of the issue domain, but also the adjacencies to other issues. The RW's research team will be in a position to contribute knowledge or conjectures about those boundaries and adjacencies.
Intellectual content
In conventional research, intellectual content takes the form of
partial drafts, notes, outlines and other ephemera that eventually lead to
a research paper. This mass of expressed knowledge is usually stored in
the author's files, and is thus inaccessible. After publication of the
research paper, much of this information is purged from the files. Even
if the files were opened to the authoring team, they would not be
searchable, and would thus present the reader with the necessity of wading
through every scrap of information in order to find what is of
interest.
In the RW, intellectual content is contained in the documents in the RW repository ( see §3.2.2). Documents include essays, models, e-mail, drafts of papers, outlines, images, annotations, and others. Essays are a formal genre, the Research Web Essay ( see §3.4), and are accessible to and annotatable by the entire team. They are organized in a hierarchy of web pages, linked hypertextually from graphic models, and subject to full text search; and the essays are annotatable. Notes may be expressed in e-mail messages that are in the team's searchable e-mail archives. Annotations are documents as well and can be found through full text searching as well as through DocReview. Outlines are a fundamental scholarly tool, a model of a document. In the RW outlines cannot only be annotated, but can form the backbone for preliminary drafts built by annotation with DocReview. Models are expressed in graphic and textual documents and are both searchable and annotatable.
Publications
Conventional research is focused on the production of research
reports, often just the smallest publishable unit. Of course the strategy
of least publishable unit could be followed in the RW as well. The RW
views research reports as derivative, not as the sole objective. The RW
can serve not only to produce research reports, but monographs and books
as well. Even criticisms of reports or books in the literature can be
generated by the team, leading to publication as brevia, reviews or
letters to the editor. Such criticism of external materials can be
archived and linked to documents in the RW repository.
It is extremely difficult to argue that the RW's research team would generate fewer publications than a conventional research team. While it is true that the effort expended in model building does not lead directly to publication, it is also true that the shared models may reduce the total expenditure of effort. The marginal cost of refining a model to include information generated in the research of a phenomenon is very small compared to the cost of reproducing the models in each research report investigation. I believe that contribution to models and the study of models will familiarize all the researchers with the issue domain to such an extent that many more hypotheses will be generated. These hypotheses are each a potential research report topic.
The purpose of a model is to rub the researcher’s nose in the problem.
Bibliographies
In conventional research, as well as in the RW, an enormous quantity
of information is gathered from the literature. The lists of references
from research reports are usually the only permanent bibliographies
produced in conventional research, references for literature investigated
but not cited is lost. In the RW, the research team has several
advantages that scholars in the conventional environment do not have.
Possession of a corporate body of literature indexed and searchable by
several methods will save considerable duplication of literature search
effort. In the well-disciplined RW, every piece of literature examined by
the team members can be incorporated into the Annotated HyperBibliography
(
see §4.4). The RW's team has the
ability to read abstracts and full text at a click along with the opinions
of colleagues. If a document is seen to be less than useful then the
member who reported can annotate the entry with a warning to avoid it.
With the assistance of the facilitator, bibliographic entries can be
tagged so special purpose bibliographies can be produced. Hypertext links
can be made to the bibliography from any document in the RW, even e-mail.
The Annotated HyperBibliography can be searched by author, title, keyword,
or text in the abstract.
Glossaries
Glossaries are seldom included in research reports, even though
readers from disciplines outside the discipline of the authors may not be
familiar with the intended specialized meanings of terms. The
appropriation of widely used words for more narrowly defined disciplinary
meanings is both necessary and widespread. In the RW, the team members
have an
Annotated HyperGlossary (
see §4.5) at hand to investigate the language
of the dialog. Should a scholar use a word in a technical sense, that
word can be linked to the proper definition, in interdisciplinary teams
often one of several. Should a RW team member choose to add a gloss or
alternative meaning to the HyperGlossary, the facilitator can do that.
The Annotated HyperGlossary provides the RW team with a forum to discuss
the terms through annotation.
Services Offered
Facilitated document review
Conventional scholarship does not usually have the benefit of shared
criticism; and review is often limited to electronic file swapping of
drafts between the lead author and the authoring team. In the RW, the use
of DocReview (
see §4.3) provides the
entire team the opportunity not only to criticize most documents, but also
to discuss the commentary of others in a focused team-wide forum. Through
the guest partition (
see §3.5.1)
selected scholars outside the team may be invited to review documents that
would benefit from a review by qualified specialists.
Proposal production
The problem of controlling the costs of proposal production is widely
recognized. Indeed the cost of producing a proposal for an Institute-level
grant is very expensive and is cited as a barrier to commissioning such
organizations
(Task Force on Enhancing the Research Environment 2001, 11). Software is becoming available to
manage the paperwork of preparing a proposal, but I am aware of no
services that attempt to provide intellectual guidance to the proposal
team. The mature RW has a very valuable repository of information that is
available to support the preparation of the intellectual content of the
proposal. Since any proposals generated by the research team must be
related to the issue domain, there exists a set of points of contact with
existing research and the models of the issue domain. The proposal can
actually point to the connection of the proposed work to existing work
within the organization as well as to the outside literature.
Technology training
In conventional research teams, any training of team members is
provided by the infrastructure of their work environment. In the RW, the
facilitator is directly tasked to provide any technical training that a
team member may request. The RW is a more technical environment, but most
tools appropriated should be web-based, thus the training required is
minimal --- every page is a web page. Becoming familiar with modeling is
quite another issue. While most representations of models are
straightforward and can easily be used as templates, a real understanding
of models may require some additional training.
Wider distribution of research documents
Conventional research generally publishes only documents that meet the
standard of minimum publishable unit. The RW can publish any document
regardless of its scope. The advantages of wider distribution of
scholarly documents are now universally recognized. The RW distributes
not only the documents themselves, but also links to the participants'
home pages, thus informing others of their research interests. Many of
the RW's documents may be available for WWW distribution from the public
partition (
see §3.5.1). Always
requesting all the commercial search engines to index them aggressively
publicizes those documents. The important research reports may be
presented in several formats: in PDF format, for direct imaging of the
document; in HTML for presenting the document in hyperdocument format,
thus offering the reader the advantages of sidebars, the hyperbibliography
and hyperglossary; and finally, in DocReview format, allowing the wider
world to annotate the document. Many documents will not be published in
journals; and others will be published in journals that may not be
available to many scholars.
Central e-mail repository
In conventional research, e-mail is not universally archived. In the
RW, unless e-mail is private, all email is archived and is searchable.
This brings a new source of intellectual content to the research team; all
that is needed to recall information are keywords or an approximate
date.
5.3.3 The Tools of the Research Web
Finally we dispense with the technology. The Internet and World Wide
Web have provided a home for the application programming that will make
the Research Web possible. There is no argument that can object to the
efficacy of the large body of software that is available to facilitate any
kind of research on any topic. If the software isn’t there, it can be
built. Do the tools proposed for the RW really serve the research team?
The core tools proposed for the RW: DocReview, RW Essays, Annotated
HyperBibliography and Annotated HyperGlossary, have been placed in service
and were found to be both useful and accepted by researchers. While only
DocReview received enough use to collect empirical data and support
research questions (
see §5.1.1), no
negative opinions were received on the other tools (aside from an unusual
total rejection of the technology by two senior researchers). Both the
Annotated HyperBibliography and the RW essays have received praise from
members of the academic community. The real question here is the efficacy
of the Research Web concept.