Laying the Groundwork: UARC Progress 1992-95

UARC began in 1992 with a focus on providing real-time collaborative access to a suite of instruments in Greenland. While we recognized that this was only one aspect of space science, it was a start on a journey to a more comprehensive collaboratory vision. The initial focus of UARC development was on a NeXT -based system that allowed coordinated real- time viewing of data from multiple instruments at the Sondrestrom Upper Atmospheric Research Facility. During the 1993- 96 period this NeXT- based system was used extensively for distributed scientific campaigns by scientists in Europe and North America. We learned important lessons about the functionality and usability of the software, and discovered limits on the scalability and reliability of the architecture when used under real Internet conditions. Among the lessons learned were:

• The heavy traffic on the Internet, the heterogeneity of network capabilities in different regions of the Internet, and the diversity of platforms and local network environments available to individual users means that collaborative systems running over the Internet must be designed in ways that allow continuous operations despite data loss or slow links. Our early client/server architecture using standard communication protocols worked well during the early years of UARC, before network traffic dramatically increased. But this architecture became increasingly problematic during the 95-96 UARC campaign season.
  
  
• The UARC system must scale well as the number of users and the number of sites increased. We found in the early years of UARC that our architecture failed when we had more than about a dozen active users, and that these numbers themselves declined when we had a full complement of five Sondrestrom instruments active.
  
  
• As collaborations increased in the number of users involved and in the complexity of the science attempted, it became clear that there was a need for the software to support different roles among the participants (e.g., campaign organizers, active researchers, observers, developers) and different subclusters of work (e.g., different scientific foci).
  
  
• Screen real estate is a critical issue. Even with only a few instruments at a single site, users quickly fill their screen with multiple views of data from single instruments (a feature to them), windows indicating the status of participants and instruments, and communication windows. Users spent considerable time arranging their screens, but still ran into serious limits. Of course, these pressures increase as more sites and capabilities are added to the software.

In addition, we knew from the outset that expanding UARC beyond our initial small community of users, the single site at Sondrestromfjord, and the real-time data collection phase of science would entail a number of technical challenges, such as:

• Users need to be able to add instrument viewers, analysis tools, and access to archival data on their own, including during the course of a collaborative session, meaning that UARC must provide dynamic configurability and extensibility.
  
  
• Scientists must be able to move in a seamless way between individual work and collaborative work.
  
  
• Both synchronous and asynchronous interactions are important, because the scientists span multiple time zones and need to be able to come and go from collaborative sessions on different schedules.
  
  
• The UARC capabilities must be accessible from as broad a base of platforms as possible.

Enlarging the Vision: UARC Changes in 1996

In early 1996 the UARC team began a re-design effort intended to address the weaknesses of the NeXT-based system described above and to lay the groundwork for inclusion of more complex and widespread use of the collaboratory software. The principle advances were:

• A systematic redesign of the architecture led to specification of a Collaboratory Builder's Environment (CBE), a programmer's toolkit for rapid development of robust, platform-independent collaborative applets. The new architecture was designed from the outset to be scaleable under realistic Internet conditions.
  
  
• A Java applet approach to the building of the CBE was adopted, so that the entire set of UARC capabilities would be accessible via a Web browser. This framework allows easy incorporation of other Web- based tools into the collaboratory.
  
  
• Intensive exploration of the performance issues involved in data transport for a real-time collaboratory over the Internet led to the design and building of Corona, a scaleable distributed data transport mechanism designed to provide reliable user-controlled quality of service for large numbers of users over the Internet. Corona provides a framework for the inclusion of semantically based quality of service policies to end users.
  
  
• A new session manager framework was developed in which users could configure both public and private workspaces so that a variety of scientific goals could be pursued with users free to choose which workspaces they wish to enter.
  
  
• The session manager also included the capability of defining specific roles for various participants, with varying access privileges defined by these roles.
  
  
• The DistView toolkit in Java for building collaborative applications using object replication schemes was developed so that the collaboratory could provide coordinated views as well as consistency of data to various participants.
  
  
• Support for persistence of state of applications in a collaboratory and the state of shared workspaces was developed, so that users can join and leave at different times. Persistence is provided by the Corona multicast servers in a generic way so that all collaboration tools built using the DistView toolkit in CBE automatically get persistence features.
  
  
• Access control and security issues were investigated, and a theoretical framework now exists that can be integrated into the system.
  
  
• New multi-viewer interface metaphors were explored as a first step toward managing the increased complexity of the displays.

An important milestone in the evolution of the CBE/Corona-based UARC was a four day telescience campaign in October, 1996, in which the UARC software for the Sondrestrom facility was integrated with web-based data viewers from several other radars (Millstone Hill, EISCAT, SuperDARN) and a suite of model outputs and general solar and atmospheric data in order to demonstrate a capability for the use of multi-site data sources and theoretical models to conduct interactive studies of a wide range of upper atmospheric phenomena across the northern hemisphere. The UARC software successfully demonstrated the substantial advances in the scalability and reliability of the new architecture through continuous operation over the Internet of applets for real-time viewing of density plots from the Sondrestrom incoherent scatter radar and magnetometer, and UARC Chat, an applet for real-time exchange of text messages. Response from scientists who used the new UARC tools during the October campaign was enthusiastic, and indicated that some new ways of doing a more global kind of space physics were possible. Of course we also learned that there are some critical areas for future development including: a) more reliable applet performance; b) integration of real-time global models with real-time data collection; c) ability to display archived data via applets; d) integration and reduction of data displays to maximize screen real estate and to minimize cognitive demands on users; and e) improved ease-of-use of applets.

Realizing the Vision: UARC Specific Goals for 97-98

The October 1996 campaign and subsequent discussion of the experience at our December 1996 workshop revealed to the space physicists a remarkable opportunity for doing some revolutionary science. By integrating a large variety of data sources and model outputs into a unified UARC framework, campaigns with a much more global reach become possible. With these visions in mind, a series of scientific objectives were discussed in some detail at the December workshop, and the following plan has emerged. On April 8-10 we will conduct a multi-data source, multi-model campaign that focuses on several scientific objectives. We expect scientists from Michigan, SRI, Millstone Hill, Applied Physics Lab, DMI, JPL, SwRI, and other locations to participate, and we have targeted this campaign for a number of substantial improvements in the UARC software (see below). Then, some time in the summer, we will use the UARC software to hold a Collaboratory Workshop that will be an occasion to reflect upon the upper atmospheric phenomena that were observed in April. If we obtain the supplement we would hold another Collaboratory Campaign in October of 1997, to be followed by another Collaboratory Workshop shortly thereafter. We would use the annual UARC workshop in December 1997 to formulate plans for the next round of campaigns and workshops in the spring and summer of 1998.

More specifically, data sources will include the Arecibo incoherent scatter radar, optical systems at Arecibo and Magnetometer data from the Antarctic research community, as well as the full suite of measurements from Sondrestrom, Millstone Hill, EISCAT and SuperDARN. We anticipate that the complete SuperDARN array will be made available through UARC, including the Southern Hemisphere radars. We will also include satellite measurements from the NASA POLAR and WIND spacecraft and possibly the FAST mission as well as "space weather" space-derived information available through the NOAA Space Environment Laboratory and other on- line services. In addition, the system will be configured to accept and display archived information stored in the CEDAR data base, as well as "experiments of opportunity" that are enabled through the initiative of individual investigators. Finally, we will incorporate a suite of theoretical models into the UARC workspace. In addition to the available TING high- resolution general circulation model (based on the NCAR TIGCM), we will investigate use and applicability of various ion convection models (IZMEM, Weimer, APL, Heelis, AMIE, etc.); ionospheric precipitation models (Hardy, etc.); Airglow models (Solomon, Link, TIGCM, etc.); interhemispheric plasma models (FLIP) and other operational ionospheric models (HWM, IRI, PRISM, etc.). Models will be brought into the UARC framework at an appropriate pace, governed by the expressed needs of the collaborating scientists and model availability. In short, a synchronous discussion involving a wide range of real-time and archival data sources along with model outputs can engage a broad community of space physicists in an electronically mediated discussion of global phenomena. Furthermore, through the capability to use these tools to bring scientists together for a retrospective discussion, we can create the Internet equivalent of a CDAW (Coordinated Data Analysis Workshops), something we call the Collaboratory Workshop.

Research Challenges

To accomplish these ambitious scientific goals, substantial development of the UARC CBE/Corona software must occur. The system goals identified through our earlier experience must be augmented with further capabilities that allow such revolutionary new science to be done. First, UARC data gathering and visualization systems must function efficiently. Scientists must be able to concentrate on scientific activity foremost, and this will require improvement of interactive performance by data-visualization applets. Second, UARC should accelerate the trend toward integration of real-time data with real-time models. In the past, much of space science was dominated by interpretation of local phenomena. Now we want to move to more global analyses. Third, UARC should provide the capacity for groups of geographically distributed scientists to return to key upper atmospheric events by replaying archived data. That is, the power of drawing on a worldwide network of data collection stations brings with it the additional burden of coordinating interpretation of multiple complex data streams. Much of this burden can be mitigated through synchronous or asynchronous collaboratory workshops in order to conveniently and retrospectively combine complementary data sources and complementary expertise. Fourth, critical national scientific resources must be easily integrated into UARC. For example, most UARC activity to date has revolved around data from a single incoherent radar site, yet the NSF funds a chain of these observatories throughout the Northern Hemisphere. A key space science objective under supplemental support will be the seamless addition of additional incoherent radar sites, specifically Arecibo and Millstone Hill, through standardized collaboratory interfaces. Finally, the tools for collecting, displaying, and analyzing global data streams must be widely available, easy to operate and install, and easy to integrate with other domain-specific data visualization systems.

Key Space Science Objectives

• Create an Internet-based collaboratory that will allow the scientists to link both theory and data on the understanding of the global properties of upper atmospheric phenomena (Òtheory-data closureÓ).
  
  
• Provide a framework for the incorporation of many real-time data sources from any ground-based or satellite data source world wide. This will be tested through specific scientific campaigns in 1997 and 1998.
  
  
• Provide access to archived data sources stored in the CEDAR data base format.
  
  
• Provide access to a variety of sophisticated modeling outputs, including the capability for easy formatting for purposes of making comparisons between data and theory.
  
  
• Support both synchronous and asynchronous access to the collaboratory environment in a seamless way.
  
  
• Develop extensions of the collaboratory ÒcampaignÓ tools to allow for collaboratory workshops that would focus on scientific problems of interest to a distributed community of users.
  
  
• Provide a suite of access and session controls so users of the system can organize the participants into meaningful groups (critical as the size of the UARC community grows).
  
  
• Ensure that the collaboratory capabilities operate across multiple platforms and easily integrate with both off-the-shelf and custom tools.
  
  
• Ensure reliable performance for multiple collaboratory components, especially in the face of high loading on the Internet.

Key Computer Science Objectives

• Provide a Collaboratory Builders Environment (CBE) toolkit that will enable other communities of scientists to configure and operate collaboratories based on the Web/applet framework.
  
  
• Ensure that CBE components interoperate with emerging Web tools to allow rapid integration of commercial and legacy code into the collaboratory environment.
  
  
• Allow users to join and leave an ongoing collaboration activity, and to participate in multiple collaboration groups simultaneously.
  
  
• Support both synchronous and asynchronous collaboration.
  
  
• Provide semantically driven scaleable quality of service that enables participation by heterogeneous users over wide area networks.
  
  
• Add support for record and replay of collaborative scientific activities.
  
  
• Explore potential synergies in the coordinated development of CBE and NCSAÕs Habenero.
  
  
• Explore new user interface metaphors and data visualization techniques that allow the scientists to manage large amounts of information in a fluid manner.

Key Behavioral Science Objectives

• Establish and apply user-centered, iterative design principles that can be extended to the design of other collaboratory projects.
  
  
• Identify critical success factors for the development and use of collaboratory technology.
  
  
• Document the impact on the practice of space physics of the successive generations of UARC technology.
  
  
• Capture lessons learned from the space science community as they adopt the new collaboratory capabilities provided by UARC.