This is the text we're considering for use in the ISSI proposal. The first version of this doc presents the initial text written by HarveyRhody. This second version is according to our meeting on the same day. -- BobKrzaczek - 22 Jun 2004

---

We propose to base our information and data management on our Data Cycle System, currently under development for NASA's Stratospheric Observatory for Infra-red Astronomy (SOFIA). The DCS supports the flexible adoption of a variety of instruments by general investigators within a science center model. It includes support for observation planning, data collection, data pipelining, and archival. The RIT Laboratory for Imaging Algorithms and Systems is the lead organization in the development of the SOFIA DCS. A prototype system has been demonstrated and delivered, and a full demonstration at the NASA Ames Research Center is scheduled for Fall, 2004.

Not entirely sure about that paragraph, even stripped down a bit like that. I think what I'd really like to say is that the DCS is under development, but that it's a functional working system today. The trouble is, that sounds too close to "not done" from the reader's point of view. Also, is RIT really still the lead organization? Since USRA took over and put Fran in place, we're really just a player like UCLA. Thoughts? -- BobKrzaczek

Central to our system is the notion of an "experiment". Using online tools hosted at the data center, investigators can propose new observations; the tools support both structured information to aid in the management of the proposal itself, as well as free form content including whatever documentation needs the investigator wishes to provide with the proposal (typically PDF files). When approved, these proposals are used to generate initial experiment for the investigators.

The investigators then develop the experiments further, using tools downloaded from the data center. In developing the experiments, the investigators specify instrument and sensor combinations which will gather raw data. The tools are smart with respect to these instruments and sensors, and assist the investigator in the specification of observations that have a high probability of success. This is achieved by including rules provided by the instrument designers in the observation tool, encouraging the user to only use parameters and settings that are known to work correctly for a given instrument.

Along with data acquisition, the DCS also allows the investigator to specify the automated reduction of new raw data when collected. Rather than the more traditional approach that requires data center staff to manually reduce new data for dissemination to the investigators, the DCS supports the automatic creation of data products as soon as the raw data is acquired at the data center.

Of course, data reduction is not limited to the automated execution of the pipelines specified when the experiments are created. The investigator is also free to visit the archive, which contains all previously captured data. Selecting groups of data indexed by a variety of methods, the investigator can then specify new reductions to be performed on this old data, exploring new relationships between previously executed experiments. All data reductions, it should be noted, are performed at the data center; the investigator needs no special processing equipment in order to take full advantage of the observatory's resources.

The underlying structure of the DCS is modular; new algorithms, data pipelines, even whole new models of investigator interaction with the DCS can be added without disturbing the rest of the system. The DCS is also designed as a distributed system; deployed over CORBA, new compute hardware can be easily added to the running system without incurring downtime or reconfiguration. In fact, much of the system can be maintained, taken down, or replaced without affecting the rest of the running system. Finally, it is a goal of the DCS that it can run on a variety of systems, including low-cost COTS PC hardware.

Maybe we should say something about "non-proprietary standards? -- BobKrzaczek

The specification of instruments, sensors, and data reduction hardware in the DCS is accomplished via a small set of descriptive XML documents. These descriptions are referenced by other documents in the system, such as experiments and observations. By specifying hardware interations and their data formats in these simple documents, new hardware (such as sensors) can be adopted by the DCS with a minimum of staff effort. Thus, the DCS can be continuously upgraded to support new hardware as well as new software pipelines and algorithms over time. Where these documents are exposed to our end users, the investigators, tools are provided that can be executed online or downloaded to local workstations to work with the files; these users need never interact directly with the XML contained in this data.

Ugh. Seriously awkward sentences, there. -- BobKrzaczek

In proposing to adopt the SOFIA DCS for use in the ISSI initiative, we expect the majority of the system to not require modifications. Instead, most most of the effort in such an adoption would be the description of the existing large body of physics-based algorithms for single-sensor applications in a format usable by the DCS. We believe that by adopting the DCS approach to data management for ISSI, we can provide a viable system that facilitates the reuse of software and maintenance of sensor data products over the long term.

---

• Set DENYTOPICREAD = TWikiGuest