Data Sometimes Is Part Of A Set

Suppose you do some search on Google about how to use the Crypto package in java. Often, in searches like this, the actual Google hit is part of a thread from an e-mail list that's archived on the web. And, often, the hit is not the beginning of the thread. It's somewhere in the middle of the thread.

From Google's point of view, this is acceptable. You searched for ZZZ, it was in document YYY. Google doesn't care that to really understand document YYY, you should have started reading at document XXX or WWW.

For our part, we're going to have to care. When data comes together in a set, we're not going to want to lose track of which set the data belongs to. Sometimes, the data may belong to more than one set.

A given WASP frame from the middle IR camera belongs in a set with the simultaneous frames from the other two IR cameras. Together, those belong in a set with the nearest (temporally or spatially?) Terrapix image. Together, those belong in a set with all of the other sets on that pass. Together, that set belongs with all of the other passes on that same region during that same flight. Additionally, these might also have an explicit association with ground-measurements taken in conjunction with them.

When someone matches a particular image, we're not going to want to send them back results that explicitly list every image in the same set. We're going to want to return the handle to the set, instead, as a single result.

Put another way, Google is indexing data that it can presume is already linked together in a way where you can explore on your own once you get to the page. We're dealing with data where the links will not all be explicitly present in the data and the browsing tools wouldn't necessarily know what to do with the links that are in the data anyway. So, we're going to have to track these linkages externally, and have smart enough tools to navigate these external linkages.

-- PatrickStein - 25 Mar 2005