Open Data Interface - PM 1 - Leuven 2009-02-12

• Peter and Daniel explained the ODI metadata structure and the relation to CDF/ISTP/IACG/PRBEM.
• It was agreed that the CDF files generated by ODI always only contain zVariables.
• The epoch variable exists in every dataset.
• CDF files are always ingested into ODI in two steps:
  1. read CDF files using IDL,
  2. store data into ODI using IDL/Dataminer.
• The origins (SEDAT or other) of the data sets that shall be ingested into ODI are specified in the LRD.txt file.
• The SREM data shall be downloaded from http://srem.web.psi.ch/data/.
• All the raw data files, whether it is CDF or ASCII, shall be stored on the ODI system.
• An SQL table shall contain the names of the raw data files and the start and end epochs for each file.
• New data records with epoch values identical to existing ODI epoch values will overwrite the old records.
• New data records with epoch values different from the existing ODI epoch values shall always be stored in ODI without taking any account to whether the new values have a different cadence or timing.
• The multiple entries for the global attributes must be handled.
• Index data will be stored in tables reflecting the time resolution, e.g. index_1h, index_3h, or index_24h.
• ODI will use its own variables names but the original CDF variable name will be stored in the ISTP variable attribute "VAR_NOTES".
• It was decided to ignore the PRBEM restriction of coordinate system to GEI.
• PRBEM has no mechanism for specifying ion species. It was decided to use a single ion flux z variable, which can be multi-dimensional when more than one ion species is present. Ion definition (A, Z, Q, ...) will be stored in the variable attributes.
• Data sets that are "complete" (i.e. the mission has terminated) will be ingested from SEDAT when possible. "Live" datasets will be ingested from the original sources, even the portions already in SEDAT.
• DH will visit HE at ESTEC on 23 Feb to review in detail the current contents of SEDAT, to review the functionality of the relevant Perl scripts in SEDAT, and to discuss technical matters related to setting up the system at ESTEC.
• HE will investigate where the system should be hosted at ESTEC, and will provide sufficient disk space for storing the datasets. He will also arrange for the necessary licences (IDL, Dataminer, ...) and servers (Apache, MySQL, ...) to be set up at ESTEC.
• The next meeting will be a one day meeting at ESTEC around mid May. The exact date will be decided later.