Exporting data with gcexport

  • by

The simplest usage exports all data for GCDIR (/data/site-instrument). It uses the gcwerks-report.conf and peak.list files in the config directory:

gcexport  GCDIR

A custom report.conf and peak.list may be used:

gcexport  GCDIR  -format report.conf  -peaklist peak.list

Instead of report.conf, the fields to export may be specified on the command-line:

gcexport  GCDIR  time type port co2_C n2o_C

Here exporting data for run type air on inlet port 1:

gcexport  GCDIR  -type air -port 1 -format report.conf -peaklist peak.list

gcexport options

  • -format report.conf: use report.conf to define the fields to export rather than the default config/gcwerks-report.conf file. If this is not specified, fields to export may be specified on the command-line.
  • -peaklist peak.list: use peak.list for species to export instead of config/peak.list.
  • -csv: export as comma-separated values and values reported with on more decimal place (note: -csv implies -noflags)
  • -noheader: export data without a header

These options select data to be reported (by date range, run type, standard tank, etc.).

  • -type type: match run type
  • -tank tank: match first part of tank name (e.g. “-tank -J”)
  • -stdtank tank: match first part of standard tank name
  • -port N: match port number
  • -mean N: select N-minute mean for CRDS data (N=1,5,20,60)
  • -mindate yymmdd: starting date (may be yymmdd.hhmm)
  • -maxdate yymmdd: ending date

These options affect how missing or flagged data are reported.

  • -skipmissing: Do not report fields not in the database. Normally fields which are not in the database are reported as a column containg all NaNs (for fixed format).
  • -nanvalue -999: report this value instead of NaN
  • -noflags: Prevents display of flags and flagged values are reported as NaN.

report.conf and peak.list example

The default config/gcwerks-report.conf file contains the default export (or GCWerks report) format. It also contains a complete commented list of possible fields to export. This file may be copied to a local directory to create a custom report.conf file.

Example report.conf:

#name        flag        format  csv_format
#----------  ----------  ------  ----------
time         -           4       2
type
tank
#
%s_C         %s_flag     9.2
%s_stdev
%s_N

Only the first column is required in the report.conf file. The two format columns over-ride the default format for each field.

Example peak.list:

co2
n2o

Using the above report.conf and peak.list would results in exported columns:

time type tank  co2_C co2_stdev co2_N   n2o_C n2o_stdev n2o_N

Flags and -noflags

If a field has a flag associated (%s_C %s_flag above), the flag is displayed after the value:

123.456F    - flag = 'F'  (flagged value)
123.456*    - flag = '*'  (filtered value)

Time formats

The following time format numbers may be used in the report.conf file.

  • 1: yyyy-mm-dd hh:mm
  • 2: yyyy-mm-dd hh:mm:ss
  • 3: yymmdd hhmm
  • 4: yymmdd hhmmss
  • 5: yyyy mm dd hh mm
  • 6: yyyy mm dd hh mm ss
  • 7: yyyy.ffffff (year and fractional year)