Application Programming Interfaces (APIs)

Smart SMEAR API provides an way to access programmatically the SMEAR database, that works as a storage for public timeseries data from SMEAR Stations. SMEAR database is a relational database (MariaDB) where timeseries data is stored in multiple tables. Every row of the table contains timestamp of the measuring moment and the values of the measured variables at the given timestamp. The intervals between rows/timestamps in tables vary from 1 min to 30min. The columns of the tables represent the measured variables. The measured variables is stored into different tables by their type and measuring station. Additionally, metadata that describes the measured variables and the measuring stations is stored in SMEAR DB. This metadata is also accessible via SMEAR API.

Smart Smear API consists three different APIs: Smeardata API, Metadata API and Trajectory API:

  • SMEAR Data API is for accessing timeseries data stored in SMEAR Database. The same data is also accessible via SMART SMEAR application. Via API the response format is CSV or TXT. The Smeardata API supports only HTTP GET requests.
  • Metadata API offers five endpoints for accessing metadata that describes variables stored in SMEAR database. Metadata API supports HTTP GET and POST requests with response format being JSON.
  • Trajectory API is for accessing trajectory data for the Smear II station in Hyytiälä. Trajectory API supports HTTP GET and POST requests with response format being JSON.

 

- SMEAR Data:

  • Basepath: https://avaa.tdata.fi
  • Endpoint: /smear-services/smeardata.jsp
  • Method: GET
  • Parameter type: Query
  • Response type: csv | txt
  • Description:

    Smeardata API is for fetching time series data stored in SMEAR Database. The timeseries data is measured for every variable stored in SMEAR DB. Information about stored variables can be found via Metadata API (see below) or via the graphical SMART SMEAR application. Smeardata API supports only HTTP GET requests and the required and optional input parameters are given as query parameters. Smeardata API returns the time series data in CSV-format as default, but txt-format is also available.

  • Input Parameters:
    Name Description Required Type Allowed Values
    variables Name of the variable (same as variable field in VariableMetadata) Yes String (CSV) All variable names from smear database. Variable names can be found from VariableMetadata endpoint in METADATA API or SmartSmear APP. Multiple variable name must be given as csv eg. Pamb0,UV_B,UV_A
    table Name of the database table where variable data is stored Yes String All table name of the table where variable is stored in smear database. Table names can be found from TableMetadata endpoint by variables tableIDs. TableID for every variable can be found from variable's metadata record via VariableMetadata endpoint. Multiple variable name must be given as csv eg. Pamb0,UV_B,UV_A and all the variables must be from the same table
    tablevariables Name of the table and the variable separated by dot (e.g. HYY_META.Pamb0). User can fetch data from multiple tables by using tablevariable instead of separate table and variable parameters Optional String All variable names and table names where variable are stored in smear database. Table names can be found from TableMetadata endpoint by variables IDs. TableID for every variable can be found from variable's metadata record via VariableMetadata endpoint. Multiple variables must be given as csv eg. HYY_META.Pamb0,HYY_META.UV_B,
    from starting point of the time series data Yes datetime yyyy-mm-dd hh:mm:ss
    to ending point of the time series data Yes datetime yyyy-mm-dd hh:mm:ss
    quality Is the timeseries data quality checked or not Yes String ANY | CHECKED
    averaging What time gap is used when aggregating values Yes String NONE | 30Min | 60Min | 1HOUR
    type Type of the aggregation Yes String none | arithmetic | geometric | sum | median | min | max | circular (= vector mean for wind direction)
    format The file format of the response No String csv | txt
    cuv_no version number of measurements on HYY_SLOW table. Required only for HYY_SLOW Integer cuv_no values as in Smeardatabase, separated with comma (,)
  • Query builder:
    Variables
    Tablevariables

  • Example query:
    https://avaa.tdata.fi/smear-services/smeardata.jsp?variables=Pamb0,UV_B&table=HYY_META&from=2016-02-11%2000:00:00.989&to=2016-02-12%2009:06:07.989&quality=ANY&averaging=30MIN&type=ARITHMETIC
    Try It!

  • Example response:

    The CSV response contains Timestamp divided in different columns (Year, Month,...,Second) and the value of measured data for variable at the time. The name of the variable in header row contains the Tablename and the variable name of the variable (=table.variable).

    Year,Month,Day,Hour,Minute,Second,HYY_META.Pamb0,HYY_META.UV_B
    2016,2,11,0,0,0,969.22266,4.6666668E-5
    2016,2,11,0,30,0,969.3223,-1.3333333E-5
    2016,2,11,1,0,0,969.539,5.3333333E-5
    2016,2,11,1,30,0,969.97235,8.666667E-5
    2016,2,11,2,0,0,970.284,6.0E-5...

- Metadata API

Metadata API is for fetching metadata that describes variables stored in SMEAR Database. The Metadata API is divided in five API endpoints: Variables, Events, Tags, Stations and Tables. Every endpoint supports HTTP GET and POST requests, but parameters are given only as query parameters. The response is in JSON -format.

- Variable metadata

  • Basepath: https://avaa.tdata.fi
  • Endpoint: /smart-smear-portlet/variablemeta.jsp
  • Method: GET
  • Parameter type: Query
  • Response type: json
  • Description:

    VariableMetadata -endpoint returns general metadata that describes variables stored in SMEAR DB. The metadata can be fetch by giving unique identifers of the variables (=variableID) or by the name of the variable (=variable) as query paramater. If query parameters is not given, the endpoint returns all the metadata records stored in the database. The variableID is unique, so endpoint return metadata record that matches given variable, but the variable name is ambiguous in SMEAR DB, so endpoint return all the records that match the variable name.

  • Input Parameters:
    Name Required Type Allowed Values Description
    variable Optional String All variable names from smear database. Multiple values can be given as csv (e.g Pamb0,d112e1,d112e2). Variable names can be found from this endpoint or SmartSmear APP The name of the variable in smear database.
    file (deprecated) Optional String See variable See variable
    varid Optional Integer All variableids from smear database. Multiple values can be given as csv (e.g 2,3,47) The unique identifier of the tag in the smear database.
    tablevariables Optional String All tablename and variablename combinations from smear database. Multiple values can be given as csv (e.g HYY_META.Pamb0,KUM_META.t) The table name and the variable name separated by dot (.).
    category No String All categories found in SMEAR DB. Multiple values can be given as csv (e.g aerosol,meteorology) The category of variable. This parameters works as filter when fetching variable metadata
    source No String All categories found in SMEAR DB. Multiple values can be given as csv (e.g aerosol,meteorology) The source of variable. This parameter works as filter when fetching variable metadata
    tableid No String All tableids found in SMEAR DB. Multiple values can be given as csv (e.g 2,4). The tableids can be found via tablemetadata-endpoint The unique identifer of a Smear database table. This parameters works as filter when fetching variable metadat
    allmeta No boolean true|false When allmeta parameter is set true, tags and table metadata is added into variable metadata response for each variable. This parameter is allowed only for json response
    format No String csv | json Format parameter defines the response format
  • Output Parameters:
    Name Type Description
    _variableID Long The unique identifer of given variable
    _tableID Long The unique identifer of the table where given variable is stored
    _variable String The name of the variable
    _description String Human readable description of the variable
    _unit String The unit of the time series data in SMEAR database
    _title String title of the variable
    _source String The instrument(s) or the source used in measurements
    _period_start Datetime The start time of the measurements
    _coverage Integer The coverage of the data stored in database
    _rights String The rights of the data usage
    _mandatory Boolean  
    _derivative Boolean  
    _vtimestamp Datetime  
    _category String The category of the variable. Multiple categories can be found in the Database, such as Aerosol, Meteorolgy or Radiation
    _cachedModel Boolean  
    _new Boolean  
  • Example query:
  • Example response:

    Because the variable name is used in three tables, there are three results.

    [{"_variableID":915,"_tableID":22,"_variable":"d112e1","_description":"Particle concentration dNdlogDp Dp\u003d1.12nm","_unit":"1/cm3","_title":"dN/dlogDp","_source":"HYY-DMPS","_period_start":"Jan 20, 1996 12:00:00 AM","_coverage":100,"_rights":"Public","_mandatory":false,"_derivative":false,"_vtimestamp":"Dec 1, 2014 4:12:09 PM","_category":"Aerosol","_cachedModel":false,"_new":false},{"_variableID":983,"_tableID":23,"_variable":"d112e1","_description":"Particle concentration dNdlogDp Dp\u003d1.12nm","_unit":"1/cm3","_title":"dN/dlogDp","_source":"VAR-DMPS","_period_start":"Dec 10, 1997 12:00:00 AM","_coverage":100,"_rights":"Public","_mandatory":false,"_derivative":false,"_vtimestamp":"Dec 1, 2014 4:08:18 PM","_category":"Aerosol","_cachedModel":false,"_new":false},{"_variableID":1051,"_tableID":24,"_variable":"d112e1","_description":"Particle concentration dNdlogDp Dp\u003d1.12nm","_unit":"1/cm3","_title":"dN/dlogDp","_source":"KUM-DMPS","_period_start":"May 5, 1997 12:00:00 AM","_coverage":100,"_rights":"Public","_mandatory":false,"_derivative":false,"_vtimestamp":"Dec 1, 2014 4:04:45 PM","_category":"Aerosol","_cachedModel":false,"_new":false}]
                                    

- Table metadata

  • Basepath: https://avaa.tdata.fi
  • Endpoint: /smart-smear-portlet/tablemetadata.jsp
  • Method: GET
  • Parameter type: Query
  • Response type: json
  • Description:

    TableMetadata endpoint returns metadata that describes measuring stations (=database tables). The data is queried by tableID. SMEAR variable’s TableID is stored in variables metadata and it can be found from variable metadata records via VariableMetadata endpoint.

  • Input Parameters:
    Name Required Type Allowed Values Description
    id Yes long All The unique identifiers of the tables in smear database. Multiple values can be given as csv (e.g 2,4).

    The unique identifier of the variable in the smear database.

  • Output Parameters:
    Name Type Description
    tableID Long The unique identifies of the database table. tableID is used in variable metadata record to link variables with database table
    identifier String The URN identifer of the database table that can be used to reference the data from the table
    title String The human readable title for the database table
    spatial_coverage String The coordinates of the measuring station as DCMI-point
    name String Name of the table in SMEAR database
    ttimestamp Datetime  
    period String  
  • Example query:
    https://avaa.tdata.fi/smart-smear-portlet/tablemetadata.jsp?id=11
     
  • Example response:
    [{ "tableID":11, "identifier":"URN:NBN:fi-fe201207066179", "title":"Värriö SMEAR I meteorology, gases and soil", "spatial_coverage":"DCMI-point: name=Värriö; east=29.610137; north=67.755044; elevation=390;", "name":"VAR_META", "ttimestamp":"2015-11-26 12:23:00.0", "period":1 }]
                                    

- Metadata

  • Basepath: https://avaa.tdata.fi
  • Endpoint: /smart-smear-portlet/metadata.jsp
  • Method: GET
  • Parameter type: NONE
  • Response type: json
  • Description:

    Metadata endpoint returns general info about Smart SMEAR application. Any query parameters is not needed in the request.

  • Input Parameters:

    General metadata query doesn't have parameters

  • Output Parameters:
    Name Type Description
    _title String The name for the SMART SMEAR Application
    _rightsCategory String The rights category for SMART SMEAR Application
    _access_rights String The url for the licenenses used in SMART SMEAR
    _project String The url for the SMEAR project
    _maintaining_organisation String The Maintaining organisation of SMART SMEAR
    _contact String E-mail address to contact maintaining organisation
    _ref String The citation for referencing to SMART SMEAR
    _creator String The Orcid identifer of the creator
    _discipline String The formal discipline of the SMART SMEAR
    _timestamp Datetime  
    _cachedModel String  
    _new String  
  • Example query:
    https://avaa.tdata.fi/smart-smear-portlet/metadata.jsp
     
  • Example response:
    [{"_title":"SMEAR Station for measuring ecosystem-atmosphere relations","_rightsCategory":"LICENSED","_access_rights":"https://creativecommons.org/licenses/by/4.0/","_project":"http://www.atm.helsinki.fi/SMEAR/","_maintaining_organisation":"University of Helsinki, Department of Physics, Division of Atmospheric Sciences","_contact":"atm-data@helsinki.fi","_ref":"Cite: Junninen, H; Lauri, A; Keronen, P; Aalto, P; Hiltunen, V; Hari, P; Kulmala, M. Smart-SMEAR: on-line data exploration and visualization tool for SMEAR stations.| BOREAL ENVIRONMENT RESEARCH (BER) Vol 14, Issue 4, pp.447-457","_creator":"http://orcid.org/0000-0001-8826-9108","_discipline":"http://www.yso.fi/onto/okm-tieteenala/ta114","_timestamp":"May 4, 2015 11:54:57 AM","_cachedModel":false,"_new":false}]

- Tags

  • Basepath: https://avaa.tdata.fi
  • Endpoint: /smart-smear-portlet/tags.jsp
  • Method: GET
  • Parameter type: Query
  • Response type: json
  • Input Parameters:
    Name Required Type Value Description
    varid No long

    All variableids from smear database. Multiple values can be given as csv (e.g 2,3,47)

    The unique identifier of the variable in the smear database.

  • Description:

    Tags are keywords for SMEAR variables. Tags are used to link variables/columns to vocabularies or other standards. Tags endpoint returns tags linked to SMEAR variable by variable’s unique identifier (=variableID).

  • Output Parameters:
    Name Type Description

    tagID

    Long The unique identifier of the tag in the smear database.
    vocabulary String Vocabulary is the URL of the formal vocabulary that the tag is based on.
    tag String Tag is a formal keyword describing the variables
    displaykeyword String Display keyword is the alternative word for the tag.
  • Example query:
    http://avaa.tdata.fi/smart-smear-portlet/tags.jsp?varid=1
  • Example response:
    [  {
     "tagID":1,
      "vocabulary":"http://cfconventions.org/Data/cf-standard-names/29/build/cf-standard-name-table.html#",
      "tag":"air_temperature",
      "displaykeyword":"ilma - lämpötila" 
     } ,  {
     "tagID":1,
      "vocabulary":"http://www.yso.fi/onto/yso/",
      "tag":"p19483",
      "displaykeyword":"ilma - lämpötila" 
     } ,  {
     "tagID":77,
      "vocabulary":"http://www.yso.fi/onto/yso/",
      "tag":"p5394",
      "displaykeyword":"meteorologia" 
     }    ]
                                    

- Events

  • Basepath: https://avaa.tdata.fi
  • Endpoint: /smart-smear-portlet/events.jsp
  • Method: GET
  • Parameter type: Query
  • Response type: json
  • Description:

    Events describes changes in time series data. The event types in SMEAR database are based on DDI data lifecycle events and ENVRI reference model. The time range of the event inicates the time frame when the event affects the data. Events endpoint returns events linked to SMEAR variable by variable’s unique identifier (=variableID).

  • Input Parameters:
    Name Required Type Value Description
    varid No long

    All variableids from smear database. Multiple values can be given as csv (e.g 2,3,47)

    The unique identifier of the variable in the smear database.

  • Output Parameters:
    Name Type Description

    eventID

    Long

    The unique identifier of the event in the smear database.

    event_type

    String

    The type of the event. The vocabulary used on event types is based on DDI data lifecycle events and ENVRI reference model:

    1a) ObjectChange

    1b)InstrumentConfiguration

    1c) Calibration

    2a) DataProcessing

    2b) NewVersionRelease

    event

    String The description of the event

    who

    String Name of the person who is responsible for the event

    period_start

    datetime (yyyy-mm-dd hh:ss:mm.ms) period_start indicates the start of time frame when the event affects the data.

    period_end

    datetime (yyyy-mm-dd hh:ss:mm.ms) period_start indicates the end of time

    frame when the event affects the data.

    etimestamp

    datetime (yyyy-mm-dd hh:ss:mm.ms)

    etimestamp indicates when the event was inserted to the database or (in case of NewVersionRelease) when the data was updated.

  • Example query:
    https://avaa.tdata.fi/smart-smear-portlet/eventsmeta.jsp?varid=43
                                    
     
  • Example response:
    [   {
     "eventID":41,
      "event_type":"NewVersionRelease",
      "event":"Processing and quality check of Smear 3 gas data ",
      "who":"Petri Keronen",
      "period_start":"2014-01-01 00:00:00.0",
      "period_end":"2014-12-31 23:59:59.0",
      "etimestamp":"2015-02-17 00:00:00.0"
     } ]
                                    

- PUI DMPS sizeranges

Particle sizer distributions measured with Differential Mobility Particle Sizer (DMPS). The table contains particle diameters for DMPS diameter channels/bins (both int and tot).

Size distributions have been measured using different diameters ranges at different periods:

  1. from 10 to 500 nm, 30 channels, 2.6.2006 - 31.3.2007
  2. from 7 to 800 nm, 40 channels, 1.4.2007 - 31.1.2012
  3. from 3 to 800 nm, 40 channels, from 1.2.2012

“sizerange_id” in PUI_dmps_int/tot tables tells which size range used and it's parameter of the query:

https://avaa.tdata.fi/smart-smear-portlet/PUI_dmps_sizeranges.jsp?id=3
                        [{"_sizerange_id":1,"_dp_min_nm":10.0,"_dp_max_nm":500.0,"_bins":30,"_start_time
":"Jun 2, 2006 12:00:00 AM","_stop_time":"Mar 21, 2007 11:59:59 PM","_dmps_table
s":"PUI_dmps_int_1 PUI_dmps_tot_1","_comments":"Particle diameters in nm","_ch01
":9.95,"_ch02":11.472,"_ch03":13.101,"_ch04":14.982,"_ch05":17.146,"_ch06":19.62
,"_ch07":22.46,"_ch08":25.713,"_ch09":29.419,"_ch10":33.672,"_ch11":38.526,"_ch1
2":44.106,"_ch13":50.459,"_ch14":57.774,"_ch15":66.103,"_ch16":75.649,"_ch17":86
.516,"_ch18":99.086,"_ch19":113.358,"_ch20":129.758,"_ch21":148.587,"_ch22":169.
89,"_ch23":194.368,"_ch24":222.533,"_ch25":254.768,"_ch26":291.517,"_ch27":333.6
04,"_ch28":382.044,"_ch29":436.948,"_ch30":499.867,"_ch31":0.0,"_ch32":0.0,"_ch3
3":0.0,"_ch34":0.0,"_ch35":0.0,"_ch36":0.0,"_ch37":0.0,"_ch38":0.0,"_ch39":0.0,"
_ch40":0.0,"_cachedModel":false,"_new":false}]
                    

- Trajectories:

  • Description:

    Trajectories API is for accessing Trajectory data that has been calculated only for the Hyytiälä station and until autumn 2012. The Trajectories API supports HTTP GET and POST requests and The response is in JSON format.

  • Input Parameters:
    Name Required Type Value Description
    Year Yes String YYYY  
    Month Yes String MM  
    Day Yes String DD  
    Hour Yes String HH  
  • Output Parameters:
    Name Type Description
    _arrTimeID String (yyyyMMddHHmm)  
    _arrTime Date (MMM d, yyyy h:mm:ss)  
    _matTime double  
    _year int  
    _month int  
    _day int  
    _hour int  
    _doy int  
    _backTime int  
    _lat100 double  
    _lon100 double  
    _alt100 double  
    _azim100 double  
    _lat250 double  
    _lon250 double  
    _alt250 double  
    _azim250 double  
    _lat500 double  
    _lon500 double  
    _alt500 double  
    _azim500 double  
    _cachedModel boolean  
    _new boolean  
  • Example query:
    https://avaa.tdata.fi/map-portlet/query.jsp?year=2012&month=07&day=16&hour=16
                            
     
  • Example response:
    [{"_arrTimeID":201207161600,"_arrTime":"Jul 16, 2012 4:00:00 PM","_matTime":735067.0,"_year":2012,"_month":7,"_day":16,"_hour":16,"_doy":198,"_backTime":0,"_lat100":61.85,"_lon100":24.28,"_alt100":100.0,"_azim100":292.7,"_lat250":61.85,"_lon250":24.28,"_alt250":250.0,"_azim250":292.7,"_lat500":61.85,"_lon500":24.28,"_alt500":500.0,"_azim500":292.9,"_cachedModel":false,"_new":false},
    ...