WSL/SLF GitLab Repository

Commit bfd33efd authored by Mathias Bavay's avatar Mathias Bavay
Browse files

Documentation update. A new section "how to build your io.ini configuration...

Documentation update. A new section "how to build your io.ini configuration file" has been added to the end user section of the documentation.
parent b51311f0
......@@ -20,53 +20,6 @@
using namespace std;
namespace mio {
/**
* @page filters Filters overview
* The filtering infrastructure is described in FilterAlgorithms (for its API). The goal of this page is to give an overview of the available filters and their usage.
*
* @section filters_modes Modes of operation
* It should be noted that filters often have two modes of operations: soft or hard. In soft mode, all value that is rejected is replaced by the filter parameter's value. This means that for a soft min filter set at 0.0, all values less than 0.0 will be replaced by 0.0. In hard mode, all rejected values are replaced by nodata.
*
* @section filtering_section Filtering section
* The filters are specified for each parameter in the [Filters] section. This section contains
* a list of the various meteo parameters (see MeteoData) with their associated choice of filtering algorithms and
* optional parameters.The filters are applied serialy, in the order they are given in. An example of such section is given below:
* @code
* [Filters]
* TA::filter1 = min_max
* TA::arg1 = 230 330
*
* RH::filter1 = min_max
* RH::arg1 = -0.2 1.2
* RH::filter2 = min_max
* RH::arg2 = soft 0.0 1.0
*
* HNW::filter1 = min
* HNW::arg1 = -0.1
* HNW::filter2 = min
* HNW::arg2 = soft 0.
* @endcode
*
* @section filters_available Available filters
* NOTE: this is now obsolete, but not all the filters have been ported to the new infrastructure.
* The documentation has not (yet) been ported to the new structure either...
* The filters that were originally available were the following:
* - rate: rate of change filter, see FilterAlgorithms::RateFilter
* - min_max: range check filter, see FilterAlgorithms::MinMaxFilter
* - min: minimum check filter, see FilterAlgorithms::MinValueFilter
* - max: maximum check filter, see FilterAlgorithms::MaxValueFilter
* - mad: median absolute deviation, see FilterAlgorithms::MedianAbsoluteDeviationFilter
* - stddev: reject data outside mean +/- k*stddev, see FilterAlgorithms::StandardDeviationFilter
* - Tukey53H: Tukey53H spike detection, based on median, see FilterAlgorithms::Tukey53HFilter
*
* A few data transformations are also supported besides filtering:
* - accumulate: data accumulates over a given period, see FilterAlgorithms::AccumulateProcess
* - exp_smoothing: exponential smoothing of data, see FilterAlgorithms::ExpSmoothingProcess
* - wma_smoothing window moving average smoothing of data, see FilterAlgorithms::WMASmoothingProcess
* - median_avg: running median average over a given window, see FilterAlgorithms::MedianAvgProcess
* - mean_avg: running mean average over a given window, see FilterAlgorithms::MeanAvgProcess
* - wind_avg: vector average over a given window, see FilterAlgorithms::WindAvgProcess
*/
std::map<std::string, FilterProperties> FilterAlgorithms::filterMap;
const bool FilterAlgorithms::__init = FilterAlgorithms::initStaticData();
......
......@@ -29,7 +29,11 @@ namespace mio {
*/
/*! \defgroup stats Statistical calculations
Documentation for available statistical calculations. This is heavily used by the \subpage filters "Filters" as well as the \subpage resampling "1D interpolations" and \subpage interpol2d "2D interpolations".
Documentation for available statistical calculations. This is heavily used by the \subpage processing "Available data processing elements" as well as the \subpage resampling "1D interpolations" and \subpage interpol2d "2D interpolations".
*/
/*! \defgroup processing Data processing elements
Documentation for available data processing components. These can be used on incoming meteorological data. See \subpage processing "Available data processing elements".
*/
/*! \defgroup data_str Data structures
......@@ -55,16 +59,17 @@ namespace mio {
* -# \subpage general "General concepts"
* -# \subpage plugins "Available plugins" and usage
* -# \subpage coords "Available coordinate systems" and usage
* -# \subpage filters "Available filters" and usage
* -# \subpage processing "Available processing elements" and usage
* -# \subpage resampling "Available temporal interpolations" and usage
* -# \subpage interpol2d "Available spatial interpolations" and usage
* -# \subpage build_io "How to build your io.ini configuration file"
* -# Programing using MeteoIO
* -# \subpage workflow "Example Workflow"
* -# \subpage quick_overview "Quick overview" of the functionnality provided by MeteoIO
* -# \subpage examples "Usage examples"
* -# Expanding MeteoIO
* -# How to \subpage dev_plugins "write a Plugin"
* -# How to \subpage dev_filters "write a Filter"
* -# How to \subpage dev_processing "write a processing element"
* -# How to \subpage dev_2Dinterpol "write a spatial interpolation algorithm"
*/
......@@ -84,6 +89,7 @@ namespace mio {
* - a module to deal with configuration files. The program that you are using might be using this module for other configuration files.
*
* @section Config Configuration file
* @anchor config_doc
* @subsection Config_syntax Configuration file syntax
* The configuration inputs/outputs file is divided in sections. Each section name is enclosed in brackets.
* @code
......@@ -112,6 +118,7 @@ namespace mio {
* [Input]
* COORDSYS = CH1903 #mandatory: which coordinate system is used for the geographic coordinates
* COORDPARAM = -999 #extra arguments for the chosen coordinate system (often, none)
* TIME_ZONE = +1 #default time zone for inputs
*
* DEM = ARC #plugin to use for reading DEM information
* #this might be followed by any number of arguments that are specific to this plugin
......@@ -141,6 +148,37 @@ namespace mio {
* As can be seen from the previous example, each plugin, each filter or each interpolation algorithm might have its own parameters. Therefore, this is the documentation of each specific plugin/filter/algorithm that has to be used in order to figure out what can be configured when it (see the next sections in the welcome page).
*/
/**
* @page build_io "How to build your io.ini configuration file"
* As shown in \ref config_doc , the operation of MeteoIO is driven by a configuration file. This section will show you
* how to practically set up your first configuration file. Please read \ref general documentation page before starting!
*
* You first need to create the various sections:
* - [General] : the documentation about this section is found in ??
* - [Input] : This section contains the list of all the plugins that you want to use as well as their parameters. You can
* use one plugin for the meteorological data (key=METEO), one for grids (key=GRID2D), one for special points
* (key=SPECIALPTS), one for data assimilation (key=DA), one for landuse (key=LANDUSE) and one for Digital
* Elevation Model (key=DEM). Please see \ref plugins for the available plugins. Afterwards, each plugin comes
* with its own set of keys, as specified in the plugin's documentation. Morevover, the geographic coordinate
* system should often be specified, as explained in \ref coords .
*
* - [Output] : This section is very similar to the [Input] section, but (obviously) for outputing the data.
*
* - [Filters] : This section lists the pre-processing that has to be performed on the incoming meteorological data.
* It builds a stack of processing elements one after the other one, for each meteorological parameter.
* See \ref processing for more information.
*
* - [Interpolations1D] : This section deals with temporal resampling of the incoming meteorological data.
* See \ref resampling .
*
* - [Interpolations2D] : This section deals with the spatial interpolation of meteorological data, based on a provided
* Digital Elevation Model. See \ref interpol2d .
*
*
* The application that you are using might also need its own section(s), check this with your application.
*
*/
/**
* @page workflow Workflow
* Here is a workflow example showing how meteorological data is requested by the user's application and delivered. This is a simplified view, in order to show the general structure. Requesting grids (2D grids, DEM, etc) is very similar but does not perfom filtering or resampling.
......@@ -217,24 +255,26 @@ namespace mio {
//2D interpolation development given in Meteo2DInterpolator.h
/**
* @page dev_filters Filter developer's guide
* @page dev_processing Processing elements developer's guide
*
* In order to add a new filter to the already existing set of filters the developer only needs to edit
* the class FilterAlgorithms. It is important to understand that the filter operate on a "per parameter" basis.
* This means that a filter might be executed for the parameter TA and another one for the parameter HNW, so the filter
* algorithm only has to deal with a generic filtering method based on double values.
* In order to add a new filter/processing element to the already existing set of components, the developer only needs to
* add a class derived from either ProcessingBlock, FilterBlock or WindowedFilter in meteoio/meteofilters.
* It is important to understand that the processing elements operate on a "per parameter" basis.
* This means that an element might be executed for the parameter TA and another one for the parameter HNW, so the
* algorithm only has to deal with a generic processing method based on double values.
*
* To implement a new filter, the following steps are necessary:
* To implement a new processing element, the following steps are necessary:
*
* -# Implementing the filter, as a derived class of FilterBlock, by creating two files: the header file and its
* -# Implementing the element, as a derived class of ProcessingBlock or FilterBlock or WindowedFilter, by creating
* two files: the header file and its
* implementation file, in the meteofilters subdirectory of the source code.
* The class will contain two public methods: a constructor
* that takes a vector of strings containing the filter arguments and a method
* that takes a vector of strings containing the element's arguments and a method
* @code
* process(const unsigned int& index, const std::vector<MeteoData>& ivec, std::vector<MeteoData>& ovec)
* @endcode
* that
* applies the filter to the provided vector of values, for a meteo parameter pointed to by index. This index
* applies the element to the provided vector of values, for a meteo parameter pointed to by index. This index
* is the MeteoData parameter that this filter shall be run upon (see MeteoData for the enumeration of
* parameters). The constructor must set up properties.for_second_pass to mark if the filter can be applied
* a second time during a second pass, that is after resampling.
......
......@@ -28,9 +28,18 @@ namespace mio {
/**
* @class FilterMax
* @brief
* @ingroup processing
* @author Thomas Egger - Mathias Bavay
* @date 2011-01-02
* @brief Max range filter.
* Reject all values greater than the max. Remarks:
* - the maximum permissible value has to be provided has an argument (in SI)
* - the keyword "soft" maybe added, in such a case all data greater than the max would be assigned
* the maximum permissible value
* @code
* TA::filter1 = max
* TA::arg1 = 330
* @endcode
*/
class FilterMax : public FilterBlock {
......
......@@ -26,9 +26,25 @@ namespace mio {
/**
* @class FilterMeanAvg
* @brief
* @ingroup processing
* @author Thomas Egger
* @date 2011-01-24
* @brief Mean average processing.
* The mean average filter returns the mean value of all values within a user given time window. Remarks:
* - nodata values are excluded from the mean
* - Two arguments expected (both have to be fullfilled for the filter to start operating):
* - minimal number of points in window
* - minimal time interval spanning the window (in seconds)
* - the two arguments may be preceded by the keywords "left", "center" or "right", indicating the window position
* - the keyword "soft" maybe added, if the window position is allowed to be adjusted to the data present
*
* @code
* Valid examples for the io.ini file:
* TA::filter1 = mean_avg
* TA::arg1 = soft left 1 1800 (1800 seconds time span for the left leaning window)
* RH::filter1 = mean_avg
* RH::arg1 = 10 600 (strictly centered window spanning 600 seconds and at least 10 points)
* @endcode
*/
class FilterMeanAvg : public WindowedFilter {
......
......@@ -28,9 +28,26 @@ namespace mio {
/**
* @class FilterMedianAvg
* @brief Median average filter
* @ingroup processing
* @author Thomas Egger
* @date 2011-01-24
* @brief Median average processing.
* The median average filter returns the median value of all values within a user given time window. Remarks:
* - nodata values are excluded from the median
* - if there is an even number of window elements the arithmetic mean of the two central elements is used to calculate the median
* - Two arguments expected (both have to be fullfilled for the filter to start operating):
* - minimal number of points in window
* - minimal time interval spanning the window (in seconds)
* - the two arguments may be preceded by the keywords "left", "center" or "right", indicating the window position
* - the keyword "soft" maybe added, if the window position is allowed to be adjusted to the data present
*
* @code
* Valid examples for the io.ini file:
* TA::filter1 = median_avg
* TA::arg1 = soft left 1 1800 (1800 seconds time span for the left leaning window)
* RH::filter1 = median_avg
* RH::arg1 = 10 600 (strictly centered window spanning 600 seconds and at least 10 points)
* @endcode
*/
class FilterMedianAvg : public WindowedFilter {
......
......@@ -28,9 +28,18 @@ namespace mio {
/**
* @class FilterMin
* @brief
* @ingroup processing
* @author Thomas Egger - Mathias Bavay
* @date 2011-01-02
* @brief Min range filter.
* Reject all values smaller than the min. Remarks:
* - the minimum permissible value has to be provided has an argument (in SI)
* - the keyword "soft" maybe added, in such a case all data smaller than the min would be assigned
* the minimum permissible value
* @code
* TA::filter1 = min
* TA::arg1 = 230
* @endcode
*/
class FilterMin : public FilterBlock {
......
......@@ -28,9 +28,19 @@ namespace mio {
/**
* @class FilterMinMax
* @brief
* @ingroup processing
* @brief Min/Max range filter.
* @author Thomas Egger
* @date 2011-01-02
* Reject all values greater than the max or smaller than the min. Remarks:
* - two arguments have to be provided, min and max (in SI)
* - the keyword "soft" maybe added, in such a case all data greater than the max would be assigned
* the maximum permissible value and all data smaller than the min would be assigned the minimum permissible value
* @code
* TA::filter1 = min_max
* TA::arg1 = 230 330
* @endcode
*
*/
class FilterMinMax : public FilterBlock {
......
......@@ -28,9 +28,18 @@ namespace mio {
/**
* @class FilterStdDev
* @brief Standard Deviation filter
* @ingroup processing
* @author Mathias Bavay
* @date 2011-02-07
* @brief Standard deviation filter.
* Values outside of mean ± 2 std_dev are rejected.
* @code
* Valid examples for the io.ini file:
* TA::filter1 = stddev
* TA::arg1 = soft left 1 1800 (1800 seconds time span for the left leaning window)
* RH::filter1 = stddev
* RH::arg1 = 10 6000 (strictly centered window spanning 6000 seconds and at least 10 points)
* @endcode
*/
class FilterStdDev : public WindowedFilter {
......
......@@ -25,6 +25,53 @@
#include <meteoio/meteofilters/RateFilter.h>
namespace mio {
/**
* @page processing Processing overview
* The pre-processing infrastructure is described in ProcessingBlock (for its API). The goal of this page is to give an overview of the available filters and processing elements and their usage.
*
* @section processing_modes Modes of operation
* It should be noted that filters often have two modes of operations: soft or hard. In soft mode, all value that is rejected is replaced by the filter parameter's value. This means that for a soft min filter set at 0.0, all values less than 0.0 will be replaced by 0.0. In hard mode, all rejected values are replaced by nodata.
*
* @section processing_section Filtering section
* The filters are specified for each parameter in the [Filters] section. This section contains
* a list of the various meteo parameters (see MeteoData) with their associated choice of filtering algorithms and
* optional parameters.The filters are applied serialy, in the order they are given in. An example of such section is given below:
* @code
* [Filters]
* TA::filter1 = min_max
* TA::arg1 = 230 330
*
* RH::filter1 = min_max
* RH::arg1 = -0.2 1.2
* RH::filter2 = min_max
* RH::arg2 = soft 0.0 1.0
*
* HNW::filter1 = min
* HNW::arg1 = -0.1
* HNW::filter2 = min
* HNW::arg2 = soft 0.
* @endcode
*
* @section processing_available Available processing elements
* The filters are being ported to the new filtering infrastructure. Only the filters whose key is capitalized have been
* ported and are ready to use in the current version.
* The filters that are currently available are the following:
* - RATE: rate of change filter, see RateFilter
* - MIN_MAX: range check filter, see FilterMinMax
* - MIN: minimum check filter, see FilterMin
* - MAX: maximum check filter, see FilterMax
* - STD_DEV: reject data outside mean +/- k*stddev, see FilterAlgorithms::StandardDeviationFilter
* - mad: median absolute deviation, see FilterAlgorithms::MedianAbsoluteDeviationFilter
* - Tukey53H: Tukey53H spike detection, based on median, see FilterAlgorithms::Tukey53HFilter
*
* A few data transformations are also supported besides filtering:
* - accumulate: data accumulates over a given period, see FilterAlgorithms::AccumulateProcess
* - exp_smoothing: exponential smoothing of data, see FilterAlgorithms::ExpSmoothingProcess
* - wma_smoothing window moving average smoothing of data, see FilterAlgorithms::WMASmoothingProcess
* - MEDIAN_AVG: running median average over a given window, see FilterMedianAvg
* - MEAN_AVG: running mean average over a given window, see FilterMeanAvg
* - wind_avg: vector average over a given window, see FilterAlgorithms::WindAvgProcess
*/
std::set<std::string> BlockFactory::availableBlocks;
const bool BlockFactory::__init = BlockFactory::initStaticData();
......
......@@ -26,6 +26,7 @@ namespace mio {
/**
* @class RateFilter
* @ingroup processing
* @author Thomas Egger
* @date 2011-01-26
* @brief Rate of change filter.
......
Supports Markdown
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment