WSL/SLF GitLab Repository

Commit 11181ce7 authored by Mathias Bavay's avatar Mathias Bavay
Browse files

Yesterday's commit was breaking some of the includes. This has been fixed, the...

Yesterday's commit was breaking some of the includes. This has been fixed, the documentation updated and the install/cleanup commands for cmake updated
parent 3ddc6025
......@@ -108,6 +108,7 @@ SET(DATA_QA OFF CACHE BOOL "Data Quality Assurance outputs ON or OFF")
#for the install target
FILE(GLOB hfiles "meteoio/*.h")
FILE(GLOB hdatafiles "meteoio/dataClasses/*.h")
FILE(GLOB hgeneratorsfiles "meteoio/dataGenerators/*.h")
FILE(GLOB hpluginfiles "meteoio/plugins/*.h")
FILE(GLOB hlawsfiles "meteoio/meteoLaws/*.h")
FILE(GLOB hfilterfiles "meteoio/meteoFilters/*.h")
......@@ -115,6 +116,7 @@ FILE(GLOB hstatsfiles "meteoio/meteoStats/*.h")
INSTALL(FILES ${hfiles} DESTINATION include/meteoio COMPONENT headers)
INSTALL(FILES ${hdatafiles} DESTINATION include/meteoio/dataClasses COMPONENT headers)
INSTALL(FILES ${hgeneratorsfiles} DESTINATION include/meteoio/dataGenerators COMPONENT headers)
INSTALL(FILES ${hpluginfiles} DESTINATION include/meteoio/plugins COMPONENT headers)
INSTALL(FILES ${hlawsfiles} DESTINATION include/meteoio/meteoLaws COMPONENT headers)
INSTALL(FILES ${hfilterfiles} DESTINATION include/meteoio/meteoFilters COMPONENT headers)
......@@ -129,6 +131,7 @@ ADD_CUSTOM_TARGET(distclean make clean
COMMAND cmake -E remove_directory Testing
COMMAND cmake -E remove_directory meteoio/CMakeFiles
COMMAND cmake -E remove_directory meteoio/dataClasses/CMakeFiles
COMMAND cmake -E remove_directory meteoio/dataGenerators/CMakeFiles
COMMAND cmake -E remove_directory meteoio/meteoFilters/CMakeFiles
COMMAND cmake -E remove_directory meteoio/meteoLaws/CMakeFiles
COMMAND cmake -E remove_directory meteoio/plugins/gsn/CMakeFiles
......
......@@ -21,70 +21,14 @@
#include <meteoio/Config.h>
#include <meteoio/dataClasses/MeteoData.h>
#include <meteoio/GeneratorAlgorithms.h>
#include <meteoio/dataGenerators/GeneratorAlgorithms.h>
#include <vector>
#include <map>
#include <set>
namespace mio {
/**
* @page dev_DataGenerator How to write a data generator
* Once the data has been read, filtered and resampled, it can be that some data points are still missing.
* These are either a few isolated periods (a sensor was not functioning) that are too large for performing
* a statistical temporal interpolation or that a meteorological parameter was not even measured. In such a case,
* we generate data, generally relying on some parametrization using other meteorological parameters. In a few
* cases, even fully arbitrary data might be helpful (replacing missing value by a given constant so a model can
* run over the data gap).
*
* @section structure_DataGenerator Structure
* The selection of which data generator to use at any given time step, for a given parameter is
* performed by the DataGenerator class. This class acts as an interface, presenting a higher level view to the
* caller. The data generators themselves derive from the GeneratorAlgorithm class that standardizes their
* public API. An object factory creates the generator during intialization (keeping all constructed generators
* in a vector during the whole life time of the DataGenerator object), based on the strings contained in the user's
* io.ini configuration file.
*
* The API also defines two public "generate" methods, taking a meteorological parameter index (see MeteoData) and either
* a set of meteo data for one station and at one point in time or a meteo time series for one station.
* These methods walk through the meteo data looking for nodata values for the requested meteo parameter index.
* If the generator could successfully generate data for <b>all</b> the nodata values it found, it returns <i>true</i>,
* <i>false</i> otherwise. If <i>false</i> was returned, the DataGenerator object that manages the process would
* call the next data generator, <b>in the order that was declared by the user</b>. For a given meteo parameter, the
* whole process stops as soon as a <i>true</i> is returned or there are no more data generators to try
* (as declared by the user in his configuration file).
*
* @section implementation_DataGenerator Implementation
* It is therefore necessary to create in GeneratorAlgorithms.cc (and declared in the .h) a new class,
* nammed after the generator that will be implemented and inheriting GeneratorAlgorithm. Three methods need
* to be implemented:
* - the constructor with (const std::vector<std::string>& vecArgs, const std::string& i_algo)
* - bool generate(const size_t& param, MeteoData& md)
* - bool generate(const size_t& param, std::vector<MeteoData>& vecMeteo)
*
* The constructor is responsible for parsing the arguments as a vector of strings and saving its own name internally, for
* error messages, warnings, etc. It should set all internal variables it sees fit according to the parsed arguments. The
* goal is <i>to not</i> do any parsing anywhere else (for performances reasons).
*
* The <i>generate(const size_t& param, MeteoData& md)</i> method compares <i>md(param)</i> with <i>IOUtils::nodata</i> and replaces
* it by its generated value if necessary. It returns <i>true</i> if no further processing is needed
* (ie. no replacement was needed or the replacement could be done) or <i>false</i> otherwise.
*
* The <i>generate(const size_t& param, std::vector<MeteoData>& vecMeteo)</i> method compares
* <i>vecMeteo[ii](param)</i> with <i>IOUtils::nodata</i> for each timestamp in the vector and tries to generate data when necessary.
* If all missing data points could be generated (or if no data point required to be generated), it returns <i>true</i>,
* and <i>false</i> otherwise.
*
* Finally, a new entry must be added in the object factory GeneratorAlgorithmFactory::getAlgorithm method at the top of file
* GeneratorAlgorithms.cc.
*
* @section doc_DataGenerator Documentation
* The newly added data generator must be added to the list of available algorithms in
* GeneratorAlgorithms.h with a proper description. Its class must be properly documented, similarly to the other data
* generators. An example can also be given in the example section of the same file.
* Please feel free to add necessary bibliographic references to the bibliographic section!
*/
/**
* @class DataGenerator
* @brief A class to generate meteo data from user-selected models or parametrizations.
......
......@@ -26,6 +26,7 @@
#include <meteoio/TimeSeriesManager.h>
#include <meteoio/GridsManager.h>
#include <meteoio/Meteo2DInterpolator.h>
#include <meteoio/meteoLaws/Sun.h>
#include <vector>
#include <string>
......
......@@ -47,7 +47,7 @@
#include <meteoio/DataGenerator.h>
#include <meteoio/exports.h>
#include <meteoio/FileUtils.h>
#include <meteoio/GeneratorAlgorithms.h>
#include <meteoio/dataGenerators/GeneratorAlgorithms.h>
#include <meteoio/Graphics.h>
#include <meteoio/InterpolationAlgorithms.h>
#include <meteoio/IOExceptions.h>
......
......@@ -756,6 +756,7 @@ WARN_LOGFILE =
INPUT = ./meteoio \
./meteoio/dataClasses \
./meteoio/dataGenerators \
./meteoio/plugins \
./meteoio/meteoLaws \
./meteoio/meteoFilters \
......
......@@ -30,10 +30,85 @@
#include <meteoio/dataGenerators/TauCLDGenerator.h>
#include <meteoio/dataGenerators/TsGenerator.h>
using namespace std;
namespace mio {
/**
* @page generators Data generators
* Once the data has been read, filtered and resampled, it can be that some data points are still missing.
* These are either a few isolated periods (a sensor was not functioning) that are too large for performing
* a statistical temporal interpolation or that a meteorological parameter was not even measured. In such a case,
* we generate data, generally relying on some parametrization using other meteorological parameters. In a few
* cases, even fully arbitrary data might be helpful (replacing missing value by a given constant so a model can
* run over the data gap).
*
* Another usage scenario is to create new parameters fully based on parametrizations. In such a case, the "generator" is called
* a "creator" and behaves the same way as a generator, except that it creates an additional parameter. It is declared as
* {new_parameter}::%create = {data generators}.
*
* @note it is generally not advised to use data generators in combination with spatial interpolations as this would
* potentially mix measured and generated values in the resulting grid. It is therefore advised to turn the data generators
* off and let the spatial interpolations algorithms adjust to the amount of measured data.
* @note it is also possible to make a copy of a given parameter under a different name. This is explained in section
* \ref data_manipulations "Raw data editing".
*
* @section generators_section Data generators section
* The data generators are defined per meteorological parameter. They are applied to all stations
* (if using multiple meteorological stations). If multiple dat generators are specified for each parameter,
* they would be used in the order of declaration, meaning that only the data points that could not be
* generated by the first generator would be tentatively generated by the second generator, etc. Please also keep
* in mind that at this stage, all data <b>must be in SI</b> units!
* @code
* [Generators]
* RH::generators = CST
* RH::Cst = .7
*
* P::generators = STD_PRESS
*
* ILWR::generators = AllSky_LW ClearSky_LW
*
* TAU_CLD::create = CST
* TA_CLD::Cst = 0.5
* @endcode
*
* @section generators_keywords Available generators
* The keywords defining the algorithms are the following:
* - STD_PRESS: standard atmospheric pressure as a function of the elevation of each station (see StandardPressureGenerator)
* - RELHUM: relative humidity from other humidity measurements (see RhGenerator)
* - TS_OLWR: surface temperature from Outgoing Long Wave Radiation (see TsGenerator)
* - ISWR_ALBEDO: ISWR from RSWR or RSWR from ISWR with either a snow or a soil albedo, depending on HS (see IswrAlbedoGenerator)
* - CST: constant value as provided in argument (see ConstGenerator)
* - SIN: sinusoidal variation (see SinGenerator)
* - CLEARSKY_LW: use a clear sky model to generate ILWR from TA, RH (see ClearSkyLWGenerator)
* - ALLSKY_LW: use an all sky model to generate ILWR from TA, RH and cloudiness (see AllSkyLWGenerator)
* - ALLSKY_SW: generate the incoming short wave radiation from the potential radiation, corrected for cloudiness if possible (see AllSkySWGenerator)
* - TAU_CLD: generate the atmospheric transmissivity based on cloud cover fraction (see TauCLDGenerator)
* - ESOLIP: generate precipitation from snow height changes (see ESOLIPGenerator)
* - PPHASE: generate precipitation phase with a user-selected method (see PPhaseGenerator)
*
* @section generators_biblio Bibliography
* The data generators have been inspired by the following papers:
* - Brutsaert -- <i>"On a Derivable Formula for Long-Wave Radiation From Clear Skies"</i>, Journal of Water Resources
* Research, <b>11</b>, No. 5, October 1975, pp 742-744.
* - Prata -- <i>"A new long-wave formula for estimating downward clear-sky radiation at the surface"</i>, Q. J. R. Meteorolo. Soc., <b>122</b>, 1996, pp 1127-1151.
* - Dilley and O'Brien -- <i>"Estimating downward clear sky long-wave irradiance at the surface from screen temperature and precipitable water"</i>, Q. J. R. Meteorolo. Soc., Vol. 124, 1998, doi:10.1002/qj.49712454903
* - Clark & Allen -- <i>"The estimation of atmospheric radiation for clear and cloudy skies"</i>, Proceedings of the second national passive solar conference, <b>2</b>, 1978, p 676.
* - Tang et al. -- <i>"Estimates of clear night sky emissivity in the Negev Highlands, Israel"</i>, Energy Conversion and Management, <b>45.11</b>, 2004, pp 1831-1843.
* - Idso -- <i>"A set of equations for full spectrum and 8 to 14 um and 10.5 to 12.5 um thermal radiation from cloudless skies"</i>, Water Resources Research, <b>17</b>, 1981, pp 295-304.
* - Kasten and Czeplak -- <i>"Solar and terrestrial radiation dependent on the amount and type of cloud"</i>, 1980, Solar energy, 24.2, pp 177-189
* - Omstedt -- <i>"A coupled one-dimensional sea ice-ocean model applied to a semi-enclosed basin"</i>, Tellus, <b>42 A</b>, 568-582, 1990, DOI:10.1034/j.1600-0870.1990.t01-3-00007.
* - Konzelmann et al. -- <i>"Parameterization of global and longwave incoming radiation for the Greenland Ice Sheet."</i> Global and Planetary change <b>9.1</b> (1994): 143-164.
* - Crawford and Duchon -- <i>"An Improved Parametrization for Estimating Effective Atmospheric Emissivity for Use in Calculating Daytime
* Downwelling Longwave Radiation"</i>, Journal of Applied Meteorology, <b>38</b>, 1999, pp 474-480
* - Unsworth and Monteith -- <i>"Long-wave radiation at the ground"</i>, Q. J. R. Meteorolo. Soc., Vol. 101, 1975, pp 13-24
* - Meeus -- <i>"Astronomical Algorithms"</i>, second edition, 1998, Willmann-Bell, Inc., Richmond, VA, USA
* - Mair et al. -- <i>" ESOLIP–estimate of solid and liquid precipitation at sub-daily time resolution by combining snow height
* and rain gauge measurements"</i>, Hydrology and Earth System Sciences Discussions, <b>10(7)</b>, 8683-8714, 2013.
*
*
* @author Mathias Bavay
* @date 2013-03-20
*/
const double GeneratorAlgorithm::soil_albedo = .23; //grass
const double GeneratorAlgorithm::snow_albedo = .85; //snow
const double GeneratorAlgorithm::snow_thresh = .1; //if snow height greater than this threshold -> snow albedo
......
......@@ -30,7 +30,7 @@
namespace mio {
/**
* @page generators Data generators
* @page dev_DataGenerator How to write a data generator
* Once the data has been read, filtered and resampled, it can be that some data points are still missing.
* These are either a few isolated periods (a sensor was not functioning) that are too large for performing
* a statistical temporal interpolation or that a meteorological parameter was not even measured. In such a case,
......@@ -38,72 +38,54 @@ namespace mio {
* cases, even fully arbitrary data might be helpful (replacing missing value by a given constant so a model can
* run over the data gap).
*
* Another usage scenario is to create new parameters fully based on parametrizations. In such a case, the "generator" is called
* a "creator" and behaves the same way as a generator, except that it creates an additional parameter. It is declared as
* {new_parameter}::%create = {data generators}.
* @section structure_DataGenerator Structure
* The selection of which data generator to use at any given time step, for a given parameter is
* performed by the DataGenerator class. This class acts as an interface, presenting a higher level view to the
* caller. The data generators themselves derive from the GeneratorAlgorithm class that standardizes their
* public API. An object factory creates the generator during intialization (keeping all constructed generators
* in a vector during the whole life time of the DataGenerator object), based on the strings contained in the user's
* io.ini configuration file.
*
* @note it is generally not advised to use data generators in combination with spatial interpolations as this would
* potentially mix measured and generated values in the resulting grid. It is therefore advised to turn the data generators
* off and let the spatial interpolations algorithms adjust to the amount of measured data.
* @note it is also possible to make a copy of a given parameter under a different name. This is explained in section
* \ref data_manipulations "Raw data editing".
* The API also defines two public "generate" methods, taking a meteorological parameter index (see MeteoData) and either
* a set of meteo data for one station and at one point in time or a meteo time series for one station.
* These methods walk through the meteo data looking for nodata values for the requested meteo parameter index.
* If the generator could successfully generate data for <b>all</b> the nodata values it found, it returns <i>true</i>,
* <i>false</i> otherwise. If <i>false</i> was returned, the DataGenerator object that manages the process would
* call the next data generator, <b>in the order that was declared by the user</b>. For a given meteo parameter, the
* whole process stops as soon as a <i>true</i> is returned or there are no more data generators to try
* (as declared by the user in his configuration file).
*
* @section generators_section Data generators section
* The data generators are defined per meteorological parameter. They are applied to all stations
* (if using multiple meteorological stations). If multiple dat generators are specified for each parameter,
* they would be used in the order of declaration, meaning that only the data points that could not be
* generated by the first generator would be tentatively generated by the second generator, etc. Please also keep
* in mind that at this stage, all data <b>must be in SI</b> units!
* @code
* [Generators]
* RH::generators = CST
* RH::Cst = .7
* @section implementation_DataGenerator Implementation
* It is therefore necessary to create a new class as two new files in the dataGenerators subdirectory (the implementation in the .cc
* and the declaration in the .h), nammed after the generator that will be implemented and inheriting GeneratorAlgorithm. You can
* start by making a copy of the dataGenerators/template.cc (or .h) that you rename according to your generator. Please make sure
* to update the header guard (the line "#ifndef GENERATORTEMPLATE_H" in the header) to reflect your generator name.
* Three methods need to be implemented:
* - the constructor with (const std::vector<std::string>& vecArgs, const std::string& i_algo)
* - bool generate(const size_t& param, MeteoData& md)
* - bool generate(const size_t& param, std::vector<MeteoData>& vecMeteo)
*
* P::generators = STD_PRESS
* The constructor is responsible for parsing the arguments as a vector of strings and saving its own name internally, for
* error messages, warnings, etc. It should set all internal variables it sees fit according to the parsed arguments. The
* goal is <i>to not</i> do any parsing anywhere else (for performances reasons).
*
* ILWR::generators = AllSky_LW ClearSky_LW
* The <i>generate(const size_t& param, MeteoData& md)</i> method compares <i>md(param)</i> with <i>IOUtils::nodata</i> and replaces
* it by its generated value if necessary. It returns <i>true</i> if no further processing is needed
* (ie. no replacement was needed or the replacement could be done) or <i>false</i> otherwise.
*
* TAU_CLD::create = CST
* TA_CLD::Cst = 0.5
* @endcode
* The <i>generate(const size_t& param, std::vector<MeteoData>& vecMeteo)</i> method compares
* <i>vecMeteo[ii](param)</i> with <i>IOUtils::nodata</i> for each timestamp in the vector and tries to generate data when necessary.
* If all missing data points could be generated (or if no data point required to be generated), it returns <i>true</i>,
* and <i>false</i> otherwise.
*
* @section generators_keywords Available generators
* The keywords defining the algorithms are the following:
* - STD_PRESS: standard atmospheric pressure as a function of the elevation of each station (see StandardPressureGenerator)
* - RELHUM: relative humidity from other humidity measurements (see RhGenerator)
* - TS_OLWR: surface temperature from Outgoing Long Wave Radiation (see TsGenerator)
* - ISWR_ALBEDO: ISWR from RSWR or RSWR from ISWR with either a snow or a soil albedo, depending on HS (see IswrAlbedoGenerator)
* - CST: constant value as provided in argument (see ConstGenerator)
* - SIN: sinusoidal variation (see SinGenerator)
* - CLEARSKY_LW: use a clear sky model to generate ILWR from TA, RH (see ClearSkyLWGenerator)
* - ALLSKY_LW: use an all sky model to generate ILWR from TA, RH and cloudiness (see AllSkyLWGenerator)
* - ALLSKY_SW: generate the incoming short wave radiation from the potential radiation, corrected for cloudiness if possible (see AllSkySWGenerator)
* - TAU_CLD: generate the atmospheric transmissivity based on cloud cover fraction (see TauCLDGenerator)
* - ESOLIP: generate precipitation from snow height changes (see ESOLIPGenerator)
* - PPHASE: generate precipitation phase with a user-selected method (see PPhaseGenerator)
* Finally, a new entry must be added in the object factory GeneratorAlgorithmFactory::getAlgorithm method at the top of file
* GeneratorAlgorithms.cc.
*
* @section generators_biblio Bibliography
* The data generators have been inspired by the following papers:
* - Brutsaert -- <i>"On a Derivable Formula for Long-Wave Radiation From Clear Skies"</i>, Journal of Water Resources
* Research, <b>11</b>, No. 5, October 1975, pp 742-744.
* - Prata -- <i>"A new long-wave formula for estimating downward clear-sky radiation at the surface"</i>, Q. J. R. Meteorolo. Soc., <b>122</b>, 1996, pp 1127-1151.
* - Dilley and O'Brien -- <i>"Estimating downward clear sky long-wave irradiance at the surface from screen temperature and precipitable water"</i>, Q. J. R. Meteorolo. Soc., Vol. 124, 1998, doi:10.1002/qj.49712454903
* - Clark & Allen -- <i>"The estimation of atmospheric radiation for clear and cloudy skies"</i>, Proceedings of the second national passive solar conference, <b>2</b>, 1978, p 676.
* - Tang et al. -- <i>"Estimates of clear night sky emissivity in the Negev Highlands, Israel"</i>, Energy Conversion and Management, <b>45.11</b>, 2004, pp 1831-1843.
* - Idso -- <i>"A set of equations for full spectrum and 8 to 14 um and 10.5 to 12.5 um thermal radiation from cloudless skies"</i>, Water Resources Research, <b>17</b>, 1981, pp 295-304.
* - Kasten and Czeplak -- <i>"Solar and terrestrial radiation dependent on the amount and type of cloud"</i>, 1980, Solar energy, 24.2, pp 177-189
* - Omstedt -- <i>"A coupled one-dimensional sea ice-ocean model applied to a semi-enclosed basin"</i>, Tellus, <b>42 A</b>, 568-582, 1990, DOI:10.1034/j.1600-0870.1990.t01-3-00007.
* - Konzelmann et al. -- <i>"Parameterization of global and longwave incoming radiation for the Greenland Ice Sheet."</i> Global and Planetary change <b>9.1</b> (1994): 143-164.
* - Crawford and Duchon -- <i>"An Improved Parametrization for Estimating Effective Atmospheric Emissivity for Use in Calculating Daytime
* Downwelling Longwave Radiation"</i>, Journal of Applied Meteorology, <b>38</b>, 1999, pp 474-480
* - Unsworth and Monteith -- <i>"Long-wave radiation at the ground"</i>, Q. J. R. Meteorolo. Soc., Vol. 101, 1975, pp 13-24
* - Meeus -- <i>"Astronomical Algorithms"</i>, second edition, 1998, Willmann-Bell, Inc., Richmond, VA, USA
* - Mair et al. -- <i>" ESOLIP–estimate of solid and liquid precipitation at sub-daily time resolution by combining snow height
* and rain gauge measurements"</i>, Hydrology and Earth System Sciences Discussions, <b>10(7)</b>, 8683-8714, 2013.
*
*
* @author Mathias Bavay
* @date 2013-03-20
* @section doc_DataGenerator Documentation
* The newly added data generator must be added to the list of available algorithms in
* GeneratorAlgorithms.h with a proper description. Its class must be properly documented, similarly to the other data
* generators. An example can also be given in the example section of the same file.
* Please feel free to add necessary bibliographic references to the bibliographic section!
*/
/**
......@@ -132,7 +114,8 @@ class GeneratorAlgorithm {
virtual void parse_args(const std::vector<std::string>& i_vecArgs);
const std::string algo;
static const double soil_albedo, snow_albedo, snow_thresh; //to try using rswr if not iswr is given
//These are used by several generators in order work with radiation data by looking at HS and deciding which albedo should be used
static const double soil_albedo, snow_albedo, snow_thresh;
};
class GeneratorAlgorithmFactory {
......
Supports Markdown
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment