The GridBuffer class has been expanded and is now used by Meteo2Dinterpolator for interpolated grids

The virtual stations have been re-enabled. A new generator has been implemented to generate ISWR from RSWR or the opposite. Some documentation has been improved. The Black Globe Temperature index should now work.

In order to make it more modular and allow more advanced processing (virtual stations, downscaling, multiple io sources) the IOManager is now a proper wrapper: the processing is fully delegated to either a GridsManager or a TimeSeriesManager. These can in turn be reused at some other places (like in the Meteo2DInterpolator) to offer full buffering and processing of "local" time series. The buffering is now done by these two new classes (therefore the BufferedIOHandler class has been removed) and it is planned that dataClasses will be implemented for proper ringbuffers (one for time series, one for grids and one for isolated points) and receive the generic buffering code later on. 

A new fastPow(double, int) has been implemented for positive exponents. The begining of a generic handling of units also appeared in IOUtils. Finally, the ProcessingLevels are now defined in IOUtils instead of IOManager.

When caching interpolated grids, the info string was not cached, leading to empty info strings when reading from cache. Now the info strings are properly cached.

Removing nodata values seemed like a good idea when doing a sort(), but this created some bugs (sometimes it is still valuable to know that y=nodata after sorting). This is now an optional behavior. The interpolated grids are now buffered in Meteo2DInterpolator. The grid buffering code has been cleaned up and backported to BufferedIOHandler. The spatial interpolations specific points (for virtual stations, etc) has been moved to Meteo2DInterpolator in order to make IOManager smaller and more focused on general data flow handling.

A new spatial interpolation algorithm has been implemented: "NONE". This just fills the grid with nodata. Otherwise, an unnecessary check has been removed in Meteo2DInterpolator and some comments editing, white spaces...

Fixing some warnings for Visual C++ and also using ostringstream instead of stringstream when appropriate

More size_t fixes (ie trying to be consistent) and a bug was found when spatially interpolating using a few specific algorithms that would not reset their internal vectors between two calls (thanks Matteo for helping uncover this bug!)

Now the toString() method of Meteo2DInterpolator works with the new structure and is called by IOManager.toString(). Small cleanup (white spaces, etc) in 2D_interpolations and ResamplingAlgorithms.

Finally, here we are: the spatial interpolation algorithms are constructed only once and remain as long as the Meteo2DInterpolator remains (generally, as long as IOManager remains). This means that an interpolation algorithm can now keep its own copy of whatever data it wants (for example, dem) to tweak it between calls.

One more step closer to the goal: the dem is now only passed when calling calculate() and the simpleDEMWindInterpolate now computes the curvatures if necessary.

Small optimizations in IOHandler.

Getting closer to the goal of persistent spatial interpolation objects. There are no "initialize" and "getQualityRating" anymore, the initializations are performed by "getQualityRating". The arguments that used to be passed to the constructors are now split between constructor and getQualityRating. A few hacks will have to be removed: we should be able to get rid of some of the callbacks, the plain copy of Date and Dem will have to be avoided (most probably by working with pointers) and the map of objects that has been implemented (but commented out) will be enabled. So, a few hacks remain, but this is already a nice, big step forward!

The structure of Meteo2DInterpolator has been brought closer to the structure of DataGenerators. More methods are static.

In order to implement an efficient and portable object serialization, it has been decided that all serializable objects would be manually serialized to an std::iostream. Then, the parallelization framework (mpi, popc, etc) can take over this stream and move it to another node. Therefore, the operators "<<" and ">>" are now reserved for serialization (as seems to be standard) and the debug outputs are obtained by printing "object.toString()". This means that all debug code has potentially to be fixed, by replacing "std::cout << object" by "std::cout << object.toString()".

The METEO_TIMSERIES and STATION_TIMESERIES have been renamed in METEO_SET and STATIONS_SET in order to better reflect their content. 

The concept of data generators is introduced. A new section has been created [Generators] to define how to generate data for each parameter when everything else failed. Soon, you will be able to generate ILWR directly by declaring it in your io.ini!

Putting the code into shape for major changes in the spatial interpolations as well as pushing the minor version nunmber up

My previous commit was terrible... it simply made it impossible to define optional keys. This has now been fixed but required a change in the API. So, don't use Config::nothrow but IOUtils::nothrow instead.

In order to try to be closer to the spirit of the STL, now the empty strings/vectors/lists are tested with .empty(), the first/last elements of vectors accessed with front()/back(), the empty strings are not contructed with string("") but with string(), etc This makes a benchmark based on meteo_reading ~4% faster and this makes the code suprisingly easier to read. 

Some extra constification took place as well as some minor code cleanup (optimizing the comparison between two vectors in order to perform less tests, etc).

The Date class can now properly generate and parse full ISO timestamps, including time zone information. The full specification is now supported by convertString() with a helper method in the Date class. Ouputing an ISO string with its time zone is done by requesting the ISO_TZ format.

Lots of small fixes after running cppcheck on Windows: unused variables, use ++i instead of i++ for complex types (ie mostly iterators for us) for more performance, a few variables that could be passed as const&, use vector.empty() instead of vector.size==0 (for clarity and potentially performances).

After compiling with -Weffc++, it appeared that lots of things could be done in initialization lists. Some of it has been done for this commit (but quite a few remain).

Fixed some spelling mistakes in the tests, improved USE_IMIS_HNW comment in ImisIO, improved the serailize method in Meteo2DInterpolator (but it is not working yet and anyway of little interest), fixed a casting warning in MathOptim. Some warnings are now turned off for POPC.

The smoothing filters now behave differently depending on the centering that has been specified. The IOManager backreference for Meteo2DInterpolator (pointer this given in the initialization list) has been replaced by a setter.

An invalid flag for Visual C++ has been fixed.

2d interpolations: changed verbosity for the point wise 2d interpolations (only one message will be printed for all points), moved the printing of the info_string from Meteo2DInterpolator to the IOmanager.

Because POPC does not properly support std::cout and requires cout instead, this has been fixed in the code, with the necessary "using namespace" added. Only one std::cout remains (in IOUtils) and will be removed if absolutely necessary later.

The OLWR hack for Norway has been implemented in SMETIO (a cleaner, long term solution will still have to be developed). Various types issues have been identified by Visual C++ and should have been fixed.

The VW_MAX field was not properly handled in SMETIO. The failure count has been slightly improved in SNIO (to decrease the number of failures when a model could recover: if ILWR is not provided but TSS is, we don't count it as a failure anymore). 

The implementation of the krigging is moving forward. A bug has been found and fixed in Fit1D, a variogram is now computed (but not using a true covariance, which prevents our krigging from properly working). 

The #define NOSAFECHECKS have been deleted and replaced by a definition by the CMakeLists when in debug mode. This means that when compiling in debug mode, the code will be quite slower but will properly check and report out of bounds access in tables and matrix.

Fixing some unsigned int / size_t conversion issues by using more widely size_t (as it seems to be recommended and done in the STL)

The emissivity models have been cleaned up: they now take RH, TA, etc but NOT the water vapor pressure (it is internally calculated). Please, update your code!

A specific spatial interpolation algorithm has been introduced for ILWR that relies on emissivity (we convert ILWR to emissivity, then interpolate the emissivity, then reconvert to ILWR). As mentionned in issue 156, a better method for interpolations by proxies will have to be designed and implemented...

The SMET specification has been changed and brought to version 1.1. The units_multiplier and units_offset order has been changed: now, the multiplier is applied FIRST and then the offset. This allows to disable a SMET column by setting the mutiplier to 0 and the offset to nodata. A warning is displayed by SMETIO for all files version <1.1 urging users to check their files and update them to 1.1.

Code cleanup in IMISIO, grid constraint for wind that it must be greater than 0 in Meteo2DInterpolator.

The custom grids spatial interpolations method is now working! The user can force loading his own grids for specific parameters and timesteps.

The (advanced) configuration of the BufferedIOHandler through io.ini (General section) has been added. It currently introduces two keywords (BUFF_CHUNKS_SIZE and BUFF_CHUNKS) to drive how much data to read at once and how much data to buffer. Currently, their product is directly used in the read, thus not reading anything by chunks smaller than a full buffer.

Fixed some << outputs, two compilation warnings found on Hera as well as introduced.

The definition of the "<<" operator has been written for IOManager and made more consistent for others

Added functionality to the Meteo2DInterpolator: it is now possible to access the iomanager and hence to access historical data points from within any spatial interpolation.

The IOManager has been expanded to:
1) cache already filtered data points and
2) provide a getAvgSamplingRate() function that returns the average data points seen per day.

The documentation and examples have been updated to reflect the latest changes in MeteoIO (namely: the merging of MeteoData with StationData). A documentation bug has been fixed, that involved deleting the doxygen comments for the commented "clone()" method in IOInterface. Since this is not used anywhere and has been commented out for a while, this method has been fully deleted.

Meteo2DInterpolator now has its own << operator for debugging purposes.