Now the dates can be rounded to the day (there was a bug preventing it before). A new parametrization has been implemented in SunObject that computes the daily sum of TOA radiation. This is used by a new resampling method that computes subdaily values out of a daily sum of radiation. This is not yet doing such a great job, but this is a start...

A new test has been created to cover basic statistical functions on vectors. Two bugs have been found and fixed when computing the derivative on a vector of data (the right derivative was in fact a left derivative and the case of X containing nodata was not covered).

Documentation fixes in libinterpol2D and ResamplingAlgorithms. The cmake macro for finding MeteoIO now looks first in the home directory of the user on Mac (as is done on Linux).

Now the toString() method of Meteo2DInterpolator works with the new structure and is called by IOManager.toString(). Small cleanup (white spaces, etc) in 2D_interpolations and ResamplingAlgorithms.

Renaming once more the nearest neighbor algorithm: it is now "nearest". That should be both quick and easy.

This adresses issue 173: now the window_size can be defined per meteo parameter and per resampling algorithm. The Interpolations1D section's syntax has been changed in order to accomodate the change and enable specifying multiple resampling algorithms in the future. An exception is thrown when using the old syntax (until we would consider that everybody should have migrated). The nearest_neighbour algorithm key has been renamed to n_neighbor. 

Overall, this commits breaks compatibility with previous ini file but the calculated results remain the same after migrating the config files.

In order to avoid re-parsing the resampling parameters for every point, the resampling algorithms have been restructured as classes inheriting from an interface class alongside an object factory. This should slightly increase the performances of resampling and clears the way (90% of the work being done) for a per-parameter window size (issue 173).

A useful link has been added to the comments of Sun and some comments improved in DataGenerators

Removing some unnecessary pointers (for things like "const double a = vecM[ii](param)") and replacing a vector push_back by a proper allocation in constructor followed by assignments

A rounding bug has been found in the Date class: when using a Date object in a loop where it gets incremented, it accumulates errors and after ~70000 iterations, the error would get bigger than one second, making date comparisons with a fixed date impossible. This has been solved by always rounding the internal gmt Julian date to the closest second, after each assignment (from the constructors, the setDate() calls or any arithmetic operation). Moreover, the rounding methods have been re-written to be more efficient as well as other roundings (specially when computing the date decomposition from Julian). 

The seek() call performing a binary search has also been simplified, in order to remove unnecessary tests, put some similar tests together and make the code clearer. 

Some constification and better vector usage took place in ResamplingAlgorithms and Meteo1DInterpolator.

Some extra documentation has been added for the UPS coordinate system. The coordinates test has been expanded. A missing default value has been added to the Suntrajectory.

An optimization from Yesterday was actually breaking the whole resampling... This has been fixed (thanks to Nander and David for finding this problem!)

A few "consts" have been added. The dates are now considered to be positive which allows comparison operators to be optimized: converting 1 year of hourly data with data_converter got almost 30% faster. Moreover, when looking for resampling window, two extra checks have been added to quickly see if the start and end point are even in the input vector. This reduces data read miss (because we start by reading vecM[0]) and skips browsing through the whole vector if we are in a large data gap. On the same 1 year data set with one large gap for one parameter, the gain is of almost 20%. Overall, these two optimizations lead to almost a doubling of the speed with data_converter... Soon we'll be so fast that we will get the data before it is even measured!

Fixed a division by zero in libsmet and the "none" resampling was broken (it was deleting all data instead of propagating available data). Fixed!

Investigating issue 210, it has been found that the IMIS precipitation had a division by 2 that was not required (halving the precipitation compared to what it should have been). This has been fixed.

Small documentation improvement in ResamplingAlgorithms.

A new option has been added to the accumulate interpolation: "strict". When this is active, if one of the accumulated value is nodata, the whole sum will be set to nodata. Without this option, all values that can be aggregated are aggregated (ie: best effort). When using multiple stations, it might make sense to enable "strict" in order to avoid polluting the spatial interpolation with potentially under-estimated values.

A documentation issue has been fixed in Meteo1DInterpolator.

The re-accumulation should now work properly for down sampling (tested with data at h+40 converted to h+00 and with 10min data converted to hourly).

Fixed some warnings (about unused variables/parameters). Added a new position for ResamplingAlgorithms: begin to describe that a point is at the begining of the data set (so we don't have to look for this condition again in the algorithm).

Bugfix: The new linear resampling always extrapolated at the end of the meteo data vector, this is fixed now.

TODO: verify and comment

Further improvements for the resampling within the Meteo1DInterpolator: no longer is an element inserted into the filtered_cache and then deleted, instead we have a MeteoData reference handed to the Interpolator representing the MeteoData element to be resampled. This interface change needed to be propagated into the different affected classes. Accumulate resampling currently disabled.

The re-accumulation was broken and producing bogus results. Moreover, its handling of nodata was sub-optimal (assuming nodata=zero lead to artificially low measurements for very noisy stations). This has been fixed. Accumulating over periods larger than in the original data set (ie:up sampling) has been validated. Down sampling has NOT been validated yet.

Documentation, default value for WINDOW_SIZE=10 days, small speed optimizations (1% faster!!), code cleanup to prepare the way for re-accumulation work

The resampling has been deeply transformed in order to produce more consistent results. By doubling the data cut window, the resampling can now properly fill a 23 hours gap with a 24 hours window. The resampling now either fills the whole gap or do not fill anything (if the gap is too large). When activating "extrapolate", a gap is now filled by the sides instead of by the middle (which was producing weird results). The drawback is that the cut window had to be doubled, which is quite costly (going from 1 day to 10 days makes it 7.5 times slower, and going to 20 days makes it 15 times slower). However, this more logical and consistent behavior will allow the user to specify the window size that he needs instead of always picking something extremely large to be on the safe side. Finally, the resampling code has been cleaned up so that each algorithm is built along similar lines.

A new getMeteo() method has been implemented that transparently performs spatial interpolations and returns a grid for a given parameter. This should be prefered to interpolate(), since it allows the reading plugin to directly provide gridded data (such as coming out of GRIB).
Code cleanup in ResamplingAlgorithms, improved error message in A3DIO. Several keys have been updated/added to the plugins, in order to be more consistent between plugins. Therefore, meteo file extensions are given with METEOEXT, grid2d extensions with GRID2DEXT, grid2d prefix with GRID2DPREFIX (not all plugins support them, though), etc

A bug was fixed in SMETIO: when given OLWR, it should have computed TSS, but it was not... (using the wrong MeteoData object).

Documentation and error messages improvements. A new method was added to Atmosphere: virtualTemperatureFactor.

Performance boost: Redesign of class MeteoData. The individual public double members like tss, ta, rh have been removed and instead of the rather complicated internal representation with the help of std::map and pointers, the MeteoData object now holds all its important information in one double vector (meteo data) and one string vector (parameter names). Thus the overloaded copy constructor could be slashed radically. This leads to huge performance gains, especially when dealing with huge vectors of MeteoData (which are needed for filtering, resampling, etc).

Furthermore the access to the meteo parameters has been simpleified by overloading the operator(), replacing the old .param(size_t index) and .param(string paramname) functions:

MeteoData md;
double ta = md(MeteoData::TA);

OR

double ta = md("TA");

These changes were propagated into all of MeteoIO.

The OLWR hack for Norway has been implemented in SMETIO (a cleaner, long term solution will still have to be developed). Various types issues have been identified by Visual C++ and should have been fixed.

Some methods have been moved into another class (like the simple linear interpolation between two points that is now in ResamplingAlgorithms), some renammed (like the new weightedMean that replaces the ill-nammed "linearInterpolate"). The regression model based on two segments is now part of libinterpol1D and has been renammed. More regression models have been implemented. 

The "initFit" method of Fit1D has been renammed in "fit" and the setGuess method now turns fit_ready to true, so that a user can force his provided parameters to be used by a given model (do a setGuess() with the parameters, then call f(x) without calling "fit", so no internal fit of the parameters would be performed).

In SNIO, the METAFILE can now be optional (which means that the metadata remain nodata, the user has therefore to fill them himself). This is used by Snowpack.

After the Visual C++ compilation log from Raphael, some changes have been implemented: lots of warnigs about signed/unsigned fixed, some unreachable code removed, and some methods reimplemented for Windows. We are not there yet, but moving closer...

Another commit related to timezones... The Unix date always being in UTC, this has been fixed. The resampling was not using dates in a proper way, this has been fixed. The GSN plugin has also been fixed, it now rounds time to the closest minute (in order to be consistent with our advertised "1 minute time resolution"). A rounding method has been implemented for dates.

The timezone parameter in io.ini is now named TIME_ZONE instead of TZ for clarity. 

Undefined dates should now be fully handled (ie: also in arithmetic).

The full support for timezones has been brought back. This fixes issue 71. The SMET, SMET, IMIS, BORMA plugins have been validated. Geotop should be validtaed if an input data set is available. A new class will have to be implemented to represent time periods. This is currently only a typedef pointing to the Date class, but a specific class would be welcomed.

GSN has been fixed for some changes on the web service, but does not retrieve the correct timestamps (is this a GSN issue?). The GSN compilation has been improved: the warnings related to GSOAP have been removed. 

A new long wave radiaiton model using cloudiness as input has been added (Omstedt, 1990). 

The smoothing algorithms have been renammed as Process instead of Filter. The rate and the Tukey filters have been improved so that they correctly handles nodata.

A new resampling algorithm has been introduced: "none". It does just that: nothing. This is an easy way to debug filters by not performing any temporal interpolations.

The getMedian method has been re-implemented using the standard nth_element method for increased performances.

The doxygen documentation has been updated to reflect the latest changes.

MeteoData now has an extra public member variable: StationData meta. It will hold the meta information of the measurment instead of dragging along a second vector<StationData> alongside the vector<MeteoData> everywhere.

This has huge implications: the IOInterface has been rewritten, subsequently changes need to be propagated into all plugins and all modules dealing with meta data.

Small documentation/indentation update. CPack can now create source packages, by typing "make package_source"

The ImisIO plugin now is fully capable of dealing with the ANETZ precipitation data in the "old" way. Furthermore the wind drift data is read from the correct stations and added to the MeteoData as VW_DRIFT and DW_DRIFT.

There was a small bug in the accumulation resampling algorithm, sometimes turing the sum into a negative number, this should be fixed now.

Fixed a few trivial compiler warnings, indenation.