WSL/SLF GitLab Repository

  1. 30 May, 2012 1 commit
    • Mathias Bavay's avatar
      A few "consts" have been added. The dates are now considered to be positive... · b79a85a4
      Mathias Bavay authored
      A few "consts" have been added. The dates are now considered to be positive which allows comparison operators to be optimized: converting 1 year of hourly data with data_converter got almost 30% faster. Moreover, when looking for resampling window, two extra checks have been added to quickly see if the start and end point are even in the input vector. This reduces data read miss (because we start by reading vecM[0]) and skips browsing through the whole vector if we are in a large data gap. On the same 1 year data set with one large gap for one parameter, the gain is of almost 20%. Overall, these two optimizations lead to almost a doubling of the speed with data_converter... Soon we'll be so fast that we will get the data before it is even measured!
      b79a85a4
  2. 16 May, 2012 1 commit
  3. 14 May, 2012 1 commit
  4. 11 May, 2012 3 commits
  5. 10 May, 2012 5 commits
  6. 08 May, 2012 1 commit
    • Mathias Bavay's avatar
      The re-accumulation was broken and producing bogus results. Moreover, its... · e8799828
      Mathias Bavay authored
      The re-accumulation was broken and producing bogus results. Moreover, its handling of nodata was sub-optimal (assuming nodata=zero lead to artificially low measurements for very noisy stations). This has been fixed. Accumulating over periods larger than in the original data set (ie:up sampling) has been validated. Down sampling has NOT been validated yet.
      e8799828
  7. 17 Apr, 2012 2 commits
    • Mathias Bavay's avatar
      Documentation, default value for WINDOW_SIZE=10 days, small speed... · 1a77eec7
      Mathias Bavay authored
      Documentation, default value for WINDOW_SIZE=10 days, small speed optimizations (1% faster!!), code cleanup to prepare the way for re-accumulation work
      1a77eec7
    • Mathias Bavay's avatar
      The resampling has been deeply transformed in order to produce more consistent... · 15f08c18
      Mathias Bavay authored
      The resampling has been deeply transformed in order to produce more consistent results. By doubling the data cut window, the resampling can now properly fill a 23 hours gap with a 24 hours window. The resampling now either fills the whole gap or do not fill anything (if the gap is too large). When activating "extrapolate", a gap is now filled by the sides instead of by the middle (which was producing weird results). The drawback is that the cut window had to be doubled, which is quite costly (going from 1 day to 10 days makes it 7.5 times slower, and going to 20 days makes it 15 times slower). However, this more logical and consistent behavior will allow the user to specify the window size that he needs instead of always picking something extremely large to be on the safe side. Finally, the resampling code has been cleaned up so that each algorithm is built along similar lines.
      15f08c18
  8. 14 Apr, 2012 1 commit
    • Mathias Bavay's avatar
      A new getMeteo() method has been implemented that transparently performs... · a54fb8a8
      Mathias Bavay authored
      A new getMeteo() method has been implemented that transparently performs spatial interpolations and returns a grid for a given parameter. This should be prefered to interpolate(), since it allows the reading plugin to directly provide gridded data (such as coming out of GRIB).
      Code cleanup in ResamplingAlgorithms, improved error message in A3DIO. Several keys have been updated/added to the plugins, in order to be more consistent between plugins. Therefore, meteo file extensions are given with METEOEXT, grid2d extensions with GRID2DEXT, grid2d prefix with GRID2DPREFIX (not all plugins support them, though), etc
      a54fb8a8
  9. 10 Nov, 2011 1 commit
  10. 07 Nov, 2011 1 commit
  11. 19 Sep, 2011 1 commit
    • Thomas Egger's avatar
      Performance boost: Redesign of class MeteoData. The individual public double... · 8d46d04d
      Thomas Egger authored
      Performance boost: Redesign of class MeteoData. The individual public double members like tss, ta, rh have been removed and instead of the rather complicated internal representation with the help of std::map and pointers, the MeteoData object now holds all its important information in one double vector (meteo data) and one string vector (parameter names). Thus the overloaded copy constructor could be slashed radically. This leads to huge performance gains, especially when dealing with huge vectors of MeteoData (which are needed for filtering, resampling, etc).
      
      Furthermore the access to the meteo parameters has been simpleified by overloading the operator(), replacing the old .param(size_t index) and .param(string paramname) functions:
      
      MeteoData md;
      double ta = md(MeteoData::TA);
      
      OR
      
      double ta = md("TA");
      
      These changes were propagated into all of MeteoIO.
      8d46d04d
  12. 23 Aug, 2011 1 commit
  13. 20 Jul, 2011 1 commit
    • Mathias Bavay's avatar
      Some methods have been moved into another class (like the simple linear... · 7797ea1c
      Mathias Bavay authored
      Some methods have been moved into another class (like the simple linear interpolation between two points that is now in ResamplingAlgorithms), some renammed (like the new weightedMean that replaces the ill-nammed "linearInterpolate"). The regression model based on two segments is now part of libinterpol1D and has been renammed. More regression models have been implemented. 
      
      The "initFit" method of Fit1D has been renammed in "fit" and the setGuess method now turns fit_ready to true, so that a user can force his provided parameters to be used by a given model (do a setGuess() with the parameters, then call f(x) without calling "fit", so no internal fit of the parameters would be performed).
      7797ea1c
  14. 19 May, 2011 1 commit
  15. 09 Mar, 2011 1 commit
    • Mathias Bavay's avatar
      In SNIO, the METAFILE can now be optional (which means that the metadata... · e9063b21
      Mathias Bavay authored
      In SNIO, the METAFILE can now be optional (which means that the metadata remain nodata, the user has therefore to fill them himself). This is used by Snowpack.
      
      After the Visual C++ compilation log from Raphael, some changes have been implemented: lots of warnigs about signed/unsigned fixed, some unreachable code removed, and some methods reimplemented for Windows. We are not there yet, but moving closer...
      e9063b21
  16. 08 Feb, 2011 1 commit
    • Mathias Bavay's avatar
      Another commit related to timezones... The Unix date always being in UTC, this... · 3fdc84f5
      Mathias Bavay authored
      Another commit related to timezones... The Unix date always being in UTC, this has been fixed. The resampling was not using dates in a proper way, this has been fixed. The GSN plugin has also been fixed, it now rounds time to the closest minute (in order to be consistent with our advertised "1 minute time resolution"). A rounding method has been implemented for dates.
      
      The timezone parameter in io.ini is now named TIME_ZONE instead of TZ for clarity. 
      
      Undefined dates should now be fully handled (ie: also in arithmetic).
      3fdc84f5
  17. 04 Feb, 2011 1 commit
    • Mathias Bavay's avatar
      The full support for timezones has been brought back. This fixes issue 71. The... · 904aeaa8
      Mathias Bavay authored
      The full support for timezones has been brought back. This fixes issue 71. The SMET, SMET, IMIS, BORMA plugins have been validated. Geotop should be validtaed if an input data set is available. A new class will have to be implemented to represent time periods. This is currently only a typedef pointing to the Date class, but a specific class would be welcomed.
      
      GSN has been fixed for some changes on the web service, but does not retrieve the correct timestamps (is this a GSN issue?). The GSN compilation has been improved: the warnings related to GSOAP have been removed. 
      
      A new long wave radiaiton model using cloudiness as input has been added (Omstedt, 1990). 
      904aeaa8
  18. 10 Dec, 2010 1 commit
    • Mathias Bavay's avatar
      The smoothing algorithms have been renammed as Process instead of Filter. The... · 11f5bc95
      Mathias Bavay authored
      The smoothing algorithms have been renammed as Process instead of Filter. The rate and the Tukey filters have been improved so that they correctly handles nodata.
      
      A new resampling algorithm has been introduced: "none". It does just that: nothing. This is an easy way to debug filters by not performing any temporal interpolations.
      
      The getMedian method has been re-implemented using the standard nth_element method for increased performances.
      
      The doxygen documentation has been updated to reflect the latest changes.
      11f5bc95
  19. 09 Dec, 2010 1 commit
    • Thomas Egger's avatar
      MeteoData now has an extra public member variable: StationData meta. It will... · 3f940ba5
      Thomas Egger authored
      MeteoData now has an extra public member variable: StationData meta. It will hold the meta information of the measurment instead of dragging along a second vector<StationData> alongside the vector<MeteoData> everywhere.
      
      This has huge implications: the IOInterface has been rewritten, subsequently changes need to be propagated into all plugins and all modules dealing with meta data.
      3f940ba5
  20. 05 Nov, 2010 1 commit
  21. 04 Nov, 2010 3 commits
  22. 25 Oct, 2010 1 commit
    • Mathias Bavay's avatar
      First (non working) implementation of the ordinary kriging. The variogram fit... · 9cbc0659
      Mathias Bavay authored
      First (non working) implementation of the ordinary kriging. The variogram fit still has to be done... But a few const issues have been fixed in Matrix and a new method added (to read a 1x1 matrix as a scalar).
      
      Fixed some doxygen warnings, added a howto for writing spatial interpolations.
      
      Better Oracle OCCI detection, renammed into FindOCCI.cmake and used by the ImisIO plugin compilation.
      9cbc0659
  23. 03 Oct, 2010 1 commit
  24. 22 Sep, 2010 1 commit
  25. 08 Sep, 2010 2 commits
  26. 07 Sep, 2010 1 commit
  27. 16 Aug, 2010 1 commit
  28. 20 Jul, 2010 1 commit
    • Mathias Bavay's avatar
      Several bugs have been fixed in Coords: the setEPSG was not working, the... · 3e406fd0
      Mathias Bavay authored
      Several bugs have been fixed in Coords: the setEPSG was not working, the altitude was not displayed with the << operator and the formatting of decimal_to_dms was broken in its handling of seconds...
      
      The documentation for the resampling has been expanded and several other modules' documentation have been updated so their structure is built along the same lines. 
      
      An error message in IOHandler (when failing to load a plugin) was not clear and has been changed.
      
      Several bugs in SMEIO have been fixed (improper column names mapping, improper handling of easting/northing, unclear error message).
      
      The example data set has been tagged as GMT+1 and the meteo_reading and 2D_interpolations modified accordingly (so the argument read on the command line is assumed to be in GMT+1 too)
      3e406fd0
  29. 07 Jul, 2010 1 commit
  30. 26 Jun, 2010 1 commit
    • Thomas Egger's avatar
      The new filtering and resampling framework. Filtering/Processing and... · c2435a6a
      Thomas Egger authored
      The new filtering and resampling framework. Filtering/Processing and Resampling have been separated from each other, the Resampling is now done with the class Meteo1DInterpolator which utilizes the static algorithms within ResamplingAlgorithms. The filtering remains a task of class MeteoFilter. The calling of the resampling and filtering as well as the data selection is now based in the facade class MeteoProcessor. MeteoProcessor::processData function calls the filtering algorithms, then passes the filtered data to the resampling algorithms and finally runs a "check only" filtering pass.
      
      The filtering system underwent some changes in order to simplify the tasks at hand, the interface has changed.
      
      The RateFilter is currently disfunctional, needs to be reimplemented.
      c2435a6a