Commits · 8528f36d14ca8eb6a55e547fda51e3720bc902df

A small bug has been found and fixed in the Gradient class (when a data range is zero). A few extra output specifications have been added to PNGIO.

PNGIO now compiles with libpng 1.5 (as well as 1.4). Names demangling has been implemented in IOExceptions, making the backtrace much friendlier! (but this only works for gcc)

Oups, fixed a few mistakes in io.ini: PNG should not be enabled by default (since this is an optional plugin) and one PNG key was spelled wrongly...

Fixed a legend labelling issue (the zero was some times appearing as a very small number) and a legend under/over range issue for heat (that was not checking that the data was in [0,1])

Fixed the PNG world file (so that we return the coordinates of the center of the cell) and updated the examples so that they perform correctly with the latest version

Speed improvement for IDW based on the profiling performed by HES-SO's V. Grivel: the stations' positions are saved in two vectors (eastings and northings) before walking the grid. For low number of stations, this makes no visible difference. For large number of stations, this makes a huge difference (as seen on the Graubünden simulation, with 35 stations).

After playing with pngcrush, a new filter has been enabled for the PNG compression. A specific compression strategy for deflate is also now enabled for indexed images. This keeps run time identical but offers a significantly smaller file size (compared to the full color, initially generated PNG of the last benchmarks, it now shows a 40% run time reduction for the whole loading+png output and 87% size reduction).

Indexed gradients are now handled by the Gradient class and used by the PNGIO plugin to generate indexed PNGs (some consts in the code can toggle it back to non-indexed images). This leads to a significant speedup and large file size reduction. On a DEM test case, while reading a 2D ARC grid and writing it back, the following have been measured:

After commit 758, support for indexed PNGs. This leads to major speed and size improvements:

version                  duration                   file size
full color                 0.530                       617k    
reduced to 30        0.460   13%             156k   75%
indexed, 30           0.352   34%               90k   85%
indexed, 30           0.320   40%             123k   80%
(optimized for speed)

The "freeze" color gradient now has a fixed point at zero celsius. A pointer error (related to the FILE pointer) has been fixed in PNGIO by making this pointer a class member. A much improved error message is now generated when a file can not be opened.

Introducing smart legends: this tries to find a meanigful range and steps within that range. The choice of rounding the lower and upper bounds (thus not guaranteeing that the whole data fits in the legend) has been made as it could lead to more suitable legends.

Changing the case of A3D_VIEW in ARCIO, fixed an image corruption issue in PNGIO (libpng just *forgets* to flush the file buffer...). Added a missing include in MeteoIO.h

A new key has been added that was missing: GRID2DPATH that can be in intput and output. This key is now properly handled by the appropriate plugins. A new color reduction strategy has been implemented in the gradients and the PNG plugin: instead of reducing the colors coming out of the gradient, the values coming in are discretized. This leads to much more effective size reduction for the same output quality, with 5% speed increase compared to the original code not doing this reduction (test: reading and writing back a grid). The file size is now 1/3 of the full color file.

A few hard coded parameters have been declared as static const (color depth per channel, max color per channel) in order to make the code cleaner. A new method has been introduced in the Gradient class: setNrOfColors. This allows to specify a reduced number of unique colors (even if still coded on 8 bits per channel). By using this call in PNGIO (with 8000 unique colors), the output remains visibly almost the same (only some very slow varying parts see a flat color instead of a slow gradient) while the run time is reduced by up to 5% and the file size more than halved.

The gradients now don't provide a transparency value (since it had to be either fully opaque or fully transparent) but only a boolean. PNGIO now uses a transparent color to encode transparency instead of a true alpha channel, leading to improved run times and smaller file size. The drawback is that pure white is now the transparent color.

A new gradient type has been added (blue-green isomorphic) and some small things improved thanks to valgrind (one potential out of bounds read and with callgrind some small speed improvements)

New methods "moveByXY" and "moveByBearing" have been added to Coords, addressing issue 39. Code cleanup and extensive documenting have been performed on Graphics and PNGIO, that now supports creating world file (optionnally). A new option has also been added to ARCIO, "A3D_view", to generate file names compatible with the Alpine3D's grid viewer (false by default). A new function to remove the extension from a file name has been implemented in IOUtils.

Better handling of "autoscale": the min and max are almost always taken into account (this is usefull to set fixed points, like the snow line, the sea level, etc) and the autoscale bool is used to tweak a given gradient for better display in autoscale (ie: removing the snow line in autoscale). The client is then responsible for provided the relevant min and max that he wants to use (constant -> no autoscale or variable -> autoscale).

Now, the plots are fitted/optimized depending on the meteo parameter (if available). The proper metadata is written into the file (again, if available). More gradients have been implemented.

Now the color gradient can be chosen (given as an argument in the Gradient constructor). This is not yet really used by PNGIO, but this will come!

The legend class is now free of hard-coded values in the middle of the code (all depends on the static consts).

The NearestNeighbour 2D resampling algorithm has been reshaped in order to offer the same interface as BiLinear. A typo in its name has been fixed. But NearestNeighbour does not work properly (bug not found yet)

A better way of submitting metadata to libpng has been implemented (ie: two vectors of strings that are then converted to char* to be given to libpng). The plugin basic options have been implemented, including size specifications. 

Some fonts improvements.

The purely graphical parts of PNGIO have been isolated in a new file, Graphics, that contain the Legend class, a Color namespace and a Gradient class. The gradient code is not so clean yet, but the PNGIO plugin is now much cleaner. The color gradients will actially need some serious re-actoring and design in order to be both flexible and efficient (the odds are, it would be based on a object factory and several derived classes like for libFit1D).

The 2D_interpolations example has been updated to reflect recent changes (see previous commit). Improvements in the legend for PNGIO.

This addresses the 2D part of issue 168: the ability to specify which meteo parameter and time step to read/write from/to a 2D grid. This should be the prefered reading method in the future, since it leaves to the plugin the task of building the file name, extension, database table, etc This also provides added metadata and more control over the writing process.

The legend text has now its own color and the legend background can also have its own color (or remain transparent).

Much improved legends: the font size is twice as big, the verticals are bold, the exponent is half-height. And spacing is more pleasant.

The png plugin can now generate a legend. This is still very primitive, the code is not so nice, but it compiles cleanly and works. Looking forward for the next version!

First version of a PNG plugin. It only writes a grid for now, and nothing is user configurable. In the process, a small bug has been found in the RH spatial interpolation method (not handling nodata correctly)

A lot of writes were using std::endl for end of line, which forces a buffer flush. This lead to a visible performance cost (as shown by the profiling project made in collaboration with HES_SO). These have been replaced by "\n" that lets ostream manage its buffer as it wants.

More error checking: the offsets and multipliers must match the number of fields and the number of fields must match the true number of columns at each line

The imisio USE_IMIS_HNW option is getting simpler... winter condition detection is not applied anymore (leaving the application do it, or an extra filter be implemented for it later).

Very small changes (ie: excluding exception backtrace) so that MeteoIO compiles for Android (follow the "Getting-started" documentation page if you're want to witness it by yourself!). Basically, android-cmake adds a new plateform variable (named "ANDROID", how surprising) that we use to skip the backtrace and its include.

Added parameter check to the HNW_MELT filter and changed its handling of filtered points: since when we can diagnose that the non-zero measurements are coming from melt in the sensor, we now set the measured value to zero. It gets set to nodata only when the melt conditions could not be established (ie: missing measurements on RH, TA, etc). We will have to see if it really makes sense by using it for a while...

The "RateFilter" filter has been renammed "FilterRate" in order to be more consistent with other filters (this does not change anything user-side). A new filter has been created, HNW_MELT. This applies the Snowpack criteria on a precipitation measurement: if relative humidity is not high enough or TA and TSS too far apart, it is considered that any measured precipitation comes from snow melting in the (unheated) rain gauge, therefore the measurement is deleted.

The last occurences of NROFSTATIONS have been deleted in BormaIO, closing issue 163.

Some code cleanup took place in CosmoIO and FilterMax.

In IMISIO, a new option and behavior has been implemented to properly manage precipitation measurements: ANETZ stations keep their HNW (they are considered reliable). IMIS stations by default don't produce HNW anymore. A key (USE_IMIS_HNW) can force using them, but a detection of winter conditions (ie: TA>0 and TSS>0) removes measurements known to be invalid. If neither TA nor TSS are available, then HNW is deleted (in order to prevent injecting HNW=0 in the system during a freeze event if the station would happen to miss TA and TSS). USE_IMIS_HNW should always be followed by the HNW_MELT filter.