Commits · c0ff5b87c81bd69ef49e84ab931cd9ca23354546

Trying to close some potential security flaws (using FlawFinder). All files opened for reading are now checked and special files (devices, etc) are refused.

Data packing has been implemented (packing from "double" to "int"). The loss of precision is fully negligible with practical data (1/10 of a mm on an Europe-wide DEM for example) while leading to 50% smaller file sizes.

The new parameters/tables definitions have been taken into account in order to solve the problem of radiations not being properly read any more. This should now work...

Grids can not properly be written out. The data packing is not supported yet and the units handling is still not finished (accumulated values are tricky to handle from within MeteoIO).

The reading support is now more generic: the time units are read and parsed from the file. The units are always corrected when necessary. If no date is provided and the file only contains the grid at one time step, it will be read by default. The DEM extraction from pressure data now works properly for any file (ie no more hard coded dates).

Removed some commented out code, improved some documentation and recomputed a test that had been invalidated by changing the implementation of the HNW_Snow spatial interpolation method.

Some doxygen comments have been improved, A few physical constants have been added. The DEM extraction from pressure fields in NETCDFIO has been improved (the previous calculation was mostly wrong).

Now the ISO year can be returned alongside the ISO week number. The ISO_WEEK print format properly uses it, making its output standard compliant. The issue 466 can now be closed!

A bug in handling the seconds had been found a while ago by Aurélien. His fix is now committed. After a long fight, the ISO week number now works, addressing issue 466 (although it might be expanded in order to also provide the matching year since last/first days might belong to the year after/before).

All the PNGs that we written were corrupt: the first line of data was missing. This has been fixed. for NetCDF, the ability to calculate the altitude from the pressure (and sea level pressure) has been added with a controlling key. This is not very precise but this could help for troubleshooting problems when no DEM has been provided (such as with ECMWF data sets).

A new date format has been added: RFC868. Two MeteoGrids have been renamed: ISW_DIR as ISWR_DIR and ISW_DIFF as ISWR_DIFF while a new grid has been declared (P_SEA). A few new constants have been declared. Some new links into the GRIB documentation as well as support for TAU_CLD. The NetCDF support is much improved, although there are still things left to do (writing grids out has been disabled for the moment, the parsing of the time units is currently not done, there is no support for automatically getting the data out of multiple files in a directory, etc).

Several warnings have been fixed. The (slightly) wrong geolocalization calculation such as found in fixed in NetCDF has also been fixed in GRIB. A bug that was preventing GRIB DEM from being read has been fixed.

The CNRM plugin can now be enabled and properly works. Its documentation has been updated and improved. The NetCDF plugin can now read DEMs using various schema/conventions and its documentation now matches the code. A few more MeteoGrids have been defined that are required for meteorological model outputs and for some future spatial interpolations (ISW_DIR, ISW_DIFF for example). The PNG plugin creates a few more EXIF fields in order to make it very easy to integrate in google earth (but it seems that google earth does not like indexed pngs). A bug computing the geolocalization of grids coming from NetCDF has been fixed (all grids were slightly off).

In order to make it easier to support ECMWF and CF1 NetCDF grids, there are now two NetCDF plugins: one for the CNRM schema (focusing on the meteo time series) and one for grids (NetCDFIO). The later is still a copy of CNRMIO for the moment, but this will change. At some point, both plugin might be merged again.

In the precipSnow spatial algorithm, one of the original steps was missing: setting the mean back to its original level.

The original PrecipSNOW spatial interpolation algorithm has been reimplemented since the version implemented by Rob is unreliable

Improved the "usage examples" section of the documentation (some of the examples were way too old), added some advices

fixed a bug when computing the amplitude with the "daily_avg" resampling algorithm. Very minor improvement to the MeteoData::merge method.

A new Date print format has been added (ISO_DAY). The "daily average" resampling now properly works with avg, avg+min or avg+max using the user provided range as a fallback.

Some code cleanup into the MeteoData::merge methods. A new method has been implemented to merge all stations close to each other within a single vector<MeteoData>. Practically: several data sets from the same location can then be read and merged on the fly by a call to MeteoData::merge(vec).

A new "NOISE" filter has been added, to add noise to the inputs. An improved check and error message has been implemented in FilterSuppr. An updated reference added to the ESOLIP generator.

It is now possible to only remove a specified fraction of a data set with the SUPPR filter (at random)

Added a range check on RH before computing the dew point temperature in the RH spatial interpolation method. Renammed and documented the ESOLIP precipitation generator after adding the reference.

fixed another bug reading files in a directory on Windows. I never thought this could be so tricky...

On some Windows versions, reading a directory was failing, simply reporting "could not open directory". This seems to have been dependent on the character encoding of the system. This has been solved by forcing the use of ANSI calls for the Win API.

Fixed a very stupid bug in Undercatch_WMO when dealing with 5 arguments (bad copy/paste). Thanks Matteo!

Cleaning up the CMake files from unecessary things, proper support for x86_64 (it needs to be named as "amd64" on Debian), a few extra options for the cpack generators.

Small tweaks for Windows: no version information in path anymore but in the name as it appears in "add/remove software"

The ALPUG plugin now uses a better way of validating the provided station_ids, by checking if at least one file matching the station ID can be found in the METEOPATH. If not, most probably the station ID is wrong. If yes, then some years are simply missing (and no error message or warning is generated). The silent skip of duplicates is now commented out in order to shame the owners of messed up data.

The warnings have been fixed. The ALPUG plugin now supports metadata (a specific file format has been designed for it), supports duplicates in the data files (since this seems to happen quite often) with either no reporting (for limited number of duplicates) and reporting a warning for the begining of a large duplicated block of data. The plugin also now handles better missing files: if some files have already been found for this station, a warning is printed out but the execution continues. If no single file was found for a given station, an exception is thrown. The documentation has also been written.