GitLab

The units conversionm now work for 'simple' units, with prefix and exponents. 'Composite' units (such as 'K/m' or 'N/m2') are currently not supported.

Finally adressing issue 72 with second resolution in dates. The ISO strings now contain the seconds when written out and by reading them, the seconds can be provided (but this is not mandatory). In ARC, grids are now named with the seconds (except when enabling the Alpine3D naming scheme). The adresses a bug that was found when requesting" raw" processing level from a plugin that constructs its timestamps from a decomposed time: time-1s and time+1s were the same and therefore this could trigger problems in the plugin.

Some fixes to make MeteoIO compile with Visual C++ 2013. The linker still complains about exported symbols (this also generate lots of warnings), but all the other issues have been fixed.

In order to make it more modular and allow more advanced processing (virtual stations, downscaling, multiple io sources) the IOManager is now a proper wrapper: the processing is fully delegated to either a GridsManager or a TimeSeriesManager. These can in turn be reused at some other places (like in the Meteo2DInterpolator) to offer full buffering and processing of "local" time series. The buffering is now done by these two new classes (therefore the BufferedIOHandler class has been removed) and it is planned that dataClasses will be implemented for proper ringbuffers (one for time series, one for grids and one for isolated points) and receive the generic buffering code later on. 

A new fastPow(double, int) has been implemented for positive exponents. The begining of a generic handling of units also appeared in IOUtils. Finally, the ProcessingLevels are now defined in IOUtils instead of IOManager.

better file structure: all the classes purely focused on data storage (such as grids, dates, etc) have been moved into a "dataClasses" directory. All subdirectories are nhow named similarly to the classes and methods: the words after the first one are capitalized.

Since IOUtils was getting bigger and bigger, all the methods dealing with low level file handling have been moved into a new file, FilesUtils. It remains in the IOUtils namespace. One method has been moved to Date. A few includes have been cleaned up.

In order to be able to read arc grids produced by R, the headers are now read in a case insensitive manner

speeding up IOUtils convertString<double>: Adding an explicit implementation for the conversion of strings to doubles, based on strtod from the cstdlib. This function cuts the runtime of this function to about a third.

The filters have been slightly simplified, some IOUtils functions optimized (after reading an interesting post about the handling of rvalues), a bug fixed in PNGIO on Windows, the macros MAX and MIN are not used anymore (relying on std::max and std::min instead). A new data generator is on its way to generate HNW from HS differences (but this will require architectural changes to the DataGenerator).

Fixing some warnings for Visual C++ and also using ostringstream instead of stringstream when appropriate

More size_t fixes (ie trying to be consistent) and a bug was found when spatially interpolating using a few specific algorithms that would not reset their internal vectors between two calls (thanks Matteo for helping uncover this bug!)

Adding a link in the documentation, moving libsmet to the plugins subdirectory and small fixes in GRIBIO

The IMPORT feature in Config now works properly on Windows. It is also now possible to call IMPORT_BEFORE from within a section (but it still needs to be the first key in a file). 

In order to properly import config files while supporting relative paths, it is necessary to internally expand the paths (as well as the symlinks) to check for multiple inclusions, to properly look for a file located relative to another one (instead of related to the execution directory). The IOUtils::cleanPath function has been expanded to support this kind of processing (with a flag to turn it on of off) and a few functions added (getPath, getFilename). The code for Windows has been written but not tested yet. The Posix implementation should also be tested on osx.

The default ilwr model in Atmosphere as been changed to Unsworth. The fixed snow albedo has been set to 0.85 for all filters/generators using it. 

The Config class can now import an ini file from another ini file, this could greatly simplify the management of operational deployments (a station that has a special configuration now can simply import a standard config file and redefine what it needs). This is either done with IMPORT_BEFORE or IMPORT_AFTER in order to choose who has precedence. Multiple (ie recursive) inclusions are supported with some enforced limitations to prevent circular dependencies.

In order to implement an efficient and portable object serialization, it has been decided that all serializable objects would be manually serialized to an std::iostream. Then, the parallelization framework (mpi, popc, etc) can take over this stream and move it to another node. Therefore, the operators "<<" and ">>" are now reserved for serialization (as seems to be standard) and the debug outputs are obtained by printing "object.toString()". This means that all debug code has potentially to be fixed, by replacing "std::cout << object" by "std::cout << object.toString()".

Hoping that the valgrind tests would finally work properly... And small tweak to the IOUtils::seek optimization

A rounding bug has been found in the Date class: when using a Date object in a loop where it gets incremented, it accumulates errors and after ~70000 iterations, the error would get bigger than one second, making date comparisons with a fixed date impossible. This has been solved by always rounding the internal gmt Julian date to the closest second, after each assignment (from the constructors, the setDate() calls or any arithmetic operation). Moreover, the rounding methods have been re-written to be more efficient as well as other roundings (specially when computing the date decomposition from Julian). 

The seek() call performing a binary search has also been simplified, in order to remove unnecessary tests, put some similar tests together and make the code clearer. 

Some constification and better vector usage took place in ResamplingAlgorithms and Meteo1DInterpolator.

fixing some of the warnings found on Hera (sorry for not noticing before). All the size_t / unsigned int stuff will have to be addressed later

My previous commit was terrible... it simply made it impossible to define optional keys. This has now been fixed but required a change in the API. So, don't use Config::nothrow but IOUtils::nothrow instead.

In order to try to be closer to the spirit of the STL, now the empty strings/vectors/lists are tested with .empty(), the first/last elements of vectors accessed with front()/back(), the empty strings are not contructed with string("") but with string(), etc This makes a benchmark based on meteo_reading ~4% faster and this makes the code suprisingly easier to read. 

Some extra constification took place as well as some minor code cleanup (optimizing the comparison between two vectors in order to perform less tests, etc).

The Date class can now properly generate and parse full ISO timestamps, including time zone information. The full specification is now supported by convertString() with a helper method in the Date class. Ouputing an ISO string with its time zone is done by requesting the ISO_TZ format.

Now, when libsmet attemps to write a station that does not contain any data, it will complain but will not throw an exception (otherwise when working with large numbers of stations, the odds of getting an exception are too high). An implementation for fast cubic root is now part of Optim, but still has to be tested. Some code cleanup (using .empty(), using copy constructor instead of construct+copy) occured in IOUtils.

Lots of small fixes after running cppcheck on Windows: unused variables, use ++i instead of i++ for complex types (ie mostly iterators for us) for more performance, a few variables that could be passed as const&, use vector.empty() instead of vector.size==0 (for clarity and potentially performances).

All the "#ifdef(_WIN)" have been replaced by "#ifdef(WIN)" for consistency. The message box code for Windows and osX has been reorganized, the console messages are now better colorized and the backtrace comes before the error message itself.

The FileIndexer class still had some bug (such a surprise). It is now working properly. Some constification in libsmet

Some warnings have been fixed in the tests and all 'string tmp=""' or 'tmp=""' have been replaced by either an empty constructor (that does the same ) or tmp.clear() (that does the same too)

The FileIndexer class has been expanded to cater to the needs of SMET (it is not the most efficient, but for now...). A bug has been fixed that was leading to segfault. The explanation for the remaining slowness of SNIO has been found: the Config keys were re-read for every parsed line... It is now 10x faster, ie up to par with SMET (more or less).

The indexing for SNIO and SMETIO was not working (because of BUFF_READ_BEFORE, we are almost never restarting exactly at the last read time step, but before). Moreover, SNIO was always reading the whole file, even past the specified dateEnd. This was extremely slow on large input file. This has been fixed. SNIO is still quite slow (it seems something like 10x compared to SMETIO), but it is much more acceptable. A file indexer class has been implemented in IOUtils.

some code cleanup in GSNIO, the warnings generated by soap are now turned off (with a hack to make old versions of gcc able to do it) and the plugins directory and component are now removed from the packaging

Some extra documentation has been added for the UPS coordinate system. The coordinates test has been expanded. A missing default value has been added to the Suntrajectory.

Cleaner definition of epsilon in MeteoData. Finally, working 2D_interpolations test (with reference files and relying on numdiff).

The MeteoData == operator now compares with an epsilon. The chekEpsilonEquality function has been rewritten, according to best practise and inlined. The powN functions have been moved to MathOptim and inlined. This makes the benchmarks 2% faster, that's such a victory...

Fixed a warning with XCode 4.3 on osX. Tried to make a more standard use of c++ (relying on std::fill for filling vectors, relying on vector.resize(sixe, value) instead of manually initializing a vector, etc). A few more const added where appropriate. It makes the benchmarks slightly faster...

Pushed documentation version number, more robust file globing for the libraries, more standard uppercase/lowecase conversion and the extra parameters in SMET are now forced to uppercase (so all parameters are uppercase when they come out of the plugins).