WSL/SLF GitLab Repository

Commit e5d482a4 authored by Mathias Bavay's avatar Mathias Bavay
Browse files

The SMET specification has been changed and brought to version 1.1. The...

The SMET specification has been changed and brought to version 1.1. The units_multiplier and units_offset order has been changed: now, the multiplier is applied FIRST and then the offset. This allows to disable a SMET column by setting the mutiplier to 0 and the offset to nodata. A warning is displayed by SMETIO for all files version <1.1 urging users to check their files and update them to 1.1.

Code cleanup in IMISIO, grid constraint for wind that it must be greater than 0 in Meteo2DInterpolator.
parent 910f2340
......@@ -2,7 +2,7 @@
%%TODO: GSN has trouble with space delimited formats, it needs comma delimited...
%opening
\title{Format specification for the SMET Weather Station Meteorological Data Format\\version 1.0}
\title{Format specification for the SMET Weather Station Meteorological Data Format\\version 1.1}
\author{Mathias Bavay}
\begin{document}
......@@ -11,6 +11,8 @@
\begin{abstract}
The goal of this data format is to ease the exchange of meteorological point measurements by providing both the data and the metadata in an easy to interpret and to manipulate format. Data manipulations should be possible manually using a standard text editor or using some tools (shell scripting, awk, perl, as well as spreadsheets, R, as well as stadard programing languages). The format is versioned, which means that format updates could break backward compatibility if necessary.
Compared to specification 1.0, when using a unit offset/multiplier, the multiplier is now applied \emph{before} the offset so that it is possible to easily force a whole column to nodata.
\end{abstract}
\section{Requirements}
......@@ -96,7 +98,7 @@ The following are optional:
\item creation: timestamp of the file's creation date (ISO 8601 Combined date and time formatted)
\item source: string describing the origin of the file
\item units\_offset: a vector of decimal numbers to add to each value of the matching column in order to get back to MKSA
\item units\_multiplier: a vector of decimal numbers to mutiply each value of the matching column (\emph{after} potential offset addition) in order to get back to MKSA
\item units\_multiplier: a vector of decimal numbers to mutiply each value of the matching column (\emph{before} potential offset addition) in order to get back to MKSA
\item comment: a free string to write any comment
\end{itemize}
......
......@@ -46,7 +46,6 @@ Meteo1DInterpolator::Meteo1DInterpolator(const Config& in_cfg) : cfg(in_cfg) {
}
if (window_size <= 0.01)
throw IOException("WINDOW_SIZE not valid", AT);
}
void Meteo1DInterpolator::getWindowSize(ProcessingProperties& o_properties)
......
......@@ -118,6 +118,8 @@ void Meteo2DInterpolator::interpolate(const Date& date, const DEMObject& dem, co
Meteo2DInterpolator::checkMinMax(0.0, 10000.0, result);
} else if (meteoparam == MeteoData::HS){
Meteo2DInterpolator::checkMinMax(0.0, 10000.0, result);
} else if (meteoparam == MeteoData::VW){
Meteo2DInterpolator::checkMinMax(0.0, 10000.0, result);
}
}
......
......@@ -170,12 +170,8 @@ void ImisIO::getDBParameters()
cfg.getValue("DBUSER", "Input", oracleUserName_in);
cfg.getValue("DBPASS", "Input", oraclePassword_in);
string tmp = cfg.get("USEANETZ", "Input", Config::nothrow);
if (tmp != "") {
cfg.getValue("USEANETZ", "Input", useAnetz);
} else {
useAnetz = false;
}
useAnetz = false;
cfg.getValue("USEANETZ", "Input", useAnetz, Config::nothrow);
/*cfg.getValue("DBNAME", "Output", oracleDBName_out);
cfg.getValue("DBUSER", "Output", oracleUserName_out);
......
......@@ -51,7 +51,7 @@ namespace mio {
//TODO: keep a pointer to last read position, so we can fseek for the next read?
const std::string SMETIO::smet_version = "1.0";
const std::string SMETIO::smet_version = "1.1";
const unsigned int SMETIO::buffer_reserve = 23*24*2; //kind of average size of a buffer for optimizing vectors
map<string, MeteoData::Parameters> SMETIO::mapParameterByName;
const bool SMETIO::__init = SMETIO::initStaticData();
......@@ -443,7 +443,7 @@ void SMETIO::readDataBinary(const char&, const std::string&, const double& timez
if(val==plugin_nodata)
SMETIO::getParameter(vecDataSequence[ii], md) = IOUtils::nodata;
else
SMETIO::getParameter(vecDataSequence[ii], md) = (val + vecUnitsOffset[ii]) * vecUnitsMultiplier[ii];
SMETIO::getParameter(vecDataSequence[ii], md) = (val * vecUnitsMultiplier[ii]) + vecUnitsOffset[ii];
}
}
}
......@@ -524,7 +524,7 @@ void SMETIO::readDataAscii(const char& eoln, const std::string& filename, const
if(val==plugin_nodata)
SMETIO::getParameter(vecDataSequence[ii], md) = IOUtils::nodata;
else
SMETIO::getParameter(vecDataSequence[ii], md) = (val + vecUnitsOffset[ii]) * vecUnitsMultiplier[ii];
SMETIO::getParameter(vecDataSequence[ii], md) = (val * vecUnitsMultiplier[ii]) + vecUnitsOffset[ii];
}
}
......@@ -667,9 +667,13 @@ void SMETIO::checkSignature(const std::vector<std::string>& vecSignature, const
throw InvalidFormatException("The signature of file " + filename + " is invalid", AT);
std::string version = vecSignature[1];
if ((version != "0.9") && (version != "0.95") && (version != "0.99") && (version != smet_version))
if ((version != "0.9") && (version != "0.95") && (version != "0.99") && (version != "1.0") && (version != smet_version))
throw InvalidFormatException("Unsupported file format version for file " + filename, AT);
if(version=="0.9" || version=="0.95" || version=="0.99" || version=="1.0") {
std::cout << "[W] SMET specification 1.1 changes the priorities of units_multiplier and units_offset. Please check/update your files and bring them to 1.1!!\n";
}
const std::string type = vecSignature[2];
if (type == "ASCII")
isAscii = true;
......@@ -783,6 +787,7 @@ void SMETIO::writeDataAscii(const bool& writeLocationInHeader, const std::vector
fout << "[DATA]" << endl;
fout.fill(' ');
fout << right;
fout << fixed;
for (unsigned int ii=0; ii<vecMeteo.size(); ii++){
if(out_dflt_TZ!=IOUtils::nodata) {
Date tmp_date(vecMeteo[ii].date);
......@@ -837,8 +842,8 @@ void SMETIO::writeHeaderSection(const bool& writeLocationInHeader, const Station
if (sd.getStationName() != "")
fout << "station_name = " << sd.getStationName() << endl;
fout << fixed;
if (writeLocationInHeader){ //TODO: only write if != nodata
fout << fixed;
fout << "latitude = " << setw(14) << setprecision(6) << sd.position.getLat() << "\n";
fout << "longitude = " << setw(14) << setprecision(6) << sd.position.getLon() << "\n";
fout << "altitude = " << setw(9) << setprecision(1) << sd.position.getAltitude() << "\n";
......
Supports Markdown
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment