Logging data formats

[smaria]

As we are 'enhancing' our logs with additional data, the introduction of new data formats is necessary!
Current suggestions for added data in the logs:

• remote controlled
• water depth
• wave information

Two main models can be distinguished here:

• time synchronous CSV format; as no sensors will be time-synchronous, this will mean data that has been post-processed on the boat already
• time a-syncronouse; every log line starts with a timestamp, then a string indicating the type of data ('RC', 'DEPTH', 'WAVE' etc.), followed by the measurement in a given format

[smaria]

It may also be worth attaching an open licence to those data formats - then we will end up generating a lovely set of widely usable reference data!

[takluyver]

I was about to write that we should ask previous competitors about the capabilities of their systems to store logs, but I've just changed my mind: the requirements shouldn't need to be fitted to what teams can do.

We might want to explicitly say that it's OK to process logs from your boat into the standard format, so the boat doesn't have to write the standard format directly, in case that's a challenge for anyone. But obviously the processing stage can't generate any new data.

[smaria]

Another question worth asking: Is the binary format still required? It is difficult to give examples for, and not readable without the added big/little endian information. As data storage gets cheaper, I would expect this to be obsolete nowadays.