Add support for FIT files generated by the Garmin Descent Mk3.
This just fixes a field number limitation, it has not yet been
verified that the Mk3 file format is properly interpreted by Subsurface.
Signed-off-by: Michael Keller <mikeller@042.ch>
When importing FIT files, we may not have serial numbers or firmware
versions in the result, so don't report them when they don't exist.
Also, add the product name to the FILE message field list, which can
contain relevant information. Not that we report it right now, but now
we *could* do so.
This concludes the Suunto FIT file export saga. It's not great, but it
looks like it should be usable.
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
So FIT files seem to have many many different ways to describe time
offsets. And I'm not talking about the overall Garmin time offset of
631065600, which is the conversion from the "Unix Epoch" (Jan 1, 1970)
to the "Garmin Epoch" (Dec 31, 1989).
No, I'm talking just about "device time" to "local time" to "UTC"
conversions.
The DEVICE_SETTINGS message has two different fields for time offsets:
there's a "UTC offset" (presumably this is the timezone the device is
set to), and a "time offset" which we actually use to transform the
recorded time of the dive into the local time that we report.
But the Suunto FIT export doesn't seem to use either of those, and
instead Nick Clark points outthe Suunto FAQ:
"Timestamp fields are deliberately defined as UTC time so that they may
be conveniently displayed in the local time if so desired.
In some instances it is useful to know the UTC offset when the file
was generated (possibly different from when it is decoded). This can
be accomplished by logging a single message containing both a
local_timestamp and a timestamp field. This will establish the UTC
offset of the file.
Presently these fields are predefined for activity and monitoring
messages"
so to get the actual local time, instead of getting it from the
DEVICE_SETTINGS message, we now have to parse the ACTIVITY message, and
take the difference between the regular timestamp and the
"local_timestamp" field.
The great thing about standards is that there are so many to choose
from.
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
We don't use them, but they seem to be trying to describe what the
developer fields are used for. We may now parse the developer fields
enough to skip over them gracefully, but it looks like we migth some day
want to _really_ parse them, and while I haven't figured it out (at
all!) yet, this may some day help.
For example, we get things like this:
FIELD_DESCRIPTION_name (STRING): "Depth"
FIELD_DESCRIPTION_unit (STRING): "feet"
FIELD_DESCRIPTION_original_mesg (UINT16): 20
FIELD_DESCRIPTION_data_index (UINT8): 0
FIELD_DESCRIPTION_field_definition (UINT8): 0
FIELD_DESCRIPTION_base_type (UINT8): 136
which doesn't tell me anything at all right now, but looks like maybe it
should some day.
It looks like this is defining a developer field for depth in feet
(duh), and the data format may be the same as a RECORD message (20),
which does indeed normally contain the depth (but in mm as an UINT32,
and it's field number 92, so..)
End result: not useful right now, because I'm much too confused about
it. But the debug printout looks interesting.
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
Not because we use them, but because it makes it clearer from the debug
output that we know what they are and that they aren't interesting..
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
This also gets the passing of string values right, even if we don't
actually use this right now.
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
This was part of me trying to figure out the compressed formats and the
developer fields, it's just noise now.
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
The ACTIVITY message contains a "local timestamp" in addition to the
regular timestamp. That gives us the timezone information.
Or rather, it would give us the timezone info if we actually used it.
But now we at least parse it, so that we *could* use it.
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
We end up using the FIT file name as the "fingerprint" for the dive, and
include it at the beginning of the dive data as such. And because of
how Garmin encoded the FIT files, we ended up having a fixed 24-byte
length for this, which is normally the date encoding:
YYYY-MM-DD-HH-MM-SS.FIT
with the terminating NUL character.
Of course, then Garmin started using a short-form encoding too
(presumably due to FAT filesystem limits), and we have magic code to
sort the dates properly, using the name format
YMDHMMSS.FIT
with the numbers encoded in a shorter format (eg "C4ND0302.fit" is
equivalent to "2022-04-23-13-03-02.fit"). See name_cmp() and
parse_short_name() for details.
Anyway, because we use the (zero-padded) 24 characters of the name as
the fingerprint, we used a fixed-size buffer for the filename that was
limited to that maximum size Garmin creates.
But then you download those things, and have multiple vendors, and
suddenly that 24-character limit on the filename is very annoying.
Instead of fixing this in some clean and generic way, let's just raise
the namelength limit to something bigger, and continue to use the first
24 characters of the name for the fingerprint.
Pretty it isn't, but it makes it slightly easier to import random FIT
files that don't conform exactly to the traditional Garmin format.
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
This was the _actual_ reason why the Suunto FIT file import fell flat on
its face: it adds records with developer fields in them, and I just had
no idea how to parse them.
It turns out that they aren't all *that* horrible to parse: they are
kind of like a special case of the regular FIT event fields.
And no, this does not really parse them: it only parses the layout, and
using that it can then skip the developer fields without causing the
decoder to go all wonky and lose stream synchronization.
At least it works for the specific case of the Suunto FIT files, and the
code makes some amount of sense. The FIT format may be odd, but at the
same time it's most definitely designed for pretty simplistic devices,
so it's not some kind of crazy XML thing.
This gets us parsing those Suunto FIT files at least partially.
That said, it is all very rough indeed, since you have to lie and claim
you're downloading from a Garmin, and have to set up the whole magic
'Garmin/Activity/' directory structure and limit the file size to the 24
characters that Garmin uses.
So this is by no means the real solution.
Considering that Jef doesn't want the Garmin parser in libdivecomputer
anyway, the proper solution might be to move this all to subsurface, and
make it be a "FIT file import" thing instead. Annoying, but on the
other hand it has also been a bit awkward to have it in libdivecomputer.
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
This turns out to have been another false turn: mis-parsing the FIT file
caused us to think it had compressed records.
So I spent much too much time trying to figure out how those compressed
records actually work. This is the result.
It looks like the only difference between a compressed record and a
regular one is that the compressed record has a single-byte "this is the
record type and the time offset" field at the start. That basically
avoids the need of then having a full 4-byte absolute time for such a
record.
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
This is an empty stub for the HRM profile message, which isn't actually
used by anything I know of, but came up as a result of some mis-parsing
of odd FIT files generated by the Suunto mobile app.
Losing synchronization in the FIT file then caused the parser to think
it needed this message type, and not having it then caused an early
abort.
While it's not actually needed once parsing things correctly, since I
looked up the message number and name for this message type, let's just
keep it around. It won't hurt, and maybe it avoids me having to look it
up in the future.
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
Instead of returning an error on unrecognized input when parsing, just
skip to the end of the buffer. This makes at least partially parsed
data available, which can help figure out what ended up happening.
This was part of my "Suunto also does FIT files now, and does them very
differently from Garmin" series. I think I parse the Suunto files right
now, but next time this happens, I'd rather get partial data than no
data at all.
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
Merge upstream updates from Jef Driesen:
- Deepblu Cosmiq+ support has been merged upstream
- Oceans S1 support has been merged upstream
- Various new models supported: Cressi Donatello, Scubapro G2 TEK, new
Excursion v6+ firmware.
- misc core changes, most notably supporting a new annoying specialized
binary format for "decomode", because Jef still can't deal with
strings.
- lots of small details
* https://github.com/libdivecomputer/libdivecomputer: (58 commits)
Keep open-circuit and diluent gas mixes separately
Parse some extra gas mix information
Limit the index to the fixed gas mixes
Handle dives without a valid gas mix more explicit
Ignore all gas mixes for freedives
Always include all gas mixes defined in the header
Add support for the new Excursion v6+ firmware
Add support for the HP CCR tank pressure
Use the correct field for the setpoint sample
Add support for the Oceans S1
Add support for the Deepblu Cosmiq+
Add missing functions for accessing big/little endian values
Move the snprintf functions to the platform module
Repeat the handshake every few packets
Enable big page support
Remove the model number from the vtpro struct
Add the model number to the version table
Move all model numbers to the common header
Remove a duplicated include statement
Add support for the 300bar pressure sensor
...
DECLARE_FIELD() uses array_uint_endian() to turn an integer type into
the right endianness. It's all conditional on being an integer type,
but the compiler still sees the assignment (with a cast) for other
types, and complains about casting the 'unsigned int' return value to a
pointer, even when that case is not actually dynamically ever taken.
Fix the compiler warning by just changing the return type of this
conversion function to 'unsigned long' instead, which will match the
size of pointers on all relevant architectures.
Don't look at that macro too closely, you'll go blind.
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
As a CCR diver I would like to see the ability to properly track the 'on
loop' / 'bailed out to open circuit' status directly in Subsurface,
because this is an important (or even the most important) bit of status
information during a CCR dive. This should extend to the use of the
correct ppO2 / gas mix in the deco ceiling / tissue model calculation.
My idea for how to do this would be to track the 'type' ('diluent' / 'OC
bailout') for every gasmix / tank that is reported by the dive computer.
Most CCR capable dive computers that I am familiar with require the user
to enter two different gas lists for diluent and bailout, so this should
work with the existing libdivecomputer API for these. Unfortunately I
think making this change in Subsurface will require a bit of work, as
the libdivecomputer field capable of tracking the 'type' of a gas or
tank (`cache->tankinfo[]`) does not seem to be consumed at all in
Subsurface.
So this just provides a prerequisite for the change in Subsurface by
populating `tankinfo[]`. In addition to this it also triggers a message
on every switch from CC to OC and back, at least giving a visual
indication of these diver triggered events. The messages can probably
be removed from libdivecomputer again once 'loop status' tracking has
been added to Subsurface.
Also included is a fix of the tab expansion mess that I created in
commit 2129403 ("Added facility to detect and interpret manual setpoint
switches for Garmin Descent computers"). Apologies for this, I've
switched to using a custom `.vimrc` for this project now.
[ Edited up the commit message a bit further - Linus ]
Signed-off-by: Michael Keller <github@ike.ch>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
The garmin FIT file parser verified that a string entry fit in the field
size, but it turns out that the check is wrong: a FIT file string field
is not necessarily NUL-terminated at all, and it's ok to have a string
that fills the entire field.
We never actually then use the string length we just checked, so with
the checks being bogus, all of this code just goes away. But let's
update the debug printout to follow these rules.
This makes parsing the example FIT file that Cédric sent us work just
fine (at least superficially, in that I don't see anything obviously
wrong with the result: I don't actually know what Cédric's dive was
supposed to look like to verify).
Reported-by: Cédric BAREYT <bareytcedric@gmail.com>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
Also improved the display of setpoint information - switch mode is shown, and auto switch depth is only shown in auto mode.
All of this has been tested.
Signed-off-by: Michael Keller <github@ike.ch>
This is adding support for parsing the setpoint information in logfiles downloaded from Garmin Descent devices.
The Garmin devices do not have support for ppO2 sensor input, so they only work in 'fixed setpoint' mode for CCR dives, and dive data records do not contain actual ppO2 values.
The ppO2 values are retrofitted to the dive data based on 'setpoint change' events reported by the device. With this change CCR dives downloaded from a Garmin device are correctly classified as CCR dive, and the calculated ceiling / tissue loading graphs are accurate and match the deco stops reported by the device.
Before this change, CCR dives were classified as open circuit dives, often resulting in a massively overstated calculated ceiling.
This has been tested for logs with only automated setpoint changes - more test dives are needed to reverse engineer the log file format for manual setpoint changes, as the setpoint fields are not documented in Garmin's documentation for the FIT file format.
Signed-off-by: Michael Keller <github@ike.ch>
The only examples I have seen so far are small incomplete files that
don't contain the dive profile, so I'm unsure if this is sufficient (and
why there are big endian files in the first place).
Signed-off-by: Dirk Hohndel <dirk@hohndel.org>
Merge Jef's upstream updates:
- Add support for Seac Screen and Action
- Add support for Cressi Michelangelo
- misc small fixes
* https://github.com/libdivecomputer/libdivecomputer:
Add support for the Seac Screen and Action
Add an address parameter to the memory dump helper function
Ignore invalid gas mixes
Add support for the Cressi Michelangelo
Move the C_ARRAY_SIZE macro to a common place
Fix the timezone offset in the xml output
Emit a devinfo event when downloading a memory dump
Read the info and more info data during startup
Use helper functions to decode multibyte values
Fix changing the OSTC settings
This should always be tested for - the only other user of the generic
helper currently didn't test for it, either.
Signed-off-by: Dirk Hohndel <dirk@hohndel.org>
The generic function already handles all the cases (treating gasmix and
tank count as being the same and responding UNSUPPORTED to the same set
of types). The only thing missing is the check of the value parameter
and the extraction of the cache pointer from the context.
Suggested-by: Linus Torvalds <torvalds@linux-foundation.org>
Signed-off-by: Dirk Hohndel <dirk@hohndel.org>
The code used sub_sport to figure out if this was a dive in the first
place, but then never reported back the specific dive mode, leading to
all dives being interpreted as open circuit.
Reported-By: Anton van Rosmalen
Signed-off-by: Dirk Hohndel <dirk@hohndel.org>
The gas mixes are numbered from 0 and up and the gas change event uses
this number directly, so no need to subtract one.
Signed-off-by: Michael Andreen <michael@andreen.dev>
Very early trial at parsing the pressure data from the Garmin tank pods.
It seems to work for the test-cases I have.
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
This adds the field definitions, but doesn't actually report the data
yet. But just having the definitions makes the debug output a lot
easier to read through.
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
This actually got broken long ago (and before the new "Subsurface-DS9"
branch: when I converted the Garmin parser to use the generic field
cache, I missed the fact that the parser now cleared _only_ that generic
part of the per-dive cache in between different dives.
The GPS information, and the general dive information would be left
alone in between dives, and could leak from one dive to the next.
Most of the time you don't notice, because dives tend to all have the
same information, so the stale data would get overwritten when parsing
the next dive. But that isn't _always_ true - particularly for the GPS
information, not all the data necessarily always exists.
So clear all the per-dive data when starting to parse a new dive.
This puts the non-gps data all in one sub-structure, so that it's easy
to clear that too in one go - even though that part of the data probably
always does get opverwritten by the new dive data.
Reported-by: @brysconsulting
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
This uses pretty much all of our new infrastructure: the USB storage
iostream for the actual IO, the field-cache for the divecomputer fields,
and the string interface for the events.
It's also a very fast downloader.
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
