Synchronizing FIT files using a Raspberry Pi

:: activitylog2

I wrote the ActivityLog2 as an application to analyze training data on the local computer, to avoid using a cloud service, and I also wanted a convenient way to download data off my Garmin watch without having to hook up USB cables to my laptop. The result is the PiFitSync project which I had running for several years now, evolving to download data from three generations of Garmin devices.

These days, all Garmin devices (and any other fitness device) will connect to the users phone and automatically upload the data to a clould service, where the user can analyze it. This process is convenient and happens automatically, with no user intervention. This is an opaque process — data goes from the device directly to the cloud provider, and usually there is little documentation on how things work, which prevents creation of alternative data download applications.

Garmin devices also allow the user to download the data directly off the device via USB: connecting the device to a computer with a USB cable will show all files on the device in the file explorer and the user can copy the files to the local computer. This is useful for applications such as ActivityLog2 and GoldenCheetah as they keep the training data on the users computer only.

However, the above process is inconvenient, since the user has to connect the device to the laptop using a USB cable. As an alternative, I decided to use a Raspberry Pi to download the data while recharging the device: the Pi is connected to the home network and the data is accessible over Wifi as a network drive, making it easy to import the data. This works as follows:

  • The Raspberry Pi is connected to the local home network, accessible over the local WiFi connection.
  • The USB charging cradle for the Garmin Watch is connected to the Raspberry Pi
  • When a Garmin Device is connected to the cradle for charging, all the FIT files are copied off the device onto a local folder.
  • The downloaded FIT files are available over a network share from laptop and can be imported into ActivityLog2 or GoldenCheetah.

Under the Hood

The PiFitSync project contains all the files for setting up the download application, and as I updated it for the latest generation of Garmin Watches, I looked back on the different strategies I used to implement the download mechanism.

A Basic Daemon Service

The first Garmin device I owned was a FR310 XT, this was before Bluetooth connectivity was common on computers, so the device used to sync using the ANT+ protocol, the same one that the device is used to communicate with sensors, such as Heart Rate monitors or Speed sensors on a bike. This required an ANT+ dongle, which is plugged into a USB port on the computer. The sync protocol was documented by Garmin (it was called ANT-FS) and I could easily write an application, fit-sync-ant, to download the files from my FR310. I even dusted off my APUE book and made the application a Linux daemon process.

Organizing for the service to be started on linux meant writing a service file and registering it with systemd:

[Unit]
Description=Download FIT files from Garmin devices (ANT-FS version)
After=fit-sync-setup.service

[Service]
Type=forking
PIDFile=/run/fit-sync/fit-sync-ant.pid
ExecStart=/usr/local/bin/fit-sync-ant -d
Restart=no
WorkingDirectory=/
User=ubuntu
Group=ubuntu

[Install]
WantedBy=multi-user.target

Since the service would run as a normal user instead of root, it would not be able to write to the /run/ folder to write its PID file, as daemon processes are supposed to do. To fix this, I created another service to create a subfolder in /run with the appropiate permissions to write a PID file:

[Unit]
Description=Setup PID folder for fit sync service

[Service]
Type=simple
ExecStart=/usr/bin/install -o ubuntu -g root -m 755 -d /run/fit-sync
Restart=no
WorkingDirectory=/

[Install]
WantedBy=multi-user.target

Finally, the USB device for the ANT stick does not have permission to be read by a non-root user. To adjust that, I had to add an UDEV rule to assign more liberal permissions, here are the rules for the two ANT sticks I used. Here, they are just made world writeable, but the UDEV rules can also be used to change ownership to the ubuntu user, or to an ant group:

SUBSYSTEM=="usb", ACTION=="add", ATTR{idVendor}=="0fcf", ATTR{idProduct}=="1008", MODE="0666", SYMLINK+="ttyANT2"
SUBSYSTEM=="usb", ACTION=="add", ATTR{idVendor}=="0fcf", ATTR{idProduct}=="1009", MODE="0666", SYMLINK+="ttyANT3"

This setup worked for many years, the Rasppery PI downloading FIT files form the device and making them available over a network share.

Copying files off a USB drive

The next Garmin device I got was the FR920, and by this time, Bluetooth was more common. The device no longer supported downloading files over the ANT+ protocol and used WiFi or Bluetooth instead. Unfortunately, none of these protocols were documented anymore, so it was no longer possible to write an independent downloader. However, when connected over USB (the device used USB for charging), the device showed up as a USB drive on the computer and the FIT files were readily available for copying from the device.

Copying files could be done using the cp -r command, but for extra sophistication, I wrote a small utility, fit-sync-usb which would look inside the FIT files and sort them by type and device serial number — this allowed copying data from multiple devices into the same folder hierarchy, without mixing the files.

The tricky part was setting up the Raspbery PI, such that the utility is invoked when the device is plugged in and this was more complex than the previous setup.

First, UDEV rules can be used to detect that a disk device was attached. This shows up as a "/dev/sda" drive, but, in order to identify that it is indeed a Garmin device, the blkid utility is used to get information about the drive and look for its file system label (ID_FS_LABEL), which should be “GARMIN” for a garmin device. If the device is the one we are looking for, a “mount” command is used to actually mount the drive than the fit-sync-usb service is started to actually download the files:

KERNEL=="sd[a-z]", GOTO="start_automount"
KERNEL=="sd[a-z][0-9]", GOTO="start_automount"
GOTO="end_automount"

LABEL="start_automount"
IMPORT{program}="/sbin/blkid -o udev -p /dev/%k"
ENV{ID_FS_LABEL}=="GARMIN", GOTO="garmin_automount"
GOTO="end_automount"

LABEL="garmin_automount"

ACTION=="add", RUN+="/bin/mount -t vfat -osync,noexec,noatime,nodiratime,uid=ubuntu /dev/%k /media/garmin"
ACTION=="add", RUN+="/bin/systemctl start fit-sync-usb.service"

ACTION=="remove", RUN+="/bin/umount -l /media/garmin"

GOTO="end_automount"

LABEL="end_automount"

The actual downloading of the file needs to be done as a separate service, since the udev daemon will expect any programs it launched to finish quickly and it is able to track and kill even processes that fork and become daemons. The fit-sync-usb service is a simple service, with no restart, as the utility will simply exit once it finishes downloading the files.

[Unit]
Description=Download FIT files from Garmin devices (USB version)
After=fit-sync-setup.service

[Service]
Type=forking
PIDFile=/run/fit-sync/fit-sync-usb.pid
ExecStart=/usr/local/bin/fit-sync-usb -d /media/garmin/GARMIN
Restart=no
WorkingDirectory=/media/garmin
User=ubuntu
Group=ubuntu

Again, this setup worked well for several years, until I upgraded my Garmin Device to the newest generation one.

Copying files off a MTP device

Recent generation Garmin devices, such as the FR945 will no longer show up as a USB drive, instead they are “Media Transfer Protocol” (MTP) devices. They can still be mounted as file systems (and fit-sync-usb used to transfer files), but this is done ouside of the Linux kernel, using a userspace program, jmtpfs and it requires using a set of UDEV rules, two services, and four shell scripts. This is considerably more complex than the previous “USB drive” approach.

UDEV Rules

The first subsystem that “notices” that the device was connected is UDEV and we need to add rules to trigger commands when MTP devices are added or removed. Here are the relevant to MTP device mounting/unmounting rules, they launch two scripts, on-mtp-added when a device is added and on-mtp-removed when the device is removed:

SUBSYSTEM=="usb", ACTION=="add", ATTR{interface}=="MTP", RUN+="/usr/local/libexec/fit-sync/on-mtp-added %p"
SUBSYSTEM=="usb", ACTION=="remove", RUN+="/usr/local/libexec/fit-sync/on-mtp-removed %k"
UDEV Rule for adding the device

An udev rule is used to detect when a MTP device from the “usb” subsusystem is added, to call on-mtp-added:

SUBSYSTEM=="usb", ACTION=="add", ATTR{interface}=="MTP", RUN+="/usr/local/libexec/on-mtp-added %p"

on-mtp-added receives the device path (sys has to be prepended to it), it will than:

  • determine the parent device (this is just the parent folder in the tree). The UDEV rule matches against the MTP interface, but the actual device is the parent — this parent device is simply the parent folder.

  • determine the vendor id and product id from the idVendor and idProduct files in the parent device folder, and check them against the desired device for a FR945 (script will exit if it does not)

  • determine the bus and device numbers (contents of the busnum and devnum files) — this information is needed by the jmtpfs utility to mount the correct MTP device. This information is saved into /var/run/fit-sync/mtp-device, and will be used by the device mount script, mount-fr945

  • determine the kernel device name — this is the folder name of the parent device. This will be saved in /var/run/fit-sync/mtp-kname and will be used by on-mtp-removed script to determine if our device was removed and the drive needs to be unmounted.

  • start the “mount-fr945.service” using systemctl start mount-fr945.service — the jmtpfs utility needs to keep running while the drive is mounted and udev scripts neeed to run in a short amount of time so we cannot launch servers directly from here.

  • start the “sync-fr945.service” using systemctl start sync-fr945.service — this will copy the FIT files into ~/FitSync to be shared over the network. This service has a dependency listed on the mount-fr945.service so, systemd will make sure the start is delayed until the file system is mounted.

UDEV Rules for removing the device

There is no remove action against the MTP device directly, so we need to add a rule against any USB device and call on-mtp-removed script to figure out the rest:

SUBSYSTEM=="usb", ACTION=="remove", RUN+="/usr/local/libexec/on-mtp-removed %k"

on-mtp-removed receives the kernel device name for the device that was removed, it will than:

  • verify that the kernel name is the same as the one saved in /var/run/fit-sync/mtp-kname

  • if it is, stop the “mount-fr945.service” using systemctl stop mount-fr945.service

The MTP Drive Mounting/Unmounting Service

Mounting (and unmounting) the MTP device is done using a systemd service. This is required, because the jmtpfs utility needs to be running while the MTP device is mounted, so it is really a service not a one-off program like the mount command. The contents of the service file are as follows:

[Unit]
Description=Mount the fr945 MTP device

[Service]
Type=forking
ExecStart=/usr/local/bin/mount-fr945
ExecStop=/usr/bin/fusermount -u /media/fr945
Restart=no
WorkingDirectory=/
User=root
Group=root

In particular, the service is setup as forking (this ensures the system is considered started only after the file system is mounted). Also, rather than calling jmtpfs directly, the mount-fr945 shell script is used. This script will read the device from /var/run/fit-sync/mtp-device and ensures that the correct MTP device is mounted.

The service is set up to call fusermount to unmount the drive when the service is deactivated.

The sync service

The same fit-sync-usb utility is used to copy FIT files from the FR945 to the ~/FitSync folder, which is shared over the network. This is done using another systemd service, sync-fr945.service which launches the service in the background:

[Unit]
Description=Download FIT files from a FR945
After=fit-sync-setup.service
Requires=mount-fr945.service
After=mount-fr945.service

[Service]
Type=forking
PIDFile=/run/fit-sync/fit-sync-fr945.pid
ExecStart=/usr/local/bin/fit-sync-usb -p /run/fit-sync/fit-sync-fr945.pid -d /media/fr945/Primary/GARMIN
Restart=no
WorkingDirectory=/media/fr945/Primary/GARMIN
User=pi
Group=pi

Final Thoughts

The Garmin devices became more user friendly with newer models: for the first ones, one would need specialized hardware (an ANT+ dongle) and specialized software to download the data off them, with the more recent ones, the user can simply plug in the device and copy files using Finder or File Explorer, however it is interesting to see that this simplicity was achieved by increasing the underlying complexity of the system.

In any case, the system is reliable and offers a convenient way to download data off the device without involving any cloud services.