Tuesday 22 October 2019

FIT File Decoder for Di2 Data

I have written a .fit file decoder to read the Di2 gear shift data from it:

To use it press the "Load Fit File" button, browse to a .fit file and wait a few seconds.

Here's a few screenshots of the Stats Data tab:






These screen shots show the "Stats Page" of the same data file-this shows a data table and a bar chart that shows either the number of times or the duration of each gear selected. There are 2 radio buttons to switch the chart between Time and Count

There's a few information text boxes to report the name of the .fit file processed, the number of shifts found and the processing time required to decode the .fit file and generate the reports and charts.



The tab "Ride data" one shows a time based shift profile for which gear has been selected on both the rear and front gears. Above the chart is a data table of all time based data read from the .fit file. Right clicking on the data table will show a menu allowing the data to be exported as a text file

The "File" menu offers 3 options:





  • "Quit" is self-explanatory 
  • "Set Verbosity" allows you to generate some log files during decoding. The default option is Normal which will generate minimal log files. High will generate more files and a very large file that will help debug issues (for the programmer at least). Please note that this High Verbosity option will take a very long time even on quite Small .fit file. With Normal selected decoding may take 5-10 seconds, but the same file with High selected will take 10-15 minutes potentially
  • "Show Working Folder" will open an Explorer window showing the files generated:


For information:
  • _messages is the log file
  • _event_dt is the file containing Di2 shift data
  • _gear_dt is the file containing the summary (count and time) of the gear shifts
  • _record_dt is the file containing the time based data (speed, power etc AND the event data (shifts) combined)
The core fit file reader code came from the ANT SDK - it provides a library and some examples in various languages. I have adapted the "decode" example and created this Windows application

It seems robust and reliable with the testing I have done.

Here is some description of what it does and some potential issues
  • Its only been tested on .fit files from a Garmin 1030
  • Occasionally an unrealistic gear number is read in the Event data - like 255. In this case the software overwrites this and replaces it with the previous valid gear number. A message is placed in the _messages file, e.g.:
  • Correcting an error state in FrontGrearNumber at row number: 365
  • It currently only processes Records (time based data) and Event. Other data is available like "UserProfile" for example. This is future work
  • Some data fields in Records are "unknown". No data for these records is used/stored.
  • The date and time in the .fit file are based on "FIT_Base_Time" - this excludes localisation from what I can see so some of my Strava data files are 1 hour different to the time stamp in the .fit file. This will need to be addressed to align the .fit data with Strava data (see below)
  • Processing with the High verbosity setting is very very slow. I need to reduce the file IO to speed this up. With the Record and Event data this is stored in memory during the decoding and then written to file on completion. These files are a few 110Kbyte so this is OK but the High Verbose _message file is 10's of Mbyte so an alternative is required
FeedbackIf you use this and have issues or suggestions please add as a comment to the Blog and I'll try and address. If you have issues and you don't use a Garmin 1030 I might need an example of the file you use

Download
Please see Di2 Analyser Application (280 KByte) to get a ready built executable (in a zip file) to copy to a folder and run. (Link is on my Google Drive)


Source code:
Please see Di2 Analyser Project to (2102 KByte) get a MS Visual Studio Solution and source files ready to be edited, built, recompiled etc. to your own desire. Note I built it with VS 2017 Community. I have a copy of VS 2019 Community but did not use itr for this application
I am NOT a programmer, I am an engineer who CAN programme - please don't criticise any of the coding in the Main_Form.c file (the one I wrote). (Link is on my Google Drive)

Planned Further Work:
I will add this feature into my Strava Data Tool also available here on this Blog
I will improve how the High Verbosity level writes the data to improve the performance 
I will add decoding of some of the other data fields 
I will add further error checking 
Convert to use a database rather than files (offers better control, management and post processing)

2 comments:

  1. Is this still an active project? Google reports that the links point to files in your "trash".

    ReplyDelete
    Replies
    1. It's active as far as I am concerned. This post is not the latest version - so please go to the latest post and try the link in the post. This is the latest post and the link there works:
      https://midlandscyclist.blogspot.com/2020/05/di2-fit-file-decoder-v1104.html

      Delete