Visualize JSON timeseries from Fitbit

Question

I'm looking to visualize some Fitbit data which comes in json format (file) . Can someone suggest a nice way to massage this into DateListPlot?

[{
  "dateTime" : "03/18/21 00:00:00",
  "value" : "724"
},{
  "dateTime" : "03/19/21 00:00:00",
  "value" : "569"
},{
  "dateTime" : "03/20/21 00:00:00",
  "value" : "594"
},{
  "dateTime" : "03/21/21 00:00:00",
  "value" : "667"
},{
  "dateTime" : "03/22/21 00:00:00",
  "value" : "576"
},{
  "dateTime" : "03/23/21 00:00:00",
  "value" : "532"
},{
  "dateTime" : "03/24/21 00:00:00",
  "value" : "602"
},{
  "dateTime" : "03/25/21 00:00:00",
  "value" : "634"
},{
  "dateTime" : "03/26/21 00:00:00",
  "value" : "632"
}]

MarcoB · Accepted Answer · 2021-04-08T00:54:00.873

The JSON dataset is imported as a list of rules:

{
  {"dateTime" -> "07/07/14 00:00:00", "value" -> "1440"}, 
  {"dateTime" -> "07/08/14 00:00:00", "value" -> "1128"}, ...
}

So take the second part of each Rule for each point to give string pairs. Then use DateObject with an appropriate template on the first string of the pair to obtain a date, and ToExpression on the second string to obtain a number. Then feed to DateListPlot.

data = Import["sedentary_minutes.json"][[All, All, 2]];
formatted = 
  {
    DateObject[{#1, {"Month", "/", "Day", "/", "YearShort", " ", "Hour", ":", "Minute", ":", "Second"}}], 
    ToExpression[#2]
  }& @@@ data;
DateListPlot[formatted]

A more performant alternative might be to avoid DateObject and ToExpression.

Since the data appears regularly spaced with one minute intervals, one can use the datespec argument to DateListPlot to specify a start date and a granularity, with an end date automatically calculated from the number of data points present.
A faster alternative to ToExpression is Internal`StringToDouble.

Combining the two, but only showing four points a day, i.e. every six hours rather than every minute:

data = Import["calories.json"][[All, All, 2]];
values = Internal`StringToDouble /@ data[[All, 2]];
DateListPlot[
  values[[;; ;; 360]], 
  {{2014, 07, 07, 0, 0, 0}, Automatic, {0, 0, 0, 6, 0, 0}},
  PlotRange -> All
]

Consider that including more points won't lead to much more information being conveyed by your plot: choosing to include only four points a day already results in close to 10,000 points being plotted. You would need an extremely high resolution medium to distinguish more than those.

Thanks for the tip! Mostly worked but didn't scale to a larger dataset (200M json for calories burned https://www.dropbox.com/s/dbh9jteggqreko9/calories.json?dl=1), wondering if the issue is the many DateTime objects being constructed — Yaroslav Bulatov, Apr 07 '21 at 20:12
@Yaroslal Take a look at the alternative approach that avoid DateObject and ToExpression. That should be quite a bit faster. — MarcoB, Apr 07 '21 at 22:04

Visualize JSON timeseries from Fitbit

1 Answers1