snehan kekre · posts

In macOS Seatbelt, bind and listen are two different permissions

Thu, 06 Aug 2026 00:00:00 GMT

Seatbelt is the sandbox macOS puts a command inside when you run it under sandbox-exec. It has two rules that look like they cover the same ground. network-bind authorizes bind(). network-inbound authorizes listen() and accept(). They are separate operations that fail at different syscalls, and under one condition the second silently stands in for the first.

I got this wrong twice in three days. First I decided network-bind was ignoring its own address filter and refusing the bind, and I wrote that into a source comment. Then I wrote a test, and the test convinced me network-bind authorized nothing at all and network-inbound was doing the work.

Both readings were wrong, and the second one had a table behind it, which is why I believed it for as long as I did. My probe called bind() and listen() inside one try block and printed “bind FAILED” whichever of them the kernel refused, so every row in that table named the wrong syscall. Everything below is the re-test with the two calls separated.

What I needed the sandbox to allow

I maintain a harness that runs documented quickstart commands inside a sandbox and records which documentation pages an agent reads. Quickstarts frequently end at “start the dev server and open it”, so the sandbox has to let a process listen on loopback and poll itself, while still refusing to reach any documentation host directly.

On macOS the enforced backend is sandbox-exec, which runs a command inside a policy you hand it as a file. Apple’s manual calls the machinery underneath it the sandbox facility, and Seatbelt is the name it goes by everywhere else. The tool is marked DEPRECATED, in a man page last revised in 2017, and developers are pointed at App Sandbox instead. App Sandbox covers apps you ship. I need to wrap whatever command a quickstart tells a reader to run. Neither man page documents the profile language, so every operation name below comes from Apple’s own profiles under /System/Library/Sandbox/Profiles/ and from testing.

Every task that started a server failed. The same tasks passed under the Docker backend, so the tasks themselves were fine. The failure looked like this:

PermissionError: [Errno 1] Operation not permitted

My profile had one rule with “bind” in the name, so I widened its filter from localhost:* to *:*, added inbound and outbound rules for loopback, and shipped all three edits in one commit. The tasks passed. That left me three changes and a single pass result, which cannot tell you which of the three did the work. I credited the one I had an explanation for, and the explanation went into the comment above those lines in the same commit.

The test that agreed with me

Later I went back to do it properly. One base profile with no network rules, then append rules one combination at a time and try to bind. The probe was two calls in a row:

s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
try:
  s.bind((addr, port))
  s.listen(1)
  print("bind ok")
except Exception as e:
  print("bind FAILED -> %r" % (e,))

That produced a clean table. Here are its three rows, re-run today against the same profiles:

bind only        bind FAILED -> PermissionError(1, 'Operation not permitted')
bind + inbound   bind ok
inbound only     bind ok

Row one carries a network-bind rule and fails. Row three has no network-bind rule anywhere in the profile and passes. The conclusion wrote itself: network-bind does not authorize a bind, network-inbound does.

I had the post drafted around that conclusion, and nothing I was going to do next would have dislodged it. Every test I could think of running was a test I had designed while believing it. So before publishing I went back at the draft with the opposite instruction. Assume every claim in here is wrong, and go find the reason. The headline claim lasted twenty minutes.

Read the probe again. Nothing in it can tell a refused bind() from a refused listen(). Those three rows are equally consistent with the answer I reached and with the answer that turned out to be true, so the table could not have come out any other way and it settled nothing. What I needed was a run where the two readings predict different output, and I had no such run until I pulled the two calls apart.

Which rule authorizes which syscall

Separate the two calls and the picture inverts:

$ sandbox-exec -f bindwild.sb python3 split.py 127.0.0.1 56412
  BIND_OK  LISTEN_FAIL PermissionError(1, 'Operation not permitted')

bind() succeeded. listen() is what the kernel refused. Every “denied” cell in my table that had a network-bind rule in it was a successful bind followed by a denied listen.

The rules map to syscalls the way their names suggest:

network-bind authorizes bind().
network-inbound authorizes listen() and accept().

Three syscalls, two operations, and one bracket covering the pair my probe could not tell apart. Everything I concluded from that table came from reading a listen() denial as a bind() denial.

The reason a profile with no network-bind rule can still bind is that network-bind sits underneath network-inbound in Seatbelt’s operation hierarchy. A rule written on network-inbound propagates down to network-bind when the profile has no network-bind rule of its own. That inheritance produced row three of my table, the inbound-only profile that printed bind ok with no bind rule written anywhere in it. I read that row as proof that network-bind was doing nothing. It was the inbound rule standing in for a bind rule that was not there.

Precedence is by specificity rather than by position, which is worth knowing separately because Seatbelt is usually described as last-rule-wins:

(deny network-bind)(allow network-inbound)   -> BIND_FAIL
(allow network-inbound)(deny network-bind)   -> BIND_FAIL

Same outcome either way. The rule naming the more specific operation wins no matter where it sits.

Apple’s own profiles use the split exactly this way. From /System/Library/Sandbox/Profiles/com.apple.rpcbind.sb:

;; Allow binding on our ticotsord transport
(allow network-bind (literal "/private/var/run/rpcb.ticotsord"))
;; Allow receiving on our ticotsord transport
(allow network-inbound (literal "/var/run/rpcb.ticotsord") (local tcp) (remote tcp))

One socket, two rules. The bind rule names the transport and nothing else. The inbound rule covers that same socket and adds (local tcp) (remote tcp), because receiving is a separate authorization that takes its own filters. If one operation covered both syscalls, the second rule would be redundant, and Apple would not have written it.

The kernel had been logging the answer

Both denials arrive in Python as the same object. PermissionError(1, 'Operation not permitted') is what you get for a refused bind() and for a refused listen(), and the exception carries nothing else to tell them apart. That is the layer I was reading, and it is the wrong layer. One level down, the kernel writes the name of the operation it refused to the unified log, every time.

Two profiles, the first allowing network-bind alone and the second allowing network-inbound while denying network-bind:

$ sandbox-exec -f bindonly.sb python3 split.py 127.0.0.1 57111
BIND_OK  LISTEN_FAIL PermissionError(1, 'Operation not permitted')

$ sandbox-exec -f denybind.sb python3 split.py 127.0.0.1 57112
BIND_FAIL           PermissionError(1, 'Operation not permitted')

$ log show --last 25s --predicate 'eventMessage CONTAINS "deny(1) network"' --style compact
kernel (Sandbox) Sandbox: Python(80245) deny(1) network-inbound local:*:57111
kernel (Sandbox) Sandbox: Python(80246) deny(1) network-bind    local:*:57112

Identical errors in Python, and the log separates them by operation name and prints the filter it evaluated. It needs no privileges, no modification to the profile, and no change to the program under test. Run against row one of my table, it would have printed network-inbound beside a probe whose own output said bind FAILED, and three days would have ended that afternoon.

One thing to know before leaning on it. The kernel collapses identical denial messages, so five refused listen() calls on the same port from one process produce a single log line, and a loop that fails on every iteration reads as though it failed once. Vary the port if you want to count them.

Under the pair of rules I ended up shipping, that same predicate stays silent through a full bind, listen, accept, and poll on loopback, and prints exactly one line for a dial to 1.1.1.1: deny(1) network-outbound remote:*:80.

The two rules you need

Two rules are enough to serve and poll on loopback:

(allow network-inbound (local ip "localhost:*"))
(allow network-outbound (remote ip "localhost:*"))

The outbound rule is separately required. Without it the server comes up and nothing can talk to it, including the process that started it:

step 1 bind/listen ok on 127.0.0.1:50580
step 2 accept+poll FAILED -> URLError(PermissionError(1, 'Operation not permitted'))

Remote egress stays denied under all of these, which I checked with a raw dial to a literal IP rather than a hostname, since a hostname failure could be DNS and tells you nothing about which rule refused:

raw TCP 1.1.1.1:80 -> refused: PermissionError(1, 'Operation not permitted')

Both rules lean on the localhost token, which does not mean what it looks like.

The filter is stranger than it looks

(local ip "localhost:*") is not an address match. The host component accepts two tokens and the compiler rejects everything else:

$ sandbox-exec -f bogus.sb python3 split.py 127.0.0.1 57999
sandbox-exec: host must be * or localhost in network address

localhost here means “an address belonging to this machine” rather than 127.0.0.0/8. Under an inbound rule naming localhost, a process can bind 0.0.0.0 and the machine’s LAN address, because those are addresses this host owns. The filter is doing its job. For inbound it is simply vacuous, since every address you can bind belongs to this host.

The same token genuinely constrains outbound, where the distinction is observable:

-> 192.168.50.1:80    EPERM          (the router, a different host)
-> 192.168.50.205:9   ECONNREFUSED   (this host's LAN address, permitted)
-> 1.1.1.1:80         EPERM

The port component does filter on both. Under (allow network-inbound (local ip "localhost:12345")), binding 12345 works and binding 12346 returns EPERM.

A demo I should not have trusted either

Having found that a sandboxed process can bind 0.0.0.0, I curled its LAN address from outside the sandbox, got a response, and was ready to write that a sandboxed command can publish a service to everyone on your network.

$ route get 192.168.50.205
   route to: mac
  interface: lo0

Traffic to my own LAN address never leaves the machine. My curl went over loopback, and so did every other client I could produce locally. A wildcard listener with the application firewall off is a real reason to expect LAN exposure, and netstat shows tcp4 *.40030 LISTEN. I haven’t demonstrated it. Settling it needs a second machine, which is a test I have not run.

If you are copying these rules

The (allow network-bind (local ip "*:*")) line that looks inert is the rule authorizing the bind. Deleting it is safe only when the profile has no other bind rule, because inbound then propagates down and brings its own filter with it, including the port constraint. In a profile carrying an explicit (deny network-bind) or a narrower inbound filter, removing that line breaks the server.

The comment I wrote above those lines claims three things: that network-bind does not match a localhost:* filter, that it silently refuses the bind, and that all three rules are needed. So I rebuilt the profile as it stood before that commit and added the three edits back one at a time, against a probe that binds, listens, accepts a connection, and polls itself:

bind localhost:*                        BIND_OK  LISTEN_FAIL
bind localhost:* + inbound              BIND_OK  LISTEN_OK   POLL_FAIL
bind localhost:* + inbound + outbound   BIND_OK  LISTEN_OK   POLL_OK
bind *:*         + inbound + outbound   BIND_OK  LISTEN_OK   POLL_OK

The filter I called broken matched the bind in every row, including the first, where no other rule in the profile could have authorized it. The inbound rule and the outbound rule each cleared a real failure. The last two rows are identical, so widening from localhost:* to *:* bought me nothing, and the comment explaining why I had done it was explaining a change that did nothing. Four runs, and I could have had them before I wrote a word of it. The comment comes out.

Two things that cost me time

Use a fresh port on every run. A port in TIME_WAIT from a previous successful run reports Address already in use, and when you already suspect your policy that reads exactly like a denial.

Do not let two syscalls share a try block when you are working out which one the kernel refused. I had a table, the table had real numbers in it, and it was measuring something other than what its column header said. Three days and a source comment came out of that. Both of my wrong answers survived every test I thought to run, because I designed those tests while believing the thing they were supposed to check. Neither survived twenty minutes of trying to break them on purpose.

All results are macOS 15.7.3, build 24G419, Darwin 24.6.0. None of this is documented or promised, so pin your own results to a version and re-check after upgrades.

My dive computer records 17 channels. Subsurface exports 4.

Wed, 05 Aug 2026 00:00:00 GMT

My Perdix 2 records seventeen channels per sample. The open formats can carry sixteen of them. Subsurface, the best open-source dive log there is, exports four.

Divers blame the formats for that gap. I did too, in my own README. The gap is in the exporters.

One dive, seventeen channels

A823C228#13, 101.5 metres, 101 minutes, 616 samples, open-circuit trimix. Counting the columns actually populated for that dive rather than what the schema permits:

depth, temperature, GF99, deco ceiling, time to surface, next stop depth, next stop time, CNS percent, battery percent, battery voltage, average ppO2, oxygen fraction, helium fraction, an in-deco flag, a packed status byte, a solenoid fire count, and one raw oxygen sensor reading.

GF99 is the channel divers assume is trapped in vendor formats. It reads out how close your leading tissue is to its Bühlmann limit, it lives in byte 25 of Shearwater’s sample record, Shearwater has never documented it, and I spent a fortnight proving it was what I thought it was.

Filled means the format has somewhere to put that channel. The three specifications between them cover sixteen of the seventeen. The exporter column is the one your data passes through.

UDDF standardised GF99 in 2018

UDDF is the Universal Dive Data Format, an XML schema for dive logs. From uddf_3.2.3.xsd, inside waypointType, the per-sample record:

<xs:element name="gradientfactor" minOccurs="0">
  <xs:complexType><xs:simpleContent>
    <xs:extension base="xs:float">
      <xs:attribute name="tissue" type="xs:int"/>
    </xs:extension>
  </xs:simpleContent></xs:complexType>
</xs:element>

A per-sample gradient factor, with an attribute naming which of the sixteen compartments is leading. My dive computer does not record that attribute. On this field the XML standard beats the binary format I reverse engineered.

The prose around it is ambiguous. It calls the value “80 % of the maximum tolerated saturation”, which is not how Shearwater defines GF99, and the two quantities agree only at zero ambient pressure. The usage settles it. Subsurface ships an APD Inspiration log in its test data where <gradientfactor> appears 695 times across nineteen values, climbing from 0.50 to 0.70, while that dive’s <setgflow> is 0.50 and its <setgfhigh> is 0.70. A number that walks from the diver’s low setting to their high setting over a dive is a live GF readout.

<batterychargecondition>, <cns>, <otu>, <decostop>, <nodecotime>, <setpo2>, <measuredpo2>, <heartrate> and <tankpressure> are waypoint children too. Twelve of my seventeen. <decostop> is the next stop, which is a different number from the current ceiling. UDDF has an element for the first and none for the second.

DL7 had the ceiling in 2006

UDDF has no ceiling and no status byte. DL7 had both in 2006, twelve years earlier.

DL7 is Divers Alert Network’s format, built for Project Dive Exploration. The specification is a Word document by Petar J. Denoble dated July 2006, and it survives in the docs directory of an archived Python library. Segment attribute tables, HL7-derived delimiters, coded value tables for computer identifiers and gas codes.

Its per-sample segment is ZDP. Two of its fields:

7 | 6 | NM | O | Current Ceiling | Value
9 |   | ST | O | Warning number  | Warning number(s) in Hex as defined within the
                                   specific Unit. May contain more that one Warning
                                   Number separated by character ^

A per-sample deco ceiling, and a per-sample hex warning word whose bits each computer defines for itself. The second one is a packed status byte under another name. DAN specified both eight years before my dive computer existed.

UDDF kept growing after 2018. None of the growth is on its website.

UDDF 3.3.0 was never published. Software writes it anyway.

UDDF 3.2.3, dated 15 November 2018, is the newest published release. A 3.3.0 exists too, in files people are writing today.

Subsurface’s test data contains dives/test-apd-inspiration.uddf. It opens <uddf version="3.3.0"> and its waypoints carry this:

<ppo2 ref="o2sensor_c1_1">35000.0</ppo2>
<batteryvoltage ref="battery_c1_1">5.6</batteryvoltage>
<batteryvoltage ref="battery_c1_2">5.7</batteryvoltage>

Battery voltage per waypoint, per device, so a rebreather with two controllers records both. Elsewhere in the same file, <timetosurface>60.0</timetosurface>. Individual oxygen cells get their own values instead of one averaged number.

That revision was never released. The UDDF site lists 3.2.1 as current and serves a certificate that expired in July 2025 for an unrelated hostname. The schema directory stops at 3.2.2. Meanwhile commercial dive software writes 3.3.0 files and an open-source dive log keeps one as a test fixture.

Across all three specifications, sixteen of my seventeen channels have somewhere to go. The seventeenth is the count of times a rebreather’s solenoid fired to inject oxygen, and nothing has a field for it.

The standard’s most recent useful version is a private agreement between vendors. You find it by reading files.

Subsurface reads more UDDF than it writes

Subsurface’s internal sample structure, core/sample.h, holds time, stoptime, ndl, tts, rbt, depth, stopdepth, temperature, ten cylinder pressures, setpoint, six oxygen sensor readings plus one computer-reported ppO2, bearing, cns, heartbeat, sac, and an in-deco flag. That is enough for most of my dive.

Its UDDF exporter is xslt/uddf-export.xslt, 773 lines with two waypoint-emitting branches. Between them they write seven elements: depth, divetime, temperature, tankpressure, switchmix, alarm, heading. The file contains no xsl:element and no dynamic construction, so that list is complete.

Subsurface holds ndl, tts, cns, setpoint, oxygen sensors and heart rate in memory. UDDF has elements for all of them. The exporter writes none.

Import is a different story. It runs through a second stylesheet, xslt/uddf.xslt, 1050 lines, which handles alarm, heading, cns, otu, batteryvoltage, setgflow, setgfhigh, scrubber and divemode. It will ingest the per-cell <ppo2> values and battery voltages from a 3.3.0 file that no published schema describes.

So one project ships a reader that understands the rich half of UDDF and a writer that emits seven elements of it. Whatever a Subsurface user exports, the next program reads a thinner dive than Subsurface could have described.

For my 101 metre dive: seventeen recorded, sixteen representable, four written. Depth, time, temperature, gas.

None of this is a complaint about Subsurface, which is free, excellent, and the reason most divers have their data at all. That XSLT is old, XSLT is a miserable language to extend, and UDDF export matters far less to its users than supporting the next computer over USB. Jef Driesen, who maintains libdivecomputer, is thanked in UDDF’s own author list. These people have done far more for open dive data than I have. The data is thinner at the end of the chain anyway, and UDDF’s authors saw that coming.

UDDF has a slot for the raw bytes. Nobody fills it.

UDDF has an element called <dcdump>: the dive computer’s raw memory, bzip2-compressed and base64-encoded. The rationale:

Such file can be useful as dive computer data backup; of course the backup usually cannot be restored to original device but UDDF software can access it later, i.e. to perform another, more complete conversion into UDDF format

Somebody in 2018 had watched this happen often enough to add a place for the original bytes, so a better decoder could revisit them. It sits under <divecomputercontrol>, beside the log instead of inside it. I have never seen a tool emit one.

That is the design of my own store, which keeps original bytes content-addressed and treats the decoded database as a rebuildable view. I built it because I thought the formats had failed me. The formats had done their part. Every program that touches them implements the subset its author needed that week, that subset becomes the format in practice, and you learn which subset it was years later, when you finally have a question.

The same thing happens outside diving

Dive logs are small enough that you can count every field and read every exporter in an afternoon. That is the only unusual thing about this case.

RFC 5545 gives calendars six component types and three alarm actions. Google’s CalDAV API supports four of the six and two of the three, and Google’s own developer documentation is where you find that out: “Doesn’t support VTODO or VJOURNAL data”, “Doesn’t support the AUDIO action”. Any format where a committee wrote a large specification and a few libraries implemented the parts their authors needed works this way, healthcare records and geospatial data included.

So stop grading a format by its specification. Grade it by what the tools you actually use will read and write. That intersection is smaller than the spec, smaller than any single tool’s internal model, and you can measure it in an afternoon. Then keep the original bytes, because the intersection shrinks to whatever two programs happened to agree on, and it is never the part you will want.

Sources

UDDF 3.2.3, the newest published release:

The documentation index, dated 15 November 2018
uddf_3.2.3.xsd, the schema itself, which the documentation’s own schema chapter does not link
<waypoint>, <gradientfactor> and <dcdump>

DL7:

DL7 Standard, Divers Alert Network, Petar J. Denoble, July 2006. A Word file in the archived PyDL7 repository, which is the only copy I could find.

Subsurface, read at master:

core/sample.h, what it holds per sample
xslt/uddf-export.xslt, what it writes
xslt/uddf.xslt, what it reads
dives/test-apd-inspiration.uddf, the UDDF 3.3.0 file

Calendars:

RFC 5545, the component types in sections 3.6.1 to 3.6.6 and the alarm actions in section 3.8.6.1
Google’s CalDAV API developer guide, under Specifications, where the unsupported list lives

Channel counts come from my own store, one query per column across the 616 samples of A823C228#13. The decoder is bottomtime and the format work behind it is pnf.

Two wrists, one dive

Thu, 23 Jul 2026 00:00:00 GMT

Technical divers wear two computers for redundancy. If one dies 75 m inside a cave or at 90 m on a wreck, and the other can get you out. Mine are a Garmin Descent Mk3i on the left wrist and a Shearwater Perdix 2 on the right, and they both log everything. Which means every dive I do produces two files that describe the same hour underwater and agree on almost nothing administrative. Neither the start time nor the duration, and sometimes not even how many dives it was. bottomtime’s job is to figure out which logs are the same physical dive without ever merging them. This post is about how.

Nobody agrees what time it is

Take my deepest dive, #202, 101.5 m at the Drop Off dive site in Tulamben. The Perdix says it started at 2025-09-27 11:55:43. The Garmin says 2025-09-27 03:56:02. Both are right.

The Perdix logs its epochs on a wall clock with no timezone attached. That 11:55:43 is Bali time wearing a UTC costume. The Garmin records true UTC and, separately, the local offset (+480 minutes). Subtract eight hours from the Perdix’s number and you get 03:55:43 UTC, 19 seconds before the Garmin’s start. Nineteen seconds is me pressing two buttons on two wrists while I do my surface swim to the entry point.

But “subtract eight hours” was the answer, and the matcher’s first problem is that the Shearwater log never states the question. So it borrows the answer from the other wrist. For each Shearwater dive, try the UTC offsets observed on Garmin dives nearby in calendar time (all 84 of my matched pairs so far are +480, but the code doesn’t assume that. When I dive somewhere else, the Garmin will have logged the new offset). Under each candidate offset, look for interval overlap of at least half the shorter log, with a sanity check that the shorter log isn’t somehow deeper than the longer one.

Passing the overlap check only makes a pair a candidate. Acceptance rides on the depth profiles.

The depth profile is the fingerprint

Two computers strapped to the same body sample the same physical curve, and depth over time is the only channel they share. (GPS is no help. Seawater absorbs the signal within a metre of the surface, so the Garmin has a fix for the entry point and nothing after it.) The matcher takes both depth series, lays them on an in-memory 10-second grid, and slides one against the other across lags of ±300 s, scoring each shift with normalized cross-correlation. The stored samples are never resampled. The grid exists only long enough to compute the score.

Here are the two real series from dive #202. Drag the lag and watch the score respond:

(The widget runs the same resample-and-correlate code as the matcher, ported to ~40 lines of JS, on the actual logs downsampled to the 10 s grid. If it’s not rendering, the data file didn’t load, and hiding beats faking.)

Two things show up while you drag. The score peaks at +20 s. After the whole-hours offset is removed, the residual disagreement between my two wrists is 20 seconds, and the matcher stores that number (residual_skew_s) rather than correcting either series. And the peak is gentle. Even 120 seconds off, this dive still scores above 0.95 because a deco profile is mostly slow monotonic segments. That gentleness is why the acceptance threshold is a greedy 0.95 with an ambiguity rule. If two candidate Garmin dives score within 0.02 of each other, the matcher refuses to choose and flags the dive for human review instead.

On the full-resolution series the winning score for this pair is 0.99976, with a duration disagreement of 43 seconds and a max-depth disagreement of 0.37 m.

Then I took it to a pool

Cross-correlation has a failure mode. It measures the agreement of variation, and a pool session has almost none. Go 3 m down, stay there, come up. Normalize a flat line and you get noise.

My worst real case: one afternoon of skills practice on 2025-12-17. The Garmin logged it as a single 113-minute activity, max depth 8.2 m, because the Descent keeps recording through surface pauses. The Perdix, which cuts a log whenever I surface for more than a moment, cut eight logs over the same afternoon, two of them 46-second and 116-second blips too brief to pair with anything. The six that could pair ran from 345 s to 1,382 s. The two deeper drills matched on profile shape (0.991 and 0.975). The four shallow ones scored as low as 0.559, hopeless as correlations, while being blatantly the same water on the same day.

So there’s a second acceptance path for this shape of failure. If the interval overlap is near-total (at least 0.85) and the median absolute depth difference at the best lag is 0.75 m or less, accept. Two instruments on one body must agree on depth even when the profile is too boring to correlate. Thirteen of my 84 matches came in through that door, all of them pool sessions and shallow drills.

None of these thresholds is principled. They’re placed in the gaps of the observed distributions. Profile-accepted pairs score between 0.962 and 0.99997 against the 0.95 bar, and the pool path’s observed depth deltas run 0.07 to 0.20 m against the 0.75 m ceiling, so each rule has room on both sides. Duration is not used as evidence at all. The deltas run from 1 second to 6,463 seconds (median 48 s, but 26 of the 84 pairs disagree by more than five minutes), all of it the Descent’s surface-pause habit, none of it saying anything about identity.

A match is just a row

When a pair is accepted, nothing is merged. The match is a row: which Garmin log, which Shearwater log, the clock offset (+28,800 s), the residual skew, the score, and which rule accepted it. Both sample series stay exactly as their computers wrote them. Canonical dives are then just the connected components of those links, which handles the pool afternoon gracefully. One canonical dive with seven members, one Garmin log and six Shearwater logs, nothing pretending the Perdix’s six dives didn’t happen.

And 84 matches does not mean 84 out of 84. My Shearwater computers have logged 142 dives, and the 58 unmatched ones are all accounted for. Forty-nine happened on days I wasn’t wearing the Garmin at all, mostly early confined-water sessions and a CCR (closed-circuit rebreather) course where my second computer was a Petrel 3 instead (the matcher only links Garmin to Shearwater, so those two-Shearwater days stay unlinked, a limitation I haven’t needed to fix yet). Eight are bench tests under two metres or two minutes. Exactly one, a six-minute 2.5 m bob during a surface interval on 2025-12-19, overlapped a real Garmin log and was still rejected, correctly, by both rules.

The residual skews are their own small dataset. The median is +20 s across my 84 pairs, 53 of them within ±30 s, worst case +190 s on a day I apparently took my time pressing the second button. If I ever want to know how my two computers’ crystals drift apart over months, the numbers are sitting in a table, because correcting a disagreement in place would have destroyed the record of it.

The verification suite closes the loop. After alignment, the median depth disagreement across all 84 matched pairs, median-of-medians, is 0.162 m. When I ask bottomtime plot for dive #202, both profiles land on one time axis using the stored offset and skew, no resampling, and the two lines sit on top of each other.

Proving byte 25

Wed, 22 Jul 2026 00:00:00 GMT

In the last post I claimed that byte 25 of a Shearwater dive sample is GF99 (the live readout of how close your tissue load is to the decompression model’s limit), byte 24 is the deco ceiling, byte 18 is battery voltage, and bytes 26-27 are the @+5 prediction, the time to surface if you stayed five more minutes. Shearwater has never documented any of that. Which means the claim is exactly as good as the evidence behind it, and “the curve looked right on a few dives” is the kind of evidence that decodes your tissue loading wrong for a year before you notice.

So bottomtime treats every byte mapping as a hypothesis and ships a verification suite, bottomtime verify, that tries to falsify all of them against every dive in the store. The suite has four independent oracles, and a fifth turned up later from outside it. None of them is my own opinion of what a plausible dive looks like.

Oracle 1: the vendor’s other rendering

Shearwater Cloud can export a dive as XML. The XML is lossy (it’s missing the very channels I had to reverse), but the channels it does carry were produced by Shearwater’s own code reading the same native log. That makes it a per-sample answer key for the documented offsets. If my decoder and their exporter disagree about sample 371’s depth, my decoder is the one that’s wrong.

The suite pairs each XML file with its dive by start time and compares nine channels sample by sample:

XML field	decoded column	tolerance
`currentDepth`	`depth_m`	0.051 m
`ttsMins`	`tts_min`	0.001
`firstStopDepth`	`stop_depth_m`	0.11 m
`fractionO2` / `fractionHe`	`o2_pct`, `he_pct`	0.001
`averagePPO2`	`avg_ppo2`	0.011
`waterTemp`	`temp_c`	0.51 C
`batteryVoltage`	`battery_v`	0.011 V
`currentNdl`	`ndl_min`	0.001

The tolerances exist because the two renderings round differently. The log stores depth as decimetres and the XML prints it as a float, so half a decimetre of slack covers the rounding without covering any real mistake. Byte 18 earns its place here too. The XML’s batteryVoltage equals the byte’s value divided by 100 on every sample, which is what promoted it from guess to mapping.

The current run verbatim against my 142 dives with Shearwater dive computers:

xml crosscheck: 142 dives, worst deviation 0.0000 at #13[86].depth_m

Every shared sample of every dive agrees exactly. The “worst” deviation is a rounding artifact so small it prints as zero.

Oracle 2: the vendor’s own arithmetic

The XML can’t vouch for GF99, since GF99 is one of the fields it omits. But Shearwater Cloud computes a summary statistic per dive called EndGF99, the gradient factor at surfacing, and stores it in the database next to the raw blob. Shearwater’s code computed that number from the same samples I’m decoding. So if byte 25 really is GF99, then the maximum of my decoded byte-25 values over the last minutes of a dive should equal their EndGF99.

Across all 142 of my Shearwater dives:

The median disagreement is zero. The worst is a single GF point, and it shows up on two dives. On one, Cloud reads 5 where I read 6. On the other, Cloud’s EndGF99 is 0 where my tail reads 1. Both readings sit at the bottom of the scale, a leading tissue only a few percent of the way to its Bühlmann limit, so a one-point gap there is the same clean surfacing counted two ways. I take the window over the last 60 samples (ten minutes) rather than the literal final sample because GF99 keeps moving during the last metre of ascent, and Cloud’s snapshot and mine need not land on the same instant.

Oracle 3: physics

Some checks need no vendor at all. A decoded ceiling that comes out deeper than the displayed stop depth would be nonsense. Deco stops land only on a 3 m grid (3 m, 6 m, 9 m…), and the stop is the ceiling pushed to the next grid depth at least as deep, so the decoded ceiling can never sit deeper than the decoded stop. That invariant holds on all 33,773 samples in the store with zero violations. If I had picked the wrong byte for either field, this is the check that would scream, because an arbitrary byte has no reason to stay quantized-consistent with its neighbor for a hundred hours of diving.

The 0xFF sentinel behaves physically too. On my 101.5 m dive, byte 25 reads 0xFF from second 10 to second 1050, one contiguous run, and then never again. That span is the descent and the deep segment. Tissues sit under ambient pressure the whole time, with no supersaturation to report. The sentinel switches off exactly when the physics says a gradient factor starts to exist.

Oracle 4: the other wrist

I log every dive on two computers, a Garmin Descent Mk3i and the Perdix. To my knowledge, they share no firmware, sensor, nor vendor. After the matcher aligns their clocks, both series claim to describe the same physical body in the same water column, so their depth channels have to agree. For each of the 84 matched pairs the suite takes the median absolute depth difference at aligned timestamps:

twin agreement: 84 matches, median-of-medians 0.162 m, worst 0.840 m

The tolerance scales with depth, max(0.6 m, 4.5% of max depth), because the two computers convert pressure to depth with their own assumed water density (fresh, salt, or the EN13319 dive-gauge standard), and those differ by up to about 3%. Sixteen centimetres of median disagreement between two unrelated instruments is, honestly, better than I expected from physics that starts with “assume a density”.

What else byte 25 could have been

A byte that merely correlates with GF99 would sail through a sloppy check, so the live alternatives each needed disproving separately. The strongest was SurfGF, Shearwater’s other gradient number, the same supersaturation recomputed as if I surfaced right now. The two converge at the moment of surfacing, so the EndGF99 oracle alone can’t tell them apart. The sentinel can. SurfGF is well-defined and climbing through the descent and bottom phase, which is precisely the 1,040 seconds byte 25 spends reading 0xFF. CNS, the oxygen-toxicity clock, was disproven twice. It’s already documented at byte 23, and it never decreases during a dive, while byte 25 decays through every long stop. Timers, counters, and battery-like slow channels fail the same shape test. GF99 is the only quantity I know of that is undefined at depth, wakes on ascent, decays at stops, spikes at surfacing, and lands on Shearwater’s own EndGF99 142 times out of 142.

Oracle 5: the vendor’s own source

The four oracles above are behavioral. They judge byte 25 by how it acts. A fifth arrived later and judges it by construction. Shearwater Cloud ships its parser as a .NET assembly, DiveLogParser.dll, and the macOS app carries the same managed file as the Windows one, so I could read the class that decodes these records (the how is in the last post). It names byte 25 GF99_OFFSET and byte 24 DECOCEILING_OFFSET, confirming all four empirical bytes at once.

It also closed the SurfGF question harder than the sentinel did. The parse method only treats byte 25 as GF99 when the deco model uses gradient factors, and reads it as a safe-ascent-depth fraction on DCIEM, a decompression model that doesn’t use gradient factors. A generic gradient number like SurfGF would never get that branch, which pins byte 25 as GF99 in particular and closes the one gap the behavioral oracles left open. That is the strongest evidence I have, which is why I put it last. If I’d read the source first, I’d have trusted a name and skipped the checks that would have caught me decoding the DCIEM case wrong. The source tells you what the bytes are called. Only the data tells you whether your code is right.

Byte 11, which no oracle could reach

Byte 11 defeated all four behavioral oracles. It was nonzero in 32,544 of my 33,773 samples, sat between 82 and 94 through the deep dive, and correlated with no exported field, computed summary, invariant, or second computer. For a long time it stayed raw in an extra_json column, unlabeled, as one of 392,959 undecoded payloads the store keeps verbatim. The fifth oracle is what finally named it as BATTERY_PERCENT_REMAINING. And because a name in a binary is still a hypothesis, I checked it the same way as the rest, and the values bound cleanly to 0-100 across every sample, which an arbitrary byte has no reason to do. While behavior can pin some bytes down, the source has to name the rest, and both still have to survive the data.

The discipline underneath all of it is that decoded and verified are different states, and only the checks are allowed to move a byte from one to the other. What I never do is guess plausibly and move on, because a dive log is the one dataset where plausible-but-wrong is worse than missing. Nobody re-checks a number that looks right.

Every trip re-litigates everything

bottomtime verify runs against the live store, so every new trip re-litigates every mapping: 142 dives of XML agreement, 142 EndGF99 convergences, 33,773 invariant samples, 84 twin comparisons, and the counts and foreign keys besides. The empirical bytes are validated on a Perdix 2 and a Petrel 3 so far. Every mapping that clears the suite is what the pnf spec writes down and the pnf package (pip install pnf) decodes. If your Teric or Nerd 2 populates them differently, the suite is designed to catch it rather than silently mis-decode, and a failing blob in the issue tracker is the data I can’t generate myself. So please, if you have a Shearwater that disagrees with my mapping, run bottomtime verify and file the blob.

The bytes my dive computer keeps to itself

Tue, 21 Jul 2026 00:00:00 GMT

My Shearwater Perdix 2 computes a number called GF99 every ten seconds of every dive. It’s the leading tissue’s supersaturation as a percentage of its Bühlmann M-value at my current depth. In plainer words, it says how close my dissolved gas load is to the decompression model’s limit, where 100 means I’m standing on the line. It’s one of the most interesting numbers a dive computer produces, a live readout of how loaded I actually am, and Shearwater Cloud plots it. But the XML export, the thing you’d reach for to get your data out, doesn’t contain it, and neither does the CSV. Same for the deco ceiling, the per-sample battery voltage, and the @+5 prediction. The app is clearly reading them from somewhere.

So I went looking. This post is about what I found: Shearwater’s Petrel Native Format, the 32-byte record stream every Petrel-family computer logs internally, and the four bytes in it I first mapped by experiment and only later confirmed against Shearwater’s own code. The decoder ships in bottomtime, the dive-log store I wrote for my two-computer logging habit (pip install bottomtime).

Where the real log lives

Shearwater Cloud’s desktop app has three export options. XML and CSV are lossy renderings. The third, “Export Database”, writes a SQLite file called dive_data.db, and that one contains the real log. Inside there’s a log_data table with a BLOB column, one blob per dive, tagged sw-pnf. The blob layout is a 4-byte little-endian length followed by a gzip stream:

expected = struct.unpack_from("<I", blob, 0)[0]
data = gzip.decompress(blob[4:])
assert len(data) == expected
assert len(data) % 32 == 0

For my dive #202 (101.5 m at Drop Off dive site Tulamben, 102 minutes runtime), the blob is 16,388 bytes, the prefix promises 26,496, and gzip delivers it. That unpacks to 828 records of 32 bytes. The whole dive which is one record every 10 seconds plus bookkeeping, is compressed smaller than a photo on your phone. Wow \(*o*)/!

Byte 0 of each record is its type. In that dive the stream looks like this:

type	meaning	count in dive #202
`0x01`	dive sample	616
`0x10`-`0x19`	opening (config) records	1 each
`0x20`-`0x29`	closing records	1 each
`0x30`	info event	7
`0x00`	all-zero padding, skipped	4
`0xFF`	final record (serial, model, firmware)	1
`0x51`, `0x70`-`0x75`, `0x80`-`0x87`, `0xA0`, `0xA1`	no idea (yet)	180

The opening records carry the dive’s configuration: the ten programmed gases, gradient factor settings, units, water density, atmospheric pressure, sample interval. The closing records mirror them with end-of-dive state.

The last row is the one I care about. Every record type the decoder doesn’t understand is stored verbatim in the database, per dive, in stream order. Nothing is dropped just because I can’t read it today.

Standing on libdivecomputer

Luckily, I didn’t have to start from scratch. The open-source libdivecomputer project has supported Petrel-family computers for years, and its shearwater_predator_parser.c handles PNF explicitly. A comment in the parser notes that relative to the legacy Predator format “the samples are simply offset by one”, because PNF prepends the record-type byte, and it documents the sample layout: depth in the first two bytes of a sample, then next stop, TTS, gas fractions, temperature, CNS. Reading that parser is how I knew what the documented bytes were, and how to check my own mapping against them.

That parser is as close to a public spec as PNF gets. Shearwater does write protocol documentation, but hands it only to registered developers through a formal process. Subsurface imports the desktop database by querying its pre-decoded tables and never opens the blob. So for the undocumented bytes, there was nothing left to look up.

Lay the documented offsets over a PNF sample record and 23 of its 32 bytes are accounted for, counting the type byte. The other nine are either padding or they’re the fields the exports omit, and Shearwater Cloud clearly reads something more, because it plots GF99.

Reading the record, byte by byte

Here is the actual record from the deepest sample of dive #202, the moment the gauge read 101.5 m. Hover (or tap) any byte:

documented (libdivecomputer) found empirically named later, from Shearwater's own DLL

01 03 f7 00 21 00 35 79 0b 3f 01 5b 12 00 19 00 00 00 98 46 ff ff ff 04 1f ff 00 55 ff ff ff ff

The documented bytes check out against what my wrist showed at 101.5 m: 11/63 trimix, TTS 53 minutes, next stop 33 m, CNS 4%, water 25 C. The arithmetic is nothing fancier than big-endian shorts and a divide by ten.

The four bytes I had to find myself

Bytes 18, 24, 25 and 26-27 are the ones libdivecomputer doesn’t read and the exports don’t include. Mapping them was an empirical process where I had to stare at the values across hundreds of dives, form a hypothesis, and try to falsify it.

Byte 25 was an interesting light bulb moment. It reads 0xFF early in a dive, wakes up as tissues load, decays during long shallow stops, and spikes right at surfacing. That is exactly how GF99 should behave, and it converges on the EndGF99 value Shearwater Cloud computes for each dive. Byte 24 hugs the depth profile from below during deco, always sitting at or just under the 3 m-quantized stop depth, and zeroes the moment deco clears. That’s the ceiling! Byte 18 matched the XML export’s batteryVoltage times 100 on every sample I checked. Bytes 26-27 tracked TTS but ran ahead of it by a sensible margin, which is the @+5 display field, TTS if I stayed five more minutes.

Here’s dive #202 with the empirical bytes plotted. Ceiling in red riding under the grey quantized stop depth and GF99 waking up at the 17-minute mark once the tissues actually have something to report:

But a curve that behaves right is still circumstantial. Hmm (╭ರ_•́)..

So I wrote a verification suite that cross-checks every decoded channel, per sample, against Shearwater’s own outputs across all 142 of my Shearwater dives, and it deserves its own post. Short version: the worst disagreement between my decoder and Shearwater’s XML across every shared sample is 0.0000, and byte 25’s surfacing value agrees with Cloud’s EndGF99 on 142 of 142 dives within one GF point.

Then I read Shearwater’s own code

I could have stopped at the verification suite. Then a Reddit comment pointed out that Shearwater Cloud ships its parser as a .NET assembly, DiveLogParser.dll, and that you can open the class PetrelNativeLogV14Parser in a decompiler and read the format directly. That’s a Windows workflow, and I’m on a Mac, but the macOS build of Shearwater Cloud is a Unity app and it ships the same managed DLL under Shearwater Cloud.app/Contents/Resources/Data/Managed/. It turned out to be real .NET bytecode which I could read.

The byte offsets are stored as named constants right in the assembly metadata, so I didn’t even need a decompiler. dnfile, a pure-Python reader, dumps them straight out. Here is what Shearwater calls the bytes I had mapped by hand:

GF99_OFFSET                = 25
DECOCEILING_OFFSET         = 24
BATTERYVOLTAGE_OFFSET_LSB  = 18
ATPLUSFIVE_1ST_BYTE_OFFSET = 26

Four for four, sweet! And three bytes I’d filed under “no idea” got names in the same table. Byte 11 is BATTERY_PERCENT_REMAINING, which is why it sat in the eighties and nineties and, I checked, never once exceeds 100 across all 32,544 samples. Byte 17 is BATTERYVOLTAGE_OFFSET_MSB, the high byte of a sixteen-bit battery voltage whose low byte is 18, reading zero only because 1.5 volts never needs the high byte. Bytes 30-31 hold surface air consumption. Shearwater’s constant calls it RESPIRATORY_MINUTE_VOLUME but the parse method reads it as SAC and divides by 100. Either way it’s 0xFFFF for me, since I dive without gas integration.

Decompiling the actual parse method with ILSpy caught something that wasn’t obvious from the constants alone. Bytes 24 and 25 are ceiling and GF99 only when the decompression model uses gradient factors:

if (headerDecoModel == 3) {   // DCIEM
    byte b = logData[offset + 24];
    float frac = logData[offset + 25] / 100f;
    sample.SafeAscentDepth = b - frac;
} else {                      // GF, VPM-B, VPM-B/GFS
    sample.DecoCeiling = logData[offset + 24];
    sample.Gf99 = logData[offset + 25];
}

On a DCIEM dive those two bytes are a safe-ascent depth instead, and my decoder, which reads byte 25 as GF99 no matter what, would mis-decode it. All 142 of my dives run gradient factors, so the bug has never fired. My data just never contained the case that breaks my code. It’s on the fix list now though, found by reading the vendor instead of my own logs lol.

What’s still dark

Even the record types I’d shrugged at earlier were mostly named once I looked. 0x70-0x75 are tissue-loading snapshots, 0x80-0x88 are validity bitmaps for the opening and sample records, 0x60 is a surface ppO2 log. Three types in my dives are still unexplained. 0x51, 0xA0, and 0xA1.

They stay in the undecoded_payloads table (392,959 rows and counting, Garmin’s mystery FIT messages included), verbatim, because the blobs are archived and the decoder is versioned. When someone works one out, I re-decode from the archive and every past dive gains the channel, no re-download needed, even if the computer that logged it is long dead by then. That archive-first design is its own story.

I wrote all of this up as a standalone format specification, and pulled the decoder into its own pure-Python package for anyone who wants to read Petrel logs without the rest of my dive store: pnf (pip install pnf).

The mappings above are validated on a Perdix 2 and a Petrel 3 (the latter driving a Choptima CCR). Other models in the family may populate these bytes differently. If yours disagrees, bottomtime verify will say so loudly, and I would love the failing blob. Issues welcome.

My homepage has a pulse

Thu, 16 Jul 2026 00:00:00 GMT

There is a small red heart on my homepage now that beats. The number next to it is my actual heart rate, and the heart pulses at that frequency: 48 bpm means one thump every 1.25 seconds, 80 bpm means one every 0.75. I wrote this up because I wanted to, and because the design ended up containing more interesting decisions than I expected.

The watch

I wear a Garmin Descent Mk3i, 43 mm, carbon gray DLC titanium. I bought it as my primary dive computer, alongside my Shearwater Perdix 2, because you need redundancy in technical diving and overhead environments like caves and wrecks. If a computer dies at 75 m inside a cave, you can’t just surface and charge it. Cave divers have backups for their backups. Since April 2025 it has been on 400+ dives with me, including to 100 m.

I don’t take it off unless I’m charging it or showering. Between dives it tracks my heart rate, sleep score, HRV, steps, calories, and a dozen other vitals around the clock, and syncs everything to the Garmin Connect app on my Pixel 9 Pro. My gym and streetlifting sessions get logged on it too. Which means Garmin already holds a continuous, granular physiological record of my life. I just needed the data out of Garmin and onto my homepage.

A more personal “online” status indicator

Old messaging apps had the green dot. Facebook Messenger, Gchat before it: a small indicator that said this person is here right now. I remember seeing it in HexChat in 2012, before I moved to Irssi. I wanted that for this site, except the green dot always felt like a statement about a socket instead of a person. A heartbeat is the least abstract “online” there is. It’s the difference between “his browser has a connection open” and “he is alive, and here is the proof”. In the end it’s just a number on a page, I get that. But it is my number, from my chest, and that makes it the most personal thing on the site. It might be a little cringe to a lot of you, but I like the idea :P.

The longer-term goal, when I get around to it, is a section of this site that shows a granular ledger of my life made public: workouts, sleep, HRV, calories burned, steps walked, etc. I understand what that leaks. All-day heart rate reveals when I sleep and wake. Gaps and timezone shifts reveal travel. A resting heart rate that jumps 8 bpm flags illness, stress, or last night’s drinks. HRV trends are a decent proxy for mental state, and a regular gym schedule advertises exactly when I am not home. I have sat with all of that and decided I am fine with it. I consent to it, and if you have a problem with it, that is your problem, not mine ¯\(ツ)/¯.

Surely someone has done this. Yup, they have.

My first thought was that this must already exist, and I was right. Twitch streamers have been putting live heart rate on screen for years through services like Pulsoid and HypeRate, and both support Garmin watches. They work by putting the wearable into its broadcast heart rate mode, where it streams over BLE/ANT+ to a phone app that relays the number to their servers. Seemed promising at first, but it was a deadend for me. Garmin’s manuals warn that broadcasting decreases battery life, and this watch is my dive computer. I’m not going to trade dive-day battery for a website widget. And broadcast is a mode you switch on and stay in range of. Seems like too much of a chore to me. Plus, I like that I charge the watch once a week and it lasts for 10 days. I don’t want to charge it every night just to keep a widget alive.

The browser itself can speak Bluetooth, which briefly seemed promising. The Web Bluetooth API can read the standard heart rate GATT service directly from a nearby device. Still a no-go because it connects the visitor’s browser to a device near the visitor. It would only ever work on my own machine, in a Chromium browser, with the watch in the broadcast mode.

Surely then there is a real API. There is: Garmin’s Health API serves this data, BUT access is gated behind a business application and review. It is built for companies integrating wearables into products. What about a guy who wants a heart on his homepage? Fahhhh (╯’□’)╯︵ ┻━┻.

At this point I did what I should have done first and searched GitHub and Reddit. Two projects kept surfacing in Reddit threads. GarminDB downloads your entire Garmin history into a local SQLite database and gives you analysis and plots on top. Cool, but wrong for this. An archive with a schema seems like overkill… I probably just want one day of heart rate as JSON every morning. python-garminconnect is the right tool because it is a Python client that speaks the same private API the Garmin Connect app uses. It lets you log in the way the app does and calls the same endpoints as the app. It exposes essentially everything I can see on my phone, which incidentally solves the someday-in-the-future-project problem, because workouts, sleep, and HRV are all reachable from the same client. Nice, thank you @cyberjunky!

Why not live?

Live was never actually on the table, but I didn’t know that at the start. The watch does not stream anywhere by default: all-day heart rate accumulates on the wrist and reaches Garmin’s servers in batches, whenever the watch↔phone sync happens. Without broadcast mode there is no feed to subscribe to. And my site is static, Astro built once and served as files from GitHub Pages. There’s no server to hold a websocket open, so even if the watch did stream, I didn’t figure out a way to relay it to visitors. The only option is to fetch the data once a day and replay it.

So the compromise is that the widget replays yesterday. When you load my homepage at 14:32 your time, you see my heart rate at 14:32 yesterday, interpolated from real samples. The “live” label is a white lie, but it’s mine to tell. But at least the lie is precise: the offset is exactly 24 hours, the data is real, and nothing is made up beyond drawing straight lines between measurements.

python-garminconnect logs in with an email and password, but what it hands back is a pair of OAuth tokens. The OAuth1 token is the long-lived one, good for about a year. The OAuth2 token is what actually authorizes API calls, and it expires after a few hours. The library signs every request with the OAuth2 token, and whenever that one expires, it uses the OAuth1 token to generate a fresh one behind the scenes (my limited understanding). So my creds get typed once on my laptop, and never go anywhere near CI. CI only holds tokens which refresh themselves for a year.

The one-time part looks like this:

python3 -m venv .venv
source .venv/bin/activate.fish
pip3 install garminconnect
python3 scripts/garmin_login.py

garmin_login.py is a short script that prompts for credentials and handles the token plumbing. The interesting part:

garmin = Garmin(email=email, password=password, return_on_mfa=True)
status, client_state = garmin.login()
if status == "needs_mfa":
    mfa = input("MFA code: ")
    garmin.resume_login(client_state, mfa)

garmin.client.dump(str(tokendir))            # token files -> ~/.garminconnect
b64_path.write_text(garmin.client.dumps())   # same tokens, one base64 string

return_on_mfa=True makes the library pause and hand control back instead of dying when Garmin asks for a two-factor code, and resume_login finishes the handshake with whatever I type. At the end the tokens land in two forms: as files in ~/.garminconnect for local runs, and as a single base64 blob (client.dumps() serializes the whole token store into one string) that becomes a GitHub Actions secret:

gh secret set GARMINTOKENS_BASE64 --repo snehankekre/snehankekre.github.io < ~/.garminconnect.b64

I had problems with the login. Garmin rate-limited my IP mid-attempt, even though I rotated through Mullvad’s servers. The library tries a chain of login strategies, and the first two (mobile+cffi, then mobile+requests) both came back with a 429 (“IP rate limited by Garmin”). Thankfully the library’s fallback strategy further down the chain succeeded. It was a one-time toll either way: CI never logs in at all, it only replays tokens, so the login endpoint never sees it again.

Fetching yesterday, every morning

My site uses Astro, built and deployed to GitHub Pages by an Actions workflow. I figured if CI builds the site, CI can fetch data right before the build, and the data would ship as a static file like everything else. So there’s a small Python script, fetch_hr.py, that runs before every build. The heart of it:

tokens = (
    os.environ.get("GARMINTOKENS_BASE64", "").strip()
    or os.environ.get("GARMINTOKENS", "~/.garminconnect")
)
garmin = Garmin()
garmin.login(tokens)

tz = ZoneInfo(os.environ.get("HR_TZ", "Asia/Makassar"))
target = (datetime.now(tz) - timedelta(days=1)).date()

data = garmin.get_heart_rates(target.isoformat())
points = [
    [ts, bpm]
    for ts, bpm in (data.get("heartRateValues") or [])
    if bpm is not None
]

garminconnect 0.3.x accepts either a token directory or the base64 blob in the same login() call: any string longer than 512 characters is treated as a blob, anything shorter as a path. Which means the identical script runs on my Mac (reading ~/.garminconnect) and in CI (reading the secret), with no branching on my side.

The timezone part was a little tricky for me to figure out. It would have silently served wrong data. GitHub runners live in UTC. My 3 a.m. Bali cron fires at 19:00 UTC, which is still the previous day in UTC. So on Wednesday 3 a.m. my time, a naive date.today() - timedelta(days=1) on the runner computes Monday, and I’d be replaying a two-day-old heart. “Yesterday” has to be computed in the timezone where my wrist lives, hence HR_TZ=Asia/Makassar, passed in from the workflow so it’s a one-line change next time I move.

The if bpm is not None filter matters too because whenever the watch is off my wrist, Garmin records nulls rather than nothing, and I want holes in the data to look like holes (more on that below). What survives gets written to public/hr.json, about 14 KB:

{"date":"2026-07-13","resting":48,"points":[[1783872000000,56],[1783872120000,56],...]}

Roughly 717 samples, one every ~2 minutes, timestamps in absolute epoch milliseconds GMT. That absoluteness matters later on.

A note on that 2 minute cadence, because it confused me: it is not how often the watch measures. The optical sensor samples multiple times per second, all day, and Garmin’s official Health API (the gated one) serves all-day heart rate as 15 second representative samples. The 2 minute series is what the Connect app’s daily chart endpoint aggregates down to, and since that is the endpoint python-garminconnect reads, that is what I get. If I ever want finer than 2 minute resolution without Health API access, there is a path I may look into: recorded activities (dives, gym sessions) are stored as FIT files with 1 second samples, and python-garminconnect can download those too. But for now 2 minutes is plenty for a heartbeat widget.

The other rule I gave the script: it is not allowed to fail. The entire body runs inside one try/except that ends like this:

except Exception as e:  # never break the build
    print(f"hr fetch skipped: {type(e).__name__}: {e}")
return 0

If the watch didn’t sync, tokens expired, or Garmin is down, it logs the reason, exits 0, and writes nothing. The build proceeds, the widget finds no data and hides.

The workflow side is two cron triggers and one step in front of the Astro build:

schedule:
  # 03:00 in Bali, once the day is complete,
  # and 12:00 to catch a late phone sync.
  - cron: '0 19 * * *'
  - cron: '0 4 * * *'

- name: Fetch heart-rate data
  env:
    GARMINTOKENS_BASE64: ${{ secrets.GARMINTOKENS_BASE64 }}
    HR_TZ: Asia/Makassar
  run: |
    pip install --quiet garminconnect==0.3.6
    python scripts/fetch_hr.py

The version is pinned to 0.3.6 because this is an unofficial client of a private API: I validated the token handling and response shapes against that exact version, and I’d rather a future breaking change fail loudly than drift silently. The JSON lands in public/, Astro copies public/ verbatim into the build, and GitHub Pages serves it like any other file. I don’t have to commit anything: public/hr.json is gitignored, so the repo doesn’t accumulate a year of heartbeat diffs, and the data lives only inside the deployed artifact.

Replaying it exactly 24 hours later

The client side is maybe 40 lines of vanilla JS inlined in the page. The browser fetches /hr.json and, every 10 seconds, evaluates a single target instant:

const bpm = sample(Date.now() - 86_400_000);

That subtraction is why the epoch-milliseconds detail from earlier mattered. Garmin’s timestamps and Date.now() are both absolute GMT milliseconds, so a visitor in Tokyo and a visitor in New York ask the same question, “what was his heart doing exactly 24 hours ago”, and get the same correct answer.

Here’s sample() in full (the repo version has TypeScript annotations, trimmed here):

const first = pts[0][0];
const last = pts[pts.length - 1][0];
const DAY = 86_400_000;
const GAP = 20 * 60 * 1000; // holes wider than this are "no signal"

const sample = (t) => {
  if (t > last) t -= DAY; // just past the data day (pre-refresh); replay it
  if (t < first - GAP || t > last + GAP) return null;
  let prev = null;
  let next = null;
  for (const p of pts) {
    if (p[0] <= t) prev = p;
    else { next = p; break; }
  }
  if (!prev) return next && next[0] - t <= GAP ? next[1] : null;
  if (!next || next[0] - prev[0] > GAP) return t - prev[0] <= GAP ? prev[1] : null;
  const f = (t - prev[0]) / (next[0] - prev[0]);
  return Math.round(prev[1] + f * (next[1] - prev[1]));
};

Yeah, I know that’s a O(n) linear scan to find the bracketing pair. It’s 717 points once every 10 seconds. But a binary search here would be cleverness with no payoff.

Garmin samples every ~2 minutes, and a number that changes once per 2 minutes reads as dead. It would be boring from a visitor’s perspective, and it would be boring from my perspective too: I want to see the widget move. So between the two real samples bracketing the target instant, the widget reads off the straight line connecting them. Say the target t falls at 10:00:50, between a sample of 62 bpm at 10:00:00 and one of 68 bpm at 10:02:00:

68 ┤                      ● next
   │          ◌ ← reading at t
62 ┤ ● prev
   └─┬────────┬───────────┬──
   10:00  10:00:50      10:02

The fraction of the way across is f = 50/120 ≈ 0.417, and the reading is 62 + 0.417 × (68 − 62) = 64.5, rounded to 65:

const f = (t - prev[0]) / (next[0] - prev[0]);   // 0 at prev, 1 at next
return Math.round(prev[1] + f * (next[1] - prev[1]));

Ticking every 10 seconds walks the reading along that chord: 62, 63, 63, 64, 65. Every displayed value lies on a straight line between two real measurements, which is what I gathered Garmin app’s own daily graph draws.s

But interpolation has a hidden assumption that the bracketing samples are adjacent, two minutes apart. The data has real holes because sometimes the watch is on a charger or on my sink while I shower, and Garmin records nulls instead of nothing. Suppose it was off my wrist from 14:00 to 17:00. The naive code would take 71 bpm at 14:00 and 58 bpm at 17:00 and happily “interpolate” three hours of smooth, plausible, entirely invented readings between them. The guard is a threshold I set at 20 minutes. So if the two samples around the target are further apart than that, the widget refuses to interpolate and hides instead. Twenty minutes is comfortably wider than the data’s normal cadence. I checked a full day of my own data: 715 of 716 consecutive samples were exactly 2 minutes apart, through the night included, and the one exception was an 8 minute gap at 09:20 that was almost certainly my shower. So ordinary data never trips the guard, and it’s narrow enough that a real gap hides the widget quickly rather than presenting a three-hour-old number as current.

The last problem is midnight. The file covers one Bali calendar day and refreshes at 3 a.m., so between midnight and the refresh, now − 24h points past the end of the data, and the widget would vanish every single night. It ends up being a one-line fix in the same place as the staleness guard:

if (t > last) t -= DAY;  // just past the data? replay the same time, one day earlier

A visitor at 1:30 a.m. briefly gets 1:30 a.m. from two days ago instead of nothing, which for a widget whose entire premise is a replay seems like a fair trade. The part I like is that the same line is the staleness guard, because it rewinds at most once. If the pipeline dies and the file goes stale for days, even the rewound target misses the data and the widget hides. The wraparound grants at most 24 hours of grace and can never dress up week-old data as current.

I decided to hide the widget on failure rather than show a stale number. The widget’s markup ships with the hidden attribute already on it, and only a successful fetch plus a valid sample removes it. Failed fetch, empty file, stale data, hole in the samples all result in the widget staying hidden.

Making it beat

The pulse is a CSS animation that scales the heart glyph up and down at the same frequency as the data. Every tick, the script writes the reading into the DOM and sets a CSS custom property to the beat interval, 60/bpm seconds:

bpmEl.textContent = String(bpm);
heartEl.style.setProperty('--beat', `${(60 / bpm).toFixed(3)}s`);

The animation just reads that variable, so the heart on the page beats at whatever rate the data says. 48 bpm visibly beats slower than 80:

.hr-heart {
  display: inline-block;
  color: var(--accent);
  transform-origin: 50% 60%;
  animation: hr-beat var(--beat, 1s) ease-out infinite;
}
@keyframes hr-beat {
  0%        { transform: scale(1); }
  14%       { transform: scale(1.32); }
  28%       { transform: scale(1); }
  42%       { transform: scale(1.18); }
  56%, 100% { transform: scale(1); }
}

The keyframes mirror the systolic/diastolic cycle where you see a big bump, then a smaller one, then rest for the remainder of the cycle, because a single symmetric scale pulse looked like a notification badge rather than a heart. transform-origin: 50% 60% makes the glyph swell from its visual center instead of its bounding-box center, which for ♥ are not the same place. And prefers-reduced-motion disables the animation entirely; the number still updates for anyone who has asked their OS for less motion.

What does this cost me? Nothing, really.

There’s no server (besides GitHub Pages), no third-party relay, and no hit to the watch battery, since it syncs exactly like it did before. I wear the watch, it syncs to my phone, and the phone syncs to Garmin’s servers. The added difference is that CI fetches the data from Garmin’s servers once a day and ships it to the site. The whole thing is one Python script, two cron lines, and some JS. The only upkeep I expect is the tokens expiring in about a year, at which point the heart will quietly vanish from my homepage and I’ll rerun the login script. I’ve asked Google Assistant to remind me in 11 months :D.

Someday, when I get around to it, I’ll add the workouts and the rest of the ledger. The same client that fetches my heart rate can fetch every gym and streetlifting session I’ve ever logged, so most of the plumbing here carries over. And because activities come down as FIT files with 1 second samples, the workouts section could be far more granular than the all-day series ever was. Until then, you get my heartbeat exactly 24 hours late.