pigeon: my experimental PHY (Part 0/12) 🕊️

Anyone who claims to understand RF is lying to you, doesn’t actually understand RF, or is lying to themselves.

I debated if I should have this be my cold open or not, since I feel like what I’m saying here could easily be read as meaning the opposite of what I actually mean here. I don’t mean to say “you will never understand this because it’s magic”, I’m trying to say “no one understands it, so learn as much as you can”.

I feel like I now know enough that RF is basically just witchcraft. There’s a lot of crystals that those holding the obscure knowledge can use along with rods of specific lengths of specific materials to channel EM fields using earth ground to have a conversation with someone on the other side of the world by sending out specific vibrations through space. Is the resonance of your equipment slightly off? No worries, you can tune it by changing how much electricity you capture in a capacitor made with an electrolyte fluid.

Oh right, also this force is somehow also the same as light (except when it’s not and it’s actually tiny particles, i won’t be taking questions, thank you). Shit, also, heat too. But only sometimes. Oh and it only travels on the surface of things when it’s not going through freespace, cool? Cool. Glad we got that out of the way.

Look, all I’m saying is maybe the witches are onto something, actually.

I’ve been to Salem as a kid. I know what things I wouldn’t want to say out loud around those parts – and RF is definitely one. Just like airplanes flying (that shit looks way too heavy), and nuclear energy (pile these rocks together and boil this water? Yeah, ok buddy, i’ll get right on that) you really can’t “just” explain it to someone who’s never seen evidence that it works (at least, not without sounding nuts). That means there’s only one thing I really can do: understand the mysteries of the universe through experimentation myself. Since I can’t easily build and fly a plane (I live in a TFR; too much paperwork) or build a nuclear reactor recreationally (I feel like someone would take issue with this one; also, too much paperwork), that just leaves me with RF (some paperwork required). The fact we use RF in the same way cavemen used rocks shouldn’t stop us from learning about RF – in fact, it should excite us, even if truly understanding the physics involved (that we all know, love and use) escapes all but the chosen few.

I, of course, am not one of the chosen few, but I don’t give up easily. I have chosen to continue to work to understand what I can in RF through force of will alone, building slowly on top of each step, over the course of what has now become years of effort. I’m doing this for the explicit purpose of attempting to humanize and provide a new set of explanations that make sense to me, so others can learn from what I’ve worked through. It’s my sincere hope this series is interesting, accessible and having had been worth your time to read.

Wait, that reminds me, I had a point here. Let me get back on track, sorry - right, so:

Those of you who have read my posts from time to time may remember my (perhaps ill-advised) fascination with networks. I have repeatedly decided that playing with networks is a fun and cool thing to do, and have even built a (very simple) SDR based Layer 1 and Layer 2, called “PACKRAT”. I like building networks, since, while they can seem daunting to many, they can generally be implemented and understood by a single human, whereas things like completely implementing a web request yourself these days completely requires a brief detour into quantumn physics. As a result, PACKRAT is/was conceptually simple (to match), and straightforward to implement without a lot of prior knowledge – however, it does not look very much like any modern protocol, and evolving it forward was a nonstarter. It just can’t handle it.

The layer 1 specifies the physical way the signal moves from one place to another (in our case, the RF signal). Examples of layers 1 would be things like the way power travels on an Ethernet line, the light wavelength and way that light changes in a fiber optic line, or the way radio signals are generated for wifi 802.11ax, lte, 5g, or even digital tv (DVB-T or ATSC).

Meanwhile, the layer 2 is what defines the structure of the data carried by the signal, and means to address a node (or group of nodes) that can communicate over the layer 1. Some examples of this are ethernet frames for the many layers 1 (from the various layer 1s from 10baseT up to 10GbE), wifi frames (you guessed it, from the layer 1 802.11a all the way to ax), or how 5g and lte layer 1 both use the ‘Radio Link Control’ layer 2 standard.

Anyway – the point here is the old PACKRAT code was fun, but couldn’t really be meaningfully iterated forward very well – there was no room to evolve it while making it interoperable. The only normal, well-adjusted and rational solution is to shed the shackles of past-me, and make a new Layer 1 and Layer 2, but this time, make it look like something modern – or at least something that could be experimented with in the direction of something modern. That’s what I started to do (back in 2022). I’ve had a few major life events since then, so progress took a bit of a hiatus, as well as a number of rewrites in an effort to isolate and fix bugs that would crop up (without a clear culprit).

Playing whack-a-mole with the bugs was easily 90+% of the work in making this real – part of the fun (and also most infuriating) thing about RF is you can be very fucking wrong and still have things work enough to think you were right (until either it being good enough isn’t true anymore, or you work through it again). It’s, in a lot of ways, just like cryptography – without a lot of specialized knowledge, bad crypto and good crypo look like the same thing to a human. I’m very happy to report that pigeon is now working (for some value of working), and generated charts off my signals match the textbooks. Not to spoil the predictable ending, but I even have a cURL to prove it will transit data:

$ curl http://[2602:810:6008:401::1]:8000/
      o                        
   _      __,  _   __   _  _   
 |/ \_|  /  | |/  /  \_/ |/ |  
 |__/ |_/\_/|/|__/\__/   |  |_/
/|         /|                  
\|         \|                  

For those of you who want some words right around here to determine if this post is interesting to you, the PHY is OFDM based, subcarriers are modulated using QPSK (or QAM-16), with 802.3an’s LDPC matrix for FEC. If that’s not useful information for you – that’s OK, we’ll get into painstaking detail about all of that anyway.

Given that I have now finished the major features I was setting out to implement, it’s a good time to document my progress. The good news is that if you’re reading this, I’ve done that! Well done me. The bad news is that what started off as a pretty long blog post has now turned into 16 (pretty long) posts.

The pigeon posts are written from the perspective of an novice radio engineer with basic experience in processing iq data from software defined radios, but experienced software engineer. I’ll be intentionally leaving out a lot of the technical details on the software end and specific software implementation, focusing on the general gist of the implementation in the radio critical components exclusively. The idea here is this is intended to be a framework – a jumping off point – for those who are interested in doing this themselves. I hope that this series of blog posts will come to be useful to those who embark on this incredibly rewarding journey after me.

This post is not announcing a new repo, or some tool hams out there other hams will run. By the end of this series, you will not have been able to copy-and-paste this into working as-is locally. Rather, it’s intentionally incomplete, with a lot of the specifics illustrated with the hope that you will feel like you now know how to build your own Layer 1 and 2. Think of this more like an assembly manual (along with a few missing a few parts) rather than something off the shelf. If that sounds boring, this may not be the series for you.

Actually building it is the fun part.

This is the first post in the series, and it will contain links to all the posts to follow. This is going to be the landing page I link others to. Additionally, all Pigeon posts (including future ones) will use the pigeon tag, which you can check back on, or follow along with here.

Let’s get started with the PHY! →

All Posts

Don’t like being told what post to read next? Me either! The full list of posts in this series are below, in case you want to cut your own path through the woods:

• background info
  • testing notes
  • osi networking model
  • frequency domain
• phy overview
  • ofdm symbols
  • subcarrier modulation
  • preamble generation
  • preamble detection
  • channel estimation
  • ldpc error correction
  • decoding a pigeon burst
  • pigeon Mode A
• data link layer (link)
  • cal protocol
• using layer 3