Tyler Robertson

You're carrying Soviet spy tech in your pocket. Here's how it got there.

Monday, December 13, 2021

If you want to get somewhere in London, you're most likely going to use the Underground ("Tube", sorry). So you go to a station, tap your card or phone to the turnstile to pay, and you're on your way. It's a routine that happens millions of times a day, and nobody ever seems to stop and consider the fact that they're using the product of Soviet spy technology. Because why would they? That's a ridiculous thing to spend your time thinking about.

(Hi, it's me: the person who spends time thinking about this.)

If you have a smart phone, a credit card with tap-to-pay, even a Nintendo Amiibo, you're using something that wouldn't have been possible without an infamous Soviet spy program, the grandfather of electronic music, barcodes on trains, and Star Wars Episode 1: The Phantom Menace.

Here's the story of how near-field communication (NFC), which powers nearly all "touchless" technology today, came to be.

1. Lev and The Thing

In TV and films about the Cold War, there's this recurring image that I absolutely love: person intently listening to headphones.

During the Cold War (and probably still), there was a whole sector of people whose job was to just listen. Silently scrubbing through air traffic frequencies, timing sonar pings, analyzing the word choices of local numbers stations… you never knew when you might hear something important, so listening was a key part of the war effort. And the reason why became especially clear in 1951, when a British radio engineer stationed in Moscow, dutifully scrolling through UHF frequencies, heard an American voice. Not a radio program, not a passing airplane, just… a conversation. Somebody had been bugged.

The only American worth spying on in Moscow at that time was the U.S. Ambassador, one George F. Kennan. And, sure enough, after doing more targeted sweeps of the ambassador's residence (the "Spaso House") in the same frequencies used by the radio engineer, they found the hidden microphone: in a large, wooden carving of the U.S. Grand Seal, hanging in the study.

The Thing - Image from Crypto Museum

The Thing - Image from the Crypto Museum

It became known as "The Thing", and what was most remarkable about it was that the Seal had been a gift from the Young Pioneer Association (the USSR equivalent to the Boy Scouts of America) to one of Kennan's predecessors: the man who held the position three Ambassadors prior, Averell Harriman, who would later go on to be Governor of New York. It was given to him just weeks before the end of World War II, in August of 1945, as a gesture of friendship between the United States and the Soviet Union. It hung in that study, which was checked for listening devices every day, for seven years.

Keep that in mind while we rewind the clock back a little further, to 1939.

In 1939, Lev Termen just wanted to make music.

More specifically, Lev wanted to make musical instruments. He had spent the last eight years touring the United States, showing off his inventions and meeting with the greatest minds and artists of his generation (Albert Einstein and Sergei Eisenstein, to name two). Then, he set up shop in New York, and began working on a new, very big project: a room which was itself an instrument, to be played by four dancers.

I'm sure you could name all sorts of reasons why Lev would want to make this new instrument (it sounds cool as hell, for starters), but my personal theory is that he made it for love, both for the musician and for the audience.

The musician in this case was Lev's wife, Lavinia Williams, a prima ballerina in the American Negro Ballet. Like Lev, Lavinia held a deep appreciation for her art, both its science and its mystery. She and Lev also shared a love of their respective heritages, and Lavinia would spent much of her later life teaching Caribbean and Haitian dance. Her precision as a dancer helped Lev to see that not only could one's hands manipulate an instrument, but one's whole body could create music, if given the right space in which to do so.

Lavinia Williams - Date unknown, probably around 1950, image from 120years.net Lavinia Williams - Date unknown, probably around 1950, image from 120years.net

The audience was Lev's home country, Russia, and more specifically, Vladimir Lenin. During the Bolshevik Revolution, which served as the backdrop to much of Lev's early life, there was a renewed interest in the blending of art and science, and in 1922 Lev was invited to the Kremlin for a performance. Lev's instrument, which would later bear his name, "Termenvox", seemed to Lenin to be a perfect merger of artistry and electrical engineering. And when Lenin requested to play it himself, and quickly proved to be proficient at it, Lev's heart was forever rooted in Leningrad. While Lev didn't get to meet Lenin again before he died in 1924, I believe it was this memory that drew Lev back to Russia, to bring back all he'd learned and built in a grand performance for his home country.

Lev never got to make that performance. War was coming, and fascist troops were moving ever-closer to Leningrad. By the time Lev's request to return to the Soviet Union had been granted in 1938, what would become World War II was beginning. Instead of returning home, Lev was brought to a sharashka, a secret military lab. He and Lavinia exchanged letters for a time, but eventually their communication was cut off. Lev's very existence became a national secret, and by 1945 he was presumed dead.

Now, I've been purposefully a little obtuse here, because I wanted to talk about Lev as a man before we talked about Lev as an inventor. He was, by all accounts, and intensely caring man, passionate about people and science and art. He believed that there could be a one-to-one physical connection between man and music, and devoted his life to an almost spiritual pursuit of it. It was said on several occasions that he had "the most beautiful hands in the world". He was, in addition to all that, one of the most important inventors of the last century.

You may have heard of him via the anglicized version of his name: Leon Theremin. The instrument that made him famous also bore that name when it came to America: the Theremin. Here he is, performing "Deep Night" in 1930:

The Theremin was the first fully-electronic instrument, and the parent of synthesized sound as we know it. You've heard it used in countless science fiction soundtracks, such as The Day the Earth Stood Still, or more recently on Disney's Loki series.

You've heard it on records, too: the Beach Boys used one in "Good Vibrations", and Led Zeppelin toured with one in the '70s. Bob Moog, founder of Moog Music and inventor of the modern synthesizer, was inspired by (and started his career by selling) the Theremin. Without the Theremin, whole genres of music (electronic, techno, dubstep, etc.) would sound drastically different, or might not exist at all.

I say all of this to mean that the Soviet Union didn't just secret away a well-intentioned artist; they captured the man who permanently changed the way the world made music, and made him create tools for espionage.

And, as we've already covered, he was very good at it. Theremin brought together his mastery of both electricity and music to create The Thing (dubbed "Reindeer" by the Russians), in a way that only he could have come up with. Here's how it worked:

Inside a wooden copy of the US Great Seal, a circular hole was cut, to act as a sounding board, similar to the opening in a guitar. In it, Theremin placed a small membrane, attached to a short antenna.

Interior of The Thing - Image from Wikipedia Interior of The Thing - Image from Wikipedia

The antenna was tuned in such a way that when you aimed a radio signal of a certain frequency at it, it would be "illuminated", resonate at a corresponding harmonic frequency, and send the harmonic signal back. Sort of like an echo, but at very high frequencies. This was already a known concept among military scientists, and the British Royal Air Force was attempting to use it during World War II as a rudimentary "friend-or-foe" system. Basically, they would shoot a radio signal at an aircraft, and if that other aircraft was equipped with the right antenna, it would send a signal back, and they'd know that plane was friendly. If there was no signal, or the wrong signal was returned, that plane would be shot down. (The plan was full of problems, like, "what if the antenna breaks" or "what if the plane is flying too fast/slow", and was eventually dropped for a different method called "pip-squeak", which had planes broadcasting their own signal for a few seconds every minute.)

The difficult part was getting sound back from the device, as most listening devices of that time required a power source for a microphone and return signal, like how each walkie-talkie in a set requires its own pair of AA batteries. To be an effective listening tool, The Thing couldn't have any electricity inside the actual device, which might be detected by its target. That's where Theremin's knowledge of sound and radio frequencies came in.

The real genius of The Thing was a thin metal disc, just under 20mm across. It was situated in the middle of the cavity inside the Great Seal, just behind some small holes that had been drilled into the eagle's beak. It acted as a microphone, and vibrated very slightly whenever sound waves hit the wood carving. That vibration caused the antenna's return frequency to change very slightly, so that the return signal would go up and down in pitch along with the sound in the room, all without requiring a power source or complex electronics. Theremin then used what he had learned while making his instrument—which turned variations in electromagnetic waves into sound—and devised a device that could detect the variations in the return signal, and use those to power a small speaker, so one could listen in from the next street over. (And if you think that's cool, he did the same thing with lasers, and so can you.)

This was the first instance of a portable device that could transfer information over radio waves, without needing an external power source.

Contrary to popular belief, Theremin didn't die in 1945, and in fact he would live a long and happy life. In the seven years that The Thing remained undetected, Theremin claimed "all we heard was everyday chatter — no government secrets", and after the war he was allowed to go back to making music and teaching at the Moscow Conservatory of Music. He re-married, had two children, and even travelled back to the United States in 1991 (there's a lovely video of his visit to Stanford at that time). I found rumors that he found and re-proposed to Lavinia during that trip, but can't verify it (it just sounds so darn romantic).

Leon Theremin passed away in 1993, at the age of 97, surrounded by his family in Moscow. But his inventions, both the Theremin and The Thing, would live on.

2. The Train Problem

Elizabeth Gilbert's Big Magic has this great chapter that talks about where ideas come from. She poses that ideas are actually unseen life forms; they exist alongside the rest of us, living and breathing, running around, trying to get noticed, and their only goal in life is to partner with a human and be realized.

In that vein, I believe that Radio-Frequency Identification (RFID) was one such idea. And while it was roommates with Theremin for a time, Leon's interests were primarily in music, so the idea had to move on. From Theremin's home in Russia, it floated around a bit, eventually making its way to America, to meet Swedish-born scientist Harry Stockman.

In 1948, Stockman wrote "Communications by Means of Reflected Power" for the Institute of Radio Engineers. Packed into just nine pages, the paper outlines the mathematical basis for taking radar technology (detecting the presence of something by listening for the radio waves reflected off it), and receiving actual information from far-off objects, just like Theremin was doing with The Thing—though no one knew that at the time. Stockman knew that the potential was there for all kinds of implementations based on "reflected power": audio and video transmission, advanced security, identification, navigation. But the technology of the time hadn't caught up to Stockman's ideas, so much of it remained theoretical.

And that wasn't for lack of trying, either. In the paper, Stockman provides pictures from several experiments performed at the Ipswich Antenna Station in Massachusetts. One of which was for a "numberplate", a tower with three reflectors that spun at different speeds. Here's the scan from the paper (if you can find a better picture of this, please let me know!):

Numberplate reflector from Stockman's experiments Stockman's "Numberplate" prototype from "Communications by Means of Reflected Power"

In the experiment, the team aimed a radio signal at the spinning reflectors, which bounced radio signals back at slightly different speeds. Using a separate antenna, Stockman could receive the reflected signal and decode the changes into numbers. By adjusting the speeds of each reflector's spin, each tower could have a unique, three-digit identifier that could be read through radio signals—the first iteration of Radio-Frequency Identification, RFID. With the numberplate, one could use radar to not only see that something was out there, but get specific information about what that thing is.

The trouble was that in the late '40s, and for some thirty years after, the parts required for setting up RFID were either too large (who wants to put a four-foot spinning antenna on a fighter plane?), or too expensive. The identification system also required power on both sides, to power the original signal and its reflector (powerless applications like Theremin's were still a closely-guarded secret). The project was eventually dropped, as a good—but impractical—idea.

Time moved on, and while interest in radio technology as a whole remained high (someday I'll write about how Hedy Lamarr invented the fundamental technology behind wifi and cell phones), the possibilities of Stockman's reflected power experiments were largely ignored for another 30 years or so. Then, on a flight from St. Paul, Minnesota, to Washington, D.C., a man called Mario Cardullo was seated next to an engineer from IBM.

Cardullo, who had spent much of his earlier career as a rocket propulsion engineer, with his hands on everything from X-15 jets to the Apollo project, had recently finished his master's degree thesis on a subject that—on its face—bears little resemblance to his work in aerospace: he developed a system for forecasting the need for Catholic Churches in the Archdiocese of Washington.

He had been raised Catholic, and later in life he would say that he was first encouraged to become an engineer by a Sister at Saint Augustus Grammar School. While working on his Master's Degree in Engineering Administration in the 1960's, he moved with his wife and children to Potomac, Maryland, where found himself in conversation with a local bishop, about whether they were going to build a new church near his house.

The bishop's initial response was, "My son, only God can tell me when we need a church." But Cardullo, a scientist, figured he could… help move God along. After doing a statistical study of the local area to see if and where a new church was needed, he presented his findings to an Archbishop, and later a Monseigneur in charge of HR for the diocese. They opened up their files to him, and he provided a 380-page document that became his Master's Degree thesis, mathematically forecasting where and when churches and priests would be needed. Thanks to those findings, the Archdiocese of Washington is now one of the largest in the world, with 139 parishes and 93 Catholic schools in the D.C. area.

After he'd established himself as a person who could think about both engineering and religion, the church began calling on him to conduct more studies along the same lines. And on one particular occasion in 1969, while he was working at Communication Sattelites (Comsat), the Archbishop asked him to study a diocese in Iowa, then present his findings to the bishop in St. Paul. So he did, and on the flight home from St. Paul to Washington, he was sat next to the engineer from IBM.

The engineer, whose name I've been unable to find (but please tell me if you know it), was assigned to a project called KarTrak.

KarTrak was a system designed to help take the manual labor out of doing inventory for freight trains. Up until the 1960's, when a train arrived at a station, it had to stop and wait for a team to manually inspect its cargo. This was both so that they could make sure nothing had been misplaced, but also so the people at the station could know where the train needed to go, and get it on the right set of rails before it took off again. The manually process, obviously, slowed things down significantly, and now that more and more things were being shipped by train, a faster solution was needed. The Association of American Railroads put together a competition to help solve the problem, and General Telephone & Electronics (which would later be acquired by Bell Atlantic and become Verizon) won with the KarTrak system.

It worked by placing large, multi-colored barcodes (which weren't as common then as they are now) on the side of each train car, customized to describe the car's contents. Scanners at regular intervals would take in the data while the train was moving, and send it to the station or a central hub. That allowed switches and stations to make any necessary adjustments ahead of time, so trains could reach their destinations faster. (Peter Dibble put together a fantastic video about KarTrak, if you want to learn more.)

Example of a KarTrak Barcode - Image from Wikipedia Example of a KarTrak Barcode - Image from Wikipedia

The problem with KarTrak, which the engineer explained to Cardullo, was that lots of things can happen to trains between check-ins. Snow, sand, dirt, or graffiti could cover the barcodes; meaning an engineer would have to come out and do a manual check anyway, creating as many (if not more) manual tasks as the previous method.

During his time on the Apollo project, and writing about satellite communications for the American Institute of Aeronautics and Astronautics (applying the same predictive models that he had developed in his thesis, this time determining the need for satellites in certain locations), Cardullo had come across the WWII "friend or foe" system several times. Although the specifics had changed slightly since the 1940's, the concept of using radio waves to identify friendly aircraft was (and is) still very much in use. And what he realized in that moment was that if you attached the radio equipment to a digital storage device, you could change what information was sent, to say "friend", "foe", or anything you like.

You might see where I'm going with this: he realized that you could identify things via radio frequency.

He pulled out a notebook, sketched out a rough diagram for what the "friend or foe" system plus a memory device could look like, and put it away.

The original sketch appears to have been lost, but this is the drawing from Cardullo's patent, which is likely quite similar.

Then, he nearly forgot about it.

Later that year, his friend Dan Webster sold his business manufacturing tape drives, netting himself a cool four or five million dollars in the process. A lot of money for the time. He told Cardullo, who was starting to grow tired of his job at Comsat, that if he put together a management team and pitched a few ideas, he might invest in him.

So Cardullo brought together a small team, and prepared to show Dan a new medical device he'd invented that analyzed EKGs. But, he and the team were hesitant about being a "one-trick pony", so they scrambled for other ideas they could bring to the table, and Cardullo uncovered that sketch he had made on the plane. The team pitched it as "The Encoder", coming up with a laundry list of potential applications seemingly on the spot, from toll booths to opening doors. Cardullo sold it as a sort of catch-all problem-solver, something that we might not ever know the bounds of.

By the end of 1969, Cardullo and his tiny team of engineers, managers, and marketers had raised over half a million dollars. They quit their jobs, started a company, and got to work.

A year later, the patent was approved, and a year after that they were ready to show the first prototype of the Encoder, with the goal of pitching it to the Port Authority of New York. It was mounted to a car, and used radio frequencies to communicate with toll booths automatically, so drivers could be charged a toll without having to stop and fiddle with change, slowing down traffic. Port Authority was facing major congestion issues, very similar to that original train problem. But the prototype was too large, and they couldn't wrap their heads around what to do with it, so they passed on the Encoder.

Then, the market crashed. The company's investors were not—save for Webster—particularly technical, and didn't fully understand what they had on their hands. Panicking, they pressured Cardullo to give them the patents for everything he'd worked on so far, Encoder included. A year later, Cardullo left the company, and a year after that, the company went under. The investors kept the patents, but again, didn't know what they had, so they sat on them. No Encoders were manufactured after that initial prototype.

Meanwhile, Port Authority of New York was still feeling the pain of their toll booth congestion issue. Now that Cardullo's company had gone under, Port Authority started shopping around, and asked companies like GE whether they could make something similar to Cardullo's Encoder. And they did—you might recognize it now as "E-ZPass".

By paying to replicate that first Encoder, they were violating Cardullo's patent. Unfortunately, it was no longer Cardullo's, and was now in the hands of investors who didn't know, and probably didn't care. Thanks to being pressured out of his business, Cardullo couldn't afford to sue the companies that were recreating his invention. And by the time he could, it was too late: several imitators had been brought to market, using it for all of those same cases that were brought to that first pitch meeting.

The RFID idea—that same enigmatic idea that travelled all this way from Leningrad—had become a reality. And Cardullo never made a cent from it.

Instead, inventor Charles Walton (formerly from IBM, just like the engineer who started this whole journey) filed a new patent in 1973, specifically focused on the identification capabilities of the concept. It was Walton who first called it "RFID", and would collect the royalties for the rest of his life.

To hear Cardullo talk about it now, however, he doesn't sound too heartbroken about it. He's one of the few inventors to have created something that gets put into use around the world during his own lifetime, which I'd like to think is some kind of compensation. For a condensed version of the above, here's a great TED Talk he gave in 2017:

3. The Phantom Merchandise

RFID was the predecessor of Near-Field Technology, but when did we start seeing it specifically used for up-close interactions, like tapping a card on the subway?

Most accounts will start with early Nokia phone models in the 2000's, but the real answer is from a long, long time ago…

References and further reading