While I am making few to no promises, I am once again attempting to fill
the hole that social media left in my heart by using Cohost. If you are
one of its ten other users please follow me: jbcrawford
After something like two years of saying I would, I finally got around to
implementing my own software for the email newsletter. I have of course
moved over existing subscribers but if you notice any issues with the email
version please let me know. If you aren't an email subscriber, now is a good
time to make my self esteem number go up and subscribe.
Most OTH radar have faded into obscurity since the end of their era of
relevance (something we will get to in the chronology soon). One, though, looms
above all the others. This is mostly a figurative expression, but it's also
literally true to some degree, as it's a pretty tall one. I am referring of
course to Duga, or as some might know it, the brain scorcher.
As I have previously mentioned, development of OTH radar proceeded in the USSR
more or less in synchronization to developments on this side of the iron
curtain. It's a bit difficult to nail down the Soviet history exactly due to a
combination of lingering Cold War secrecy and the language barrier, but it
appears that the USSR may have developed their first functional OTH radar
somewhat before the US, at least as a pilot project. Soviet OTH radar did not
become well known to the western world, though, until the commissioning of their
first full-scale ballistic missile surveillance radar in 1972. This would
become known as Duga-1.
To discuss Duga we first need to return briefly to the fundamentals of radar.
A monostatic radar system transmits and receives at the same site, observing
for reflections. A bistatic radar system transmits and receives from separate
sites. The earliest bistatic radar systems, and many later on, actually looked
less for reflections than variations in the signal received directly from the
transmit site. We can understand this concept by considering the simplest
possible type of bistatic radar: if you transmit a signal at one antenna, and
receive it at another, than any reduction in the strength of the received
signal might indicate the presence of an object in between.
This naive scheme is actually surprisingly practical and sees real use mostly
in intrusion-detection applications, where microwave or millimeter wave radar
acts as an RF analog to a laser or infrared beam gate---which has the advantage
of being more tolerant of rain and fog. A notable example is the AN/TPS-39
microwave perimeter protection radar used to protect the entrance points to
Atlas missile silos, but a similar system (made both more reliable and more
compact by digital technology) is sometimes used today, particularly around the
perimeters of prisons.
Bistatic radar can also be appreciably more complex. For example, the "Pinetree
Line" radar system deployed across southern Canada for detection of Soviet
bombers employed the simple bistatic radar principle enhanced with the use of
Doppler detection. If you imagine the field between a radar transmitter pointed
at a remote radar receiver, you might realize that objects not exactly between
the antennas, but near the line between them, might reflect RF radiation at a
glancing angle and cause a return to the receiver delayed by the angle distance
between the object and the two sites. The Pinetree line exploited this effect
as well, allowing it to detect aircraft before they reached the line proper.
At the extreme, a bistatic radar system detecting objects not directly between
the sites appears more like a monostatic system, and the separation of the
transmit and receive sites becomes more a detail of the antenna construction
than the operating principle of the radar. This blurring of the line between
monostatic and bistatic radar is apparent in OTH radar, very much as a design
detail of the antennas.
There is a general challenge in radio: transmit and receive can be difficult
functions to combine. There are several reasons for this. Considering the
practicalities of antenna design, different antennas are sometimes more optimal
for transit vs. receive. Moreover, it is very difficult to receive at the same
time as transmitting, because the transmit power will easily overwhelm the
front-end filtering of the receiver causing extremely poor reception, if not
outright damage (this is the basic reason most conventional radio applications
are "half duplex," where you cannot hear when you are talking).
For radar, this problem is typically addressed by transmitting a pulse and then
switching to receive operation immediately afterwards. The same problem rears
its head again when considering the relationship between range and pulse
repetition frequency. When a radar detects an object 1500km away (slant range
for simplicity), it takes about 0.01s for the RF pulse to make the round trip.
This means that if the radar is avoiding duplex operation, it can transmit
pulses a maximum of 100 times per second. In reality the problem is much worse,
because of the need for buffer time past the longest desired range and more
significantly the time required to switch between transmit and receive, which
in the case of radar was typically being done by "RF contactors..." huge relays
designed to tolerate RF power [1].
OTH radar systems are intended to operate at both extremely long ranges (3000km
being moderate in this context) and desire high PRFs, since high PRFs are very
useful in identifying and tracking fast-moving objects. For this reason, most
practical OTH radars actually transmit and receive at the same time. There is
only one practical way to achieve this: geographic separation, or in other
words, bistatic radar. Most of the OTH radar systems we will discuss from this
point use this method, with transmitter and receiver separated by at least 50km
or so and usually placed on opposite sides of a mountain range or other ridge
to provide additional geographic attenuation of the direct transmit power at
the receive site.
That was all a long preamble to say where Duga is, which is of course the
reason that it's so well known. The Duga-1 transmit site was located very
near Chernobyl in present-day Ukraine, and the receive site about 50km to the
northeast. The imposing Duga-1 antenna, which might be 150m tall but it's hard
to find that number for sure, is visible from many parts of the Chernobyl
exclusion zone and is a typical stop when visiting the area. The other antenna
of Duga-1 no longer survives, having been dismantled decades ago.
It's hard to find much detailed information about Duga-1. Its transmit power is
often cited as 10MW, but given the general history of OTH radar in the United
States I find it likely that that was a target peak power that may never have
been achieved. The practical average output power was probably more in the
range of 1MW or lower, similar to US systems. There are actually quite clearly
two separate antennas at the Chernobyl transmit site, something that is oddly
rarely mentioned. I haven't been able to find details on why, but I will
speculate: first, it is not at all unusual for radio transmit sites to have two
antennas, primary and auxiliary. The auxiliary antenna serves as a backup and
can be switched into use when the primary antenna is damaged or being serviced.
The auxiliary antenna is often smaller, both to save money on the antenna and
because it's often used with an auxiliary transmitter, smaller for the same
reasons.
It's also possible that the two antennas have different radiation patterns,
allowing basic steering functionality such as seen with Cobra Mist. It's most
likely that these patterns would be two altitude angles, intended for longer
and shorter range use. Considering the different sizes of the antennas it might
make sense for the smaller antenna to have a higher altitude angle, since less
transmit power would be required at shorter ranges (remember that the radar
signals reflect of the upper atmosphere, so aiming higher moves the target area
closer).
Duga-1 pointed north, towards the United States by great circle distance. It
operated from 7-19 MHz (the lower minimum frequency than Cobra Mist would
improve performance at night), and tended to operate on a given frequency for a
short time of up to ten minutes before switching to a different frequency.
This is a common operating pattern for HF radars because of the need for human
operators to find clear frequencies and because different frequencies will tend
to propagate better to certain regions. Cobra Mist tended to operate in a
similar fashion, with operators selecting a likely-looking frequency and
operating the radar on it for long enough to receive and analyze a fair amount
of data before moving on.
Duga is best known in the western world for its transmissions. There is an
aspect of OTH radar that I have not so far discussed, but is important: HF
radio can propagate over extremely long distances, and OTH radars operate at
very high power levels. The pulses transmitted by OTH radar can thus often be
received over a large portion of the planet. Duga became famous for its widely
heard chirps, earning it the nickname "Russian woodpecker." It was particularly
well-known because it operated without regard for typical European and American
band plans (but presumably according to USSR spectrum management conventions),
so it was often heard on broadcast and amateur bands where it became irritating
interference.
There was a famous fight back against the interference: some amateur radio
operators began actively jamming Duga, transmitting recorded pulses back on the
same frequency. These false returns apparently did cause frustration to the
Duga operators, because shortly after jamming started Duga would switch to a
different frequency. This casual electronic warfare went on for years.
Duga-1 apparently performed well enough in practice, as the USSR built another
one. Duga-3, also located in Ukraine, faced east instead to observe for
launches from Asia. It appears to have been a substantially similar design and
was likely located in the same region to allow for shared engineering efforts
and parts stockpiles.
Duga, at least its general design, is not unique to the USSR, and might be
considered a latecomer depending on how exactly you place the dates. Despite
the mysterious failure of Cobra Mist, the United States had nearly
simultaneously began work on a full-scale OTH radar for defense against inbound
ICBMs: OTH-B.
The history of OTH-B is fuzzier than Cobra Mist, although not quite as fuzzy as
that of Duga. Although OTH-B is no longer operational, it formally left
operation recently enough that it's likely some of the documentation is still
classified. Moreover, OTH-B became a bureaucratic embarrassment to the Air
Force for much the same reason as other late Cold War bureaucratic
embarrassments: it took so long to finish that by the time it was ready, the war
was over. There are still documents available from DTIC, though, enough that
we can put together the general details.
Work on OTH-B began in 1970, before completion of Cobra Mist, but progress was
slow for the kind of reasons that are later detailed in a GAO report. Cost and
schedule issues delayed the first working prototype to 1977, years after the
failure of Cobra Mist. OTH-B faced similar challenges: the prototype system
apparently did not perform to specification, but it was functional enough that
the Air Force made the decision to proceed with full-scale implementation.
Sometime in the late '70s, construction began on the first full OTH-B site.
Like Duga, OTH-B employed a bistatic design. The transmitter was located at
Moscow, Maine (a fitting town name), and the receiver near Columbia Falls,
Maine. While the timeline of OTH-B is often confusing, multiple sources agree
that the Moscow and Columbia Falls sites entered service in 1990---an
astoundingly late date considering the program's 1970 start (with limited
experimental validation of antenna and radio designs in the first year!). The
reasons for the tremendously slow development of OTH-B seem lost to history, or
at least deeply buried in Air Force files. We do know that over the course of
its development the GAO issued multiple reports questioning not only the
extremely long schedule and large budget of the program, but the necessity of
the program itself. In 1983 the GAO estimated a final cost of about one billion
dollars per site, and given that the program continued for another seven years
before achieving its first commissioning, it seems more likely that this was an
underestimate than an overestimate.
The same 1983 GAO report suggests that OTH-B befell much the same fate as many
ambitious military acquisition programs: the full production system was
contracted before experimental validation was complete, resulting in parallel
construction and R&D activities. This situation nearly always results in
changes being made to the design after relevant components were assembled,
creating a work-rework cycle that can easily drag on for years and hundreds of
millions.
For construction and operations purposes, OTH-B was divided into sectors. The
word sector here can be understood in the same way as "sector antenna." A
sector of OTH-B comprised a complete antenna and transmitter chain that
provided coverage of an area of roughly 60 degrees. The East Coast system
installed in Maine included three sectors to provide 180 degree coverage off of
the coast, and the minimum practical takeoff angle resulted in a range of a bit
over 3000km, placing a surprisingly large chunk of South America and a sliver
of Europe within the detection area. Each sector operated at 1MW of power,
although it's not clear to me if this was peak or average---more likely, peak.
OTH-B's late commissioning date provided the benefit of much more sophisticated
computer technology. OTH-B target discrimination appears to have been entirely
digital rather than a hybrid digital-analog system as seen in Cobra Mist. The
system implemented beam steering, and used a basic digital beam steering design
in which the transmit site emitted a 7.5 degree beam and the receive site
processed it as three, separate 2.5 degree slices simultaneously based on a
simple phased-array technique.
OTH-B's basic radio design varied appreciably from earlier systems like Cobra
Mist or Duga. Rather than a pulsed radar, it was a continuous wave radar. The
transmitter continuously emitted a signal that was frequency modulated with a
"chirp" signal that could be used to "align" received reflections with the
original transmitted signal. The CW design of the radar gave it significantly
improved sensitivity, at the cost that the "duplex problem" (of the transmitter
overwhelming the receiver) was much greater. As a result, OTH-B transmit and
receive sites required an even greater separation of 100-200km.
OTH-B worked, apparently well although details on performance are hard to come
by. It worked, though, in 1990: the year before the fall of the Soviet Union.
By 1990 the Cold War was already effectively over, having been replaced by a
period of both warming relations and Soviet economic collapse. OTH-B seemed
a system without a purpose.
The DoD seldom lets lack of a purpose come between it and a large contract, and
despite the clearly decreasing relevancy of the system construction went ahead
on the second and third OTH-B sites. In 1988, the Air Force activated two West
Coast stations for construction of three additional OTH-B sectors: Christmas
Valley, Oregon (transmit) and Tulelake, California (receive). Christmas Valley
is an unusual place, a failed real estate development in the pattern set by
Horizon and Amrep in the Southwest. Platted to be a large city, it is now a
tiny town centered on an artificial lake. A short distance from what might be
called city center, the 300+ acre OTH-B site once loomed on the horizon. Its
three sector antennas were metal frames reaching 135' feet high for the lowest
bands, supporting insulated radiating elements projected in front of the "rack"
at a jaunty angle. Each of the three antennas, nearly a mile wide, sat behind a
4000' buried ground screen. In total, each antenna sector antenna occupied
nearly a square mile. Behind each antenna, a transmitter building of 14,000
square feet housed the radio equipment, shielded behind the antenna back
screen.
Although construction of the Christmas Valley and Tulelake sites was
substantially complete, they were turned down in 1991 a few months before the
Soviet Union itself was deactivated. The radar never operated.
Meanwhile, construction had begun on the first sector of the north system, in
Alaska. The transmitter would be located at Gakona, and the receiver at Tok.
Little is known about this northern site. Radomes Inc. (an association for
veterans of air defense radar operations) reports that construction on the
first sector transmitter at Gakona had begun but was never completed, and the
Tok site was never started.
As of 1991, the constructed and activated east installation of OTH-B formed
sectors 1-3, the constructed but never activated west installation sectors 3-6,
and the partially constructed Alaska system sectors 7-8. The final planned
sector, 9, would have been located somewhere in the central United States
facing south. It is not clear to me if a final site was ever selected for the
south system, as it was canceled before construction was contracted.
The east system, fully functional when it was deactivated, won a second life.
From 1991 to 1997 it was operated in partnership with the Border Patrol on a
drug interdiction mission, detecting aircraft involved in smuggling. It was
apparently useful enough in this role that its antenna was modified in 1992
to expand coverage over a portion of the US-Mexico border.
The western site, never activated, was kept in an idle state ready for
potential reactivation until 1997. In 1997, the eastern system was deactivated
and both east and west sites were transitioned to a storage status with only a
few staff members to maintain site security and integrity against the elements.
In 2002, east and west were reduced to "cold storage" and the radio equipment
was removed. In 2007, at 17 years of disuse after their near-completion, the
Christmas Valley and Tulelake sites were partially demolished. Antennas were
removed and sold for scrap and some site remediation was performed. The
equipment buildings and some infrastructure remain, their ownership having
reverted to the respective states who have unclear plans for their future use.
The Maine sites appear to be in a similar state, with antennas removed,
although I haven't been able to find documentation on this action.
The partially constructed site at Gakona, Alaska met a more interesting fate.
In 1993, it was reused by the Air Force in partnership with other organizations
for the construction of HAARP. HAARP is a research instrument that uses RF
radiation to ionize the upper atmosphere and observes the results. While HAARP
is now operated by the University of Alaska for public research purposes, its
original lifespan under Air Force management was secretive enough to make it
the center of many conspiracy theories. While the Air Force is cagey about
identifying its research function, it seems likely that it was used for at
least two disciplines.
First, to develop an improved understanding of the radio reflection properties
of the upper atmosphere. The lack of a clear model of this behavior was
repeatedly identified as a limitation in the performance of OTH radar systems,
and so this research would have directly contributed to the Air Force's
interest in long-range radar. Second, by this time the NSA had a
well-established interest in ionospheric radio propagation. Unusual propagation
patterns sometimes allowed radio signals originating within the USSR and China
to be received in nearby US-allied territories or even within the US proper.
The NSA operated several research facilities that both performed this type of
interception and basic research into ionospheric propagation with an eye
towards making this signals intelligence method more effective and predictable.
While not proven to my knowledge, it seems likely that HAARP served as a part
of this effort, and that active ionization of the atmosphere may have been at
least evaluated as a possibility to improve the chances of interception.
Amusingly, a 2015 paper from a HAARP researcher suggests that the University of
Alaska is once again investigating use of the site for an OTH radar directed at
observing vessel traffic through the arctic ocean. The arctic ocean north of
Canada is becoming increasingly navigable due to climate change, and so is
thought likely to become a strategically important area for naval combat and
enforcement of Canadian sovereignty. Indeed, confirming a connection between
HAARP's research goals and OTH radar, the paper notes that HAARP has
successfully demonstrated the creation of an "artificial ionosphere" at lower
altitudes which can be used to perform OTH radar at shorter ranges than
previously practical---filling in the "beyond visual horizon but before
atmospheric skip" range gap in existing radar methods.
What of OTH radar as a broader art? I have framed the decline of OTH radar
systems mostly in terms of the fall of the Soviet Union. This was indeed the
killing blow for both Duga and OTH-B, but by 1983 the GAO was recommending that
OTH-B be canceled. OTH radar faced a challenge even greater than the end of
the war: satellites. By the '90s, satellite remote sensing had advanced to the
point that it could perform the functions of OTH radar, and usually better.
The MIDAS (Missile Defense Alarm System) satellites, launched in the '60s, were
the first working space-based ICBM launch detection system. MIDAS was quickly
replaced by the more covertly named Defense Support Program or DSP. These
satellite systems, based mostly on infrared imaging, remain the United States'
primary missile defense warning system today. By the '80s, they provided both
better sensitivity and wider coverage than HF OTH radar.
Today, HF OTH radar has faded nearly into obscurity. Some military OTH radar
systems still operate. The Navy, for example, built a relatively small OTH
radar system in 1987 directed south for surveillance of the Caribbean. It
remains in operation today, mostly for counter-narcotics surveillance similar
to the east OTH-B site's second chance. China constructed and operates a system
very similar to OTH-B for airspace surveillance. China's OTH-B is thought to
use interferometry methods for significantly improved resolution. The antenna
design looks extremely similar to OTH-B, although I suspect this is more a case
of common constraints producing a common design than China having based their
design on the US one.
Interestingly, one of OTH-Bs most enduring contributions is not military but
scientific: NOAA, through a partnership with the Air Force, discovered that the
"ground clutter" received by OTH-B's east site from the ocean could be analyzed
to collect information on wave heights and current directions. Today, a system
of small coastal HF radars on both sides of the United States collect
information useful for marine weather forecasting. Data from this system is
also used for improved tidal predictions and by the Coast Guard to model
surface currents, allowing improved predictions of where a drifting vessel in
distress (or its lifeboats or survivors) might be found.
Ionospheric propagation remains an area of scientific interest. For some time,
the Arecibo observatory operated an ionospheric modification observatory using
transmitters repurposed from OTH-B's eastern sectors. HAARP operates to this
day. The NSA is mum as always, but seems to still be busy at their propagation
research facilities. But large OTH radar, on the scale of OTH-B, seems unlikely
to return to the United States. It is costly, complex, and struggles to provide
anything that satellites can't. As with terrestrial radio-based navigation
systems, perhaps the specter of ASAT warfare will motivate renewed interest in
OTH methods. But I wouldn't hold my breath. For now, all we have is a set of
large trapezoids in the Oregon desert.
The end of OTH radar is, of course, far from the end of military radar.
Improved radar methods, especially phased-array systems, made it far more
practical to detect ICBMs at their extremely high, orbital approach altitudes.
Terrestrial missile warning systems re-concentrated on this method, and many are
in operation today. I'll get to them eventually.
[1] Some readers might be familiar with duplexers, filtering devices that
employ dark magic RF trickery to effectively "split" transmit and receive
power. There are two reasons that filtering-based approaches are not very
practical for radar: first, radar must receive the same frequency it transmits,
preventing the use of common cavity filters for separation. Second, RF filters
become larger and dissipate more heat as they handle higher powers, and most
radar operates at very high power levels. Even the simplest RF filters become
impractical to build when considering megawatt power levels.
Previously on Deep Space Nine, we discussed the MUSIC and MADRE
over-the-horizon radar (OTH) programs. MUSIC and especially MADRE validated the
concept of an HF radar using ionospheric (often called "skywave" in the radio
art) propagation, with a novel hybrid digital-analog computerized signal
processing system. MADRE was a tremendous success, ultimately able to detect
ICBM launches, aircraft, and ship traffic in the North Atlantic region. What
was needed next seemed simple: a very similar radar, perhaps more powerful,
aimed at the Soviet Union.
In 1964, final planning began on the full-scale OTH radar for missile defense.
Code-named "Cobra Mist," the AN/FPS-95 [1] radar was initially slated for
Turkey. Design proceeded for some time (a year or so) for a site in Turkey, but
ultimately no site could be obtained. The documents I have are somewhat vague
on the reason for this, but it seems likely that Turkey was hesitant to host a
installation that would seem a direct affront to the USSR's nuclear power.
Another host would have to be found, some time already into the design process.
Finally, a site was selected that was not quite ideal but workable: Orford
Ness, England.
Orford Ness (alternately, according to the typical complexity of English place
names, Orfordness) is a sort of long, sandy semi-island, of a type prone to
forming on the coast of Great Britain but not seen much here in the western
United States. Orford Ness has been under the control of the Ministry of
Defence since before WWII, and by the '60s was well in use as a research range
for the Atomic Weapons Establishment (AWE). As is often the case with odd bits
of land in military use, it contains a number of interesting historic sites
which include a very early marine radionavigation system, a lighthouse notable
for its involvement in the Rendlesham Forest UFO incident which occurred
nearby, and several curious instrumentation bunkers left from AWE
high-explosives research.
It also contains a lot of land, and that land would be needed for Cobra Mist.
Construction proceeded from 1966 to 1972. Due to the experimental nature of OTH
radar and the large scale of the system, during construction a set of
experiments was designed under the name Design Verification System Testing
(DVST). This was essentially field acceptance testing, but with the added
complication that Cobra Mist was advancing the state of the art in OTH radar so
much that the expected performance of the system was largely unknown. Cobra
Mist fell into sort of an uncomfortable in-between: In part a production
defense system, in part an experimental apparatus.
The Cobra Mist antenna, constructed by RCA (then a major defense contractor),
consisted of 18 strings 620 meters long arranged in a circle, radiating from
the center. A buried mesh ground plane extended forward from the antenna
strings to provide vertical shaping of the beam, for a total built length on
each string of about 900m.
A complex switching network connected six of these strings at a time to the
transmitter and receiver, applying phase shifts to four of the strings (those
not in the center of the active array) to maintain the phase of the emitted
signal despite the varying distances of the strings from the target area (due
to the arc shape of the array). This is an early version of the phased-array
technology which is heavily used in military radar today... but a very early
version indeed. The radar could be "steered" or aimed by selecting different
sets of strings, but this was a fairly lengthy process and was done while
offline. Nonetheless, it allowed the radar to cover a total azimuth of about 90
degrees with a smaller target area, about seven degrees, selected from within
that range.
The altitude of the antenna was also somewhat steerable: each string contained
two sets of radiating elements in phase relationships that would produce two
different vertical angles. Further, varying the transmit frequency across its
6 to 40 MHz range resulted in different ranges as the ionospheric propagation
shifted closer and further.
The minimum effective range was 500nmi, because the antenna could only emit
radiation upwards (being located on the ground) and the maximum elevation angle
produced a reflection from the ionosphere centered at about that distance. The
maximum range is a somewhat more theoretical matter, but was placed at about
2000 nmi assuming simple single-hop propagation at the antenna's lowest
achievable radiating angle. In practice, the vagaries of ionospheric
propagation would make this range overoptimistic in some cases but an
underestimate in favorable conditions where multi-hop or ducting patterns
occur.
Behind the antenna, a large squat building housed a transmitter specified for
600 kW average power and 10 MW peak, although a report on the project from the
Mitre Corporation (from which much of the information here is derived [2])
states that only 3.5 MW was achieved in practice. A specially designed receiver
with a very large dynamic range (140dB specified) was connected to a set of RF
filters and then an analog-to-digital converter system that provided the
received signal to the computer.
Computers are ostensibly what I write about here, and this story has a good
one. The Signal Analysis Subsystem (SAS) was a hybrid digital-analog computer
driven by a Xerox Sigma V. The Sigma V was actually a fairly low-end computer,
32-bit but without the virtual memory support found on higher-end Sigma models.
The inexpensive (relatively) computer was enabled by the fact that most actual
signal processing was performed in the SAS. The SAS was constructed by
Interstate Electric Corporation (IEC), under contract with the Army Security
Agency which would later become the NSA. Like MADRE, the SAS functioned by
converting the digitally stored signals back to analog---but at a much
accelerated speed. This higher-frequency signal was then fed through a series
of analog filters including a matched filter to discriminate reflections of the
radar's distinctive pulse shape. The results of the analog filter process were
then redigitized and returned to the computer, which allowed operators to
display the results in graphical form. Cobra Mist's designers also provided a
useful secondary mechanism: received signals could be written out to magnetic
tape, allowing them to be analyzed off-line later by other computer systems.
This would prove an important capability during DVST, when multiple other
analysis systems were developed to interpret these tapes.
DVST began in 1972 and almost immediately hit a serious snag. Cobra Mist, an
implementation of a technology reasonably well proven by MUSIC and MADRE,
turned out to be pretty much useless. It struggled to detect targets in even
the most favorable conditions, much less past Moscow.
Cobra Mist was never expected to be especially precise. Its huge antenna
produced wide beams that reflected off of the ionosphere at an uncertain point.
It would never be able to locate targets to more than general regions. But,
like the eyes of insects, Cobra Mist was expected to make up for its blurry
vision with a very fine sensitivity to motion. Its narrow Doppler filters
should have been able to discriminate any object moving at more than 1.5 knots
towards or away from the radar. The ability to discriminate target speeds very
finely should have allowed Cobra Mist to not only differentiate moving targets
from the ground, but to determine the number of moving targets based on their
varying speeds.
The Mitre corporation had devised a series of twelve DVST experiments to
demonstrate the radar's ability to detect and track aircraft. Despite its
excellent sensitivity to motion, only three of these were even performed. The
radar's poor performance on the initial three tests made the remainder
hopeless, and efforts shifted to identifying the cause of the radar's
unexpected blindness.
To make a rather long story short, DVST researchers identified a large,
consistent problem with the radar: in every configuration, a great deal of
radio noise was received at roughly the same range as the ground. This noise
was of such great magnitude, not previously observed with OTH radar, that it
drowned out the actual radar returns, preventing discrimination of the pulse
reflections.
There were other problems as well. The 6MHz frequency floor proved a challenge,
the computer display system was difficult to use to detect faint targets, and
the antenna somewhat under-performed expectations. Various adaptations were made
on most of these issues, including a significant overhaul of parts of the
antenna system to improve gain, offline analysis of the recorded tapes, and
later an overhaul of the computer system to provide a more flexible display
capability. But none of these improvements could overcome the most basic
problem, the surprising noise, which researchers labeled "range-related
noise" due to its appearance at the same range and Doppler gates as the
typical ground clutter.
I will spare a multi-page discussion of efforts to characterize and eliminate
this noise, which you can read in the original paper linked at the bottom of
this article. Suffice to say that numerous experiments showed that the noise
appeared at around the same range as the ground in nearly all operating
conditions, and that it was far greater in magnitude than would be explained by
ground reflection (the usual clutter) or typical atmospheric, galactic, or
man-made radio noise. The noise did not originate in the receiver or antenna
system, it did not appear over signals transmitted from nearby the antenna, and
the same noise could even be detected by a completely separate test antenna
system installed for this purpose.
DVST engineers felt that the problem could likely be identified, but that for
the time being Cobra Mist could not be effective in its mission. An Air Force
panel reviewed the problem, and the result was a bit surprising. Defense money
was clearly tighter in the '70s than it is in the era of the F-35, and perhaps
the recently negotiated ABM treaty's restrictions on large radar overseas was
a factor (Cobra Mist was an overseas radar, since although placed in the UK it
was built and operated by the US). In any case, the Air Force pulled the plug.
Cobra Mist shut down in 1973, without ever being operational and less than 18
months after the beginning of DVST. The project had cost about a billion
dollars in today's money, and its short life left more questions than answers.
Or, perhaps more accurately, it left one very big question: what was the noise?
History has borne out the design of Cobra Mist. Multiple similar OTH radars
have been constructed and provided good performance, including the Jindalee
radar in Australia (designed with the participation of the same NRL researchers
as Cobra Mist) and the Duga radar in the USSR, present-day Ukraine (which I
will discuss more in part III). The OTH-B system, based on similar principles
and first designed at around the same time, was greatly expanded over following
decades and remained in service into the 21st century (this too will be
discussed more in the next part). The point is that Cobra Mist should have
worked, it was all correct in principle, but no one could explain that noise.
Given the passage of a great deal of time since then, the most likely
explanation is probably some subtle defect in the receiver design. The Mitre
report, written mostly to argue against the radar's premature shutdown, admits
the possibility of a problem in the analog-to-digital conversion stage that
possibly contributed to the noise but was not thoroughly investigated before
the shutdown. The authors also considered the possibility that the noise was a
result of one or more of several speculated effects: perhaps there are so many
aircraft in the air as to create a broad set of Doppler-shifted reflections
that cannot be discriminated from one another. Or perhaps there are some sort
of objects on the ground that vibrate or rotate in such a way to produce a set
of strong and broadly Doppler-shifted reflections. Subsequent OTH radars have
not suffered from these problems, though, making it unlikely that the cause of
the noise was some incidental property of Europe at the time.
The reason I find this story so fascinating is that the Mitre authors also
suggest another possibility, one that they had intended to evaluate with
additional tests that were never performed due to the 1973 shutdown. They
thought it possible that Cobra Mist was being jammed.
The basic idea is this: if the USSR became aware of Cobra Mist (not at all a
far stretch considering its large physical size) and managed to obtain some
detailed information on its operation (via espionage presumably), it would have
been quite possible to build an active countermeasure. A system of relatively
small transmitters distributed across various locations in the USSR, the Mitre
authors estimated 15 sites would do, could use computer technology to detect
Cobra Mist's emitted pulses and then transmit the same pulse back at a wide
variety of Doppler offsets. It would be difficult or (at least at the time)
impossible to differentiate these transmitted "distractors" from actual
returns, and they would appear to the operators as random noise centered around
the target range. The slight latency the computer system would introduce even
provides an explanation for one of the observed properties of the noise, that
it occurred mostly at ranges slightly further than the peak ground reflection.
Because of the extreme sensitivity of the radar, these active countermeasures
would only require a few watts of RF power to overwhelm the radar.
The DVST team was able to perform one experiment that bolstered the theory that
the noise was a result of something on the ground, whether an intentional
countermeasure or an incidental property of something widely used in Europe.
When targeting the radar at azimuths and ranges that consisted primarily of
open sea, they did not observe the noise. The noise only seemed to occur on
land, and more generally in their target region. The Mitre paper states near
its end:
We are forced to conclude that the jamming technique is quite feasible,
and it is not clear that the experiments conducted at the AN/FPS-95
would have discovered the jamming had it occurred. If experiments
confirming or denying the possibility had been conducted, they would
have perhaps resolved the issue. They were not conducted.
With fifty years of hindsight it seems unlikely that there was any meaningful
scale electronic countermeasures effort in the USSR that has still not come to
light, but the possibility is certainly interesting. It is, at least in my
mind, within the realm of reason: the USSR were working on their own OTH radar
efforts at the time along much the same technical lines and so would have been
familiar with the principles that Cobra Mist relied on. The USSR's own premiere
OTH radar, Duga, suffered occasional jamming and the operators had to develop
countermeasures. In other words, both the means and the motivation were in
place. Considering that the scale of the jamming effort would have been
relatively small (the Mitre authors mention 15 sites would have been sufficient
and that Mitre had constructed similar antennas at the cost of about $25,000,
low for high-end RF equipment at the time), perhaps there is something to it
and the Soviet program simply faded into obscurity, little documented and known
to few. This could be a minor front of early electronic warfare that has been
lost to history.
The failure of Cobra Mist, frustrating as it was, did little to dissuade the
United States or other countries from the broader concept of OTH radar. In
fact, by the time Cobra Mist DVST was in progress, the US had already begin
major work on an OTH radar system in the United States (and thus compliant with
the ABM treaty), called OTH-B. By the end of the Cold War, OTH-B reached almost
10 MW in combined operating power across multiple sites, and was well on the
way to complete 360 degree coverage from the US extending over a large portion
of the planet.
As with many things late in the Cold War, it also suffered an ignominious fate:
repeatedly replanned at the whims of politics, partially constructed, partially
dismantled, repurposed for the war on drugs, and ultimately forgotten. We'll
talk more about OTH-B, and its Soviet contemporary Duga made incidentally
famous by the Chernobyl disaster, in Part III. As with most of my interests, it
involves secret sites in barren remote areas near failed cities... not
California City or the Rio Estates this time, but Christmas Valley, Oregon.
[1] I think I've used these part numbers several time without explaining them.
Dating back to WWII, many military technology systems have been assigned
article numbers according to this system. Initially, it was called the
Army-Navy Nomenclature System, from which the "AN" prefix is derived (for
Army-Navy). The part after the slash is a type code, for which we most often
discuss FPS: fixed (F) radar (P) search (S). P is used for "radar" because R
had already been claimed for "radio." When researching radar, you will also
often see AN/TPS - transportable search radar. The number is just, well, a
number, assigned to each project sequentially.
[2] This report, titled "The Enigma of the AN/FPS-95 OTH Radar," was helpfully
retrieved via FOIA and preserved by the Computer UFO Network---not so much due
to a latent interest in radar history but because Cobra Mist's nature as a
secret facility in close proximity to the Rendlesham Forest has suggested to
some a connection to the prominent UFO incident there. You can read it at
CUFON
One if the most interesting things about studying history is noting the
technologies that did not shape the present. We tend to think of new
inventions as permanent fixtures, but of course the past is littered with
innovations that became obsolete and fell out of production. Most of these
at least get the time to become well-understood, but there are cases where
it's possible that even the short-term potential of new technologies was
never reached because of the pace at which they were replaced.
And of course there are examples to be found in the Cold War.
Today we're going to talk about Over-the-Horizon Radar, or OTH; a key
innovation of the Cold War that is still found in places today but mostly
lacks relevance in the modern world. OTH's short life is a bit of a
disappointment: the most basic successes in OTH were hard-won, and the
state of the art advanced rapidly until hitting a standstill around the
'90s.
But let's start with the basics.
Radar systems can be described as either monostatic or bistatic, terms which
will be important when I write more about air defense radar. Of interest to us
now is monostatic radar, which is generally what you think of when someone just
says "radar." Monostatic radars emit RF radiation and then observe for a
reflection, as opposed to bistatic radars which emit RF radiation from one site
and then receive it at another site, observing for changes. Actually, we'll see
that OTH radar sometimes had characteristics of both, but the most important
thing is to understand the basic principle of monostatic radar, of emitting
radiation and looking for what bounces back.
Radar can operate in a variety of parts of the RF spectrum, but for the most
part is found in UHF and SHF - UHF (Ultra-High Frequency) and SHF (Super High
Frequency) being the conventional terms for the spectrum from 300MHz-3GHz and
3GHz-30GHz. Why these powers of ten multiplied by three? Convention and
history, as with most terminology. Short wavelengths are advantageous to radar,
because RF radiation reflects better from objects that are a large portion or
even better a multiple of the wavelength. A shorter wavelength thus means that
you can detect smaller objects. There are other advantages of these high
frequencies as well, such as allowing for smaller antennas (for much the same
reason, the gain of an antenna is maximized at multiples of the wavelength, or
at least at divisions by small powers of two).
UHF and SHF have a disadvantage for radar though, and that is range. As a rule
of thumb, the higher the frequency (and the shorter the wavelength), the
shorter the distance it will travel. There are various reasons for this, a big
one is that shorter wavelengths more readily interact with materials in the
path, losing energy as they do so. This has been a big topic of discussion in
5G telephony; since some 5G bands are in upper UHF and lower SHF where they
will not pass through most building materials. The atmosphere actually poses
the same problem, and as wavelengths get shorter the molecules in the
atmosphere begin to absorb more energy. This problem gets very bad at around
60GHz and is one of the reasons that the RF spectrum must be considered finite
(even more so than suggested by the fact that, well, eventually you get visible
light).
There's another reason, though, and it's the more important one for our
purposes. It's also the atmosphere, but in a very different way.
Most of the time that we talk about RF we are talking about line-of-sight
operations. For high-band VHF and above [1], it's a good rule of thumb that RF
behaves like light. If you can see from one antenna to the other you will have
a solid path, but if you can't things get questionable. This is of course not
entirely true, VHF and UHF can penetrate most building materials well and
especially for VHF reflections tend to help you out. But it's the right general
idea, and it's very much true for radar. In most cases the useful range of a
monostatic radar is limited to the "radio horizon," which is a little further
away than the visible horizon due to atmospheric refraction, but not that much
further. This is one of the reasons we tend to put antennas on towers. Because
of the low curvature of the earth's surface, a higher vantage point can push
the horizon quite a bit further away.
For air-defense radar applications, though, the type I tend to talk about, the
situation is a little different. Most air-defense radar antennas are quite low
to the ground, and are elevated on towers only to minimize ground clutter
(reflections off of terrain and structures near the antenna) and terrain shadow
(due to hills for example). A common airport surveillance radar might be
elevated only a few feet, since airfields tend to be flat and pretty clear of
obstructions to begin with. There's a reason we don't bother to put them up on
big towers: air-defense radars are pointed up. The aircraft they are trying to
detect are quite high in the air, which gives a significant range advantage,
sort of the opposite situation of putting the radar in the air to get better
range on the ground. For the same reason, though, aircraft low to the ground
are more likely to be outside of radar coverage. This is a tactical problem in
wartime when pilots are trained to fly "nap of the earth" so that the reverse
radar range, from their perspective, is very small. It's also a practical
problem in air traffic control and airspace surveillance, as a Skyhawk at 2000'
above ground level (a pretty typical altitude here in the mountain west where
the ground is at 6k already) will pass through many blind spots in the Air
Force-FAA Joint Surveillance System.
This is all a somewhat longwinded explanation of a difficult problem in the
early Cold War. Before the era of ICBMs, Soviet nuclear weapons would arrive by
airplane. Airplanes are, fortunately, fairly slow... especially bombers large
enough for bulky nuclear munitions. The problem is that we would not be able to
detect inbound aircraft until they were quite close to our coasts, allowing a
much shorter warning (and interception) time than you would expect. There are a
few ways to solve this problem, and we put great effort into pursuing the most
obvious: placing the radar sets closer to the USSR. NORAD (North American Air
Defense Command) is a joint US-Canadian venture largely because Canada is,
conveniently for this purpose, in between the USSR and the US by the shortest
route. A series of radar "lines" were constructed across Alaska, Canada, and
into Greenland, culminating with the DEW (Distant Early Warning) Line in arctic
norther Canada.
This approach was never quite complete, and there was always a possibility that
Soviet bombers would take the long route, flying south over the Pacific or
Atlantic to stay clear of the range of North American radar until they neared
the coasts of the US. This is a particularly troubling possibility since even
today the population of the US is quite concentrated on the coasts, and early
in the Cold War it was even more the case that the East Coast was the United
States for most purposes. Some creative solutions were imagined to this
problem, including most notably the Texas Towers, radar stations built on
concrete platforms far into the ocean. The Texas Towers never really worked
well; the program was canceled before all five were built and then one of them
collapsed, killing all 28 crew. There was an even bigger problem with this
model, though: the threat landscape had changed.
During the 1960s, bombers became far less of a concern as both the US and the
USSR fielded intercontinental ballistic missiles (ICBMs). ICBMs are basically
rockets that launch into space, orbit around to the other side of the planet,
and then plunge back towards it at terminal velocity. ICBMs are fast: a famous
mural painted on a blast door by crew of a Minuteman ICBM silo, now Minuteman
Missile National Historic Park, parodies the Domino's Pizza logo with the
slogan "Delivered worldwide in 30 minutes or less, or your next one is free."
This timeline is only a little optimistic, ICBM travel time between Russia and
the US really is about a half hour.
Moreover, ICBMs are hard to detect. At launch time they are very large, but
like rockets (they are, after all, rockets, and several space launch systems
still in use today are directly derived from ICBMs) they shed stages as they
reach the apex of their trip. By the time an ICBM begins its descent to target
it is only a re-entry vehicle or RV, and some RVs are only about the size of a
person. To achieve both a high probability of detection and a warning time of
better than a few minutes, ICBMs needed to be detected during their ascent.
This is tricky: Soviet ICBMs had a tendency of launching from the USSR, which
was a long ways away.
From the middle of the US to the middle of Russia is around 9000km, great
circle distance. That's orders of magnitude larger than the range of the best
extant radar technology. And there are few ways to cheat on range: the USSR was
physically vast, with the nearest allied territory still being far from ICBM
fields. In order to detect the launch of ICBMs, we would need a radar that
could not only see past the horizon, but see far past the horizon.
Let's go back, now, to what I was saying about radio bands and the atmosphere.
Below VHF is HF, High Frequency, which by irony of history is now rather low
frequency relative to most applications. HF has an intriguing property: some
layers of the atmosphere, some of the time, will actually reflect HF radiation.
In fact, complex propagation patterns can form based on multiple reflections
and refraction phenomenon that allow lucky HF signals to make it clear around
the planet. Ionospheric propagation of HF has been well known for just about as
long as the art of radio has, and was (and still is) regularly used by ships at
sea to reach each other and coast stations. HF is cantankerous, though. This
is not exactly a technical term but I think it gets the idea across. Which HF
frequencies will propagate in which ways depends on multiple weather and
astronomical factors. More than the complexity of early radio equipment
(although this was a factor), the tricky nature of HF operation is the reason
that ships carried a radio officer. Establishing long-distance connections by
HF required experimentation, skill, and no small amount of luck.
Luck is hard to automate, and in general there weren't really any automated HF
communications systems until the computer age. The long range of HF made it
very appealing for radar, but the complexity of HF made it very challenging.
An HF radar could, conceptually, transmit pulses via ionospheric propagation
well past the horizon and then receive the reflections by the same path. The
problem is how to actually interpret the reflections.
First, you must consider the view angle. HF radar energy reflects off of the
high ionosphere back towards the earth, and so arrives at its target from
above, at a glancing angle. This means of course that reflections are very
weak, but more problematically it means that the biggest reflection is from the
ground... and the targets, not far above the ground, are difficult to
discriminate from the earth behind them. Radar usually solves this problem
based on time-of-flight. Airplanes or recently launched ICBMs, thousands of
feet or more in the air, will be a little bit closer to the ionosphere and thus
to the radar site than the ground, and so the reflections will arrive a bit
earlier. Here's the complication: in ionospheric propagation, "multipath" is
almost guaranteed. RF energy leaves the radar site at a range of angles
(constrained by the directional gain of the antenna), hits a large swath of the
ionosphere, and reflects off of that swath at variable angles. The whole thing
is sort of a smearing effect... every point on earth is reached by a number of
different paths through the atmosphere at once, all with somewhat different
lengths. The result is that time-of-flight discrimination is difficult or even
impossible.
There are other complexities. Achieving long ranges by ionospheric propagation
requires emitting RF energy at a very shallow angle with respect to the
horizon, a few degrees. To be efficient (the high path loss and faint
reflections mean that OTH radar requires enormous power levels), the antenna
must exhibit a very high gain and be very directional. Directional antennas are
typically built by placing radiating and reflecting elements some distance to
either side of the primary axis, but for an antenna pointed just a few degrees
above the horizon, one side of the primary axis is very quickly in the ground.
HF OTH radar antennas thus must be formidably large, typically using a
ground-plane design with some combination of a tall, large radiating system
and a long groundplane extending in the target direction. When I say "large"
here I mean on the scale of kilometers. Just the design and construction of the
antennas was a major challenge in the development of OTH radar.
Let's switch to more of a chronological perspective, and examine the
development of OTH. First, I must make the obligatory disclaimer on any cold
war technology history: the Soviet Union built and operated multiple OTH
radars, and likely arrived at a working design earlier than the US.
Unfortunately, few resources on this history escaped Soviet secrecy, and even
fewer have been translated to English. I know very little about the history of
OTH radar in the USSR, although I will, of course, discuss the most famous
example.
In the US, OTH radar was pioneered at the Naval Research Laboratory. Two early
prototypes were built in the northeastern United States: MUSIC, and MADRE.
Historical details on MUSIC are somewhat scarce, but it seems to have been of
a very similar design to MADRE but not intended for permanent operation. MADRE
was built in 1961, located at an existing NRL research site on Chesapeake
Bay near Washington. Facing east towards the Atlantic, it transmitted pulses
on variable frequencies at up to 100kW of power. MADRE's large antenna is
still conspicuous today, about 300 feet wide and perhaps 100 feet tall---but
this would be quite small compared to later systems.
What is most interesting about MADRE is not so much the radio gear as the
signal processing required to overcome the challenges I've discussed. MADRE,
like most military programs, is a tortured acronym. It stands for Magnetic-Drum
Radar Equipment, and that name reveals the most interesting aspect. MADRE, like
OTH radars to come, relied on computer processing to extract target returns.
In the early '60s, radar systems were almost entirely analog, particularly in
the discrimination process. Common radar systems cleared clutter from the
display (to show only moving targets) using methods like mercury acoustic delay
lines, a basic form of electronic storage that sent a signal as a mechanical
pulse through a tube of mercury. By controlling the length of the tube, the
signal could be delayed for whatever period was useful---say one rotational
period of the radar antenna. For OTH radar, though, data needed to be stored
on multiple dimensions and then processed in a time-compressed form.
Let's explain that a bit Further. When I mentioned that it was difficult to
separate target returns from the reflection of the earth, if you have much
interest in radar you may have immediately thought of Doppler methods. Indeed,
ionospheric OTH radars are necessarily Doppler radars, measuring not just the
reflected signal but the frequency shift it has undergone. Due to multipath
effects, though, the simple use of Doppler shifts is insufficient. Atmospheric
effects produce returns at a variety of shifts. To discriminate targets, it's
necessary to compare target positions between pulses... and thus to store a
history of recent pulses with the ability to consider more than one pulse at a
time. Perhaps this could be implemented using a large number of delay lines,
but this was impractical, and fortunately in 1961 the magnetic drum computer
was coming into use.
The magnetic drum computer is a slightly odd part of computer history, a
computer fundamentally architected around its storage medium (often not only
logically, but also physically). The core of the computer is a drum, often a
fairly large one, spinning at a high speed. A row of magnetic heads read and
write data from its magnetically coercible surface. Like delay tubes, drum
computers have a fundamental time basis in their design: the revolution speed
of the drum, which dictates when the same drum position will arrive back at the
heads. But, they are two-dimensional, with many compact multi-track heads used
to simultaneously read and write many bits at each drum position.
Signals received by MADRE were recorded in terms of Doppler shifts onto a drum
spinning at 180 revolutions per second. The radar similarly transmitted 180
pulses per second (PRF), so that each revolution of the drum matched a radar
pulse. With each rotation of the drum, the computer switched to writing the
new samples to a new track, allowing the drum to store a history of the recent
pulses---20 seconds worth.
For each pulse, the computer wrote 23 analog samples. Each of these samples
was "range gated," meaning time limited to a specific time range and thus
distance range. Specifically, in MADRE, each sample corresponded to a 455 nmi
distance from the radar. The 23 samples thus covered a total of 10,465 nmi in
theory, about half of the way around the earth. The area around 0Hz Doppler
shift was removed from the returned signal via analog filtering, since it
always contained the strong earth reflection and it was important to preserve
as much dynamic range as possible for the Doppler shifted component of the
return.
As the drum rotated, the computer examined the history of pulses in each range
gate to find consistent returns with a similar Doppler shift. To do this,
though, it was first necessary to discriminate reflections of the original
transmitted pulse from various random noise received by the radar. The signal
processing algorithm used for this purpose is referred to as "matched
filtering" or "matched Doppler filtering" and I don't really understand it very
well, but I do understand a rather intriguing aspect of the MADRE design: the
computer was not actually capable of performing the matched filtering at a high
enough rate, and so an independent analog device was built to perform the
filtering step. As an early step in processing returns, the computer actually
played them back to the analog filtering processor at a greatly accelerated
speed. This allowed the computer to complete the comparative analysis of
multiple pulses in the time that one pulse was recorded.
MADRE worked: in its first version, it was able to track aircraft flying over
the Atlantic ocean. Later, the computer system was replaced with one that used
magnetic core memory. Core memory was random access and so could be read faster
than the drum, but moreover GE was able to design core memory for the computer
which stored analog samples with a greater dynamic range than the original
drum. These enhancements allowed MADRE to successfully track much slower
targets, including ships at sea.
The MUSIC and MADRE programs produced a working OTH radar capable of surveiling
the North Atlantic, and their operation lead to several useful discoveries.
Perhaps the most interesting is that the radar could readily detect the
ionospheric distortions caused by nuclear detonations, and MADRE regularly
detected atmospheric tests at the NNSS despite pointing the wrong direction.
More importantly, it was discovered that ICBM launches caused similar but much
smaller distortions of the ionosphere which could also be detected by HF radar.
This improved the probability of HF radar detecting an ICBM launch further.
AND THAT'S PART ONE. I'm going to call this a multi-part piece instead of just
saying I'll return to it later so that, well, I'll return to it later. Because
here's the thing: on the tails of MADRE's success, the US launched a program
to build a second OTH radar of similar design but bigger. This one would be
aimed directly at the Soviet Union.
It didn't work.
But it didn't work in a weird way, that leaves some interesting questions to
this day.
[1] VHF is 30-300MHz, which is actually a pretty huge range in terms of
characteristics and propagation. For this reason, land-mobile radio technicians
especially have a tendency to subdivide VHF into low and high band, and
sometimes mid-band, according to mostly informal rules.
One of my chief interests is lighting. This manifests primarily as no end of
tinkering with inexpensive consumer IoT devices, because I am cheap and running
new cabling is time consuming. I did nearly end up using DMX for my
under-cabinet lighting but ultimately saw sense and stuck to a protocol that is
even more unfamiliar to the average consumer, Z-Wave.
I worked in theater (at a university conference center) only briefly but the
fact that it was a very small operation gave me a great deal of exposure to the
cutting edge of theatrical control last time a major capital expenditure had
been authorized, in the '90s. This was an ETC Sensor dimmer system with an ETC
Express 48/96 console for which we had to maintain a small stash of 3.5"
diskettes. The ETC Express is still, in my mind, pretty much the pinnacle of
user interface design: it had delightfully tactile mechanical buttons that you
pressed according to a scheme that was somehow simultaneously intuitive and
utterly inscrutable. Mastery of the "Thru," "And," "Except," "Rel" buttons made
you feel like a wizard even though you were essentially typing very elementary
sentences. It ran some type of non-Microsoft commercial DOS, maybe DR-DOS if I
remember correctly, and drove the attached 1080p LCD display at 1024x768.
The integration with the Lutron architectural lighting control system had never
actually worked properly, necessitating a somewhat complex pattern of
button-mashing to turn on the lobby lights that sometimes turned into sending a
runner upstairs to mash other buttons on a different panel. There was an
accessory, the Remote Focus Unit, that was a much smaller version of the
console that was even more inscrutable to use, and that you would carry around
with you trailing a thick cable as you navigated the catwalks. This was one of
two XLR cables that clattered along the steel grating behind you, the other
being for the wired intercom system.
My brief career in theater was very influential on me: it was a sort of
Battlestar Galactica-esque world in which every piece of technology was from
the late '90s or early '00s, and nothing was wireless. You unplugged your
intercom pack, climbed the spiral staircase (which claimed many a shin) to an
alarmingly high point in the flyloft, and plugged your intercom pack into the
wall socket up there. Then you fiddled around for a moment and had to walk back
to the wall socket, because the toggle switch that changed the socket between
buses was always set wrong, and you never thought to check it in the first
place. Truly a wonderful era of technology.
The spiral staircase exists in a strange liminal space in the building: the
large open area, behind the flyloft, which primarily contained the air handlers
for the passive solar heating system installed as a pilot project in the '80s.
It had apparently never worked well. The water tubing was prone to leaking, and
the storage closets under the solar array had to be treated as if they were
outdoors. Many of the counterweights and older fixtures were rusty for this
reason. It would rain indoors, in the back of the Macey Center: not because of
some microclimate phenomena, but by the simple logic of a university that
occasionally received generous grants for new technology, but never had the
money for maintenance. Of course today, the passive solar array has been
removed and replaced by a pointless bank of multicolored architectural panels
curiously aimed at the sun. Progress marches on.
Well that's enough nostalgia. Here's the point: I think lighting control is
interesting, chiefly because it involves a whole lot of low-speed digital
protocols that are all more or less related to RS-422. But also, there is
history!
I am going to sort of mix theatrical and architectural lighting control here,
but it is useful to know the difference. Theatrical lighting control is
generally the older field. Early theaters had used chemical light sources
(literal limelight) to light the stage, later theaters used primitive
electrical lights like carbon-arc spotlights. Just about as soon as electrical
lights were introduced, electrical light dimming came about. Theatrical
lighting controls have certain properties that are different from other
lighting systems. They are usually designed for flexibility, to allow light
fixtures (sometimes called luminaires but moreso in architectural lighting)
to be moved around and swapped out between shows. They are designed with the
expectation that relatively complex scenes will be composed, with a single
show containing a large a number of lighting cues that will be changed from
show to show. Theatrical lighting is largely confined to the theater, mostly
in very traditional forms of footlights, side lights, and numbered catwalks
or bridges extending both directions from the proscenium (upstage and into
the house).
Architectural lighting systems, on the other hand, are intended to make
buildings both more dramatic and practical. There are similarities in that
architectural lighting control systems mostly involve channels which are
dimmed. But there are significant differences: architectural lighting is mostly
permanently installed and unmovable. There is a relatively limited number of
channels, and more significantly there is a relatively limited number of
scenes: maybe a half dozen in total. Control is sometimes automated (based on a
solar calendar, sunset and sunrise) and when manual is intended to be operated
by untrained persons, and so usually limited to a row of buttons that call up
different scenes. You, the reader, probably encounter architectural lighting
control most often in the completely automated, scheduled systems used by large
buildings, and second in the wall panel scene control systems often used in
conference rooms and lecture halls.
There exists, of course, an uncomfortable in between: the corporate or
university auditorium, which has some elements of both. Theaters are also often
found inside of buildings with architectural lighting controls, leading to a
need to make the two interoperate. Because of both the similarities and the
need for interoperability there are some common protocols between theatrical
and architectural systems, but for the most part they are still fairly separate
from each other.
So how does a light control system actually work?
The primitive element of a light control system was, for a long time, the
dimmer. Early theaters used saltwater dimmers and later variac dimmers arranged
into banks and operated by levers, which could be mechanically linked to
each other to effect scene changes. Architectural systems are much the same, but
instead of backstage or in a patch bay, the dimmers are located in a closet.
Architectural systems have always been more automated and required remote
control, which of course means that they came about later.
Let's start with a very basic but very common scheme for lighting control: the
0-10V dimmer. Widely used in architectural lighting for many decades, this is
perhaps the simplest viable system. For each dimmer there is a knob which
adjusts an output voltage between 0 and 10v, and this is routed by low voltage
wiring to either a central dimmer or (more common in later systems) a
distributed system of dimmmable lighting ballasts incorporated into the
fixtures. The main appeal of 0-10v analog dimming is its simplicity, but this
simplicity betrays the basic complexity of dimming.
Some lights are very easy to dim, mostly incandescent bulbs which are capable
of a very wide range of brightnesses corresponding more or less linearly to the
power they consume (which can be moderated by the voltage applied or by other
means). Arc and discharge lights introduce a complication; they produce no
light at all until they reach a "striking power" at which point they can be
dimmed back down to a lower power level. Incandescent light bulbs can actually
behave the same way, although it tends to be less obvious. The issue is a
bigger one in architectural lighting than in theatrical lighting [1], because
architectural lighting of the early era of central control relied heavily on
fluorescent fixtures. These have a particularly dramatic difference between
striking power and minimum power, and in general are difficult to dim [2].
This has lead to a few different variations on the 0-10v scheme, the most
common of which is 1-10v fluorescent control. In this variant, 0v means "off"
while 1v means "minimum brightness." This difference is purely semantic in the
case of incandescent bulbs, but for fluorescent ballasts indicates whether or
not the bulb should be struck. The clear differentiation between "off" and
"very dim" was important for simpler, non-microcontroller ballasts, but then
became less important over time as most fluorescent ballasts switched to
computerization which could more intelligently make a threshold decision
about whether or not the bulb should be struck near 0v.
The 0-10v scheme is simple and easy to work with, so it was widely installed.
It has a major downside, though: the need to run separate control wiring to
every "zone" or set of dimmable lights. In typical architectural installations
this is pretty manageable, because in the era of 0-10v analog dimming (as in
the era before of direct dimming of the power supply wiring) it was typical to
have perhaps six distinct zones. In theatrical lighting, where a modest
configuration is more like 16 dimming channels and wiring is more often
expected to be portable and reconfigurable, it was a much bigger nuisance.
Fortunately, improving electronic technology coming mostly out of the telecom
industry offered a promising innovation: multiplexing.
If you are not familiar with the term at its most general, multiplexing
describes basically any method of encoding more than one logical channel over a
single physical channel. On this blog I have spoken about various multiplexing
methods since it has an extensive history in telecommunications, most obviously
for the purpose of putting multiple telephone calls over one set of wires. If
you, like me, have an academic education in computing you might remember the
high level principles from a data communications or networking class. The most
common forms of multiplexing are FDM and TDM, or frequency division muxing and
time division muxing. While I'm omitting perhaps a bit of nuance, it is mostly
safe to say that "muxing" is an abbreviation for "multiplexing" which is the
kind of word that you quickly get tired of typing.
While there are forms of muxing other than FDM and TDM, if you understand FDM
and TDM you can interpret most other methods that exist as being some sort of
variation on one, the other, or both at the same time. FDM, frequency division,
is best explained (at least I think) in the example of analog telephone muxing.
Humans can hear roughly from 20-20kHz, and speech occurs mostly at the bottom
end of this range, from 80-8kHz (these rough ranges tend to keep to multiples
of ten like this because, well, it's convenient, and also tends to reflect
reality well since humans interpret sound mostly on a logarithmic basis). A
well-conditioned telephone pair can carry frequencies up to a couple hundred
kHz, which means that when you're carrying a single voice conversation there's
a lot of "wasted" headroom in the high frequencies, higher than audible to
humans. You can take advantage of this by mixing a speech channel with a higher
frequency "carrier", say 40kHz, and mixing the result with an unmodified voice
channel. You now have two voice conversations on the same wire: one at 0-20kHz
(often called "AF" or "audio frequency" since it's what we can directly hear)
and another at 40-60kHz. Of course the higher frequency conversation needs to
be shifted back down at the other end, but you can see the idea: we can take
advantage of the wide bandwidth of the physical channel to stuff two different
logical channels onto it at the same time. And this is, of course,
fundamentally how radio and all other RF communications media work.
TDM, time division, took longer to show up in the telecom industry because it is
harder to do. This is actually a little counterintuitive to me because in many
ways TDM is easier to understand than FDM, but FDM can be implemented with
all-analog electronics fairly easily while TDM is hard to do without digital
electronics and, ultimately, computers. The basic idea of TDM is that the
logical channels "take turns." The medium is divided into "time slots" and each
logical channel is assigned a time slot, it gets to "speak" only during that
time slot. TDM is very widely used today because most types of communication
media can move data faster than the "realtime" rate of that data. For example,
human speech can be digitized and then transmitted in a shorter period of time
than the speech originally took. This means that you can take multiple realtime
conversations and pack them onto the same wire by buffering each one to
temporary memory and then sending them much faster than they originally
occurred during rotating timeslots. TDM is basically old-fashioned sharing and
can be visualized (and sometimes implemented) as something like passing a
talking stick between logical channels.
Why am I explaining the concepts of FDM and TDM in such depth here? Well,
mostly because I am at heart a rambler and once I start on something I can't
stop. But also because I think lighting control systems are an interesting
opportunity to look at the practicalities of muxing in real systems that are
expected to be low-cost, high-reliability, and operate over cabling that isn't
too demanding to install.
And also because I think it will be helpful in explaining a historically
important lighting control scheme: analog multiplexing, or AMX.
AMX192, the most significant form of AMX, was introduced in 1975 (or so,
sources are a little vague on this) by Strand Lighting. Strand is a
historically very important manufacturer of theatrical lighting, and later
became part of Philips where it was influential on architectural lighting as
well (along with the rest of Philips lighting, Strand is now part of Signify).
In this way, one can argue that there is a direct through-line from Strand's
AMX to today's Hue smart bulbs. AMX192 supports 192 channels on a single cable,
and uses twisted-pair wiring with two pairs terminated in 4-pin XLR connectors.
This will all sound very, very familiar to anyone familiar with theatrical
lighting today even if they are too young to have ever dealt with AMX, but
we'll get to that in a bit.
What makes AMX192 (and its broader generation of control protocols) very
interesting to me is that it employs analog signaling and TDM. Fundamentally,
AMX192 is the same as the 0-10v control scheme (although it actually employs
0-5v), but the analog control signal is sent alongside a clock signal and every
clock pulse it changes to the value for the next channel. On the demultiplexing
or demuxing side, receivers need to pick out the right channel by counting
clock pulses and then "freeze" the analog value of the signal pair to hold it
over while the control wiring cycles through the other channels.
One of the sort of neat things about AMX192 is that you can hook up your
control wiring to an oscilloscope and, once you've got the triggering set up
right, see a very neat visualization of all 192 control channels across your
scope going up and down like the faders on your control board. It's a neat and
simple system, but was still fairly cutting edge in the '70s due to the
complexity of the electronics used to track the clock pulses.
We'll take a moment here too to discuss the physical wiring topology of AMX192:
as you might guess, AMX192 is set up as a "bus" system with each dimmer
connected to the same two twisted pairs. In the '70s, dimmers were still fairly
large devices and so theaters almost exclusively used traditional dimmer rack
systems, with all dimmers installed in one central location. So while there was
a multidrop bus wiring arrangement, it was mostly contained to the rack
backplanes and not really something that users interacted with.
This idea of multi-drop bus wiring, though, might sound familiar if you have
read my other posts. It's largely the same electrical scheme as used by RS-485,
a pretty ubiquitous standard for low-speed serial buses. AMX192 is analog, but
could RS-485 be applied to use digital signaling on a similar wiring topology?
This is not a hypothetical question, the answer is obviously yes, and about ten
years after AMX192 Strand introduced a new digital protocol called DMX512. This
stands for Digital Multiplexing, 512 channels, and it employs the RS-485 wiring
scheme of one twisted pair in a shielded cable terminated with 5-pin XLR
connectors. Now, on the 5-pin XLR connector we have two data pins and one
shield/common pin. Of course there are two more pins, and this hints at the
curiously complicated landscape of DMX512 cabling.
The DMX512 specification requires that 5-pin cables include two twisted pairs,
much like AMX192. You have no doubt determined by now that DMX512 is directly
based on AMX192 and carries over the same two-twisted-pair cabling, but with
the addition of an extra pin for a grounded shield/signal reference common as
required by RS-485, which is the physical layer for DMX512. RS-485 uses
embedded clocking though, so it does not require a dedicated pair for clock
like AMX192 did. This creates the curious situation that a whole twisted pair
is required by the spec but has no specified use. Various off-label
applications of the second pair exist, often to carry a second "universe" of an
additional 512 channels, but by far the most alternative use of the second pair
is to omit it entirely... resulting in 3 pins, and of course this is a rather
attractive option since the 3-pin XLR connector is widely used in live
production for balanced audio (e.g. from microphones).
You can run DMX512 over microphone cables, in fact, and it will largely work. A
lot of cheaper DMX512 equipment comes fitted with 3-pin XLR connectors for this
purpose. The problem is that microphone cables don't actually meet the
electrical specifications for DMX512/RS-485 (particularly in that they are not
twisted), but on the other hand RS-485 is an intentionally very robust physical
protocol and so it tends to work fine in a variety of improper environments. So
perhaps a short way to put it is that DMX512 over 3-pin XLR is probably okay
for shorter ranges and if you apply some moral flexibility to standards.
Let's talk a bit about the logical protocol employed by DMX512, because it's
interesting. DMX512 is a continuous broadcast protocol. That is, despite being
digital and packetized it operates exactly like AMX192. The lighting controller
continuously transmits the values of every channel in a loop. The only real
concession to the power of digital networks in the basic DMX512 protocol is
variable slot count. That is, not all 512 channels have to be transmitted if
they aren't all in use. The controller can send an arbitrary number of channels
up to 512. Extensions to the DMX protocol employ a flag byte at the beginning
of the frame to support types of messages other than the values for sequential
channels starting at 1, but these extensions aren't as widely used and tend to
be a little more manufacturer-specific.
DMX512 has no error correction or even detection; instead it relies on the fact
that all values are repeatedly transmitted so any transient bit error should
only be in effect for a short period of time. Of course running DMX512 over
non-twisted 3-pin XLR cable will increase the number of such transient errors,
and in the modern world of more complex fixtures these errors can become much
more noticeable as fixtures "stutter" in movement.
Let's talk a bit about the fixtures. AMX192 was designed as a solution for the
controller to send channel values to the dimmer rack. DMX512 was designed for
the same application. The same digital technology that enabled DMX512, though,
has enabled a number of innovations in theatrical lighting that could all be
summed up as distributed, rather than centralized, dimming. Instead of having a
dimmer rack backstage or in a side room, where the dimmers are patched to
line-level electrical wiring to fixtures, compact digital dimmers (called
dimmer packs) can be placed just about anywhere. DMX512 cabling is then
"daisy-chained" in the simplest configurations or active repeaters are used to
distribute the DMX512 frames onto multiple wiring runs.
The next logical step from the dimmer pack is building dimming directly into
fixtures, and far more than that has happened. A modern "moving head" fixture,
even a relatively low-end one, can have two axes of movement (altitude-azimuth
polar coordinates), four channels of dimming (red, green, blue, white), a
multi-position filter or gobo wheel, and even one or two effect drive motors.
Higher-end fixtures can have more features like motorized zoom and focus,
additional filter wheels and motorized effects, cool white/warm white and UV
color channels, etc. The point is that one physical fixture can require direct
connection to the DMX bus on which it consumes 8 or more channels. That 512
channel limit can sneak up on you real fast, leading to "multi-universe"
configurations where multiple separate DMX512 networks are used to increase
channel count.
DMX, then, while cutting-edge in 1986, is a bit lacking today. Strand basically
took AMX192 and shoved it into RS-485 to develop DMX512. Could you take DMX512
and shove it into IP? Consider that a cliffhanger! There's a lot more to this
topic, particularly because I haven't even started on digital architectural
control. While DMX512 can be used for architectural lighting control it's not
really all that common and there's a universe of interesting protocols on "the
other side of the fence."
[1] Nonetheless, the effect can be noticeable in theatrical lighting even at
its small magnitude with halogen bulbs. As a result many theatrical light
controllers have a "bulb warmer" feature where they keep all fixtures at a very
low power level instead of turning them off. You can imagine that when mixing
incandescent and LED fixtures with much more noticeable minimum brightness,
making sure this is disabled for the LED fixtures can become a headache.
[2] Some may be familiar with the issue of dimmable vs. non-dimmable
fluorescent fixtures, and the similar issue that exists with LEDs to a lesser
extent. The difference here is actually less in the light than in the power
supply, which for fluorescent fixtures and sometimes LED fixtures is usually
called the (whether or not it is actually a ballast in the electrical
engineering sense, which newer ballasts are usually not). In LED fixtures it is
becoming more common to refer to it as a driver, since the prototypical form of
an LED light power supply is a constant-current driver... although once again,
many "LED drivers" are actually more complex devices than simple CC drivers,
and the term should be viewed as an imprecise one. Dimmable fluorescent and LED
drivers mostly use PWM, meaning that they rapidly switch the output on and off
to achieve a desired duty cycle. This is slightly more complicated for
fluorescent bulbs due to the need to get them warm enough to strike before they
emit light, which means that modern dimmable fluorescent ballasts usually
include "programmed start." This basically means that they're running software
that detects the state of the lamp based on current consumption and provides
striking power if necessary. This is all sort of complicated which is why the
dimmable vs. non-dimmable issue is a big one for CFLs and cheaper LED bulbs: in
these types of light bulbs the power supply is a large portion of the total
cost and simpler non-dimmable ballasts and drivers keep the product price down.
It's a much smaller issue in architectural lighting where the type of ballast
is specified up front and the ballast is a separate component from the bulb,
meaning that its price is a little less important of an issue.
Today in the mail I received the latest issue of the New Mexico Sun, a lovely
local newspaper that I have never heard of, nor received, before. An oddity of
the addressing strongly suggests that it was sent based on the same address
list used for a lot of the political advertising I've received, and the
contents are... well, we'll go over that in detail in a moment, but I
immediately got the impression that this "newspaper" was actually a piece of
political advertising. The odd thing is when it arrived: at about noon on
election day. In some ways this seems like a smart strategy because it will be
so salient in the mind of its recipients when they go to the polls, except for
the problem that I would imagine a lot of people wouldn't receive their mail
until after they had voted (doubly so since early voting and mail-in voting are
both pretty popular here). In any case, the election day timing seemed either
intentional or like it had just arrived one or two days late of the target
date.
I have a vague recollection that there is some sort of Postal Service
regulation requiring that periodicals distributed by mail provide some standard
information about the publisher, editor, etc., so I flipped through this paper
in search of a masthead. There's none to be found. In fact, the only
information the paper gives bout its origin is a domain name, NewMexicoSun.com.
A quick search of postal regulations suggests that my memory is not entirely
incorrect but also not very applicable here: Domestic mail manual, section 207,
requires that an "identification statement" appear somewhere in the first five
pages or on the editorial page, and that the identification statement include
the address of the publisher.
But... section 207 gives the rules for periodical mail, which is a specific
postage rate for items like magazines and newspapers. The address block on this
item includes "ECRWSH" in the Optional Endorsement Line or OEL, the first line
of the address on commercial mail that often has a lot of asterisks. The OEL
serves mostly to speed up handling of bulk mail by providing some sorting
information in a standard numeric format, and for many bulk mail services
contains some type of abbreviation that identifies the type of bulk postage
paid. Domestic mail manual section 240 tells us that ECRWSH indicates USPS
Marketing Mail, high density rate. Marketing mail meaning that this was mailed
at reduced rates for advertising, and high density that an additional discount
was provided in exchange for the mail piece being sent to at least 125
addresses on each route (this high count per route simplifies sorting).
I still wonder if sending something that so much resembles a newspaper under
Marketing Mail might run afoul of some postal regulations, but a qualified
opinion on that would probably require a postal lawyer, which I imagine as
being somewhat like a maritime lawyer as depicted in Arrested Development.
Let's consider the contents of the paper. The front page features a vertically
stretched portrait of the incumbent governor above the fold with the headline
"Career criminals found new victims after early release," and page 6 consists
only of mugshots of individuals released from New Mexico prisons as a result of
COVID protective orders. This forms an odd contrast with page 8, which is laid
out identically but instead features mugshots of local high school athletes who
have been recruited to college teams. The descriptions on this page are very
oddly formatted and show a lack of local knowledge, e.g. the caption "Joah
Flores played high school football at New Mexico." New Mexico what? where?
I strongly suspect that this page was automatically generated using data from a
sports scouting service, and probably minimally reviewed by a human if at all.
Like the Governor on the front page, many of the portraits have had their
aspect ratios awkwardly changed. This repeat problem is, at least in my
suspicion, indicative that this paper was mostly generated by pasting into an
Adobe InDesign or Quark Xpress template. The newspaper also includes several
bits of awkward blank space, something that newspapers habitually avoid (column
inches are money) but also points to this being a fixed layout with locally
relevant text pasted into it.
In fact the only material in the paper that doesn't have the whiff of a
political hit job (i.e. consists primarily of criticism of an incumbent
Democratic elected official) is the aforementioned "Athletes in Action" section
and an events directory, which I suspect is also software-generated due to some
telltales like blank template fields and a very odd selection of events to
cover (the headline item is the band Agent Orange playing at Sister Bar, but no
other items from Sister Bar's busy music schedule make it to this page).
Most articles have the byline "George Willis," although one opinion piece by
Pete Dinelli stands out. Dinelli is a former city councilor and writes a
somewhat prominent blog on local politics. He is also, as far as I can tell,
the only byline that is definitely a real person. Apparent lead reporter
"George Willis" has a fairly generic name but but seems most likely to be a
freelance sports journalist.
The Dinelli piece is interesting. It closely parallels, but does not match, an
article on Dinelli's blog. I reached out to Dinelli to ask how he came to
contribute to the New Mexico Sun, but I didn't hear anything back.
Let's turn to the website, newmexicosun.com. Its contents are very similar to
the printed paper, although it looks appreciable more polished and has a lot
more general news content. A somewhat buried "About" page indicates that it is
published by "PIPELINE Advisors LLC" and is part of their "family of Metro News
Sites." Bradley Cameron is named as CEO and managing editor. No such entity, or
foreign registration, exists in New Mexico, but it does exist in Texas where
the secretary of state indeed lists Bradley Cameron. It's rather confusing to
be the CEO of an LLC, and Texas records actually give the title managing
member, along with Brian Timpone. The address given is a single family home in
downtown Austin, which I always find a bit odd given the ready availability of
virtual offices.
Some readers probably know exactly where this is going by now, and the names
Cameron and Timpone might have been just a bit familiar to them. Cameron and
Timpone run Metric Media, Locality Labs (formerly Local Labs), and the Local
Government Information Service (LGIS), several organizations accused in the
press of operating large numbers of websites that appear to be local news
sources but actually operate as advertising for conservative political
interests. While the line between news, opinion, and advertising can be
somewhat thin in the world of politics, the most damning aspect of this
operation is its volume. It's no coincidence that this newspaper seems hastily
prepared, and probably mostly by the use of freelancers and automation. Cameron
and Timpone operate over a thousand such websites according to an article in
CJR,
each of which is superficially a local operation but is in fact run out of
Austin. The Guardian has reported on this group as
well.
Indeed, printed versions of these papers are apparently not unique, as an
article
details that some printed copies were produced at the printing plant of the
Des Moines Register. While it's common for newspapers to run commercial print
jobs for smaller publications and marketing, this situation certainly has a bit
of a smell to it.
So none of this is really new, and the New Mexico Sun as a website dates back
to 2020 at least. What has changed is its unprompted appearance in my mailbox.
Whether this is a new strategy on the part of Metric Media/Locality
Labs/Pipeline Advisors, or just a decision to prioritize New Mexico this year
due to the apparently close governor's race, is hard to say. It sure is a weird
piece of mail, though.
But I don't write about journalism, do I? Let's take a look at the CYBER
INTELLIGENCE.
newmexicosun.com is, unsurprisingly, registered with "domain privacy" although
I find it somewhat interesting that it's registered through the fairly small
registrar Epik. Like most of the internet these days, the domain name points at
AWS and MX records indicate the use of G-Suite. Passive DNS information for AWS
IP addresses can be questionable since they may change hands relatively
frequently, but SecurityTrails' free lookup shows about a half dozen "local news"
websites all being served up by the same IP. As with the shell companies behind
these websites, they seem to organize their infrastructure regionally: the New
Mexico Sun runs alongside the Austin Journal, the Houston Daily, and the Midland
Times. One standout is the Suburban Marquee, apparently of Chicago, but an odder
one is the Globe Banner.
The only Globe I know of is Globe, Arizona, which meets the regional theme but
is a town of under 10,000. Perhaps this explains why I am having such a hard
time determining just where the Globe Banner is supposed to be local to:
there's absolutely nothing on that website that isn't low-effort international
business news right off of PR Newswire. Whatever's going on in Globe, wherever,
the front-page headliner is "HappyCo confirmed plans to acquire Toronto-based
rental lifecycle management platform Yuhu, according to a press release." How
regionally appropriate! The #1 most read article, according to the sidebar, is
"Okonjo-Iweala: ‘We cannot afford to leave trade and WTO behind’ when tackling
climate change."
So I think the Globe Banner might actually just be a mistake, one that might
live on for years to come. It's sort of the liminal space of newspapers:
endless empty hallways of acquisition announcements and industry association
lobbying. The headline "Rubio and Gallagher call for national TikTok ban"
seemed like it might be a politically-motivated insertion but is actually just
"syndicated" straight out of a press release from those congresspeople. The
Globe Banner's ambitious Europe section is here to tell us that a German
company has introduced a new line of two-post car lifts. "According to the
automotive equipment company, the products will help customers reduce
installation times by up to 20%, feature three adjustable width positions
allowing flexible installation of lifts and offer an efficient and safe
obstacle-free working area."
Most of these websites are identical, although the Suburban Marquee also stands
out with a different theme. They have something the feeling of WordPress
templates, but I think these websites are actually running on homegrown
software. It seems a little sloppy: the single minified Javascript file used
for most functionality appears to be written by StackOverflow. The API endpoint
used by the newsletter sign-up form returns {"message":"Page Not Found"} to a
GET request, and {"message":"Page Not Found"} to a POST without the correct
form fields. It's got that fun CSS grid framework thing going on where the CSS
class list on every div is just inline styling in a less readable syntax and
not at all semantic.
One tell in the markup of an article indicates that "Froala WYSIWYG Editor" was
used to create its summary text. The developer doesn't seem to have kept the
different properties very well separated: the New Mexico Sun loads media from
Cloudfront with "houstondaily" in the path (I briefly got excited and thought
this might be an S3 bucket name, but it seems Cloudfront wisely leaves those
out).
Oh, I found an S3 bucket: jnswire, used for a few background images, seemingly
only for articles that are genuinely about New Mexico - and thus perhaps those
actually composed in this software and not syndicated automatically. Google has
indexed a healthy range of court filings in that bucket, but little use.
Looking for this term more broadly, a research service tells me that
jnswire.com was registered via GoDaddy by Brian Timpone, one of the founders of
this local news collective, in 2012. Indeed, this use of the JNS acronym refers
to Journatic News Wire, a connection made by a Twitter profile that no longer
exists.
And the New York
Times
has more on that: Journatic was apparently a service Brian Timpone developed
that generated articles automatically based on data feeds. It would seem that
this software probably backs most of these "local news websites," which the
NYT suggests as well. The software is, apparently, not very good. The Chicago
Tribune was apparently a customer and backed off of the service after it found
it was distributing plagiarized articles.
How does an automated system produce a plagiarized article? Well, one way is by
lying about the "automated" part. While I don't know that there is completely
solid evidence, several sources online (including the NYT) make the accusation
that Journatic was mechanical turking the problem, paying writers in the
Philippines on a gig-work basis to write articles that were then distributed
under either wire byline or a fake byline. Some suspect that the local news
network that rose from Journatic's ashes does the same, but honestly from
looking at the content I'm skeptical... it's not even really good enough to
have been written under a cent a word. Most of the articles stray so little
from the press release that they were less written than somewhat selectively
copied and pasted.
So, where does this leave us? About ten years ago, a journalist started a
service that would automate news by (supposedly) generating articles via
software. Today, it seems that they've done just that, but it's not really
news anyone wants to read... it's window dressing, sort of the journalistic
equivalent of those five books at Ikea, providing a generally newspaper-like
environment for the "payload" articles of the Locality/Metric/LGIS network.
The wonders of the internet.