>>> 2021-10-11 intro to burglary

One of the best-known brands in American burglar alarm systems is ADT. ADT stands, of course, for American District Telegraph. As the name suggests, ADT did not originate as a burglar alarm company. Instead, it started as a telegraph operation, mostly providing stock quotes and other financial information within cities. As happened with many early telegraph companies, ADT became part of Western Union and later part of AT&T. The history of ADT as a telegraph company is not all that interesting, and their telegraphy business is not well remembered today.

The modern ADT story starts, the company history tells us, with someone breaking into the home of one of the founders of one of ADT's constituent companies. This was the origin of the business line we know ADT for today: burglar alarms. While a number of companies have entered and exited the burglar alarm market (including AT&T itself as a separate venture from ADT), none have gained the market dominance of ADT... dominance so great that they lost an antitrust case in the '60s.

Before we get to the history of burglar alarm signaling, though, we should discuss the burglar alarms themselves.

The fundamental concept of a burglar alarm is fairly simple, and might better be explained by analogy to a similar, simpler system. Many industrial machines make use of a "safety string" or "safety circuit." This consists of a set of limit switches or other devices connected in series on things that are safety critical, like access doors. It's all arranged such that, in a "normal" state, the safety string is closed (all of the switches are in their circuit-closed state). If any unsafe condition occurs, such as an access door being opened, a switch opens the circuit. This usually cuts power to the potentially dangerous equipment.

A burglar alarm is simply the extension of this concept to multiple circuits. A set of switches and sensors are installed throughout the protected area. Any of these sensors indicating an "unsecured" state causes an alarm.

In practice, the wiring of burglar alarms is more complex than that of a simple safety string. There are various reasons for this, but the most obvious is the problem of supervision. Burglar alarms are designed to operate in a somewhat hostile environment including the possibility of tampering, and in any case insurance companies usually require that they have an adequate "self-test" facility to ensure that the system is functioning correctly during arming. This means that the alarm system needs to have a way to determine whether or not the sensor circuits are intact---whether the sensors are still connected. This is referred to as supervision.

A very common supervision method is the end-of-line resistor. A common burglar alarm circuit arrangement is the normally closed circuit with 5.6k EOL resistor. In this system, the sensors are each connected to the alarm in series and are normally closed. When a protected door is opened, for example, the sensor opens the circuit which is detected by the alarm controller. An enterprising burglar, though, might realize that they can short the two alarm wires together and thus prevent the circuit ever opening. An EOL resistor complicates this attack by creating a known, fixed resistance at the far end of the sensor circuit.

Let's imagine that the alarm controller functions by measuring the resistance of each alarm circuit (they basically do, but usually instead by putting a fixed current on the circuit and then measuring the voltage drop). If the resistance of the circuit is infinite, it is "open," which indicates an alarm. If the resistance of the circuit matches the expected EOL resistor (say 5.6k but the value just depends on what the alarm manufacturer chose), everything is normal. If the resistance of the circuit is zero (or near zero), the circuit has been shorted... and that can result in either the alarm sounding or reporting of a "trouble" condition.

This isn't the only way to use an EOL resistor. Another approach that is very common with fire alarms is a normally open EOL resistor circuit. In this system, the normal state is the fixed resistance of the EOL resistor, but the "initiating devices" such as smoke detectors are connected between the two alarm wires and actually short them together (creating a zero resistance) to cause the alarm to sound. In this case, an infinite resistance (open circuit) indicates that the circuit has been broken somewhere. In practice the difference between normally-open and normally-closed alarm circuits is more one of convention than technical merit, as both have very similar reliability characteristics. Largely for historical reasons, burglar alarms tend to be normally closed and fire alarms tend to be normally open, but there are many exceptions to both.

In early burglar alarms, this basic concept of measuring the resistance of an alarm circuit to check for a "normal" or "secure" value was very clear. A common vault alarm in the early 20th century had a large gauge showing the resistance of the single circuit and a knob which was used to adjust a variable resistor. To arm the alarm, the knob was turned (adjusting a reference resistance) until the needle fell into the green band, and then the alarm was switched on. The needle leaving the green band in either direction (indicating an open or a short circuit) resulted in an alarm, often by a mechanism as simple as the needle touching a peg (but perhaps also by things like spring-balanced magnetic coils). Modern burglar alarms are mostly just an evolution of this same design, although digital communications are becoming more common.

A burglar alarm usually uses several of these circuits, which are labeled "zones." Zones could correspond to physical areas of a protected building (and sometimes do), but for practical reasons it's very common for zones to correspond more to type of sensor than location. For example, it is very common to have one zone for perimeter detection devices (e.g. window and door closures) and one zone for presence detection devices (e.g. motion detectors) [1]. This is done so that different types of sensors can be armed separately, most often to enable a "perimeter" or "home" or "stay" mode in which the sensors on entry and exit points are armed but the sensors on the interior space are not.

Besides the sensors themselves, another important part of a burglar alarm is a means of arming and disarming it. Due to the lack of practical digital technology, early alarms took a very mechanical approach. A common arrangement in commercial buildings was a two-keyswitch system. To disarm the alarm, a key switch on the outside of the building had to be turned to the disarm position. Then, within a certain time period (measured by anything from a mechanical timer to a thermistor and heater), a second keyswitch located somewhere in the building had to be turned to the disarm position.

This somewhat complex system is actually very clever. It's quite possible that a skilled burglar will pick, dismantle, or otherwise defeat the outside keyswitch. The interior keyswitch serves as a "what you know" second factor: a burglar, not being familiar with the building's alarm system, would probably not know where the second switch was and would run out of time while looking for it. To improve this mechanism, the alarm panel and second keyswitch (it was usually right on the alarm panel) was often put somewhere non-obvious like a closet off of an office or behind a painting.

This use of two different key switches gets at a fundamental concept in the design of burglar alarm systems. The set of access points (like doors and windows) monitored by the alarm delineates a boundary between the secured space and the unsecured space. Alarm equipment within the secured space gets a certain degree of protection against physical tampering by virtue of its location: it is difficult to a burglar to tamper with something if they can't get to it without setting off the alarm. On the other hand, devices in the unsecured space, such as a keyswitch placed outside, are much more vulnerable to tampering. These devices need some kind of additional protection, or need to have their capabilities limited. The same problem exists with door locks and all sorts of other security systems [2].

In other cases a much simpler and more manual approach was taken. In some early alarm systems, there was no way to disarm the alarm. Instead, the employee opening for the morning would contact the monitoring point and exchange codewords (or otherwise identify themselves), so that the alarm center operator knew to disregard the upcoming alarm. This is actually still a fairly common practice both in military and other high security installations (where the extra manual checks are worth the resources) and in multi-tenant, infrequent access locations like radio towers where it would become frustrating to issue alarm codes to all of the different contract technicians. You will sometimes see signs along the lines of "Call 123-456-7890 before entering," which is a good indicator that this approach is still in use [3].

By the '70s the "alarm code" or "alarm PIN" was becoming the dominant approach, with users expected to disarm the alarm by entering a numeric combination. Modern alarms still mostly rely on this method, although it's getting more common to be able to arm and disarm alarms via the internet or a remote keyfob.

In both the cases of sensors and arm/disarm mechanisms we see that there has not been a great deal of technical progress over the last decades. In general the alarm space is somewhat stagnant, but it depends on the market. Consumer systems are changing very quickly, but in some ways for the worse, as newer consumer alarms are often cheaper and easier to install but also less reliable and easier to tamper with than older systems. No small number of the "ease of use" improvements in consumer alarms are directly achieved by reducing the security of the system, usually in a way that consumers don't clearly understand (more about this later).

Commercial alarm systems, on the other hand, have changed much less. Partially this is because the market is small and somewhat saturated, but partly it is because of the relatively higher security and certification requirements of commercial insurance companies. For businesses, insurance discounts are usually a bigger factor in the decision to install an alarm system, and the insurance companies are usually stricter about requiring that the alarm be certified against a specific standard. The good news is that these standards mean that commercial alarms are usually built on all of the best practices of the '90s. The downside is that the standards do not change quickly and so commercial alarms do not necessarily integrate modern cryptography or other methods of enhancing security. All in all, though, commercial alarms tend to be both more antiquated and more secure than modern consumer alarms.

This post has really just been a quick introduction to the topic of burglar alarms, giving you the basic idea that they function by monitoring strings of sensors and provide some authenticating method of arming and disarming the alarm. In the future, we'll talk a lot more about burglar alarms: about central monitoring systems (remember ADT?) and about the new landscape of DIY consumer systems, at least. Probably some rambling about different types of motion detectors as well.

[1] There are a GREAT variety of different types of perimeter and presence sensors, and they are all very interesting, at least to me. You may be detecting that technical security is an interest of mine. In the future I will probably write some posts about different types of burglar alarm sensors and the historical evolution they have gone through.

[2] While not as relevant today, this is one of the reasons that alarm keypads are usually placed inside the secured space. In older alarm systems, the keypad sometimes directly armed and disarmed the alarm via a simple electrical mechanism, making it possible to "hotwire" the alarm from the keypad. Placing the keypad inside of the secured space, such that anyone accessing it would have set off a sensor and started the entry delay timer, makes this kind of exploit more difficult by putting the burglar on the clock. In well-designed modern alarms the keypad no longer has the capability to disarm the alarm without the user entering a code (i.e. the keypad sends the code to the alarm panel elsewhere to be checked), but even today we can't take this for granted as some manufacturers have "economized" by putting the entire controller into the keypad.

[3] This is particularly common in telecom facilities because telecom companies have an old habit of implementing a "poor-man's burglar alarm." They would simply connect a switch on the door of the equipment hut to the trouble circuit used to report conditions like low backup batteries and failed amplifiers. That way any unauthorized access would result in a trouble ticket for someone to go out and inspect the equipment. Of course, authorized access just meant calling the maintenance office first to tell them to ignore the upcoming trouble alarm.

>>> 2021-10-06 street lighting and nuclear war

Let's consider, for a while, electric lighting.

The history of electric lighting is long and interesting, and I don't expect to cover it in much depth or breadth because it's somewhat off of my usual topic and not something I have a lot of expertise in. But I do have a fascination with the particular case of large-area outdoor lighting, e.g. street lighting, and there are a few things to talk about around street lighting that are really rather interesting.

As a very compressed history, the first electric street lights usually took the form of "moon towers," tall masts with arc lamps at the top. Various types of open and contained arc lamps were experimented with early on, but generally fell out of favor as incandescent lamps became cheaper and lower maintenance. The only remaining moon towers in the US are in Austin, Texas. They were expensive and high-maintenance, so they were fairly quickly replaced with low-height incandescent in most applications. Later, mercury vapor and metal halide arc lamps would begin to replace incandescent street lighting, but let's stop for a while in the era of the incandescent street light [1].

Let's say that you were installing a large number of incandescent lights, for example to light a street. How would you wire them?

In our homes, here in the United States, we wire our lights in parallel and apply 120v to them. This has convenient properties: parallel wiring means that the lights all function independently. A fixed voltage means that the brightness of a given light can be adjusted by using bulbs of different resistances, which will result in a different power (say, 60 watts). That makes a lot of sense, which is why it may be surprising to some to learn that incandescent street lights were wire in series.

Series wiring had multiple advantages, but the biggest was conductor size: in a parallel-wired system, wiring near the power source would need to carry a very large current (the combined current of all of the bulbs), and lights further from the power source would be dimmer due to voltage drop unless the conductors were prohibitively large. In a series-wired system, the current is the same (and much lower, approximately equivalent to a single bulb) at all points in the wiring, and voltage drop is seen across the entire length, and thus consistent across all of the bulbs.

These street lighting circuits are referred to as constant current circuits, as opposed to the more conventional constant voltage. All of the bulbs were designed to operate at a specific current, 6.6A was typical, and a specially equipped power supply transformer adjusted the voltage on the circuit to achieve that 6.6A. The voltage would be fairly large, typically something like 5kV, but that wasn't a problem because the streetlight wiring ran overhead with power distribution wiring of similar and higher voltages.

In early street lighting systems, the constant current regulator was a complex mechanical device using magnetic coils and levers. Today, constant-current incandescent lighting circuits are still in use in some applications and the constant current regulators have been replaced with more robust electronically controlled systems. But the regulators aren't really that important, except to understand that they simply apply whatever voltage is required to get 6.6A to flow.

Of course, connecting lighting in series introduces a major problem that you have probably realized. If any bulb burns out, the entire series circuit will be broken. That's unacceptable for street lights, and a few different solutions were invented but the most common was a cut-out disk. The cut-out disk was a small fuse-like device, but operated somewhat in the opposite way of a fuse. In fact, they're sometimes amusingly referred to as antifuses.

The cut-out disk is wired in parallel with the bulb. Should the bulb burn out, the voltage across the two sides of the disk rises, which causes an arc that "burns out" a film material separating two contacts. The contacts then touch, and electricity flows freely through the cut-out disk, restoring the circuit. When the bulb is replaced, the cut-out disk is replaced as well with a fresh one.

"Stay-Lit" Christmas light strings employ a very similar method, but use a thermistor instead of a cut-out disk so that it is not an additional consumable part. But the concept is the same: a device in the bulb base begins to pass current if exposed to the full 120v power supply, instead of the few volts normally seen when the bulb is present.

Constant-current lighting circuits have an interesting safety property, which this discussion of cut-out disks may have brought to your mind. A short circuit in a constant-current lighting circuit is actually pretty safe, as the regulator will reduce the voltage to near zero to maintain only 6.6A. But, when no current flows, the constant current regulator will increase the voltage applied to the entire circuit in an attempt to restore current flow. Modern electronic regulators mitigate this somewhat by detecting this condition, but it gives constant-current lighting circuits a particularly sharp edge. An open circuit can be rather dangerous as the regulator will increase the output voltage to the upper limit---potentially something like 10kV. Everything on the lighting circuit needs to be rated for this maximum voltage, which leads to more demanding requirements than typical 120v or 240v commercial lighting.

With that preamble on the concept of constant-current street lighting, let's take a look at a particularly interesting bit of history that closely relates to these technical details [2].

On July 9th, 1962, the United States conducted its 7th high-altitude nuclear test. While an earlier series of high-altitude demonstrations had shown the tactical potential of detonations far from the ground, significant advances had been made in the science and instrumentation in the intervening years, and greater resources had become available for geopolitical reasons (resumed Soviet testing). The new test, named Starfish Prime, was thus expected to contribute significant information on the effects of high-altitude detonations.

The approximately 1.5Mt detonation occurred at about 400 km altitude, well into space. It was expected that a detonation at such a high altitude would create a substantial electromagnetic pulse effect, since the "horizon" was far from the detonation allowing wide coverage and less cancellation by ground reflections. Prior to the Starfish Prime test, however, EMP had not typically been viewed as a major consideration in the use of nuclear weapons... while the effect clearly existed, the magnitude was not such that it was likely to cause substantial disruption [3].

Starfish Prime seemed to suggest otherwise. Effects that are out of scope for our purposes resulted in damage to several US military satellites. Moreover, unanticipated effects resulted in higher EMP field strengths at the ground than had originally been estimated. This effect extended over a very large area, including the nearly 1,500km straight-line distance from the detonation near Johnston Atoll to Honolulu.

Various effects observed in Honolulu (from which the detonation was visible in the sky) were attributed to the explosion. It can be difficult to sort out which of these reports are accurate, as something as dramatic as a nuclear detonation tends to get tied up with all kinds of unrelated events in people's memory. One thing that was particularly widely reported, though, was streetlights in Honolulu going dark at the instant of the detonation.

Was this actually an example of EMP having a significant effect on a civilian power distribution system, as has long been theorized but seldom validated?

The answer, it turns out, is yes and no. Honolulu, like many jurisdictions in the early '60s, made extensive use of series-wired street lighting circuits. Also like many municipalities, Honolulu had gone through growth and changes that resulted in various ad-hoc and sometimes confusing adjustments to the electrical distribution system. Part of this had involved changes in distribution voltage on existing lines, which created problems with the safety separation distances required between lines of different voltages attached to the same pole. The result was somewhat odd arrangements of lines on poles which complicated the installation of street lighting circuits.

On some Honolulu streets, it had become impractical to install series-wired street lighting circuits at medium voltage (6kV being fairly typical) and still have adequate safety separation from 240v split secondary wiring supplying power to homes. Honolulu adopted a solution of mixing series-wired high-voltage and parallel-wired low-voltage street lighting. On residential streets with crowded poles, street lights ran on 500v so that their lines could be run directly alongside the residential power supply.

At this point in time, the modern norm of photocell switches installed on each light did not yet exist. Instead, streetlights were turned on and off by mechanical timer switches (or occasionally manual switches) attached to the constant current regulators. So, the 500v lighting used on residential streets still needed to be connected to the series-wired system in order to run off of the same controls. The solution, which simplified the cabling in other ways as well, was to power the 500v constant-voltage lighting off of the 6.6A constant-current system. This was implemented by connecting autotransformers to the series-wired system that reduced the voltage to 500v. The constant-current regulator meant that the autotransformer could cause an arbitrary voltage drop, at 6.6A, as necessary to provide adequate power to the 500v lights connected to it.

Essentially, the autotransformer acted as a particularly large 6.6A light on the series circuit. This made it subject to the same series disadvantage: if the autotransformer failed, it would cause the entire series circuit to shut off. The solution was to equip the autotransformer with a cut-out disk just like the lights, albeit one rated for a higher voltage.

It was determined that the street lights which failed in response to the Starfish Prime EMP all seem to have been the low-voltage lights attached to constant-current circuits. Perhaps you can see what happened.

Some of the low-voltage lighting circuits were positioned such that their length was oriented perpendicular to the EMP field. The altitude and location of the detonation was such that the EMP field reached Honolulu with horizontal polarization nearly parallel to the ground. The combination of these factors created a near-worst-case for induced voltage in certain low-voltage lighting circuits. The induced voltage backfed the autotransformer, resulting in a higher voltage on its constant-current side that fused the cut-out disk, shorting the autotransformer and cutting off the power supply to the low-voltage lighting circuit.

The solution was simply to replace the cut-out disk.

This should illustrate two things about the effects of nuclear EMP: First, it is possible for nuclear EMP to have damaging effects on power infrastructure. Second, it is not especially easy for nuclear EMP to have damaging effects on power infrastructure. The Honolulu incident resulted from a specific combination of factors, and specifically from the use of a "weak-link" design element in the form of the cut-out disks, which performed as expected in response to an unusual situation.

None of this is to say that EMP effects are insubstantial, but they are mostly confined to digital and especially telecommunications systems due to their far higher sensitivity to induced voltages. Power systems with long enough wiring runs to pick up substantial induced voltages also tend to be designed for very high voltages, making damage unlikely. The same cannot be said of telephone lines.

By the '60s series-wired constant-current lighting systems were falling out of favor. The future was in mercury-vapor lamps running at 240v and controlled by local photocells, so that they could be powered directly off of the secondary distribution lines. This is still basically the arrangement used for street lighting today, but mercury vapor has given way to LED.

Constant-current lighting still has its place. While airfield marker lighting is mostly being converted to LED, it's still generally powered by constant-current loops. Many airfields still use 6.6A tungsten bulbs, which were at least perceived to have reliability and visibility advantages until relatively recently. Airfield constant-current regulators usually provide three (occasionally more) output settings with different target currents, allowing for a low, medium, and high intensity. While less common with today's better marker light optics, you will still sometimes hear pilots ask the tower (or use radio control) to turn the lights down.

When it comes to LEDs, most non-trivial LED lighting is actually constant-current. The use of a small solid-state constant-current regulator (usually called an LED driver in this context) improves efficiency and lifespan by keeping LEDs at an optimal current as temperature and supply voltage changes.

Despite staying reasonably close to their '70s state, streetlights have become hubs for technology projects. They're convenient, widely-distributed power sources that are either owned by the city or operated on contract for the city. Many "smart city" technologies like environmental sensors and WiFi/municipal LTE/smart meter radios are offered in packages intended to be clamped directly onto streetlights and powered by tapping their cables. The fairly standardized screw socket used for photocell switches on street lights has itself become a form of power outlet, with some smart city devices screwing in to use it as a power feed and incidentally also turning the light on and off.

In this way the line between "light controller" and "sensor package" can become blurry. The City of Albuquerque now installs networked controllers on new street lights that primarily allow for remote monitoring and management of the light itself---but also support environmental data collection and traffic monitoring via Bluetooth sniffing.

The humble street light today reminds me a bit of the cigarette lighter socket in cars... barely changed in decades, and yet a major factor in enabling a huge technology market.

Or at least that's an optimistic way to look at it. The pessimistic way is to observe that we now live in a world where streetlights are being repurposed to detect gunshots. Cheery.

Addendum: I am not personally a fan of aggressive street lighting. It tends to create a substantial light pollution problem that is thought to contribute to health problems in humans and environmental disturbances. Further, it's been the common wisdom for some time now that street lighting is likely not actually effective in reducing crime. That said, a recent 2019 study conducted in New York City is the first randomized controlled trial of the impact of additional street lighting on crime, and it actually did find a reduction in crime, and not a small one. That stands in opposition to a history of studies that have not found any crime reduction, but none of those studies were as rigorously designed. Hopefully more research will be conducted on this question.

[1] Incandescent street lights are typically the ones fondly remembered as "historic" street lamps anyway, with many "antique style" streetlamps installed in historic districts today being loose replicas of various common incandescent models, such as the GE or Novalux "acorn" glass globes with frilly metal reflector. That is, assuming they are replicas of electric and not gas fixtures, although makers of vaguely historic light fixtures often mix the two anyway.

[2] Much of the information on this incident comes from the aptly titled "Did High-Altitude EMP Cause the Hawaiian Street Light Incident?", SAND88-3341. Charles Vittitoe, Sandia National Laboratories, 1989. As far as I can tell this paper, prepared well after the fact, is one of the only detailed case reports of a real EMP incident in the public literature. The fact that there was, reportedly, very little rigorous investigation of the Starfish Prime test's EMP effects in Hawaii for over twenty years after has interesting implications on how seriously EMP effects on civilian infrastructure were taken at the time.

[3] I discuss this topic a bit more in my YouTube video on EMP simulation facilities at Kirtland Air Force Base.

>>> 2021-09-30 the original microcell

Many readers may know that, historically, telephone instruments (e.g. the thing you hold up to your face and talk into) have been considered part of the telephone system proper. In practice, that meant that up until the '70s most people leased their phones from the telephone company, and the telephone company maintained everything from the exchange office to the side of your head.

The biggest landmark in the shift of telephone instruments from "part of AT&T" to personal property that you can buy at WalMart is the "Carterfone decision." The Carterfone precedent, that telephone users could connect anything they want to the telephone network as long as it is reasonably well-functioning and compatible, created the entire modern industry of telephones and accessories. What's only remembered a little hazily, though, is what the Carterfone actually was---the original device that fought for telephone interconnection.

The Carterfone was an acoustic coupler that allowed a telephone handset to be connected to a two-way radio. Essentially, it was the first form of telephone patch, although it was very simple and did not provide the automation that would later be expected from phone patches. A typical use-case for the Carterfone was to allow a dispatcher to connect someone in the field (using a mobile radio) to someone by telephone, with the dispatcher doing all of the dialing and supervision of the call.

The Carterfone was a long way from the wireless telephony systems we use today, but it is a common ancestor to most of them. It, and several other early telephone radio patch systems, introduced the concept that telephones could be divorced from their wires. Of course this lead to the development of radiotelephony, cellular phones, etc, but great distances weren't required for the radiotelephone to be useful.

Well into the '90s it was common to see people walking around their homes trailing 50' of telephone wire. A great deal of couplers and long cables were available for the purpose, and it either replaced or complemented (depending on budget) the also common practice of providing a telephone jack in each room. Wouldn't it be easier to avoid the need for in-wall, under-carpet, along-baseboard, and trip-hazard wiring entirely by using some sort of local, low-power radiotelephone?

I am, of course, describing the cordless phone, and early cordless phones were not much more complex than low-power two-way radios. There is one primary way in which a telephone is different from a radio: telephones are full-duplex, while "radio" is generally taken to refer to a half-duplex system in which it is necessary to use a push-to-talk or other control to switch to transmit.

Nowadays we tend to take full-duplex communications somewhat for granted, because it is reasonably easy to simulate it through packetization. In the '80s, though, when cordless phones became popular, packetized digital voice technology was still some ways from being sufficiently small and cheap to put in a consumer phone. Instead, full-duplex communication required that the handset and base station both transmit and receive simultaneously.

So, a cordless handset contained two radios, a transmitter and a receiver, and the base station contained two radios as well. There is a basic problem with operating two radios like this: the receive radio will typically be overwhelmed by the signal emitted by the transmit radio, and so unable to receive anything. There are various ways to solve this problem, but it becomes far easier if the simultaneous transmit and receive are at very different frequencies. For this reason, early cordless phone systems made use of pairs.

A typical design was this: the base station transmitted to the handset at around 1.7MHz, and the handset transmitted back to the base station at around 27MHz. The use of two different bands made it relatively easy for each receiver to filter out the signal from the nearby transmitter. Of course, the split-system between two bands can now result in confusion as it's not always clear what a "27MHz phone" for example even means.

Signaling was extremely simple, generally constrained to something like the phone emitting one fixed tone to indicate that the base should go off-hook, and another fixed tone to indicate it should go on-hook. Since DTMF was generated on the handset, no other signaling was required. This allowed the implementation to be almost entirely analog.

Actual modulation was FM, and on early cordless phones channel selection was manual with only a small number of discrete channels available. This meant that interference and crosstalk between different cordless phones was a common problem, one worsened by cordless phones sharing frequency allocations with some devices like baby monitors. Security was a very real problem, people did indeed listen in on other people's cordless phone calls by repurposing consumer devices like baby monitors or using wide-band receive radios.

Later cordless phones continued to use this basic scheme, although a 47Mhz band was added into the mix. The use of the higher frequency bands had the major benefit of making the antennas smaller; 1.7MHz phones had required a telescopic whip antenna while 27MHz/49MHz phones could generally make do with a "rubber ducky" antenna more typical of cordless phones today.

More interestingly, though, these later phones in the VHF low-band were capable of quite a few more channels and so introduced automatic channel selection. The need to implement link negotiation lead naturally to a more sophisticated signaling system between the handset and the base. Digital signaling allowed the base and handset to exchange commands, first enabling the "find handset" button on the base and later allowing the handset to control an answering machine integrated into the base.

The introduction of 900MHz phones (900MHz is a common ISM band used by a variety of devices) occurred somewhat in a transitional period, as there are both analog and digital phones made for 900MHz with various combinations of signaling and security features. Digital encoding and the use of spread spectrum tended to improve audio quality but also security, because even if there was no encryption decoding required a specialized device... and spread spectrum is itself a form of security if the sequence is determined in a secure fashion. Digital cordless phones largely eliminated eavesdropping except by particularly determined opponents, but generally only did so by raising the attack cost (requiring a decoder) rather than by employing strong security.

The 900MHz band is crowded with devices, as are 2.4GHz and 5.8 GHz where various late-'90s and early-'00s digital cordless phones operated. This often translated into disappointing range and reliability, and moreover there was a serious lack of standardization. Standardization tends to be less important for cordless phones because the base and handsets are sold together to the extend that consumers have no real expectation of them being interchangeable (technically they often are but it is difficult to determine between which devices). A bigger issue was a lack of consumer understanding of the different features that cordless phones carried. Was any given phone secure (in that it employed some type of encryption)? Was it going to be more or less subject to interference from the microwave oven? What about range?

All of these questions would be best addressed by the industry concentrating on a standard cordless phone implementation, and the allocation of a dedicated band for these devices would significantly reduce interference problems. The confluence of these two interests lead to a major standards effort in the early '00s that culminated in the adoption of a European standard which had been the norm in Europe since the '90s: DECT.

DECT is a very interesting beast. While DECT is now strongly associated with consumer cordless phones, it was always intended for much more ambitious use-cases. DECT could serve as the entire wireless plane for a corporate PBX system, for example, using centralized base stations (ceiling mount perhaps) to set up calls and signaling between a variety of handsets carried by employees [1]. DECT was even contemplated as a cellular telephony standard, with urban-area base stations managing large numbers of subscriber handsets.

In practice, though, DECT has a much more modest role in the US. First, there is a matter of naming. DECT in the US is referred to as DECT 6.0, which has created a false impression that it is either version 6 or operates at 6.0 GHz. In fact, DECT 6.0 operates in a 1.8GHz band allocated for that purpose and the name is just marketing fluff because 6 is a bigger number than the 5.8GHz cordless phones that were on the market at the time.

DECT makes use of frequency-hopping techniques as well as digital encoding that allow for active mitigation of interference based on time-division multiplexing. The takeaway is that, generally speaking, DECT phones will not interfere with other DECT phones. This addressed the biggest performance problems seen in congested areas.

DECT is, of course, packetized, and took many tips from the simultaneous work on ISDN in Europe. The DECT network protocol, LAPC, is based on the ISDN data link layer. Both are based on HDLC, an ISO network protocol that was derived from IBM's SNA. So, in a vague historical sense, modern cordless phones speak the same language as late '70s big iron, but over a wireless medium.

LAPC is a fairly complete network protocol, even from a modern perspective, and provides both connection-oriented and connectionless communications. Like most network protocols from the telecom industry, DECT supports defined-bandwidth connections by means of allocating "slots" in the network scheduling.

On top of LAPC a variety of functionality has been implemented, but most importantly basic call control (supervision) functionality and set up of real-time media channels using various codecs. Common codecs are ADPCM and u-law PCM, which both provide superior call quality to the cellular network (when HD voice does not succeed).

Because of DECT's greater ambitions, there is also a management protocol defined which allows handsets to register with base stations and exchange subscriber information. This allows cellular-like behavior that supports environments where there are multiple base stations with handsets roaming between them.

DECT supports authentication and encryption, but implementation is very inconsistent. The standard authentication and encryption protocol until around a decade ago was one with known weaknesses. As a practical matter, the use of FSS and TDMA in DECT makes active attacks difficult, and so DECT "exploits" do not seem to have ever been particularly common in the wild... but it certainly is possible to intercept DECT traffic, which lead to a change in standards to 128-bit AES encryption with improved key negotiation. Unfortunately it's not always easy to tell if devices make use of the newer encryption capability (or any at all), so security issues remain in practical DECT systems.

That's a lot about what DECT actually does, but what about the weird things it could do and usually doesn't? Those are my favorite kinds of things.

Protocols have been defined to run IP on top of DECT. IP-over-DECT was actually very competitive with 801.11 WiFi---it was slow (0.5mbps) but comparatively very reliable. A particular strong point for DECT was its origin as a possible cellular standard: DECT has always handled roaming between base stations very well, something that multi-AP WiFi systems struggle with in practice to this day. This made DECT a particularly compelling option for data networking in large, industrial environments. DECT IP networking was integrated into some industrial hand-held PC systems but was never common, despite efforts to commercialize a very WiFi-like DECT system under the name Net3. Another DECT computer networking initiative ran under the name PADcard and seems to have been launched on a few consumer products, but fizzled out very quickly.

What about DECT as a public cellular telephony standard? This doesn't seem to have really materialized anywhere. DECT had success in large-area applications like mines, but these were all still private corporate systems. DECT development for large systems ran pretty simultaneously with the development of GSM, which quickly gained more traction in DECT's European stronghold.

Despite DECT's failure to achieve its full potential, it has brought a surprising level of sophistication to the humble cordless phone. Modern cordless phones are almost exclusively DECT and take advantage of DECT's capabilities to offer multiple handsets per base station, intercom calling between handsets (often both dialed and "page" or broadcast), and a solid-state answering machine integrated into the base station and controllable from handsets. All cool features that no one uses, because now we all have cellphones.

[1] This use-case has obtained some success in the US in retail environments. Pacific Northwest retailer Fred Meyer's, for example, at least prior to the Kroger acquisition, armed each staff member with a DECT handset and earpiece at most stores. The advantages of this type of setup are clear: DECT handsets can be similarly priced to two-way radios but allow full-duplex communication and a "hybrid" model of radio vs. telephony behavior, by either selecting an intercom channel or dialing a number to reach a specific person. DECT in retail is likely giving way to IP-based solutions like Theatro (in use at some Walgreens locations), but a great many retailers (most Wal-Marts, for example) still use basic two-way radios on MURS or color dot channels. PBX DECT systems are still available though, and I remain tempted to buy the DECT gateway for the '90s digital PBX that runs in my office closet. More in the modern era, because DECT is better established than WiFi for telephony applications there are a lot of "Cordless IP phones" that handle all the IP in the base station and use DECT to reach the cordless phones. These are basically just IP evolutions of the older approach of a DECT module connected to analog accessory ports or a dedicated board in the PBX.

A special bonus addendum for the Hacker News crowd: After I wrote this, I somehow ran into the Japanese Personal Handy-Phone System. It's a moderately successful (for a time) cellular service using a technology that was very similar to DECT. Despite the Wikipedia article sort of making it sound like it, PHS does not seem to have actually been based on DECT in any way. It looks like a case of parallel evolution at the least.

>>> 2021-09-20 iMessage for Business

Given the sources of most of my readers, some of you may have seen this article about A/UX, Apple's failed earlier effort towards delivering an Apple user experience on a POSIX operating system. A/UX was driven primarily by demands for a "serious workstation," which was a difficult category to obtain at any kind of reasonable price in the 1980s. It is also an example of Apple putting a concerted effort into attracting large enterprise customers (e.g. universities), something that is not exactly part of the Apple brand identity today.

I wanted t, expand on A/UX by discussing some of Apple's other efforts to make serious inroads into the world of Big Business. In the 2000s and 2010s, Apple was frequently perceived as focusing on consumers and small organizations rather than large organizations. This was perhaps a result of Apple's particular nexus to the personal computing ethos; Apple believed that computers should be affordable and easy to use rather than massively integrated.

This doesn't seem to have been a choice made by intention, though: through the '90s Apple made many efforts to build a platform which would be attractive to large enterprises. Many major events in the company's history are related to these attempts, and they're formative to some aspects of Apple's modern products... but they were never really successful. There's a reason that our usual go-to for an integrated network computing environment is the much maligned Microsoft Active Directory, and not the even more maligned Apple Workgroup Manager.

Indeed, somewhere around 2010 Apple seems to have lost interest in enterprise environments entirely, and has been slowly retiring their products aimed at that type of customer. On the one hand, Apple probably claims that their iCloud offerings replace some of the functionality of their on-prem network products. On the other hand, I think a far larger factor is that they just never sold well, and Apple made a decision to cut their losses.

But that's starting at the end. Let's take a step back to the '80s, during the project that would become A/UX, to Apple's first major step into the world of networking.

I have previously mentioned the topic of network operating systems, or NOS. NOS can be tricky because, at different points in time, the term referred to two different things. During the '80s, an NOS was an operating system that was intended specifically to be used with a network. Keep in mind that at the time many operating systems had no support for networking at all, so NOS were for a while an exciting new category. NOS were expected to supported file sharing, printer sharing, messaging, and other features we now take for granted [1].

Apple never released an NOS as such (stories about the 'i' in iMac standing for 'internet' aside), but they were not ignoring the increasing development of microcomputer networks. The Apple Lisa appears to have been the first Apple product to gain network support, although only in the most technical sense. Often referred to as AppleNet [2], this early Apple effort towards networking was directly based on Xerox XNS (which will be familiar to long-time readers, as many early PC network protocols were derived from XNS). AppleNet ran at 1mbps using coaxial cables, and failed to gain any meaningful traction. While Apple announced Lisa-based AppleNet file and print servers, I'm not sure whether they ever actually shipped a server product at all (at least the file server was vaporware).

Apple's second swing at networking took the form of AppleBus. AppleBus was fundamentally just a software expansion of the Macintosh serial controller into an RS-422 like multi-drop serial network scheme. Because AppleBus also served as the general-purpose peripheral interconnect for many Apple systems into the '90s, it could be seen as an unusually sophisticated interconnect in terms of its flexibility. On the other hand, consumers could find it confusing that a disk drive and the network could be plugged into the same port, a confusion that in my experience lasted into the '00s among some devoted Macintosh users.

Nonetheless, the idea that inter-computer networking was simply an evolution of the peripheral bus was a progressive one that would continue to appear in Apple-influenced interconnects like Firewire and to a lesser extent Thunderbolt, although it would basically never be successful. In a different world, the new iMac purchaser that thought USB and Ethernet were the same thing might have been correct. Like the proverbial mad prophet, they have a rare insight into a better way.

For all of my optimism about AppleBus, it was barely more successful than AppleNet. AppleBus, by that name, came and went with barely any actual computer inter-networking applications. AppleBus was short-lived to the extent that it is barely worth mentioning, except for the interesting fact that AppleBus development as a computer network was heavily motivated by the LaserWriter. One of the first laser printers on the market, the LaserWriter was formidably expensive. AppleBus was the obvious choice of interface since it was well supported by the Macintosh, but the ability to share the LaserWriter between multiple machines seemed a business imperative to sell them. Ipso facto, the peripheral interconnect had to become a network interconnect.

In 1985, Apple rebranded the AppleBus effort to AppleTalk, which will likely be familiar to many readers. Although AppleTalk was a fairly complete rebrand and significantly more successful than AppleBus, it was technically not much different. The main component of AppleTalk outside of the built-in serial controller was an external adapter box which performed some simple level conversions to allow for a longer range. Unfortunately this simple design lead to limitations, the major one being a speed of just 230kbps which was decidedly slow even for the time.

As early as the initial development of AppleBus, token ring seem to be the leading direction for microcomputer networking and, indeed, Apple had been involved in discussions with IBM over use of token ring for the Macintosh. Unfortunately for IBM, their delays in having token ring ready for market lead somewhat directly to Apple's shift towards ethernet. Since token ring was not an option for the LaserWriter, Apple pushed forward with serial AppleBus for several years, by which point it became clear that Ethernet would be the victor. In 1987, AppleTalk shifted to Ethernet as a long-range physical layer (called EtherTalk) and the formerly-AppleBus serial physical layer was rebranded as LocalTalk.

AppleTalk was a massive success. For fear of turning this entire post into a long discussion of Apple networking history, I will now omit most of the fate of AppleTalk, but LocalTalk remained in use to the late '90s even as IP over Ethernet became the norm. AppleTalk was for a time the most widely deployed microcomputer networking protocol, and a third-party accessory ecosystem flourished that included alternate physical media, protocol converters, switches, etc. Of course, this all inevitably fell out of use for inter-networking as IP proliferated.

The summation of this history is that Apple has offered networking for their computers from the mid-'80s to the present. But what of network applications?

One of the great selling points of AppleTalk was its low-setup, auto-configuring nature. As AppleTalk rose to prominence, Apple broke from conventional NOS vendors by keeping to more of a peer-to-peer logical architecture where basic services like file sharing could be hosted by any Macintosh. This seems to have been an early manifestation of Apple's dominance in zero-configuration network protocols, which is perhaps reduced today by their heavy reliance on iCloud but is still evident in the form of Bonjour.

This is not at all to say that Apple did not introduce a server product, although Apple was slow to the server market, and the first AppleTalk file and print servers were third-party. In 1987, Apple released their own effort: AppleShare. AppleShare started out as a file server (using the AFP protocol that Apple designed for this purpose), but it gained print and mail support, which put it roughly on par with the NOS landscape at the time.

But what of directory services? Apple lagged on development of their own directory system, but had inherited NetInfo from their acquisition of NeXT. NetInfo could be used via AppleTalk, and that seems to have been at least somewhat common in universities, but it's hard to find much information on this---in part because directory services have never been especially common in Apple environments, compared to UNIX and Windows ones. The story of Apple directory services becomes clearer with Apple Open Directory, an LDAP-based solution introduced in 2002. Open Directory is largely similar to competitors like MS AD and Red Hat IDM. As a result of the move to open standards, it has occasionally, with certain versions of the relevant operating systems, been possible to use OS X Server as an Active Directory domain controller or join OS X to an Active Directory domain. I have previously worked with OS X machines joined to an Oracle directory, which was fine except for all the Oracle parts and the one time that Apple introduced a bug where OS X didn't check user's passwords any more. Haha, wacky hijinx!

As I mentioned, Apple has been losing interest in the enterprise market. The primary tool for management of Apple directory environments, Workgroup Manager, was retired around 2016 and has not really received a replacement. It is still possible to use Open Directory, but no longer feels like a path that Apple intends to support.

That of course brings us naturally to the entire topic of OS X Server and the XServe hardware. Much like Windows Server, for many years OS X Server was a variant of the operating system that included a variety of pre-configured network services ranging from Open Directory to Apache. As of now, OS X Server is no longer a standalone product and is instead offered as a set of apps to be installed on OS X. This comes along with the end of the XServe hardware in 2011, meaning that it is no longer possible to obtain a rack-mount system which legitimately runs OS X.

The industry norm now appears to be affixing multiple Mac Minis to a 19" sheet of plywood with plumber's tape. And that, right there, is a summary of Apple's place in the enterprise market. In actuality, Apple has come around somewhat and now offers an official rack ear kit for the Mac Pro. That said, it's 5U for specifications that other vendors fit in 1U (save the GPU which is not of interest to conventional NOS applications), and lacks conventional server usability features like IPMI or even a cable arm. In general, Apple continues to be uninterested in the server market, even to support their own workstations, and really offers the Mac Pro rack option only for media professionals who have one of the desks with 19" racks integrated [3].

I originally set out to write this post about Apple's partnership with IBM, which has delightful parallels to Microsoft's near simultaneous partnership with IBM as both were attempting to develop a new operating system for the business workstation market. I find great irony and amusement in the fact that both Microsoft, which positioned itself as Not IBM, and Apple, which positioned itself as even more Not IBM, both spent an appreciable portion of their corporate histories desperately attempting to court IBM. While neither was successful, Apple was even less successful than Microsoft. Let's take a look at that next time---although I am about to head out on a vacation and might either not write for a while or end up writing something strange related to said trip.

[1] Amusingly, despite many efforts by vendors, local file and printer sharing can still be a huge pain if there are heterogeneous product families involved. The more things change, the more they stay the same, which is to say that we are somehow still fighting with SMB and NFS.

[2] AppleBus, AppleNet, and AppleTalk all refer to somewhat different things, but for simplicity I am calling it AppleNet until the AppleTalk name was introduced. This is really just for convenience and because it matches the terminology I see other modern sources using. The relationship between AppleBus and AppleNet was as confusing at the time as it is now, and it is common for '80s articles to confuse and intermix the two.

[3] I would absolutely be using one of these if I had room for one.

>>> 2021-09-08 W X Y Z

Let's return, for a while, to the green-ish-sometimes pastures of GUI systems. To get to one of my favorite parts of the story, delivery of GUIs to terminals over the network, a natural starting point is to discuss an arcane, ancient GUI system that came out of academia and became rather influential.

I am referring of course to the successor of W on V: X.

Volumes could be written about X, and indeed they have. So I'm not intending to present anything like a thorough history of X, but I do want to address some interesting aspects of X's design and some interesting applications. Before we talk about X history, though, it might be useful to understand the broader landscape of GUI systems on UNIX-like operating systems, though, because so far we've talked about the DOS/VMS sort of family instead and there are some significant differences.

Operating systems can be broadly categorized as single-user and multi-user. Today, single-user operating systems are mostly associated only with embedded and other very lightweight devices [1]. Back in the 1980s, though, this divide was much more important. Multi-user operating systems were associated with "big iron," machines that were so expensive that you would need to share them for budget reasons. Most personal computers were not expected to handle multiple users, over the network or otherwise, and so the operating systems had no features to support this (no concept of user process contexts, permissions, etc).

Of course, you can likely imagine that the latter situation, single-user operating systems, made the development of GUI software appreciably easier. It wasn't even so much about the number of users, but rather about where they were. Single-user operating systems usually only supported someone working right at the console, and so applications could write directly to the graphics hardware. A windowing system, at the most basic, only really needed to concern itself with getting applications to write to the correct section of the frame buffer.

On a multi-user system, on the other hand, there were multiple terminals connected by some method or other that almost certainly did not allow for direct memory access to the graphics hardware. Further, the system needed to manage what applications belonged on which graphics devices, as well as the basic issue of windowing. This required a more complicated design. In particular, server-client systems were extremely in at the time because they had the same general shape as the computer-terminal architecture of the system. This made them easier to reason about and implement.

So, graphics systems written for multi-user systems were often, but not always, server-client. X is no different: the basic architecture of X is that of a server (running within the user context generally) that has access to a graphics device and input devices (that it knows how to use), and one or more clients that want to display graphics. The clients, which are the applications the user is using, tell the server what they want to display. In turn, the server tells the clients what input they have received. The clients never interact directly with the display or input hardware, which allows X to manage multiple access and to provide abstraction.

While X was neither the first graphics system for multi-user operating systems, nor the first server-client graphics system, it rapidly spread between academic institutions and so became a de facto standard fairly quickly [2]. X's dominance lasts nearly to this day, Wayland has only recently begun to exceed it in popularity. Wayland is based on essentially the same architecture.

X has a number of interesting properties and eccentricities. One of the first interesting things many people discover about X is that its client-server nature actually means something in practice: it is possible for clients to connect to an X server running on a different machine via network sockets. Combined with SSH's ability to tunnel arbitrary network traffic, this means that nearly all Linux systems have a basic "remote application" (and even full remote desktop) capability built in. Everyone is very excited when they first learn this fact, until they give it a try and discover that the X protocol is so hopelessly inefficient and modern applications so complex that X is utterly unusable for most applications over most internet connections.

This gets at the first major criticism of X: the protocol that clients use to describe their output to X is very low level. Besides making the X protocol fairly inefficient for conveying typical "buttons and forms" graphics, X's lack of higher-level constructs is a major contributor to the inconsistent look-and-feel and interface of typical Linux systems. A lot of basic functionality that feels cross-cutting, like copy-and-paste, is basically considered a client-side problem (and you can probably see how this leads to the situation where Linux systems commonly have two or three distinct copy and paste buffers).

But, to be fair, X never aimed to be a desktop environment, and that type of functionality was always assumed to occur at a higher level.

One of the earliest prominent pseudo-standards built on top of X was Motif, which was used as a basis for the pseudo-standard Common Desktop Environment presented by many popular UNIX machines. Motif was designed in the late '80s and shows it, but it was popular and both laid groundwork and matched the existing designs (Apple Lisa etc) to an extent that a modern user wouldn't have much trouble using a Motif system.

Motif could have remained a common standard into the modern era, and we can imagine a scenario where Linux had a more consistent look-and-feel because Motif rose to the same level of universality as X. But it didn't. There are a few reasons, but probably the biggest is that Motif was proprietary and not released as open source until well after it had fallen out of popularity. No one outside of the big UNIX vendors wanted to pay the license fees.

There were several other popular GUI toolkits built on top of X in the early days, but I won't spend time discussing them for the same reason I don't care to talk about Gnome and KDE in this post. But rest assured that there is a complex and often frustrating history of different projects coming and going, with few efforts standing the test of time.

Instead of going on about that, I want to dig a bit more into some of the less discussed implications of X's client-server nature. The client and server could exist on different machines, and because of the hardware-independence of the X protocol could also be running different operating systems, architectures, etc. This gave X a rather interesting property: you could use one computer to "look at" X software running on another computer almost regardless of what the two computers were running.

In effect, this provided one of the first forms of remote application delivery. Much like textual terminals had allowed the user to be physically removed from the computer, X allowed the machine that rendered the application and collected input to be physically removed from the actual computational resources running the software. In effect, it created a new kind of terminal: a "dumb" machine that did nothing but run an X server, with all applications running on another machine.

The terminology around this kind of thing can be confusing and is not well agreed upon, but most of the time the term "thin terminal" refers to exactly this: a machine that handles the basic mechanics of graphical output and user input but does not run any application software.

Because of the relatively high complexity of handling graphics outputs, thin terminals tend to be substantially similar to proper "computers," but have very limited local storage (usually only enough for configuration) and local processing and memory capacity that are just enough to handle the display task. They're like really low-end computers, basically, that just run the display server part of the system.

In a way that's not really very interesting, as it's conceptually very similar to the block terminals used by IBM and other mainframes. In practice, though, this GUI foray into terminals took on some very odd forms over the early history of personal computers. The terminal was decidedly obsolete, but was also the hot new thing.

Take, for example, DESQview. DESQview was a text-mode GUI for DOS that I believe I have mentioned before. After DESQview, the same developer, Quarterdeck, released DESQview/X. DESQview/X was just one of several X servers that ran on DOS. X was in many ways a poor fit for DOS, given DOS's single task nature and X's close association with larger machines running more capable operating systems, but it was really motivated by cost-savings. DOS was cheap, and running an X server on DOS allowed you to both more easily port applications written for big expensive computers, and to use a DOS machine as an X server for an application running on another machine. The cheap DOS PC became effectively a hybrid thin terminal that could both run DOS software and "connect to" software running on a more expensive system.

One way to take advantage of this functionality was reduced-cost workstations. For example, at one time years ago the middle school I attended briefly had a computer lab which consisted of workstations (passed down from another better funded middle school) with their disks removed. The machines booted via PXE into a minimal Linux environment. The user was presented with a specialized display manager that connected to a central server over the network to start a desktop environment.

The goal of this scheme was reduced cost. In practice, the system was dog slow [3] and the unfamiliarity of KDE, StarOffice, and the other common applications on the SuSE server was a major barrier to adoption. Like most K-12 schools at the time the middle school was already firmly in the grasp of Apple anyway.

Another interesting aspect of X is the way it relates to the user model. I will constrain myself here to modern Linux systems, because this situation has varied over time. What user does X run as?

On a typical Linux distribution (that still uses X), the init system starts a desktop manager as root and then launches an X server to use, still with root privileges. The terminology for desktop things can get confusing, but a desktop manager is responsible for handling logins and setting up graphical user sessions. It's now common for the display manager to actually run as its own user to contain its capabilities (by switching users after start), so it may use setuid in order to start the X server with root capabilities.

Once a user authenticates to the display manager, a common display manager behavior is to launch the user's desktop environment as the user and then hand it the information necessary to connect to the existing X instance. X runs as root the whole time.

It is completely possible to run X as a non-privileged user. The problem is that X handles all of the hardware abstraction, so it needs direct write access to hardware. For numerous reasons this is typically constrained to root.

You can imagine that the security concerns related to running X as root are significant. There is work to change this situation, such as the kernel mode setting feature, but of course there is a substantial problem of inertia: since X has always run with root privileges, a great many things assume that X has them, so there are a lot of little problems that need solving and this usage is not yet well supported.

This puts X in a pretty weird situation. It is a system service because it needs to interact directly with scarce hardware, but it's also a user application because it is tied to one user session (ultimately by means of password authentication, the so-called X cookie). This is an inherent tension that arises from the nature of graphics cards as devices that expect very low-level interaction. Unfortunately, continuously increasing performance requirements for graphical software make it very difficult to change this situation... as is, many applications use something like mesa to actually bypass the X server and talk directly to the graphics hardware instead.

I am avoiding talking about the landscape of remote applications on Windows, because that's a topic that deserves its own post. And of course X is a fertile field for technology stores, and I haven't even gotten into the odd politics of Linux's multiple historic X implementations.

[1] Windows looks and feels like a single-user operating system to the extent that I sometimes have to point out to people that NT windows releases are fully multi-user. In fact, in some ways Windows NT is "more multi-user" than Linux, since it was developed later on and the multi-user concept is more thoroughly integrated into the product. Eventually I will probably write about some impacts of these differences, but the most obvious is screen locking: on Linux, the screen is "locked" by covering it with a window that won't go away. On Windows, the screen is "locked" by detaching the user session from the local console. This is less prone to bypasses, which perennially appear in Linux screensaver implementations.

[2] The history of X, multi-user operating systems from which UNIX inherited, and network computing systems in general is closely tied to major projects at a small number of prominent American universities. These universities tended to be very ambitious in their efforts to provide a "unified environment" which lead to the development of what we might now call a "network environment," in the sense of shared resources across an institution. The fact that this whole concept came out of prominent university CS departments helps to explain why most of the major components are open source but hilariously complex and notoriously hard to support, which is why everyone today just pays for Microsoft Active Directory, including those universities.

[3] Given the project's small budget, I think the server was just under-spec'd to handle 30 sessions of Firefox at once. The irony is, of course, that as computers have sped up web browsers have as well, to the extent that running tens of user sessions of Firefox remains a formidable task today.

Note: several corrections made, mostly minor, thanks to HN users smcameron, yrro, segfaultbuser. One was a larger one: I find the startup process around X to be confusing (you hopefully see why), and indeed I described it wrong. The display manager starts first and is responsible for starting an X server to use, not the other way around (of course, when you use xinit to start X after login, it goes the way I originally said... I didn't even get into that).