>>> 2020-10-18 internet to your door (PDF)

Let's talk a bit about how internet is delivered to consumers today. This will be unusually applied material for this venue, and I will be basing it on a presentation I gave a while ago, so it is not entirely original. However, it is something that impacts us in a material way that few technologists are currently very familiar with: last-mile network technology.

In the telecom industry, the term "last-mile" refers generally to the last link to the consumer premises. It may be roughly a mile a long, but as we will see it can be both longer and shorter. The "last mile" is particularly important because most, or depending on how you look at it, all internet service providers employ a "hybrid" network design in which the last mile delivery technology is different from the inner portion of the network. For example, in one of the most common cases, cable internet providers employ what they call a "hybrid fiber-coaxial" network or HFC. This concept of the network being a hybrid of the two technologies is important enough that cable devices these days often label the DOCSIS-side interface the HFC interface. In this type of network, fiber optic cable is used to connect to a "node," which then uses a DOCSIS (television cable) connection to a relatively small number of homes. This reduces the number of homes in a collision domain to allow greater bandwidth, along with other advantages such as the fiber lines being generally more reliable.

This leads us an important point of discussion: fiber to the what? There has been an ongoing trend for years of technology-centric groups wanting fiber internet service. I am unconvinced that fiber service is actually nearly as important as many people believe it to be (DOCSIS 3.1 is capable of similar or better performance compared to GPON), in reality the focus on "fiber" tends to just be a proxy for the actual demand for much higher downstream and upstream bandwidth---the delivery technology isn't really that important. The fixation on fiber has, however, provided the ISP industry an in to create uncertainty for marketing advantage by confusingly branding things as fiber. One manifestation of this is a terminology clash I call "fiber-to-the-what." These terms are increasingly used in consumer and even commercial ISP marketing and can get confusing. Here's a rough summary:

Fiber-to-the-home (FttH): fiber optic delivered to a media converter which is inside the premises of a single consumer. Generally what people mean when they say "fiber internet," and typically delivered using GPON as the technology. In most cases GPON should be considered a last-mile delivery technology and thus distinct from "fiber optics" in the sense of a telecom inside network (e.g. 10GBASE-ER), as it has many of the same disadvantages of non-fiber last-mile technologies such as DOCSIS. However, FttH virtually always means that gigabit downstream is an option, which is basically what people really want.

Fiber-to-the-premises/building (FttH/FttB): Typically applicable to multi-family housing environments, fiber optic is delivered to a central point in the structure and another technology (usually GbE) is used for delivery to individual units. Common in newer apartment buildings. The "fiber" involved may be either GPON or a "proper" full-duplex optical transit technology, for which there are numerous options.

Fiber-to-the-curb (FttC): A rare branding in the US, although cable internet using a "node+zero" architecture is basically FttC. This refers to a situation where fiber optic transport is used to connect a curbside cabinet, and then another transport technology (potentially GbE) connects a small number of homes to the cabinet.

Fiber-to-the-node (FttN): What AT&T meant when they were speciously advertising fiber internet years ago. This is the most common situation today, where a modest number of homes (up to say a couple hundred) are connected to a "node" using some other transport. The node has an optical uplink.

You will see these terms used in discussions of internet service and hopefully this explanation is helpful. As I have aimed towards, something that I would like to convey is that "fiber internet" is not nearly as important as many pro-broadband parties seem to think. Similar quality of service can often be offered by other transport technologies with a lower investment. The limiting factor is generally that cable companies are terrible, not that the delivery technology they employ is terrible.

All of that said, here is a general survey of the last-mile transport technologies currently in widespread use in the United States. Overseas the situation is often different but hard to generalize as it depends on the region---for example, fiber service seems to be far more common in Asia while very-high-speed DSL variants are more common in Europe, at least from what I have seen. I'm sure there are various odd enclaves of less common technologies throughout the world.

DSL

While the term "DSL" is widely used by consumers and providers, it's a bit nonspecific. There are actually several variants of DSL with meaningfully different capabilities. What matters, though, is that DSL refers to a family of technologies which transport data over telephone lines using frequencies above the audible range (which frequencies depends on the variant, but they generally start at around 25kHz). Unlike general landline telephony, the DSL "node" multiplexes over a large set of telephone lines, so DSL is "always connected" without any dialing involved (this is somewhat different from ISDN).

There are a few common elements of DSL technologies. Consumers will have a "DSL modem" which communicates over the telephone line with a "DSL access multiplexer" or DSLAM, which converts from DSL to another transport technology. This depends on the ISP, but most often the actual transport protocol used within DSL networks is ATM, and the DSLAM converts from ATM over DSL to ATM over ethernet. The modem handles ATM signaling so that the connection between the modem and the DSLAM---the actual DSL segment---is transparent and basically a long serial line. Ethernet frames are passed over that link, but because there is no proper addressing within the network PPPoE, or Point-to-Point Protocol over Ethernet (say that five times fast), is used to encapsulate the "payload" ethernet frames onto the DSL network. This is actually running over ATM, so we have a situation you could call PPPoEoA. Of course PPPoA exists but is not generally used with DSL, for reasons I am not familiar with but suspect are historic. This is all a long explanation of the fact that the MTU or maximum packet size on DSL connections is usually 1430, which is your standard Ethernet 1500 minus the PPPoE headers.

It is possible to directly run IP over DSL and there are providers that do this, but it is very uncommon in the United States. To add slightly more complexity, it is common for DSL providers to use VLAN tagging to segregate customer traffic from management traffic, and so DSL modems often need to be configured with both PPPoE parameters (including authentication) and a VLAN tag.

Yes, PPPoE has an authentication component. DSL networks do not generally use "low-level" authentication based on modem identities, but instead the DSLAM accepts any PPPoE traffic from modems but at a higher level internet access is denied unless PPPoE authentication is completed successfully. This means that a DSL subscriber is identified by a username and password. Most DSL providers have an autoconfiguration system in place that allows their rental modems to obtain these parameters automatically, but customers that own their own modems will often need to call support to get a username and password.

DSL providers are generally telephone companies and subject to local loop unbundling regulatory requirements, meaning that it is possible to purchase DSL internet service from someone other than your telephone provider, but if you do so you must still pay your telephone provider a monthly fee for the use of their outside plant. In practice this is rarely competitive.

This all describes the general DSL situation, but there are two fairly different DSL variants in use in the US:

[ SIDEBAR ]

An important note for those who have not picked up on it: for historical reasons, network speeds are given in bits per second rather than bytes. This has a tenuous connection to things like symbol rate and baud rate which can become quickly confusing, so bit rate tends to serve as a good common denominator across technologies. It can be annoying, though, since most other things are quoted in bytes, and so you will often need to divide network rates by eight when doing back of the envelope calculations.

[ END SIDEBAR ]

ADSL

ADSL, or Asynchronous Digital Subscriber Line, is the most common DSL service. The most recent version, ADSL2+, is capable (on paper) of full-duplex operation at 25Mbps down and 3.3Mbps up. It is possible, although not especially common, to bond two lines to double those capabilities. These speeds are rarely obtained in practice. The range of ADSL is generally limited to a few miles and achievable speeds drop off quickly with range. It is very common to see that ADSL speeds offered very clearly drop as you get further from the telephone exchange, as in small towns that may be the location of the only DSLAM. However, it is possible for providers to use various technologies to place DSLAMs "in the field" in curbside cabinets, thus reducing the range to the customer. A more robust technology such as ISDN may be used for upstream transit. There are various names for various different types of devices, but the simplest is a "loop extender" which is basically just an ADSL repeater.

Typical ADSL speed offerings range from 5 to 15Mbps depending on range from the DSLAM. Upstream is uniformly poor and less than one Mbps is common even for downstream speeds on the high end. The downstream/upstream asymmetry is designed into the standard frequency allocations. ADSL has a reputation for high latencies, which has more to do with the typical network architectures of DSL providers than the transport technology, although ADSL does have some inherent overhead.

VDSL

VDSL, Very High Speed Digital Subscriber Line, is now becoming common in urban environments in the US. VDSL, and its latest standard VDSL2, is capable of much higher bandwidths using the same telephone lines as ADSL. Up to 52Mbps downstream and 16Mbps upstream is possible on paper, and pair-bonding to double these rates is common. Use of curbside DSLAMs is also ubiquitous. As a result, common VDSL speed offerings are as high as 80Mbps downstream. The useful range of VDSL is actually shorter than ADSL, and beyond a range of one mile or so ADSL becomes a better option.

VDSL is a relatively new technology. Unfortunately, DSL providers have not generally made it clear which technology they use, although you can infer from the bandwidths advertised. CenturyLink for example is deploying VDSL in many major cities and when they do so they will begin to advertise 80Mbps service, often at a lifetime rate for extra competitive edge.

DOCSIS

The next important technology is DOCSIS. DOCSIS and DSL probably form the top two technologies in use and I suspect DOCSIS is now the leader, although DSL has a decided edge in smaller towns. DOCSIS stands for Data over Cable Service Interface Specification, and to explain it simply it functions by using the bandwidths allocated to television channels on a cable television system to transmit data. DOCSIS is very popular because it relies on infrastructure which generally already exists (although some upgrades to outside plant are required to deploy DOCSIS, such as new distribution amplifiers), and it can offer very high speeds.

DOCSIS consumers use a DOCSIS modem which communicates with a Cable Modem Termination System or CMTS. DOCSIS natively moves IP and authentication is handled within the management component of the DOCSIS protocol based on the identity (serial number) of the modem. Like DSL, modems rented from the ISP generally autoconfigure, while people who own their own modem will need to contact their ISP and provide the modem's serial number for provisioning. Some DOCSIS providers place unrecognized modems onto a captive portal network, similar to many free WiFi access points, where the user can log into their ISP account or complete some other challenge to have their modem automatically provisioned based on the source of their traffic.

The latest standard, DOCSIS 4, is capable of 10Gbps downstream and 6Gbps upstream. In practice, the limiting factor is generally the uplink at the node. DOCSIS also functions over fairly long ranges, with tens of miles generally being practical. However, as consumer bandwidth demands increase DOCSIS providers are generally hitting the limits of the upstream connection used by nodes, and to address the problem and improve reliability they are deploying more nodes. Many major DOCSIS ISPs are moving to a "node+zero" architecture, where the "plus zero" refers to the number of distribution amplifiers. The goal is for all consumers to be directly connected by a relatively short cable run to a node which serves a relatively small number of users. The node uses multi-gigabit fiber for uplink. This forms the "hybrid fiber-coaxial network" and is practically capable of providing 1Gbps or even 2Gbps symmetric service.

Unfortunately, I am not currently aware of a DOCSIS provider which actually offers symmetric Gbps service. Technical challenges related to legacy equipment make it difficult to allocate additional channels to upstream, keeping upstream limited to as low as 20Mbps. Unfortunately the "legacy equipment" involved here is set-top boxes in customer homes, which are very difficult to widely replace even besides cost.

DOCSIS provides relatively rich management capabilities as a core part of the protocol, which is why, for example, DOCSIS providers can usually remotely command the customer modem to reboot as part of troubleshooting even if it isn't ISP-owned. These protocols also allow the ISP to push firmware to the modem, and most ISPs refuse service to modems which are not running an ISP-approved firmware version. This is not entirely selfish as the nature of DOCSIS is that a malfunctioning modem could disrupt service across many users.

Further, DOCSIS ISPs often make use of a higher level management protocol called TR-069 which is based on HTTP interactions between the modem and an ISP-operated management server. TR-069 provides the ISP with much greater ability to configure and manage the modem and enables features like changing WiFi network options through the ISP's mobile app. Appreciable security concerns have been identified related to TR-069 but have been overblown in many reports. Unlike DOCSIS's integral management capabilities (which are comparatively very limited), TR-069 must be explicitly configured on the modem, there is no magical discovery of the management server. As a result, if you own your modem, TR-069 is generally not a factor.

I would assert that, from a purely technical analysis, DOCSIS is generally the best choice in urban areas. While it does have limitations compared to GPON, it is significantly less expensive to deploy (assuming existing cable television infrastructure) and can provide symmetric gigabit. Unfortunately, a set of problems including not insignificantly the immense stinginess of cable providers means that more typical DOCSIS offerings are up to gigabit downstream and 50Mbps upstream. For DOCSIS to reach its potential it is likely that the cable industry will first need to be burnt to the ground.

WISPs

An up-and-coming last-mile technology is the wireless ISP or WISPs. Although there are other options, WISP virtually always implies the use of point-to-point WiFi in the 5GHz band for last-mile delivery. Proprietary extensions or modifications of the WiFi standards are often used to improve performance and manageability, such as overlaid time-division multiplexing to allow closer positioning of antennas without interference.

While WISPs are proliferating due to the very low startup costs (less than $10k with some elbow grease), they face significant technical limitations. In practice WISPs are generally not able to offer better than 40Mbps although there are some exceptions. Weather is usually not a significant challenge but trees are and some areas may not be amenable to WISP service at all. Many WISPs are recent startups with few or no employees familiar with commercial network operations and so reliability and security are highly variable.

Less commonly, some WISPs use non-WiFi technologies. There is limited use of unlicensed low-power LTE for consumer internet service, and then a few proprietary technologies that see scattered use. There may be some potential in 11GHz and other point-to-point microwave bands for WISP use although devices to take advantage of these are fairly new to the market.

Overall, WISPs are exciting due to the flexibility and low startup costs, particularly in more sparsely populated areas, but are generally incapable of meaningfully competing with VDSL or DOCSIS providers in areas where these exist.

GPON

Fiber-to-the-home generally implies the use of a Passive Optical Network or PON, most often in the Gigabit variant or GPON. PONs use time-division multiplexing to allow multiple stations (generally one "main" and multiple "consumer") to signal bidirectionally on a single fiber optic cable. They are called "passive" because each consumer is connected to a "trunk" line using a passive splitter, which is essentially a prism. A GPON consumer has an Optical Network Terminal or ONT which communicates with an Optical Line Terminal or OLT at the service node. PON networks generally use IP natively, so the ONT and OLT are essentially just media converters.

PON networks are half-duplex at a low level, but time slots are usually allocated using a demand-based algorithm and in practice performance is very good for each consumer. Combining PON with wavelength division multiplexing can improve the situation further. The range on GPON goes up to 20km with up to 64 end users on each fiber, some variants allow more of each. Symmetric gigabit service is often offered.

GPON can offer very good service and is inexpensive compared to the fiber technologies used inside of ISP networks, but there is rarely existing infrastructure that can be used and so deploying GPON is a very expensive process. Nonetheless, for the rare ISP which has the capital to compete with the cable company and isn't, well, the cable company, GPON is generally the delivery technology of choice as it offers speeds competitive with DOCSIS without any of the overhead of legacy cable equipment.

As of recently costs for GPON equipment have become very low, but the cost of the equipment is pretty insubstantial compared to the cost of trenching or pole attachment.

Satellite

In more rural areas many people use satellite providers. In this case the consumer has a Very Small Aperture Terminal or VSAT. In modern satellite networks the VSAT is bidirectional and so both uplink and downlink move via satellite (compared to older systems in which uplink was by telephone and downlink by satellite). Satellite service typically offers up to 40mbps or so of bandwidth, but because current satellite internet technologies use geostationary satellites (which are very far away) latency is considerable, e.g. 250ms base and often quite a bit more. Of course there is promising progress in this area involving, distastefully, Elon Musk, but it is unlikely that satellite service will ever be competitive with DOCSIS or GPON in areas where they are available.

And that's the world of the internet, today! Next, let's dive into history again and talk about the cellular telephone and how it got to be the way it is. This is a very complex area where I have pretty limited knowledge of all developments since the '90s, so we will be together trying to tell our 3GPP apart from our GPRS.