Internet Consolidation: What can Standards Efforts Do?

2.1. Assessing Consolidation Risk

By default, Internet protocol designers avoid centralized designs because the Internet's very nature is incompatible with centralization. As a "large, heterogeneous collection of interconnected systems" [BCP95] the Internet is often characterised as a "network of networks". These networks relate as peers who agree to facilitate communication, rather than having a relationship of subservience to others' requirements or coercion by them. This focus on independence of action carries through the way the network is architected -- for example, in the concept of an "autonomous system".¶

However, as discussed below in Section 2.2.2, not all centralization is avoidable, and in some cases it is even desirable. With that in mind, consolidation risk on the Internet is most concerning when it is not broadly held to be necessary, when it has no checks, balances, or other mechanisms of accountability, when it selects "favorites" which are difficult (or impossible) to displace, and when it threatens to diminish the success factors that enable the Internet to thrive -- scalability to meet the demands of new users, adaptability to encompass new applications, flexibility to enable deployment of new technologies, and resilience to shocks and changes [KENDE].¶

Most often, consolidation risk is indicated when a proposal has one or more of the following damaging effects (or the potential for them):¶

• Power Imbalance: When a third party has unavoidable access to communications, the informational and positional advantages gained allow observation of behavior (the "panopticon effect") and shaping or even denial of behavior (the "chokepoint effect") [JUDGE] -- capabilities that those parties (or the states that have authority over them) can use for coercive ends [FARRELL] or even to disrupt society itself. Just as good governance of states requires separation of powers [MADISON], so too does good governance of the Internet require that power not be concentrated in one place without appropriate checks and balances.¶
• Limits on Innovation: Consolidation can preclude the possibility of "permissionless innovation" -- the ability to deploy new, unforeseen applications without requiring coordination with parties other than those you are communicating with.¶
• Constraints on Competition: The Internet and its users benefit from robust competition when applications and services are available from many providers -- especially when those users can build their own applications and services based upon interoperable standards. When a consolidated service or platform must be used because no substitutes are suitable, it effectively becomes an essential facility, which encourages abuse of power.¶
• Reduced Availability: Availability of the Internet (and applications and services built upon it) improves when there are many ways to obtain access. While service availability can benefit from the focused attention of a large consolidated provider, that provider's failure can have a disproportionate impact on availability.¶
• Monoculture: The scale available to a consolidated provider can magnify minor flaws in features to a degree that can have broad consequences. For example, a single codebase for routers elevates the impact of a bug or vulnerability; a single recommendation algorithm for content can have severe social impact. Diversity in functions' implementation leads to a more robust outcome when viewed systemically, because "progress is the outcome of a trial-and-error evolutionary process of many agents interacting freely." [ALIGIA]¶
• Self-Reinforcement: As widely noted (see, e.g., [ABRAHAMSON]), a consolidated provider's access to data allows it the opportunity to make improvements to its offerings, while denying such access to others.¶

However, these are only indicators, and each needs to be evaluated carefully on a case-by-case basis.¶

For example, it is important to distinguish consolidation risk from anticompetitive concerns (also known as "antitrust"). While there are many interactions between these concepts and making the Internet more competitive may be a motivation for avoiding centralization, only courts (and in some cases, regulators) have the authority to define a relevant market and determine that behavior is anti-competitive. Furthermore, what might be considered undesirable consolidation by the technical community might not attract competition regulation, and conversely what might attract competition regulation might not be of great concern to the technical community if other mitigations are felt to be adequate.¶

Likewise, while centralization interacts with availability, they are distinct and any relationship between them cannot be assumed without careful analysis of where and how centralization occurs. Centralized systems might be more available due to factors like the resources available to them, but also have greater impact when they encounter a fault; decentralized systems might be more resilient in the face of local failures, but less able to react to systemic issues. Furthermore, a failure due to a cut cable, power outage, or failed server is qualitatively different from the issues encountered when a core Internet function has a gatekeeper.¶

For example, a large variety of Web sites might depend upon a cloud hosting provider or content delivery network; if it were to become unavailable (whether for technical or other reasons), many people's experience of the Internet might be disrupted. Likewise, a mobile Internet access provider might have an outage that affects hundreds, thousands, or more of its users. In both cases, consolidation is not indicated by the loss of availability or its scale, but it well might be if the parties relying on the function don't have reasonable options to switch to if they are unhappy with the availability of the service provided, or if friction against switching to an alternative is too great.¶

2.2. How Centralization Occurs

A function's design can exhibit centralization in a variety of ways. The subsections below describe different contributors to and expressions of centralization in Internet functions. Note that this is not a taxonomy, in that it is not necessarily complete, and there may be overlap.¶

3.1. Decentralization Techniques

In the context of Internet standardization, decentralization is a two-step process: first assessing the nature of consolidation risk, followed by the application of techniques to reduce or mitigate it. The subsections below examine some of these techniques.¶

Choosing the appropriate techniques for decentralization requires balancing the specific goals of the function against consolidation risk, because completely precluding all forms of consolidation through technical means is rarely achievable. When performed properly, decentralization might produce an outcome that still has consolidation risk, but that risk should be understood, acceptable, and, where possible and appropriate, mitigated.¶

Notably, decentralization does not require that provision of a function need be distributed in a particular fashion, or to a particular degree. For example, the Domain Name System [RFC1035] is widely agreed to have acceptable consolidation risk, despite it being provided by a limited set of entities.¶

4.1. Bolster Legitimacy

As discussed, many efforts to address Internet consolidation are likely to take place outside of standards bodies. If the IETF wishes to contribute to these efforts to assure their compatibility with the Internet's architectural goals, it must be seen as legitimate by the relevant parties -- especially, by competition regulators. Alternatively, if the IETF is perceived as representing or being controlled by "big tech" concerns, its ability to guide decisions that affect the Internet are diminished considerably.¶

The IETF already has features that arguably provide considerable legitimacy; for example, open participation and representation by individuals rather than companies both enhance input legitimacy; a well-defined process with multiple layers of appeals and transparency contribute to throughput legitimacy, and a long history of successful Internet standards provide perhaps the strongest source of legitimacy for the IETF -- its output.¶

However, it is also widely recognized the considerable costs (in time and money) involved in successfully participating in the IETF have a tendency of favouring larger companies over smaller concerns. Likewise, the specialised and highly technical nature of the work creates barriers to entry for non-technical stakeholders. These factors have the potential to reduce the legitimacy of the IETF's decisions, at least in some eyes.¶

Efforts to address these shortcomings are ongoing; see, for example, [RFC8890]. Overall, bolstering the legitimacy of the organization should be seen as a continuous effort, with a focus on all of the aspects listed.¶

4.2. Engage with Consolidation Risk Thoroughly but Realistically

Some consolidation risks are easy to manage in standards efforts. For example, if a proprietary protocol were to be proposed to the IETF, it would be rejected out of hand. There is a growing body of knowledge and experience in managing the risk of beneficial centralization, and a strong inclination to reuse existing infrastructure where possible. As discussed above, encryption is often a way to manage inherited centralization, and has become the norm in standard protocols. These responses are appropriate ways for Internet standards to manage consolidation risk.¶

However, mitigating concentration and platform centralization is much more difficult in standards efforts. Because the IETF has no "protocol police", it's not possible to demand that someone stop building a proprietary service using a federated protocol (for example). The standards process also cannot stop someone from building services "on top" of standard protocols without abandoning architectural goals like permissionless innovation. While the imprimatur of an Internet Standard is not without value, merely withholding it cannot prevent these sources of consolidation.¶

Therefore, committing significant resources to scrutinizing protocols for latent consolidation risk -- especially for concentration and platform risks -- is unlikely to be effective in preventing Internet consolidation. Almost all existing Internet protocols -- including IP, TCP, HTTP, and DNS -- exhibit concentration or platform centralization. Refusing to standardize a newer protocol because it faces similar risks would not be equitable, proportionate, or effective.¶

When claims are made that a given proposal is "centralized" or "decentralized", the context of those statements should be examined for presuppositions, assumptions, and omissions. [BACCHI] offers one framework for such critical interrogations. [SCHNEIDERa] implores that proposals to decentralize be "really, really clear about what particular features of a system a given design seeks to decentralize" and promotes borrowing remedies from more traditional governance systems, such as separation of powers and accountability.¶

When consolidation risk is found, standards efforts should consider its relationship with other architectural goals as they consider how to address it. In particular, attention should be paid to how effective standards (as a form of architectural control) is in achieving each goal.¶

For example, privacy is often more effectively ensured by ex ante technical constraints, as compared to ex post legal regulation. Conversely (as discussed) some consolidation risks may be more effectively addressed through legal regulation. Thus, as a first order concern, a standards effort balancing these concerns might focus primarily on privacy. However, often these are not completely separable goals -- concentration can result in one or a few entities having greater volume and variety of data available exclusively to them, raising significant privacy and security concerns.¶

4.3. Decentralize Proprietary Functions

It is worthwhile to create specifications for functions that are currently only available through proprietary providers. By establishing open standards, an alternative to a consolidated function can be created.¶

Such efforts might include large-scale protocols for existing proprietary functions (e.g., chat) as well as smaller efforts to improve interoperability and portability of specific features that are often used to "lock in" users to a platform; for example, a format for lists of contacts in a social network.¶

A common objection to this approach is that adoption is voluntary, not mandatory; there are no "standards police" to mandate their use or enforce correct implementation. For example, specifications like [ACTIVITYSTREAMS]) have been available for some time without being used in a federated manner by commercial social networking providers.¶

However, while standards aren't mandatory, legal regulation is, and regulators around the globe are focusing specific efforts on some aspects of the Internet. In particular, legal mandates for interoperability are increasingly discussed as a remedy for competition issues (see, e.g., [OECD]).¶

As such, appetite for regulation presents an opportunity for new specifications to decentralize these functions, backed by a legal mandate in combination with changing norms and the resulting market forces [LESSIG]. That opportunity also presents a risk, however, if the resulting legal regulation is at odds with the Internet architecture.¶

Successfully creating standards that work in concert with legal regulation is new ground for the IETF, presents many potential pitfalls, and will require new capabilities (especially liaison, likely originating in the IAB) and considerable effort. If the Internet community does not make that effort, it is likely that regulators will turn to other sources of interoperability specifications -- most likely, with less transparency, more narrow input, limited experience, and without reference to the Internet's architectural goals.¶

4.4. Evaluate New Decentralization Techniques

The decentralization techniques listed in Section 3.1 are not a closed set; wide interest has spurred development of new approaches, both in general and as solutions to specific problems.¶

For example, secure multi-party computation techniques (see, e.g., [YAO]) can be composed to allow parties to compute over private inputs without revealing them. Protocols like [ENPA] and [PRIO] use them to limit the information available to participants in protocols to realize privacy goals; as discussed in Section 4.6 doing so might also counteract some sources of centralization. However, as in other cases these techniques do not automatically preclude all consolidation; such systems often still require trust, even if it is limited, and that might result in other forms of consolidation emerging.¶

Whether use of these techniques (or others) can meaningfully counteract consolidation is still uncertain. Standards bodies (including the IETF) can serve an important function by incubating them, applying (and, where necessary, developing) architectural guidelines for privacy, security, operability, and other goals, and assuring interoperability. When appropriate, publication on the standards track or as experimental can send signals to implementers, users, and regulators about their fitness for particular purposes.¶

4.5. Enable Switching

To minimize consolidation risk, standards-defined functions should have an explicit goal enabling users' switching between implementations and deployments of protocols.¶

One necessary condition for this is the availability of alternatives; breadth and diversity of implementation and deployment are required. Section 2.1 of [RFC5218] explores some factors in protocol design that encourage this outcome.¶

Another factor is the cost of substituting an alternative implementation or deployment by users. This implies that the standard needs to be functionally complete and specified precisely enough to allow substitution.¶

The goals of completeness and diversity are sometimes in tension. If a standard becomes extremely complex to assure easy switching, it may discourage implementation diversity because the cost of a complete implementation is too high (consider: Web browsers). On the other hand, if the specification is too simple, it may not enable easy switching, especially if proprietary extensions are necessary to complete it (see Section 4.7).¶

One objection to protocols that enable easy switching is that they reduce the incentives for implementation by commercial vendors. While a completely commoditized protocol might not allow implementations to differentiate themselves, they provide opportunities for specialization and improvement elsewhere in the value chain [CHRISTENSEN]. Well-timed standards efforts leverage these forces to focus proprietary interests on top of open technology, rather than as a replacement for it.¶

4.6. Control Delegation of Power

Some functions might see substantial benefits if they are performed by a third party in communication. When used well, adding a new party to communication can improve:¶

• Efficiency: Many functions on the Internet are more efficient when performed at a higher scale. For example, a content delivery network can offer services at a fraction of the financial and environmental cost that someone serving content themselves would otherwise pay, because of the scale they operate at. Likewise, a two-sided market platform can introduce sizeable efficiencies over pair-wise buyer/seller trading [SPULBER].¶
• Simplicity: Completely disintermediating communication can shift the burden of functions onto endpoints. This can cause increased cognitive load for users; for example, compare commercial social networking platforms with self-hosted efforts.¶
• Specialization: Having a function concentrated into a few hands can improve outcomes because of the resulting specialization. For example, services overseen by professional administrators are often seen to have a better security posture and improved availability.¶
• Privacy: For some functions, user privacy can be improved by concentrating their activity to prevent individual behaviors from being discriminated from each other.[CHAUM] Introduction of a third party can also enforce functional boundaries -- for example, to reduce the need for users to trust potentially malicious endpoints, as seen in the so-called "oblivious" protocols (e.g., [RFC9230]) that allow end users to hide their identity from services, while still accessing them.¶

However, introducing a new party to communication adds concentration and platform centralization risk to Internet functions, because it brings opportunities for control and observation. While (as discussed above) standards efforts have a very limited capability to prevent or control the resulting consolidation, designing functions with thoughtful constraints on third party functions can prevent at least the most egregious outcomes.¶

Most often, third parties are added to functions as "intermediaries" or in designated "agent" roles. In general, they should only be interposed because of the positive action of at least one of the primary parties, and should have their ability to observe or control communication limited to what is necessary to perform their intended function.¶

For example, early deployments of HTTP allowed intermediaries to be interposed by the network without knowledge of the endpoints, and those intermediaries could see and change the full content of traffic by default -- even when they are only intended to perform basic functions such as caching. Because of the introduction of HTTPS and the CONNECT method (see Section 9.3.6 of [HTTP]), combined with efforts to encourage its adoption, those intermediaries are now required to be explicitly interposed by one endpoint.¶

See [I-D.thomson-tmi] for more guidance on protocol intermediation.¶

The term "intermediary" is also used (often in legal and regulatory contexts) more broadly than it has been in protocol design; for example, an auction Web site intermediates between buyers and sellers is considered an intermediary, even though it is not formally an intermediary in HTTP (see Section 3.7 of [HTTP]). Protocol designers can address the consolidation risk associated with this kind of intermediation by standardising the function, rather than restricting the capabilities of the underlying protocols; see Section 4.3.¶

4.7. Consider Extensibility and Modularity Carefully

An important feature of the Internet is its ability to evolve, so that it can meet new requirements and adapt to new conditions without requiring a "flag day" to upgrade implementations. Typically, functions accommodate evolution by defining extension interfaces, which allow optional features to be added or change over time in an interoperable fashion.¶

However, these interfaces can also be a basis for platform centralization if a powerful entity can change the target for meaningful interoperability by adding proprietary extensions to a standard. This is especially true when the core standard does not itself provide sufficient utility on its own.¶

For example, the SOAP protocol's [SOAP] extreme flexibility and failure to provide significant standalone value allowed vendors to require use of their preferred extensions, favoring those who had more market power.¶

Therefore, standards efforts should focus on providing concrete utility to the majority of their users as published, rather than being a "framework" where interoperability is not immediately available. Internet functions should not make every aspect of their operation extensible; boundaries between modules should be designed in a way that allows evolution and discourages consolidation, while still offering meaningful functionality.¶

Beyond allowing evolution, well-considered interfaces can also aid decentralization efforts. The structural boundaries that emerge between the sub-modules of the function -- as well as those with adjacent functions -- provide touchpoints for interoperability and an opportunity for substitution of providers.¶

In particular, if the interfaces of a function are well-defined and stable, there is an opportunity to use different providers for that function. When those interfaces are open standards, change control resides with the Internet community instead of remaining in proprietary hands, further enhancing stability and enabling (but not ensuring) decentralization.¶