| Internet-Draft | draft-irtf-pearg-censorship | January 2023 |
| Hall, et al. | Expires 14 July 2023 | [Page] |
This document describes technical mechanisms employed in network censorship that regimes around the world use for blocking or impairing Internet traffic. It aims to make designers, implementers, and users of Internet protocols aware of the properties exploited and mechanisms used for censoring end-user access to information. This document makes no suggestions on individual protocol considerations, and is purely informational, intended as a reference. This document is a product of the Privacy Enhancement and Assessment Research Group (PEARG) in the IRTF.¶
This Internet-Draft is submitted in full conformance with the provisions of BCP 78 and BCP 79.¶
Internet-Drafts are working documents of the Internet Engineering Task Force (IETF). Note that other groups may also distribute working documents as Internet-Drafts. The list of current Internet-Drafts is at https://datatracker.ietf.org/drafts/current/.¶
Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet-Drafts as reference material or to cite them other than as "work in progress."¶
This Internet-Draft will expire on 14 July 2023.¶
Copyright (c) 2023 IETF Trust and the persons identified as the document authors. All rights reserved.¶
This document is subject to BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents (https://trustee.ietf.org/license-info) in effect on the date of publication of this document. Please review these documents carefully, as they describe your rights and restrictions with respect to this document. Code Components extracted from this document must include Revised BSD License text as described in Section 4.e of the Trust Legal Provisions and are provided without warranty as described in the Revised BSD License.¶
Censorship is where an entity in a position of power -- such as a government, organization, or individual -- suppresses communication that it considers objectionable, harmful, sensitive, politically incorrect or inconvenient [WP-Def-2020]. Although censors that engage in censorship must do so through legal, military, or other means, this document focuses largely on technical mechanisms used to achieve network censorship.¶
This document describes technical mechanisms that censorship regimes around the world use for blocking or impairing Internet traffic. See [RFC7754] for a discussion of Internet blocking and filtering in terms of implications for Internet architecture, rather than end-user access to content and services. There is also a growing field of academic study of censorship circumvention (see the review article of [Tschantz-2016]), results from which we seek to make relevant here for protocol designers and implementers.¶
Censorship circumvention also impacts the cost of implementation of a censorship measure and we include mentions of tradeoffs in relation to such costs in conjunction with each technical method identified below.¶
This document has seen extensive discussion and review in the IRTF Privacy Enhancement and Assessment Research Group (PEARG) and represents the consensus of that group. It is not an IETF product and is not a standard.¶
We describe three elements of Internet censorship: prescription, identification, and interference. The document contains three major sections, each corresponding to one of these elements. Prescription is the process by which censors determine what types of material they should censor, e.g., classifying pornographic websites as undesirable. Identification is the process by which censors classify specific traffic or traffic identifiers to be blocked or impaired, e.g., deciding that webpages containing "sex" in an HTTP Header or that accept traffic through the URL www.sex.example are likely to be undesirable. Interference is the process by which censors intercede in communication and prevents access to censored materials by blocking access or impairing the connection, e.g., implementing a technical solution capable of identifying HTTP headers or URLs and ensuring they are rendered wholly or partially inaccessible.¶
Prescription is the process of figuring out what censors would like to block [Glanville-2008]. Generally, censors aggregate information "to block" in blocklists, databases of image hashes [ekr-2021], or use real-time heuristic assessment of content [Ding-1999]. Some national networks are designed to more naturally serve as points of control [Leyba-2019]. There are also indications that online censors use probabilistic machine learning techniques [Tang-2016]. Indeed, web crawling and machine learning techniques are an active research idea in the effort to identify content deemed as morally or commercially harmful to companies or consumers in some jurisdictions [SIDN2020].¶
There are typically a few types of blocklist elements: Keyword, domain name, protocol, or Internet Protocol (IP) address. Keyword and domain name blocking take place at the application level, e.g., HTTP; protocol blocking often occurs using deep packet inspection to identify a forbidden protocol; IP blocking tends to take place using IP addresses in IPv4/IPv6 headers. Some censors also use the presence of certain keywords to enable more aggressive blocklists [Rambert-2021] or to be more permissive with content [Knockel-2021].¶
The mechanisms for building up these blocklists vary. Censors can purchase from private industry "content control" software, which lets censors filter traffic from broad categories they would like to block, such as gambling or pornography [Knight-2005]. In these cases, these private services attempt to categorize every semi-questionable website as to allow for meta-tag blocking. Similarly, they tune real-time content heuristic systems to map their assessments onto categories of objectionable content.¶
Countries that are more interested in retaining specific political control typically have ministries or organizations that maintain blocklists. Examples include the Ministry of Industry and Information Technology in China, Ministry of Culture and Islamic Guidance in Iran, and specific to copyright in France [HADOPI-2020] and across the EU for consumer protection law [Reda-2017].¶
Content-layer filtering of images and video require institutions or organizations to store hashes of images or videos to be blocked in databases, which can then be compared, with some degree of tolerance, to content that is sent, received or stored using centralized, content applications and services [ekr-2021].¶
Internet censorship takes place in all parts of the network topology. It may be implemented in the network itself (e.g. local loop or backhaul), on the services side of communication (e.g. web hosts, cloud providers or content delivery networks), in the ancillary services eco-system (e.g. domain name system or certificate authorities) or on the end-client side (e.g. in an end-user device such as a smartphone, laptop or desktop or software executed on such devices). An important aspect of pervasive technical interception is the necessity to rely on software or hardware to intercept the content the censor is interested in. There are various logical and physical points-of-control censors may use for interception mechanisms, including, though not limited to, the following.¶
At all levels of the network hierarchy, the filtration mechanisms used to censor undesirable traffic are essentially the same: a censor either directly identifies undesirable content using the identifiers described below and then uses a blocking or shaping mechanism such as the ones exemplified below to prevent or impair access, or requests that an actor ancillary to the censor, such as a private entity, perform these functions. Identification of undesirable traffic can occur at the application, transport, or network layer of the IP stack. Censors often focus on web traffic, so the relevant protocols tend to be filtered in predictable ways (see Section 4.2.1 and Section 4.2.2). For example, a subversive image might make it past a keyword filter. However, if later the image is deemed undesirable, a censor may then blocklist the provider site's IP address.¶
The following subsections describe properties and tradeoffs of common ways in which censors filter using application-layer information. Each subsection includes empirical examples describing these common behaviors for further reference.¶
An HTTP header contains a lot of useful information for traffic identification. Although "host" is the only required field in an HTTP request header (for HTTP/1.1 and later), an HTTP method field is necessary to do anything useful. As such, "method" and "host" are the two fields used most often for ubiquitous censorship. A censor can sniff traffic and identify a specific domain name (host) and usually a page name (GET /page) as well. This identification technique is usually paired with transport header identification (see Section 4.3.1) for a more robust method.¶
Tradeoffs: Request Identification is a technically straight-forward identification method that can be easily implemented at the Backbone or ISP level. The hardware needed for this sort of identification is cheap and easy-to-acquire, making it desirable when budget and scope are a concern. HTTPS will encrypt the relevant request and response fields, so pairing with transport identification (see Section 4.3.1) is necessary for HTTPS filtering. However, some countermeasures can trivially defeat simple forms of HTTP Request Header Identification. For example, two cooperating endpoints -- an instrumented web server and client -- could encrypt or otherwise obfuscate the "host" header in a request, potentially thwarting techniques that match against "host" header values.¶
Empirical Examples: Studies exploring censorship mechanisms have found evidence of HTTP header/ URL filtering in many countries, including Bangladesh, Bahrain, China, India, Iran, Malaysia, Pakistan, Russia, Saudi Arabia, South Korea, Thailand, and Turkey [Verkamp-2012] [Nabi-2013] [Aryan-2012]. Commercial technologies are often purchased by censors [Dalek-2013]. These commercial technologies use a combination of HTTP Request Identification and Transport Header Identification to filter specific URLs. Dalek et al. and Jones et al. identified the use of these products in the wild [Dalek-2013] [Jones-2014].¶
While HTTP Request Header Identification relies on the information contained in the HTTP request from client to server, response identification uses information sent in response by the server to client to identify undesirable content.¶
Tradeoffs: As with HTTP Request Header Identification, the techniques used to identify HTTP traffic are well-known, cheap, and relatively easy to implement. However, they are made useless by HTTPS because HTTPS encrypts the response and its headers.¶
The response fields are also less helpful for identifying content than request fields, as "Server" could easily be identified using HTTP Request Header identification, and "Via" is rarely relevant. HTTP Response censorship mechanisms normally let the first n packets through while the mirrored traffic is being processed; this may allow some content through and the user may be able to detect that the censor is actively interfering with undesirable content.¶
Empirical Examples: In 2009, Jong Park et al. at the University of New Mexico demonstrated that the Great Firewall of China (GFW) has used this technique [Crandall-2010]. However, Jong Park et al. found that the GFW discontinued this practice during the course of the study. Due to the overlap in HTTP response filtering and keyword filtering (see Section 4.2.4), it is likely that most censors rely on keyword filtering over TCP streams instead of HTTP response filtering.¶
Similar to HTTP, censors have deployed a variety of techniques towards censoring Transport Layer Security (TLS) (and by extension HTTPS). Most of these techniques relate to the Server Name Indication (SNI) field, including censoring SNI, Encrypted SNI, or omitted SNI. Censors can also censor HTTPS content via server certificates. Note that TLS 1.3 acts as a security component of QUIC.¶
In encrypted connections using TLS, there may be servers that host multiple "virtual servers" at a given network address, and the client will need to specify in the Client Hello message which domain name it seeks to connect to (so that the server can respond with the appropriate TLS certificate) using the Server Name Indication (SNI) TLS extension [RFC6066]. The Client Hello message is unencrypted for TCP-based TLS. When using QUIC, the Client Hello message is encrypted but its confidentiality is not effectively protected because the initial encryption keys are derived using a value that is visible on the wire. Since SNI is often sent in the clear (as are the cert fields sent in response), censors and filtering software can use it (and response cert fields) as a basis for blocking, filtering, or impairment by dropping connections to domains that match prohibited content (e.g., bad.foo.example may be censored while good.foo.example is not) [Shbair-2015]. There are undergoing standardization efforts in the TLS Working Group to encrypt SNI [I-D.ietf-tls-sni-encryption] [I-D.ietf-tls-esni] and recent research shows promising results in the use of encrypted SNI in the face of SNI-based filtering [Chai-2019] in some countries.¶
Domain fronting has been one popular way to avoid identification by censors [Fifield-2015]. To avoid identification by censors, applications using domain fronting put a different domain name in the SNI extension than in the Host: header, which is protected by HTTPS. The visible SNI would indicate an unblocked domain, while the blocked domain remains hidden in the encrypted application header. Some encrypted messaging services relied on domain fronting to enable their provision in countries employing SNI-based filtering. These services used the cover provided by domains for which blocking at the domain level would be undesirable to hide their true domain names. However, the companies holding the most popular domains have since reconfigured their software to prevent this practice. It may be possible to achieve similar results using potential future options to encrypt SNI.¶
Tradeoffs: Some clients do not send the SNI extension (e.g., clients that only support versions of SSL and not TLS), rendering this method ineffective (see Section 4.2.3.3). In addition, this technique requires deep packet inspection (DPI) techniques that can be computationally and infrastructurally expensive, especially when applied to QUIC where DPI requires key extraction and decryption of the Client Hello in order to read the SNI. Improper configuration of an SNI-based block can result in significant overblocking, e.g., when a second-level domain like populardomain.example is inadvertently blocked. In the case of encrypted SNI, pressure to censor may transfer to other points of intervention, such as content and application providers.¶
Empirical Examples: There are many examples of security firms that offer SNI-based filtering products [Trustwave-2015] [Sophos-2015] [Shbair-2015], and the governments of China, Egypt, Iran, Qatar, South Korea, Turkey, Turkmenistan, and the UAE all do widespread SNI filtering or blocking [OONI-2018] [OONI-2019] [NA-SK-2019] [CitizenLab-2018] [Gatlan-2019] [Chai-2019] [Grover-2019] [Singh-2019]. SNI blocking against QUIC traffic has been first observed in Russia in March 2022 [Elmenhorst-2022].¶
With the data leakage present with the SNI field, a natural response is to encrypt it, which is forthcoming in TLS 1.3 with Encrypted Client Hello (ECH). Prior to ECH, the Encrypted SNI (ESNI) extension is available to prevent the data leakage caused by SNI, which encrypts only the SNI field. Unfortunately, censors can target connections that use the ESNI extension specifically for censorship. This guarantees overblocking for the censor, but can be worth the cost if ESNI is not yet widely deployed within the country. Encrypted Client Hello (ECH) is the emerging standard for protecting the entire TLS Client Hello, but it is not yet widely deployed.¶
Tradeoffs: The cost to censoring Encrypted SNI (ESNI) is significantly higher than SNI to a censor, as the censor can no longer target censorship to specific domains and guarantees over-blocking. In these cases, the censor uses the over-blocking to discourage the use of ESNI entirely.¶
Empirical Examples: In 2020, China began censoring all uses of Encrypted ESNI (ESNI) [Bock-2020b], even for innocuous connections. The censorship mechanism for China's ESNI censorship differs from how China censors SNI-based connections, suggesting that new middleboxes were deployed specifically to target ESNI connections.¶
Researchers have observed that some clients omit the SNI extension entirely. This omitted-SNI approach limits the information available to a censor. Like with ESNI, censors can choose to block connections that omit the SNI, though this too risks over-blocking.¶
Tradeoffs: The approach of censoring all connections that omit the SNI field is guaranteed to over-block, though connections that omit the SNI field should be relatively rare in the wild.¶
Empirical Examples: In the past, researchers have observed censors in Russia blocking connections that omit the SNI field [Bock-2020b].¶
During the TLS handshake after the TLS Client Hello, the server will respond with the TLS certificate. This certificate also contains the domain the client is trying to access, creating another avenue that censors can use to perform censorship. This technique will not work in TLS 1.3, as the certificate will be encrypted.¶
Tradeoffs: Censoring based on the server certificate requires DPI techniques that can be more computationally expensive compared to other methods. Additionally, the certificate is sent later in the TLS Handshake compared to the SNI field, forcing the censor to track the connection for longer.¶
Empirical Examples: Researchers have observed the Reliance Jio ISP in India using certificate response fields to censor connections [Satija-2021].¶
Many governments pressure content providers to censor themselves, or provide the legal framework within which content distributors are incentivized to follow the content restriction preferences of agents external to the content distributor [Boyle-1997]. Due to the extensive reach of such censorship, we define content distributor as any service that provides utility to users, including everything from web sites to storage to locally installed programs.¶
A commonly used method of instrumenting content distributors consists of keyword identification to detect restricted terms on their platform. Governments may provide the terms on such keyword lists. Alternatively, the content provider may be expected to come up with their own list.¶
An increasingly common method of instrumeting content distribution consists of hash matching to detect and action images and videos known to be restricted either by governments, institutions, organizations or the distributor themselves [ekr-2021].¶
A different method of instrumenting content distributors consists of requiring a distributor to disassociate with some categories of users. See also Section 6.4.¶
Tradeoffs: By instrumenting content distributors to identify restricted content or content providers, the censor can gain new information at the cost of political capital with the companies it forces or encourages to participate in censorship. For example, the censor can gain insight about the content of encrypted traffic by coercing web sites to identify restricted content. Coercing content distributors to regulate users, categories of users, content and content providers may encourage users and content providers to exhibit self-censorship, an additional advantage for censors (see Section 6.2). The tradeoffs for instrumenting content distributors are highly dependent on the content provider and the requested assistance. A typical concern is that the targeted keywords or categories of users are too broad, risk being too broadly applied, or are not subjected to a sufficiently robust legal process prior to their mandatory application (see p. 8 of [EC-2012]).¶
Empirical Examples: Researchers discovered keyword identification by content providers on platforms ranging from instant messaging applications [Senft-2013] to search engines [Rushe-2015] [Cheng-2010] [Whittaker-2013] [BBC-2013] [Condliffe-2013]. To demonstrate the prevalence of this type of keyword identification, we look to search engine censorship.¶
Search engine censorship demonstrates keyword identification by content providers and can be regional or worldwide. Implementation is occasionally voluntary, but normally it is based on laws and regulations of the country a search engine is operating in. The keyword blocklists are most likely maintained by the search engine provider. China is known to require search engine providers to "voluntarily" maintain search term blocklists to acquire and keep an Internet content provider (ICP) license [Cheng-2010]. It is clear these blocklists are maintained by each search engine provider based on the slight variations in the intercepted searches [Zhu-2011] [Whittaker-2013]. The United Kingdom has been pushing search engines to self-censor with the threat of litigation if they do not do it themselves: Google and Microsoft have agreed to block more than 100,000 queries in U.K. to help combat abuse [BBC-2013] [Condliffe-2013]. European Union law, as well as US law, requires modification of search engine results in response to either copyright, trademark, data protection or defamation concerns [EC-2012].¶
Depending on the output, search engine keyword identification may be difficult or easy to detect. In some cases specialized or blank results provide a trivial enumeration mechanism, but more subtle censorship can be difficult to detect. In February 2015, Microsoft's search engine, Bing, was accused of censoring Chinese content outside of China [Rushe-2015] because Bing returned different results for censored terms in Chinese and English. However, it is possible that censorship of the largest base of Chinese search users, China, biased Bing's results so that the more popular results in China (the uncensored results) were also more popular for Chinese speakers outside of China.¶
Disassociation by content distributors from certain categories of users has happened for instance in Spain, as a result of the conflict between the Catalunyan independence movement and the Spanish legal presumption of a unitary state [Lomas-2019]. E-sport event organizers have also disassociated themselves from top players who expressed political opinions in relation to the 2019 Hong Kong protests [Victor-2019]. See also Section 5.3.1.¶
DPI (deep packet inspection) technically is any kind of packet analysis beyond IP address and port number and has become computationally feasible as a component of censorship mechanisms in recent years [Wagner-2009]. Unlike other techniques, DPI reassembles network flows to examine the application "data" section, as opposed to only headers, and is therefore often used for keyword identification. DPI also differs from other identification technologies because it can leverage additional packet and flow characteristics, e.g., packet sizes and timings, when identifying content. To prevent substantial quality of service (QoS) impacts, DPI normally analyzes a copy of data while the original packets continue to be routed. Typically, the traffic is split using either a mirror switch or fiber splitter, and analyzed on a cluster of machines running Intrusion Detection Systems (IDS) configured for censorship.¶
Tradeoffs: DPI is one of the most expensive identification mechanisms and can have a large QoS impact [Porter-2010]. When used as a keyword filter for TCP flows, DPI systems can cause also major overblocking problems. Like other techniques, DPI is less useful against encrypted data, though DPI can leverage unencrypted elements of an encrypted data flow, e.g., the Server Name Indication (SNI) sent in the clear for TLS, or metadata about an encrypted flow, e.g., packet sizes, which differ across video and textual flows, to identify traffic. See Section 4.2.3.1 for more information about SNI-based filtration mechanisms.¶
Other kinds of information can be inferred by comparing certain unencrypted elements exchanged during TLS handshakes to similar data points from known sources. This practice, called TLS fingerprinting, allows a probabilistic identification of a party's operating system, browser, or application based on a comparison of the specific combinations of TLS version, ciphersuites, compression options, etc. sent in the ClientHello message to similar signatures found in unencrypted traffic [Husak-2016].¶
Despite these problems, DPI is the most powerful identification method and is widely used in practice. The Great Firewall of China (GFW), the largest censorship system in the world, uses DPI to identify restricted content over HTTP and DNS and inject TCP RSTs and bad DNS responses, respectively, into connections [Crandall-2010] [Clayton-2006] [Anonymous-2014].¶
Empirical Examples: Several studies have found evidence of censors using DPI for censoring content and tools. Clayton et al., Crandal et al., Anonymous, and Khattak et al., all explored the GFW [Crandall-2010] [Clayton-2006] [Anonymous-2014]. Khattak et al. even probed the firewall to discover implementation details like how much state it stores [Khattak-2013]. The Tor project claims that China, Iran, Ethiopia, and others must have used DPI to block the obfs2 protocol [Wilde-2012]. Malaysia has been accused of using targeted DPI, paired with DDoS, to identify and subsequently attack pro-opposition material [Wagstaff-2013]. It also seems likely that organizations not so worried about blocking content in real-time could use DPI to sort and categorically search gathered traffic using technologies such as high-speed packet processing [Hepting-2011].¶
Of the various shallow packet inspection methods, Transport Header Identification is the most pervasive, reliable, and predictable type of identification. Transport headers contain a few invaluable pieces of information that must be transparent for traffic to be successfully routed: destination and source IP address and port. Destination and Source IP are doubly useful, as not only does it allow a censor to block undesirable content via IP blocklisting, but also allows a censor to identify the IP of the user making the request and the IP address of the destination being visited, which in most cases can be used to infer the domain being visited [Patil-2019]. Port is useful for allowlisting certain applications.¶
Combining IP address, port and protocol information found in the transport header, shallow packet inspection can be used by a censor to identify specific TCP or UDP endpoints. UDP endpoint blocking has been observed in the context of QUIC blocking [Elmenhorst-2021].¶
Trade-offs: header identification is popular due to its simplicity, availability, and robustness.¶
Header identification is trivial to implement, but is difficult to implement in backbone or ISP routers at scale, and is therefore typically implemented with DPI. Blocklisting an IP is equivalent to installing a specific route on a router (such as a /32 route for IPv4 addresses and a /128 route for IPv6 addresses). However, due to limited flow table space, this cannot scale beyond a few thousand IPs at most. IP blocking is also relatively crude. It often leads to overblocking and cannot deal with some services like content distribution networks (CDN) that host content at hundreds or thousands of IP addresses. Despite these limitations, IP blocking is extremely effective because the user needs to proxy their traffic through another destination to circumvent this type of identification. In addition, IP blocking is effective against all protocols above IP, e.g. TCP and QUIC.¶
Port-blocking is generally not useful because many types of content share the same port and it is possible for censored applications to change their port. For example, most HTTP traffic goes over port 80, so the censor cannot differentiate between restricted and allowed web content solely on the basis of port. HTTPS goes over port 443, with similar consequences for the censor except only partial metadata may now be available to the censor. Port allowlisting is occasionally used, where a censor limits communication to approved ports, such as 80 for HTTP traffic and is most effective when used in conjunction with other identification mechanisms. For example, a censor could block the default HTTPS port, port 443, thereby forcing most users to fall back to HTTP. A counter-example is that port 25 (SMTP) has long been blocked on residential ISPs' networks to reduce the risk for email spam, but in doing so also prohibits residential ISP customers from running their own email servers.¶
Censors sometimes identify entire protocols to be blocked using a variety of traffic characteristics. For example, Iran impairs the performance of HTTPS traffic, a protocol that prevents further analysis, to encourage users to switch to HTTP, a protocol that they can analyze [Aryan-2012]. A simple protocol identification would be to recognize all TCP traffic over port 443 as HTTPS, but more sophisticated analysis of the statistical properties of payload data and flow behavior, would be more effective, even when port 443 is not used [Hjelmvik-2010] [Sandvine-2014].¶
If censors can detect circumvention tools, they can block them, so censors like China are extremely interested in identifying the protocols for censorship circumvention tools. In recent years, this has devolved into an arms race between censors and circumvention tool developers. As part of this arms race, China developed an extremely effective protocol identification technique that researchers call active probing or active scanning.¶
In active probing, the censor determines whether hosts are running a circumvention protocol by trying to initiate communication using the circumvention protocol. If the host and the censor successfully negotiate a connection, then the censor conclusively knows that host is running a circumvention tool. China has used active scanning to great effect to block Tor [Winter-2012].¶
Trade-offs: Protocol identification necessarily only provides insight into the way information is traveling, and not the information itself.¶
Protocol identification is useful for detecting and blocking circumvention tools, like Tor, or traffic that is difficult to analyze, like VoIP or SSL, because the censor can assume that this traffic should be blocked. However, this can lead to over-blocking problems when used with popular protocols. These methods are expensive, both computationally and financially, due to the use of statistical analysis, and can be ineffective due to their imprecise nature.¶
Censors have also used protocol identification in the past in an 'allowlist' filtering capacity, such as by only allowing specific, pre-vetted protocols to be used and blocking any unrecognized protocols [Bock-2020]. These protocol filtering approaches can also lead to over-blocking if the allowed lists of protocols is too small or incomplete, but can be cheap to implement, as many standard 'allowed' protocols are simple to identify (such as HTTP).¶
Empirical Examples: Protocol identification can be easy to detect if it is conducted in real time and only a particular protocol is blocked, but some types of protocol identification, like active scanning, are much more difficult to detect. Protocol identification has been used by Iran to identify and throttle SSH traffic to make it unusable [Anonymous-2007] and by China to identify and block Tor relays [Winter-2012]. Protocol identification has also been used for traffic management, such as the 2007 case where Comcast in the United States used RST injection to interrupt BitTorrent Traffic [Winter-2012]. In 2020, Iran deployed an allowlist protocol filter, which only allowed three protocols to be used (DNS, TLS, and HTTP) on specific ports and censored any connection it could not identify [Bock-2020]. In 2022, Russia seemed to have used protocol identification to block most HTTP/3 connections [Elmenhorst-2022].¶
Another feature of some modern censorship systems is residual censorship, a punitive form of censorship whereby after a censor disrupts a forbidden connection, the censor continues to target subsequent connections, even if they are innocuous [Bock-2021]. Residual censorship can take many forms and often relies on the methods of technical interference described in the next section.¶
An important facet of residual censorship is precisely what the censor continues to block after censorship is initially triggered. There are three common options available to an adversary: 2-tuple (client IP, server IP), 3-tuple (client IP, server IP+port), or 4-tuple (client IP+port, server IP+port). Future connections that match the tuple of information the censor records will be disrupted [Bock-2021].¶
Residual censorship can sometimes be difficult to identify and can often complicate censorship measurement.¶
Trade-offs: The impact of residual censorship is to provide users with further discouragement from trying to access forbidden content, though it is not clear how successful it is at accomplishing this.¶
Empirical Examples: China has used 3-tuple residual censorship in conjunction with their HTTP censorship for years and researchers have reported seeing similar residual censorship for HTTPS. China seems to use a mix of 3-tuple and 4-tuple residual censorship for their censorship of HTTPS with ESNI. Some censors that perform censorship via packet dropping often accidentally implement 4-tuple residual censorship, including Iran and Kazakhstan [Bock-2021].¶
There are a variety of mechanisms that censors can use to block or filter access to content by altering responses from the DNS [AFNIC-2013] [ICANN-SSAC-2012], including blocking the response, replying with an error message, or responding with an incorrect address. Note that there are now encrypted transports for DNS queries in DNS-over-HTTPS [RFC8484] and DNS-over-TLS [RFC7858] that can mitigate interference with DNS queries between the stub and the resolver.¶
Responding to a DNS query with an incorrect address can be achieved with on-path interception, off-path cache poisoning, and lying by the nameserver.¶
"DNS mangling" is a network-level technique of on-path interception where an incorrect IP address is returned in response to a DNS query to a censored destination. An example of this is what some Chinese networks do (we are not aware of any other wide-scale uses of mangling). On those Chinese networks, every DNS request in transit is examined (presumably by network inspection technologies such as DPI) and, if it matches a censored domain, a false response is injected. End users can see this technique in action by simply sending DNS requests to any unused IP address in China (see example below). If it is not a censored name, there will be no response. If it is censored, a forged response will be returned. For example, using the command-line dig utility to query an unused IP address in China of 192.0.2.2 for the name "www.uncensored.example" compared with "www.censored.example" (censored at the time of writing), we get a forged IP address "198.51.100.0" as a response:¶
% dig +short +nodnssec @192.0.2.2 A www.uncensored.example ;; connection timed out; no servers could be reached % dig +short +nodnssec @192.0.2.2 A www.censored.example 198.51.100.0¶
DNS cache poisoning happens off-path and refers to a mechanism where a censor interferes with the response sent by an authoritative DNS name server to a recursive resolver by responding more quickly than the authoritative name server can respond with an alternative IP address [Halley-2008]. Cache poisoning occurs after the requested site's name servers resolve the request and attempt to forward the true IP back to the requesting device; on the return route the resolved IP is recursively cached by each DNS server that initially forwarded the request. During this caching process if an undesirable keyword is recognized, the resolved IP is "poisoned" and an alternative IP (or NXDOMAIN error) is returned more quickly than the upstream resolver can respond, causing a forged IP address to be cached (and potentially recursively so). The alternative IPs usually direct to a nonsense domain or a warning page. Alternatively, Iranian censorship appears to prevent the communication en-route, preventing a response from ever being sent [Aryan-2012].¶
There are also cases of what is colloquially called "DNS lying", where a censor mandates that the DNS responses provided -- by an operator of a recursive resolver such as an Internet access provider -- be different than what authoritative name server would provide [Bortzmeyer-2015].¶
Trade-offs: These forms of DNS interference require the censor to force a user to traverse a controlled DNS hierarchy (or intervening network on which the censor serves as a Active Pervasive Attacker [RFC7624] to rewrite DNS responses) for the mechanism to be effective. It can be circumvented by using alternative DNS resolvers (such as any of the public DNS resolvers) that may fall outside of the jurisdictional control of the censor, or Virtual Private Network (VPN) technology. DNS mangling and cache poisoning also imply returning an incorrect IP to those attempting to resolve a domain name, but in some cases the destination may be technically accessible; over HTTP, for example, the user may have another method of obtaining the IP address of the desired site and may be able to access it if the site is configured to be the default server listening at this IP address. Target blocking has also been a problem, as occasionally users outside of the censors region will be directed through DNS servers or DNS-rewriting network equipment controlled by a censor, causing the request to fail. The ease of circumvention paired with the large risk of content blocking and target blocking make DNS interference a partial, difficult, and less than ideal censorship mechanism.¶
Additionally, the above mechanisms rely on DNSSEC not being deployed or DNSSEC validation not being active on the client or recursive resolver (neither of which are hard to imagine given limited deployment of DNSSEC and limited client support for DNSSEC validation). Note that an adversary seeking to merely block resolution can serve a DNSSEC record that doesn't validate correctly, assuming of course that the client/recursive resolver validates.¶
Previously, techniques were used for e.g. censorship that relied on DNS requests being passed in cleartext over port 53 [SSAC-109-2020]. With the deployment of encrypted DNS (e.g., DNS-over-HTTPS [RFC8484]) these requests are now increasingly passed on port 443 with other HTTPS traffic, or in the case of DNS-over-TLS [RFC7858] no longer passed in the clear (see also Section 4.3.1).¶
Empirical Examples: DNS interference, when properly implemented, is easy to identify based on the shortcomings identified above. Turkey relied on DNS interference for its country-wide block of websites such Twitter and YouTube for almost week in March of 2014 but the ease of circumvention resulted in an increase in the popularity of Twitter until Turkish ISPs implementing an IP blocklist to achieve the governmental mandate [Zmijewski-2014]. Ultimately, Turkish ISPs started hijacking all requests to Google and Level 3's international DNS resolvers [Zmijewski-2014]. DNS interference, when incorrectly implemented, has resulted in some of the largest "censorship disasters". In January 2014, China started directing all requests passing through the Great Fire Wall to a single domain, dongtaiwang.com, due to an improperly configured DNS poisoning attempt; this incident is thought to be the largest Internet-service outage in history [AFP-2014] [Anon-SIGCOMM12]. Countries such as China, Iran, Turkey, and the United States have discussed blocking entire TLDs as well, but only Iran has acted by blocking all Israeli (.il) domains [Albert-2011]. DNS-blocking is commonly deployed in European countries to deal with undesirable content, such as child abuse content (Norway, United Kingdom, Belgium, Denmark, Finland, France, Germany, Ireland, Italy, Malta, the Netherlands, Poland, Spain and Sweden [Wright-2013] [Eneman-2010]), online gambling (Belgium, Bulgaria, Czech Republic, Cyprus, Denmark, Estonia, France, Greece, Hungary, Italy, Latvia, Lithuania, Poland, Portugal, Romania, Slovakia, Slovenia, Spain (see Section 6.3.2 of: [EC-gambling-2012], [EC-gambling-2019])), copyright infringement (all European Economic Area countries), hate-speech and extremism (France [Hertel-2015]) and terrorism content (France [Hertel-2015]).¶
While other interference techniques outlined in this section mostly focus on blocking or preventing access to content, it can be an effective censorship strategy in some cases to not entirely block access to a given destination, or service but instead degrade the performance of the relevant network connection. The resulting user experience for a site or service under performance degradation can be so bad that users opt to use a different site, service, or method of communication, or may not engage in communication at all if there are no alternatives. Traffic shaping techniques that rate-limit the bandwidth available to certain types of traffic is one example of a performance degradation.¶
Trade offs: While implementing a performance degradation will not always eliminate the ability of people to access a desire resource, it may force them to use other means of communication where censorship (or surveillance) is more easily accomplished.¶
Empirical Examples: Iran has been known to shape the bandwidth available to HTTPS traffic to encourage unencrypted HTTP traffic [Aryan-2012].¶
Packet dropping is a simple mechanism to prevent undesirable traffic. The censor identifies undesirable traffic and chooses to not properly forward any packets it sees associated with the traversing undesirable traffic instead of following a normal routing protocol. This can be paired with any of the previously described mechanisms so long as the censor knows the user must route traffic through a controlled router.¶
Trade offs: Packet Dropping is most successful when every traversing packet has transparent information linked to undesirable content, such as a Destination IP. One downside Packet Dropping suffers from is the necessity of blocking all content from otherwise allowable IPs based on a single subversive sub-domain; blogging services and github repositories are good examples. China famously dropped all github packets for three days based on a single repository hosting undesirable content [Anonymous-2013]. The need to inspect every traversing packet in close to real time also makes Packet Dropping somewhat challenging from a QoS perspective.¶
Empirical Examples: Packet Dropping is a very common form of technical interference and lends itself to accurate detection given the unique nature of the time-out requests it leaves in its wake. The Great Firewall of China has been observed using packet dropping as one of its primary mechanisms of technical censorship [Ensafi-2013]. Iran has also used Packet Dropping as the mechanisms for throttling SSH [Aryan-2012]. These are but two examples of a ubiquitous censorship practice. Notably, packet dropping during the handshake or working connection is the only interference technique observed for QUIC traffic so far, e.g. in India, Iran, Russia and Uganda [Elmenhorst-2021][Elmenhorst-2022].¶
Packet injection, generally, refers to a man-in-the-middle (MITM) network interference technique that spoofs packets in an established traffic stream. RST packets are normally used to let one side of TCP connection know the other side has stopped sending information, and thus the receiver should close the connection. RST Packet Injection is a specific type of packet injection attack that is used to interrupt an established stream by sending RST packets to both sides of a TCP connection; as each receiver thinks the other has dropped the connection, the session is terminated.¶
QUIC is not vulnerable to these types of injection attacks once the connection has been setup. While QUIC implements a stateless reset mechanism, such a reset is only accepted by a peer if the packet ends in a previously issued stateless reset token which is hard to guess. During the handshake, QUIC only provides effective protection against off-path attackers but is vulnerable to injection attacks by attackers that have parsed prior packets. (See [I-D.ietf-quic-transport] for more details.)¶
Trade-offs: Although ineffective against non-TCP protocols (QUIC, IPSec), RST Packet Injection has a few advantages that make it extremely popular as a technique employed for censorship. RST Packet Injection is an out-of-band interference mechanism, allowing the avoidance of the the QoS bottleneck one can encounter with inline techniques such as Packet Dropping. This out-of-band property allows a censor to inspect a copy of the information, usually mirrored by an optical splitter, making it an ideal pairing for DPI and protocol identification [Weaver-2009] (this asynchronous version of a MITM is often called a Man-on-the-Side (MOTS)). RST Packet Injection also has the advantage of only requiring one of the two endpoints to accept the spoofed packet for the connection to be interrupted.¶
The difficult part of RST Packet Injection is spoofing "enough" correct information to ensure one end-point accepts a RST packet as legitimate; this generally implies a correct IP, port, and TCP sequence number. Sequence number is the hardest to get correct, as [RFC0793] specifies an RST Packet should be in-sequence to be accepted, although the RFC also recommends allowing in-window packets as "good enough". This in-window recommendation is important, as if it is implemented it allows for successful Blind RST Injection attacks [Netsec-2011]. When in-window sequencing is allowed, it is trivial to conduct a Blind RST Injection: while the term "blind" injection implies the censor doesn't know any sensitive sequencing information about the TCP stream they are injecting into, they can simply enumerate all ~70000 possible windows; this is particularly useful for interrupting encrypted/obfuscated protocols such as SSH or Tor [Gilad]. Some censorship evasion systems work by trying to confuse the censor into tracking incorrect information, rendering their RST Packet Injection useless [Khattak-2013], [Wang-2017], [Li-2017], [Bock-2019], [Wang-2020].¶
RST Packet Injection relies on a stateful network, making it useless against UDP connections. RST Packet Injection is among the most popular censorship techniques used today given its versatile nature and effectiveness against all types of TCP traffic. Recent research shows that a TCP RST packet injection attack can even work in the case of an off-path attacker [Cao-2016].¶
Empirical Examples: RST Packet Injection, as mentioned above, is most often paired with identification techniques that require splitting, such as DPI or protocol identification. In 2007, Comcast was accused of using RST Packet Injection to interrupt traffic it identified as BitTorrent [Schoen-2007], this later led to a US Federal Communications Commission ruling against Comcast [VonLohmann-2008]. China has also been known to use RST Packet Injection for censorship purposes. This interference is especially evident in the interruption of encrypted/obfuscated protocols, such as those used by Tor [Winter-2012].¶
While it is perhaps the crudest of all techniques employed for censorship, there is no more effective way of making sure undesirable information isn't allowed to propagate on the web than by shutting off the network. The network can be logically cut off in a region when a censoring body withdraws all of the Border Gateway Protocol (BGP) prefixes routing through the censor's country.¶
Trade-offs: The impact to a network disconnection in a region is huge and absolute; the censor pays for absolute control over digital information by losing the benefits a globally-accessible Internet brings. Network disconnections are also politically expensive - citizens used to being able to access Internet platforms and services see such disconnections as a loss of civil liberty. Network disconnection is rarely a long-term solution for any censor and is normally only used as a last resort in times of substantial unrest.¶
Empirical Examples: Network Disconnections tend to only happen in times of substantial unrest, largely due to the huge social, political, and economic impact such a move has. One of the first, highly covered occurrences was with the Junta in Myanmar employing Network Disconnection to help Junta forces quash a rebellion in 2007 [Dobie-2007]. China disconnected the network in the Xinjiang region during unrest in 2009 in an effort to prevent the protests from spreading to other regions [Heacock-2009]. The Arab Spring saw the the most frequent usage of Network Disconnection, with events in Egypt and Libya in 2011 [Cowie-2011], and Syria in 2012 [Thomson-2012]. Russia indicated that it would attempt to disconnect all Russian networks from the global internet in April 2019 as part of a test of the nation's network independence. Reports also indicate that, as part of the test disconnect, Russian telecommunications firms must now route all traffic to state-operated monitoring points [Cimpanu-2019]. India was the country that saw the largest number of internet shutdowns per year in 2016 and 2017 [Dada-2017].¶
More fine-grained and potentially wide-spread censorship can be achieved with BGP hijacking, which adversarially re-routes BGP IP prefixes incorrectly within a region and beyond. This restricts and effectively censors the correctly known location of information that flows into or out of a jurisdiction and will similarly prevent people from outside your jurisdiction from viewing content generated outside your jurisdiction as the adversarial route announcement propagates. The first can be achieved by an adversarial BGP announcement of incorrect routes that are not intended to leak beyond a jurisdiction, where the latter attacks traffic by deliberately introducing bogus BGP announcements that reach the global internet.¶
Trade-offs: A global leak of a misrouted website can overwhelm an ISP if the website gets a lot of traffic. It is not a permanent solution because incorrect BGP routes that leak globally can be fixed, though within a jurisdiction only the ISP/IXP is in a position to correct them for local users.¶
Empirical examples: In 2008 Pakistan Telecom censored Youtube at the request of the Pakistan government by changing its BGP routes for the website. The new routes were announced to the ISP's upstream providers and beyond. The entire Internet began directing Youtube routes to Pakistan Telecom and continued doing so for many hours. In 2018 nearly all Google services and Google cloud customers like Spotify all lost more than one hour of service after it lost control of several million of its IP addresses. Those IP prefixes were being misdirected to China Telecom, a Chinese government-owned ISP [Google-2018]}, in a manner similar to the BGP hijacking of US government and military websites by China Telecom in 2010. ISPs in both Russia (2022) and Myanmar (2021) have tried to hijack the same Twitter prefix more than once [MANRS].¶
Distributed Denial of Service attacks are a common attack mechanism used by "hacktivists" and malicious hackers, but censors have used DDoS in the past for a variety of reasons. There is a huge variety of DDoS attacks [Wikip-DoS], but at a high level two possible impacts tend to occur; a flood attack results in the service being unusable while resources are being spent to flood the service, a crash attack aims to crash the service so resources can be reallocated elsewhere without "releasing" the service.¶
Trade-offs: DDoS is an appealing mechanism when a censor would like to prevent all access to undesirable content, instead of only access in their region for a limited period of time, but this is really the only uniquely beneficial feature for DDoS as a technique employed for censorship. The resources required to carry out a successful DDoS against major targets are computationally expensive, usually requiring renting or owning a malicious distributed platform such as a botnet, and imprecise. DDoS is an incredibly crude censorship technique, and appears to largely be used as a timely, easy-to-access mechanism for blocking undesirable content for a limited period of time.¶
Empirical Examples: In 2012 the U.K.'s GCHQ used DDoS to temporarily shutdown IRC chat rooms frequented by members of Anonymous using the Syn Flood DDoS method; Syn Flood exploits the handshake used by TCP to overload the victim server with so many requests that legitimate traffic becomes slow or impossible [Schone-2014] [CERT-2000]. Dissenting opinion websites are frequently victims of DDoS around politically sensitive events in Burma [Villeneuve-2011]. Controlling parties in Russia [Kravtsova-2012], Zimbabwe [Orion-2013], and Malaysia [Muncaster-2013] have been accused of using DDoS to interrupt opposition support and access during elections. In 2015, China launched a DDoS attack using a true MITM system collocated with the Great Firewall, dubbed "Great Cannon", that was able to inject JavaScript code into web visits to a Chinese search engine that commandeered those user agents to send DDoS traffic to various sites [Marczak-2015].¶
Often, censors implement multiple techniques in tandem, creating "censorship in depth". Censorship in depth can take many forms; some censors block the same content through multiple techniques (such as blocking a domain by DNS, IP blocking, and HTTP simultaneously), some deploy parallel systems to improve censorship reliability (such as deploying multiple different censorship systems to block the same domain), and others can use complimentary systems to limit evasion (such as by blocking unwanted protocols entirely, forcing users to use other filtered protocols).¶
Trade-offs: Censorship in depth can be attractive for censors to deploy, as it offers additional guarantees about censorship: even if someone evades one type of censorship, they may still be blocked by another. The main drawback to this approach is the cost to initial deployment, as it requires the system to deploy multiple censorship systems in tandem.¶
Empirical Examples: Censorship in depth is present in many large censoring nation states today. Researchers have observed China has deployed significant censorship in depth, often censoring the same resource across multiple protocols [Chai-2019], [Bock-2020b] or deploying additional censorship systems to censor the same content and protocol [Bock-2021b]. Iran also has deployed a complimentary protocol filter to limit which protocols can be used on certain ports, forcing users to rely on protocols their censorship system can filter [Bock-2020].¶
As the name implies, sometimes manual labor is the easiest way to figure out which content to block. Manual Filtering differs from the common tactic of building up blocklists in that it doesn't necessarily target a specific IP or DNS, but instead removes or flags content. Given the imprecise nature of automatic filtering, manually sorting through content and flagging dissenting websites, blogs, articles and other media for filtration can be an effective technique on its own, or combined with other automated techniques of detection that are then followed by an action that would require manual confirmation. This filtration can occur on the Backbone/ISP level -- China's army of monitors is a good example [BBC-2013b] -- but more commonly manual filtering occurs on an institutional level. Internet Content Providers such as Google or Weibo, require a business license to operate in China. One of the prerequisites for a business license is an agreement to sign a "voluntary pledge" known as the "Public Pledge on Self-discipline for the Chinese Internet Industry". The failure to "energetically uphold" the pledged values can lead to the ICPs being held liable for the offending content by the Chinese government [BBC-2013b].¶
Self-censorship is difficult to document, as it manifests primarily through a lack of undesirable content. Tools which encourage self-censorship are those which may lead a prospective speaker to believe that speaking increases the risk of unfavourable outcomes for the speaker (technical monitoring, identification requirements, etc.). Reporters Without Borders exemplify methods of imposing self-censorship in their annual World Press Freedom Index reports [RWB2020].¶
As mentioned in passing by [Murdoch-2011], servers must have a physical location somewhere in the world. If undesirable content is hosted in the censoring country the servers can be physically seized or -- in cases where a server is virtualized in a cloud infrastructure where it may not necessarily have a fixed physical location -- the hosting provider can be required to prevent access.¶
In many countries, legal mechanisms exist where an individual or other content provider can issue a legal request to a content host that requires the host to take down content. Examples include the systems employed by companies like Google to comply with "Right to be Forgotten" policies in the European Union [Google-RTBF], intermediary liability rules for electronic platform providers [EC-2012], or the copyright-oriented notice and takedown regime of the United States Digital Millennium Copyright Act (DMCA) Section 512 [DMLP-512].¶
Domain names are catalogued in so-called name-servers operated by legal entities called registries. These registries can be made to cede control over a domain name to someone other than the entity which registered the domain name through a legal procedure grounded in either private contracts or public law. Domain name seizures is increasingly used by both public authorities and private entities to deal with undesired content dissemination [ICANN2012] [EFF2017].¶
In addition to establishing a thorough resource for describing censorship techniques this document implicates critical areas for future work.¶
Taken as a whole the apparent costs of implementation of censorship techniques indicate a need for better classification of censorship regimes as they evolve and mature and specifying censorship circumvention techniques themselves. Censors maturity refers to the technical maturity required of the censor to perform the specific censorship technique. Future work might classify techniques by essentially how hard a censor must work, including what infrastructure is required, in order to successfully censor content, users or services.¶
On circumvention, the increase in protocols leveraging encryption is an effective counter-measure against some forms of censorship described in this document, but that thorough research on circumvention and encryption be left for another document. Moreover the censorship circumvention community has developed an area of research on "pluggable transports," which collects, documents and makes agile methods for obfuscating the on-path traffic of censorship circumvention tools such that it appears indistinguishable from other kinds of traffic [Tor-2020]. Those methods would benefit from future work in the internet standards community, too.¶
Lastly the empirical examples demonstrate that censorship techniques can evolve quickly, and experience shows that this document can only be a point-in-time statement. Future work might extend this document with updates and new techniques described using a comparable methodology.¶
This document benefited from discussions with and input from David Belson, Stephane Bortzmeyer, Vinicius Fortuna, Gurshabad Grover, Andrew McConachie, Martin Nilsson, Michael Richardson, Patrick Vacek and Chris Wood.¶