| Network Working Group | R. Gerhards |
| Internet-Draft | Adiscon GmbH |
| Intended status: Informational | C. Lonvick |
| Expires: April 18, 2012 | Cisco Systems, Inc |
| October 16, 2011 |
Transmission of Syslog Messages over TCP
draft-gerhards-syslog-plain-tcp-11.txt
There have been many implementations and deployments of legacy syslog over TCP for many years. That protocol has evolved without being standardized and has proven to be quite interoperable in practice. The aim of this specification is to explain how TCP has been used as a transport for syslog messages.
This Internet-Draft is submitted in full conformance with the provisions of BCP 78 and BCP 79.
Internet-Drafts are working documents of the Internet Engineering Task Force (IETF). Note that other groups may also distribute working documents as Internet-Drafts. The list of current Internet- Drafts is at http://datatracker.ietf.org/drafts/current/.
Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet-Drafts as reference material or to cite them other than as "work in progress."
This Internet-Draft will expire on April 18, 2012.
Copyright (c) 2011 IETF Trust and the persons identified as the document authors. All rights reserved.
This document is subject to BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents (http://trustee.ietf.org/license-info) in effect on the date of publication of this document. Please review these documents carefully, as they describe your rights and restrictions with respect to this document. Code Components extracted from this document must include Simplified BSD License text as described in Section 4.e of the Trust Legal Provisions and are provided without warranty as described in the Simplified BSD License.
Historically, the syslog protocol [RFC3164] has been run over UDP. This has been replaced with the standardized syslog protocol [RFC5424] in which the TLS transport [RFC5425] is required. Even so, there are many instances of syslog running atop TCP [RFC0793].
Two primary format options have been observed with legacy syslog being transported over TCP. These have been called non-transparent-framing and octet-counting. The non-transparent-framing mechanism has some inherent problems.
Diagram 1 shows how all of these syslog transports relate to each other. In this diagram three originators are seen, labeled A, B, and C, along with one collector. Originator A is using the TCP transport which is described in this document. Originator B is using the UDP transport which is described in [RFC5426]. Originator C is using the TLS transport which is described in [RFC5425]. The collector is shown with the capability to accept all three transports.
+---------------------+
| Originator A |
|---------------------|
| syslog application |
| |
|---------------------|
| syslog transport |
| TCP |
|---------------------|
v
|
/ +---------------------+
/ | Originator B |
/ |---------------------|
/ +----------------------+ | syslog application |
/ | Collector | | |
| |----------------------| |---------------------|
| | syslog application | | syslog transport |
| | | | UDP |
| |----------------------| |---------------------|
| | syslog transport | v
| | TCP | TLS | UDP | |
| |----------------------| |
| ^ ^ ^ |
| | | | |
\ / | \ /
--------- | ------------------
|
|
| +---------------------+
| | Originator C |
| |---------------------|
| | syslog application |
| | |
| |---------------------|
| | syslog transport |
| | TLS |
| |---------------------|
| v
\ /
---------------
Diagram 1. Syslog Layers
The terminology defined in Section 3 of [RFC5424] is used throughout this specification. The reader should be familiar with that to follow this discussion.
This document also references devices that use the syslog message format as described in [RFC3164]. Devices that continue to use that message format (regardless of transport) will be described as "legacy syslog devices". Similarly, devices that use the message format as described in [RFC5424] will be described as "standardized syslog devices".
Syslog is simplex in nature. It has been observed that implementations of syslog over TCP also do not use any backchannel mechanism to convey information to the transport sender, and consequently do not use any application-level acknowledgement for syslog receiver to sender signaling. Message receipt acknowledgement, reliability, and flow control are provided by the capabilities of TCP.
For syslog over TCP messages no indication of the character set being used is given. In these messages, various character sets have been observed, with US-ASCII being predominant.
The message header is usually expected and provided in US-ASCII, only. This has been observed even in cases where a different encoding has been used for the MSG part. However, non-US-ASCII characters may be present inside the header. In that case, some syslog applications have been know to experience problems processing those messages.
In some cases, it has been observed that characters outside of the range of %d32 to %d126 (inclusive) are often being transformed by receivers in an effort to "escape control characters". Some receiver implementations simply drop those characters. This is considered to be a poor practice as it causes problems with various character sets, most notably Unicode and Asian character sets.
It has also been observed that relays will forward messages using the character sets of messages they receive. In the case where two different senders are using different character sets, the relay will forward each message to a collector in that character set. The collector of these messages will have to be prepared to receive messages from the same transport sender with different encodings.
A syslog over TCP session is a TCP connection between a syslog transport sender and a syslog transport receiver. The syslog transport sender is the TCP host that initiates the TCP session. After initiation, messages are sent from the transport sender to the transport receiver. No application-level data is transmitted from the transport receiver to the transport sender. The roles of transport sender and receiver seem to be fixed once the session is established.
If an error occurs that cannot be corrected by TCP, the host detecting the error gracefully closes the TCP session. There have been no application level messages seen that were sent to notify the other host about the state of the host syslog application.
The TCP host that intends to act as a syslog transport receiver listens to TCP port <TBD>. The TCP host that intends to act as the transport sender initiates a TCP session to the syslog transport receiver as specified in [RFC0793].
Syslog over TCP has been around for a number of years. Just like legacy syslog over UDP, different implementations exist. The older method of non-transparent-framing has problems. The newer method of octet-counting is reliable and is usually preferred.
In both of these methods, during the message transfer phase, the syslog transport sender sends a stream of messages to the transport receiver. These are sent in sequence and one message is encapsulated inside each TCP frame. Either of the TCP hosts may initiate session closure at any time as specified in Section 3.5 of [RFC0793]. In practice, this is often seen after a prolonged period of inactivity.
This framing allows for the transmission of all characters inside a syslog message and is similar to the method used in [RFC5425]. A transport receiver uses the defined message length to delimit a syslog message. As noted in [RFC3164] the upper limit for a legacy syslog message length is 1024 octets. That length has been expanded for standardized syslog.
It can be assume that octet-counting framing is used if a syslog frame starts with a digit.
TCP-DATA = *SYSLOG-FRAME
SYSLOG-FRAME = MSG-LEN SP SYSLOG-MSG ; Octet-counting
; method
MSG-LEN = NONZERO-DIGIT *DIGIT
SP = %d32
NONZERO-DIGIT = %d49-57
DIGIT = %d48 / NONZERO-DIGIT
SYSLOG-MSG is defined in the syslog protocol [RFC5424] and may
also be considered to be the payload in [RFC3164]
All syslog messages can be considered to be TCP "data" as per Transmission Control Protocol [RFC0793]. The syslog message stream has the following ABNF [RFC5234] definition:
MSG-LEN is the octet count of the SYSLOG-MSG in the SYSLOG-FRAME.
The non-transparent-framing method inserts a syslog message into a frame and terminates it with a TRAILER character. The TRAILER has usually been a single character and most often is US-ASCII LF (%d10). However, other characters have also been seen, with US-ASCII NUL (%d00) being a prominent example. Some devices have also been seen to emit a two-character TRAILER, which is usually CR and LF.
The problem with non-transparent-framing comes from the use of a TRAILER character. In that, the traditional trailer character is not escaped within the message, which causes problems for the receiver. For example, a message in the style of [RFC3164] containing one or more LF characters may be misinterpreted as multiple messages by the receiving syslog application.
TCP-DATA = *SYSLOG-FRAME
SYSLOG-FRAME = SYSLOG-MSG TRAILER ; non-transparent-framing
; method
TRAILER = LF / APP-DEFINED
LF = %d10
APP-DEFINED = 1*2OCTET
SYSLOG-MSG is defined in the syslog protocol [RFC5424] and may
also be considered to be the payload in [RFC3164]
The ABNF for this is shown here:
A transport receiver can assume that non-transparent-framing is used if a syslog frame starts with the US-ASCII character "<" (%d60).
It has been observed in legacy implementations that the framing may change on a frame-by-frame basis. This is probably not a good idea, but it's been seen.
The SYSLOG session is closed when one of the TCP hosts decides to do so. It then initiates a local TCP session closure. Following TCP [RFC0793] it doesn't need to notify the remote TCP host of its intention to close the session, nor does it accept any messages that are still in transit.
As per the standards track documents in the syslog series, it is recommended to use the TLS transport [RFC5425] to transport syslog messages. This document is provided to document what has been seen in hopes that interoperability for transporting syslog over TCP may be achieved.
There are several advantages to using TCP: flow control, error recovery, and reliability, to name a few. These reasons and the ease of programming have lead people to use this transmission protocol to transmit syslog.
One potential disadvantage is the buffering mechanism used by TCP. Ordinarily, TCP decides when enough data has been received from the application to form a segment for transmission. This may be adjusted through timers but still, some application data may wait in a buffer for a relatively long time. Syslog data is not normally time-sensitive but if this delay is a concern, the syslog transport sender may utilize the PUSH Flag as described in [RFC0793] to have the sending TCP immediately send all buffered data.
This protocol makes no meaningful provisions for security. It lacks authentication, integrity checking, and privacy. It makes no provision for flow control or end-to-end confirmation of receipt, relying instead on the underlying TCP implementations to approximate these functions. It should not be used when the alternative [RFC5425] is available.
Service Name - syslog-tcp
Transport Protocol - TCP
Assignee - IESG <iesg@ietf.org>
Contact - IETF Chair <chair@ietf.org>
Description - syslog protocol (RFC 5424) over TCP
Reference - This document
Port Number - 10514
Note to the IANA - we're making an assumption that this document needs to be compliant with Section 8.1.1. of RFC 6335. If so, then the above table is our best guess. We'd also like to use 10514/tcp for this protocol as syslog over udp is assigned 514.
The authors wish to thank David Harrington, Tom Petch, Richard Graveman, and all other people who commented on various versions of this proposal.
The authors would also like to thank Randy Presuhn for being our reviewer and document shepherd.
These are notes to the RFC editor. Please delete this section after the notes have been followed.
Please replace the instances of <TBD> with the port number assigned by IANA.
Version -11 fixed the ABNF and was submitted in October of 2011.
Version -10 was put together based on Randy Presuhn's feedback as shepherd. A section on character sets has been added. The term "octet-stuffing" was incorrectly used and has been replaced by "non-transparent-framing". The security considerations section has been simplified. It was submitted in October of 2011.
Version -09 was put together based on IESG member feedback. The appendixes were removed and things were consolidated to be more appropriate for an informational document. It was submitted in August of 2011. Dan Romascanu is actually the IESG member who will watch this document.
Version -08 included a reference to vulnerabilities of TCP. It was submitted in February of 2011.
Version -07 was submitted in January, 2011. This clarified what was really expected from what was optional. Appendix B was added for further clarification. Additionally, the security Considerations section was edited to include a discussion about transport layer issues.
Version -06 was submitted in October, 2010. The 2119 language was removed. Also, we compared notes and couldn't find any implementations that stacked multiple messages in a frame in the octet-counting method. That paragraph was removed.
Version -05 was submitted in September, 2010 to address some items that David Harrington noted as he is becoming the document shepherd.
Version -04 was submitted in April, 2010 to clean up some items.
Version -03 was submitted in April, 2010 based upon further review comments from Tom Petch.
Version -02 was submitted in March, 2010 based upon review comments from Tom Petch.
Version -01 was submitted based upon review comments from David Harrington.
Version -00 was created in November, 2009.
| [RFC0793] | Postel, J., "Transmission Control Protocol", STD 7, RFC 793, September 1981. |
| [RFC5234] | Crocker, D. and P. Overell, "Augmented BNF for Syntax Specifications: ABNF", STD 68, RFC 5234, January 2008. |
| [RFC5424] | Gerhards, R., "The Syslog Protocol", RFC 5424, March 2009. |
| [RFC5425] | Miao, F., Ma, Y. and J. Salowey, "Transport Layer Security (TLS) Transport Mapping for Syslog", RFC 5425, March 2009. |
| [RFC5426] | Okmianski, A., "Transmission of Syslog Messages over UDP", RFC 5426, March 2009. |
| [RFC3164] | Lonvick, C., "The BSD Syslog Protocol", RFC 3164, August 2001. |