Parameters for Static Macroblocks and Aspect Ratio in the RTP Payload Format for H.264 Video
draft-ietf-avt-rtp-h264-params-01

Status of this Memo

By submitting this Internet-Draft, each author represents that any applicable patent or other IPR claims of which he or she is aware have been or will be disclosed, and any of which he or she becomes aware will be disclosed, in accordance with Section 6 of BCP 79.

Internet-Drafts are working documents of the Internet Engineering Task Force (IETF), its areas, and its working groups. Note that other groups may also distribute working documents as Internet-Drafts.

Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet-Drafts as reference material or to cite them other than as “work in progress.”

The list of current Internet-Drafts can be accessed at http://www.ietf.org/ietf/1id-abstracts.txt.

The list of Internet-Draft Shadow Directories can be accessed at http://www.ietf.org/shadow.html.

This Internet-Draft will expire on August 2, 2008.

Abstract

This document updates RFC 3984. It defines new optional parameters addressing recent extensions currently supported in H.323 systems: The signalling of the maximum rate of static macroblocks a decoder is able to process. The signalling of the sample aspect ratio supported by the sender or the receiver.

Table of Contents

1.  Introduction and Overview
    1.1.  Static macroblocks
    1.2.  Sample aspect ratio
2.  Requirements notation
3.  Payload Format Parameters
    3.1.  Media Type Registration
4.  SDP Parameters
    4.1.  Mapping of the optional parameters to SDP
    4.2.  Usage with SDP offer/answer
    4.3.  Examples
5.  IANA Considerations
6.  Security Considerations
7.  Normative References
Appendix A.  Change Log
§  Authors' Addresses
§  Intellectual Property and Copyright Statements

1.  Introduction and Overview

The ITU-T H.264 [4] (International Telecommunications Union, “Advanced video coding for generic audiovisual services,” March 2005.) codec and the extended video procedures and control signals for H.300 series terminals in ITU-T H.241 [5] (International Telecommunications Union, “Extended video procedures and control signals for H.300-series terminals,” May 2006.), continue to evolve. The IETF RTP payload formats and parameters need to be updated to include important new functionalities not covered in RFC 3984 [3] (Wenger, S., Hannuksela, M., Stockhammer, T., Westerlund, M., and D. Singer, “RTP Payload Format for H.264 Video,” February 2005.). This document describes new parameters used to signal maximal number of static macroblocks per second and the sample aspect ratio of the H.264 video stream.

The new parameters are already defined in the latest version of H.241 [5] (International Telecommunications Union, “Extended video procedures and control signals for H.300-series terminals,” May 2006.). This proposal defines media type parameters for them and allows their use in SDP based systems.

1.1.  Static macroblocks

Running H.264 encode and decode operations require large amounts of video processing power. The challenge being to sustain high frame rate (e.g. 30 frames/sec) with the large frame sizes that stems from recent demand for HD resolution. In practice, a certain amount of macroblocks in the video stream, which do not change in a frame, can be defined as static and, this way, free up additional processing cycles for the handling of non-static macroblocks.

Based on a given amount of video processing resources and a given resolution, a higher number of static macroblocks enables a correspondingly higher frame rate. A new parameter MaxStaticMBPS (maximum static macroblocks per second) was introduced in H.241 to address this issue. The MaxStaticMBPS parameter is specified in Section 8.3.2.8 of H.241 [5] (International Telecommunications Union, “Extended video procedures and control signals for H.300-series terminals,” May 2006.).

1.2.  Sample aspect ratio

Sample Aspect Ratio (SAR) is the intended ratio of the horizontal distance between the columns to the vertical distance between the rows of the luma sample array in a frame. SAR is expressed as h:v, where h is horizontal width and v is vertical height (in arbitrary units of spatial distance). A sample is an individual luma picture element ("pixel") making up the complete displayed image (including both fields in the case of interlaced-scan video). For example, the sample aspect ratio for a CIF picture with a 4:3 Picture Aspect Ratio (PAR) is 12:11. For a CIF picture with a 16:9 PAR, the sample aspect ratio is 16:11.

All systems which send H.264 video should indicate the sample aspect ratio in the VUI parameters specified in Annex E of H.264 [4] (International Telecommunications Union, “Advanced video coding for generic audiovisual services,” March 2005.).

In the absence of a H.264 VUI parameter aspect_ratio_idc value in the received H.264 bitstream, and in the case of an aspect_ratio_idc value equal to 0, the sample aspect ratio may be assumed to have a value according to Table 1b in H.241 [5] (International Telecommunications Union, “Extended video procedures and control signals for H.300-series terminals,” May 2006.).

This draft enables the receiver to indicate what sample aspect ratio it can support without distortion. This may help an encoder to support the different monitors being used today, which support different PAR such as 4:3 and 16:9.

2.  Requirements notation

The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in [1] (Bradner, S., “Key words for use in RFCs to Indicate Requirement Levels,” March 1997.).

3.  Payload Format Parameters

The optional H264 media subtype parameters max-smbps, sar and esar specified below comes in addition to the list of optional H264 media subtype parameters defined in Section 8.1 of RFC 3984 [3] (Wenger, S., Hannuksela, M., Stockhammer, T., Westerlund, M., and D. Singer, “RTP Payload Format for H.264 Video,” February 2005.).

3.1.  Media Type Registration

New optional parameters:

max-smbps:

The value of max-smbps is an integer indicating the maximum static macroblock processing rate in units of static macroblocks per second, under the hypothetical assumption that all macroblocks are static macroblocks. When max-smbps is signalled the MaxMBPS value in Table A-1 of H.264 [4] (International Telecommunications Union, “Advanced video coding for generic audiovisual services,” March 2005.) should be replaced with the result of the following computation:

1. If the parameter max-mbps is signalled, set a variable MaxMacroblocksPerSecond to the value of max-mbps. Otherwise, set MaxMacroblocksPerSecond equal to the value of MaxMBPS for the level in Table A-1 of H.264 [4] (International Telecommunications Union, “Advanced video coding for generic audiovisual services,” March 2005.).
2. Set a variable P_non-static to the proportion of non-static macroblocks in picture n.
3. Set a variable P_static to the proportion of static macroblocks in picture n.
4. The value of MaxMBPS in Table A-1 of H.264 [4] (International Telecommunications Union, “Advanced video coding for generic audiovisual services,” March 2005.) should be considered by the encoder to be equal to:

                                  1
              -----------------------------------------
                      P_non-static          P_static
               ------------------------- + -----------
                MaxMacroblocksPerSecond     max-smbps
   

   The encoder should recompute this value for each picture.

The value of max-smbps MUST be greater than the value of MaxMBPS for the level given in Table A-1 of H.264 [4] (International Telecommunications Union, “Advanced video coding for generic audiovisual services,” March 2005.). Senders MAY use this knowledge to send pictures of a given size at a higher picture rate than is indicated in the signalled level.

sar:

The value of this parameter is an integer that indicates support of sample aspect ratios corresponding to H.264 aspect_ratio_idc values in the range from 1 to N inclusive, where N is the value of this parameter (see Table E-1 of H.264 [4] (International Telecommunications Union, “Advanced video coding for generic audiovisual services,” March 2005.)).

esar:

The value of this parameter is 1 or 0. 1 indicates support for all sample aspect ratios which are expressible using the H.264 aspect_ratio_idc value of 255 (Extended_SAR, see Table E-1 of H.264 [4] (International Telecommunications Union, “Advanced video coding for generic audiovisual services,” March 2005.)). If the parameter does not exist, its value is 0.

4.  SDP Parameters

4.1.  Mapping of the optional parameters to SDP

The optional parameters max-smbps, sar and esar when present, MUST be included in the "a=fmtp" line of SDP. These parameters are expressed as a media type string, in the form of a semicolon separated list of paremeter=value pairs.

4.2.  Usage with SDP offer/answer

When H.264 is offered over RTP using SDP in an offer/answer exchange [2] (Rosenberg, J. and H. Schulzrinne, “An Offer/Answer Model with Session Description Protocol (SDP),” June 2002.) for negotiation of unicast streams, the following limitations and rules applies:

The optional parameter max-smbps is treated the same way as max-mbps, when used in both the SDP offer/answer model [2] (Rosenberg, J. and H. Schulzrinne, “An Offer/Answer Model with Session Description Protocol (SDP),” June 2002.) and declarative session descriptions. Its interpretation depends on the direction attribute. When the direction attribute is sendonly, then the parameters describe the limits of the RTP packets and the NAL unit stream that the sender is capable of producing. When the direction attribute is sendrecv or recvonly, the parameters describe the limitations of what the receiver accepts. The profile-level-id parameter MUST be present in the same receiver capability description that contains this parameter.

The interpretation of the optional parameter sar depends on the direction attribute. When the direction attribute is sendonly, then it indicates the range of sample aspect ratios the stream will contain. When the direction attribute is sendrecv or recvonly, the value of this parameter indicates the range of sample ratios that the receiver is able to display without geometric (shape) distortion. The receiver shall still display pictures encoded at other sample aspect ratios (though perhaps with geometric distortion).

H.264 compliant encoders should choose to send a SAR which can be displayed without geometric distortion. However, H.264 compliant encoders may choose to send pictures using any SAR.

The optional parameter esar is a receiver capability that permits a terminal to signal additional abilities to display any decoded video expressible using the H.264 aspect_ratio_idc value of 255 (Extended_SAR, see Table E-1 of H.264 [4] (International Telecommunications Union, “Advanced video coding for generic audiovisual services,” March 2005.)). The actual extended aspect ratio is conveyed using H.264 VUI.

4.3.  Examples

TBD

5.  IANA Considerations

This draft updates RFC 3984 by adding three new optional parameters to the H264 media subtype. The parameters are specified in Section Section 3 (Payload Format Parameters) of this document.

6.  Security Considerations

No separate considerations introduced in this document. Refer to section 9 of RFC 3984 [3] (Wenger, S., Hannuksela, M., Stockhammer, T., Westerlund, M., and D. Singer, “RTP Payload Format for H.264 Video,” February 2005.).

7. Normative References

Appendix A.  Change Log

draft-kristensen-avt-rtp-h264-params-00:

- 2007-06-19: Based on static macroblocks part of draft-kristensen-avt-rtp-h264-extension-00, restructured. Added sample aspect ratio parameter from H.241.

draft-ietf-avt-rtp-h264-params-00:

- 2007-06-30: Editorial changes. Input from G. Sullivan's review considered.

- 2007-07-01: Better descriptions, tightened text. Introduced the term PAR to explicitly distuingish it from SAR.

draft-ietf-avt-rtp-h264-params-00 to -01:

- 2007-07-03: Moved requirement/acronyms sections to after Introduction.

- 2008-01-30: Mainly a keep-alive-submission. Adding missing PAR for the first example with 12:11 SAR in Section 2.2.

Authors' Addresses

Full Copyright Statement

Copyright © The IETF Trust (2008).

This document is subject to the rights, licenses and restrictions contained in BCP 78, and except as set forth therein, the authors retain all their rights.

This document and the information contained herein are provided on an “AS IS” basis and THE CONTRIBUTOR, THE ORGANIZATION HE/SHE REPRESENTS OR IS SPONSORED BY (IF ANY), THE INTERNET SOCIETY, THE IETF TRUST AND THE INTERNET ENGINEERING TASK FORCE DISCLAIM ALL WARRANTIES, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO ANY WARRANTY THAT THE USE OF THE INFORMATION HEREIN WILL NOT INFRINGE ANY RIGHTS OR ANY IMPLIED WARRANTIES OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE.

Intellectual Property

The IETF takes no position regarding the validity or scope of any Intellectual Property Rights or other rights that might be claimed to pertain to the implementation or use of the technology described in this document or the extent to which any license under such rights might or might not be available; nor does it represent that it has made any independent effort to identify any such rights. Information on the procedures with respect to rights in RFC documents can be found in BCP 78 and BCP 79.

Copies of IPR disclosures made to the IETF Secretariat and any assurances of licenses to be made available, or the result of an attempt made to obtain a general license or permission for the use of such proprietary rights by implementers or users of this specification can be obtained from the IETF on-line IPR repository at http://www.ietf.org/ipr.

The IETF invites any interested party to bring to its attention any copyrights, patents or patent applications, or other proprietary rights that may cover technology that may be required to implement this standard. Please address the information to the IETF at ietf-ipr@ietf.org.