As per Relevance of the word transport, we have this rfc below:
Network Working Group D.
Request for Comments: 2038 G.
Category: Standards Track Sun Microsystems, Inc
V.
Precept Software, Inc
October 1996
RTP Payload Format for MPEG1/MPEG2
Status of this
This document specifies an Internet standards track protocol for
Internet community, and requests discussion and suggestions
improvements. Please refer to the current edition of the "
Official Protocol Standards" (STD 1) for the standardization
and status of this protocol. Distribution of this memo is unlimited
This memo describes a packetization scheme for MPEG video and
streams. The scheme proposed can be used to transport such a
or audio flow over the transport protocols supported by RTP.
approaches are described. The first is designed to support
interoperability with MPEG System environments. The second
designed to provide maximum compatibility with other RTP-
media streams and future conference control work of the IETF
1.
ISO/IEC JTC1/SC29 WG11 (also referred to as the MPEG committee)
defined the MPEG1 standard (ISO/IEC 11172)[1] and the MPEG2
(ISO/IEC 13818)[2]. This memo describes a packetization scheme
transport MPEG video and audio streams using the Real-time
Protocol (RTP), version 2 [3, 4].
The MPEG1 specification is defined in three parts: System, Video
Audio. It is designed primarily for CD-ROM-based applications,
is optimized for approximately 1.5 Mbits/sec combined data rates.
video and audio portions of the specification describe the
format of the video or audio stream. These formats define
Elementary Streams (ES). The MPEG1 System specification defines
encapsulation of the ES that contains Presentation Time Stamps (PTS),
Decoding Time Stamps and System Clock references, and
multiplexing of MPEG1 compressed video and audio ES's with user data
Hoffman, et. al. Standards Track [Page 1]
RFC 2038 RTP Payload Format for MPEG1/MPEG2 Video October 1996
The MPEG2 specification is structured in a similar way. However,
hasn't been restricted only to CD-ROM applications. The MPEG2
specification defines two system stream formats: the MPEG2
Stream (MTS) and the MPEG2 Program Stream (MPS). The MTS is
for communicating or storing one or more programs of MPEG2
data and also other data in relatively error-prone environments.
MPS is tailored for relatively error-free environments
We seek to achieve interoperability among 4 types of end-systems
the following specification. The 4 types are
1. Transmitting Interworking Unit (TIU
Receives MPEG information from a native MTS system
distribution over packet networks using a native RTP-
system layer (such as an IP-based internetwork). Examples
real-time encoder, MTS satellite link to Internet,
server with MTS-encoded source material
2. Receiving Interworking Unit (RIU
Receives MPEG information in real time from an RTP-
network for forwarding to a native MTS environment
Examples: Internet-based video server to MTS-based
distribution plant
3. Transmitting Internet End-System (TAES
Transmits MPEG information generated or stored within
internet end-system itself, or received from internet-
computer networks. Example: video server
4. Receiving Internet End-System (RAES
Receives MPEG information over an RTP-based internet
consumption at the internet end-system or forwarding
traditional computer network. Example: desktop PC
workstation viewing training video
Each of the 2 types of transmitters must work with each of the 2
types of receivers. Because it is probable that the TAES,
certain that the RAES, will be based on existing and
internet-connected computers, it is highly desirable for
interoperable protocol to be based on RTP
Because of the range of applications that might employ MPEG streams
we propose to define two payload formats
Hoffman, et. al. Standards Track [Page 2]
RFC 2038 RTP Payload Format for MPEG1/MPEG2 Video October 1996
Much interest in the MPEG community is in the use of one of the
System encodings, and hence, in Section 2 we propose
of MPEG1 System streams and MPEG2 Transport and Program Streams
RTP. This profile supports the full semantics of MPEG System
offers basic interoperability among all four end-system types
When operating only among internet-based end-systems (i.e., TAES
RAES) a payload format that provides greater compatibility with
Internet architecture is desired, deferring some of the system
to other protocols being defined in the Internet community (such
the MMUSIC WG). In Section 3 we propose an encapsulation
compressed video and audio data (referred to in MPEG documentation
"Elementary Streams" (ES)) complying with either MPEG1 or MPEG2.
Here, neither of the System standards of MPEG1 or MPEG2 are utilized
The ES's are directly encapsulated with RTP
Throughout this specification, we make extensive use of
terminology. The reader should consult the primary MPEG
for definitive descriptions of this terminology
2. Encapsulation of MPEG System and Transport
Each RTP packet will contain a timestamp derived from the sender'
90KHz clock reference. This clock is synchronized to the
stream Program Clock Reference (PCR) or System Clock Reference (SCR
and represents the target transmission time of the first byte of
packet payload. The RTP timestamp will not be passed to the
decoder. This use of the timestamp is somewhat different
normally is the case in RTP, in that it is not considered to be
media display or presentation timestamp. The primary purposes of
RTP timestamp will be to estimate and reduce any network-
jitter and to synchronize relative time drift between the
and receiver
For MPEG2 Transport Streams the RTP payload will contain an
number of MPEG transport packets. To avoid end
inefficiencies, data from multiple small MTS packets (normally
in size at 188 bytes) are aggregated into a single RTP packet.
number of transport packets contained is computed by dividing
payload length by the length of an MTS packet (188).
For MPEG2 Program streams and MPEG1 system streams there are
packetization restrictions; these streams are treated as a
stream of bytes
Hoffman, et. al. Standards Track [Page 3]
RFC 2038 RTP Payload Format for MPEG1/MPEG2 Video October 1996
2.1 RTP header
The RTP header fields are used as follows
Payload Type: Distinct payload types should be assigned
of MPEG1 System Streams, MPEG2 Program Streams and MPEG
Transport Streams. See [4] for payload type assignments
M bit: Set to 1 whenever the timestamp is
(such as might happen when a sender switches from one
source to another). This allows the receiver and
intervening RTP mixers or translators that are
to the flow to ignore the difference between this
and any previous timestamp in their clock phase detectors
timestamp: 32 bit 90K Hz timestamp representing the
transmission time for the first byte of the packet
3. Encapsulation of MPEG Elementary
The following ES types may be encapsulated directly in RTP
(a) MPEG1 Video (ISO/IEC 11172-2)
(b) MPEG2 Video (ISO/IEC 13818-2)
(c) MPEG1 Audio (ISO/IEC 11172-3)
(d) MPEG2 Audio (ISO/IEC 13818-3)
A distinct RTP payload type is assigned to MPEG1/MPEG2 Video
MPEG1/MPEG2 Audio, respectively. Further indication as to whether
data is MPEG1 or MPEG2 need not be provided in the RTP or MPEG
specific headers of this encapsulation, as this information
available in the ES headers
Presentation Time Stamps (PTS) of 32 bits with an accuracy of 90
shall be carried in the fixed RTP header. All packets that make up
audio or video frame shall have the same time stamp
3.1 MPEG Video elementary
MPEG1 Video can be distinguished from MPEG2 Video at the
sequence header, i.e. for MPEG2 Video a sequence_header() is
by sequence_extension(). The particular profile and level of MPEG
Video (MAIN_Profile@MAIN_Level, HIGH_Profile@HIGH_Level, etc)
determined by the profile_and_level_indicator field of
sequence_extension header of MPEG2 Video
The MPEG bit-stream semantics were designed for relatively error-
environments, and there is significant amount of dependency (
Hoffman, et. al. Standards Track [Page 4]
RFC 2038 RTP Payload Format for MPEG1/MPEG2 Video October 1996
temporal and spatial) within the stream such that loss of some
make other uncorrupted data useless. The format as defined in
encapsulation uses application layer framing information
additional information in the RTP stream-specific header to allow
certain recovery mechanisms. Appendix 1 suggests several
strategies based on the properties of this encapsulation
Since MPEG pictures can be large, they will normally be
into packets of size less than a typical LAN/WAN MTU. The
fragmentation rules apply
1. The MPEG Video_Sequence_Header, when present, will
be at the beginning of an RTP payload
2. An MPEG GOP_header, when present, will always be at
beginning of the RTP payload, or will follow
Video_Sequence_Header
3. An MPEG Picture_Header, when present, will always be at
beginning of a RTP payload, or will follow a GOP_header
Each ES header must be completely contained within the packet
Consequently, a minimum RTP payload size of 261 bytes must
supported to contain the largest single header defined in the
(that is, the extension_data() header containing
quant_matrix_extension()). Otherwise, there are no restrictions
where headers may appear within packet payloads
In MPEG, each picture is made up of one or more "slices," and a
is intended to be the unit of recovery from data loss or corruption
An MPEG-compliant decoder will normally advance to the beginning
next slice whenever an error is encountered in the stream.
slice begin and end bits are provided in the encapsulation header
facilitate this
The beginning of a slice must either be the first data in a
(after any MPEG ES headers) or must follow after some integral
of slices in a packet. This requirement insures that the
of the next slice after one with a missing packet can be
without requiring that the receiver scan the packet contents.
may be fragmented across packets as long as all the above rules
met
An implementation based on this encapsulation assumes that
Video_Sequence_Header is repeated periodically in the MPEG bit
stream. In practice (though not required by MPEG standard) this
used to allow channel switching and to receive and start decoding
continuously relayed MPEG bit-stream at arbitrary points in the
stream. It is suggested that when playing back from an MPEG
from a file format (where the Video_Sequence_Header may only
Hoffman, et. al. Standards Track [Page 5]
RFC 2038 RTP Payload Format for MPEG1/MPEG2 Video October 1996
represented at the beginning of the stream) that the
Video_Sequence_Header (preceded by an end-of-stream indicator)
saved by the packetizer for periodic injection in to the
stream
3.2 MPEG Audio elementary
MPEG1 Audio can be distinguished from MPEG2 Audio from the
ancillary_data() header. For either MPEG1 or MPEG2 Audio,
Presentation Time Stamps may be present for frames which
to either 384 samples for Layer-I, or 1152 samples for Layer-II
Layer-III. The actual number of bytes required to represent
number of samples will vary depending on the encoder parameters
Multiple audio frames may be encapsulated within one RTP packet.
this case, an integral number of audio frames must be
within the packet and the fragmentation header defined in Section 3.5
shall be set to 0.
Also, if relatively short packets are to be used, one frame may be
large that it may straddle multiple RTP packets. For example,
Layer-II MPEG audio sampled at a rate of 44.1 KHz each frame
represent a time slot of 26.1 msec. At this sampling rate if
compressed bit-rate is 384 kbits/sec (i.e. 48 kBytes/sec) then
average audio frame size would be 1.25 KBytes. If packets were to
500 Bytes long, then each audio frame would straddle 3 RTP packets
The audio fragmentation indicator header (See Section 3.5) shall
present for an MPEG1/2 Audio payload type to provide for
fragmentation
3.3 RTP Fixed Header for MPEG ES
The RTP header fields are used as follows
Payload Type: Distinct payload types should be
for video elementary streams and audio elementary streams
See [4] for payload type assignments
M bit: For video, set to 1 on packet containing MPEG
end code, 0 otherwise. For audio, set to 1 on first
of a "talk-spurt," 0 otherwise
PT: MPEG video or audio stream ID
timestamp: 32-bit 90K Hz timestamp representing
time of MPEG picture or audio frame. Same for all
that make up a picture or audio frame. May not
monotonically increasing in video stream if B
Hoffman, et. al. Standards Track [Page 6]
RFC 2038 RTP Payload Format for MPEG1/MPEG2 Video October 1996
present in stream. For packets that contain only a
sequence and/or GOP header, the timestamp is that of
subsequent picture
3.4 MPEG Video-specific
This header shall be attached to each RTP packet after the RTP
header
0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| MBZ | TR |MBZ|S|B|E| P | | BFC | | FFC |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
FBV
MBZ: Unused. Must be set to zero in
specification. This space is reserved for future use
TR: Temporal-Reference (10 bits). The temporal reference
the current picture within the current GOP. This
ranges from 0-1023 and is constant for all RTP packets of
given picture
MBZ: Unused. Must be set to zero in
specification. This space is reserved for future use
S: Sequence-header-present (1 bit). Normally 0 and set to 1
the occurrence of each MPEG sequence header. Used
detect presence of sequence header in RTP packet
B: Beginning-of-slice (BS) (1 bit). Set when the start of
packet payload is a slice start code, or when a slice
code is preceded only by one or more of
Video_Sequence_Header, GOP_header and/or Picture_Header
E: End-of-slice (ES) (1 bit). Set when the last byte of
payload is the end of an MPEG slice
P: Picture-Type (3 bits). I (1), P (2), B (3) or D (4).
value is constant for each RTP packet of a given picture
Value 000B is forbidden and 101B - 111B are reserved
support future extensions to the MPEG ES specification
FBV: full_pel_backward_
BFC: backward_f_
FFV: full_pel_forward_
FFC: forward_f_
Hoffman, et. al. Standards Track [Page 7]
RFC 2038 RTP Payload Format for MPEG1/MPEG2 Video October 1996
Obtained from the most recent picture header, and
constant for each RTP packet of a given picture. None
these values are used for I frames and must be set to
in the RTP header. For P frames only the last two
are present and FBV and BFC must be set to zero in the
header. For B frames all the four values are present
3.5 MPEG Audio-specific
This header shall be attached to each RTP packet at the start of
payload and after any RTP headers for an MPEG1/2 Audio payload type
0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| MBZ | Frag_offset |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
Frag_offset: Byte offset into the audio frame for the
in this packet
Hoffman, et. al. Standards Track [Page 8]
RFC 2038 RTP Payload Format for MPEG1/MPEG2 Video October 1996
Appendix 1. Error Recovery and Resynchronization Strategies
The following error recovery and resynchronization strategies
intended to be guidelines only. A compliant receiver is free
employ alternative (or no) strategies
When initially decoding an RTP-encapsulated MPEG Elementary Stream
the receiver may discard all packets until the Sequence-header
present bit is set to 1. At this point, sufficient state
is contained in the stream to allow processing by an MPEG decoder
Loss of packets containing the GOP_header and/or Picture_Header
detected by an unexpected change in the Temporal-Reference
Picture-Type values. Consider the following example GOP sequence
In display order: 0B 1B 2I 3B 4B 5P 6B 7B 8P GOP_HDR 0B ...
In stream order: 2I 0B 1B 5P 3B 4B 8P 6B 7B GOP_HDR 2I ...
Consider also two counters
ref_pic_temp (Reference Picture (I,P) Temporal Reference
dep_pic_temp (Dependent Picture (B) Temporal Reference
At each GOP beginning, set these counters to the temporal
value of the corresponding picture type. For our example
sequence, ref_pic_temp = 2 and dep_pic_temp = 0. Keep
BOTH counters by unity with each following picture. Ref_pic_
should match the temporal references of the I and P frames,
dep_pic_temp should match the temporal references of the B frames
dep_pic_temp: - 0 1 2 3 4 5 6 7 8 9
In stream order: 2I 0B 1B 5P 3B 4B 8P 6B 7B GOP_H 2I 0B 1B ...
ref_pic_temp: 2 3 4 5 6 7 8 9 10 ^ 11
-------------------------- | ^
Match Drop |
in ref_pic_
The loss of a GOP header can be detected by matching the
counter (based on picture type) to the temporal reference value.
mismatch indicates a lost GOP header. If desired, a GOP header can
re-constructed using a "null" time_code, repeating the closed_
flag from previous GOP headers, and setting the broken_link flag
1.
The loss of a Picture_Header can also be detected by a mismatch
the Temporal Reference contained in the RTP packet from
appropriate dep_pic_temp or ref_pic_temp counters at the receiver
Hoffman, et. al. Standards Track [Page 9]
RFC 2038 RTP Payload Format for MPEG1/MPEG2 Video October 1996
After scanning to the next Beginning-of-slice the Picture_Header
reconstructed from the P, TR, FBV, BFC, FFV and FFC contained in
packet, and from stream-dependent default values
Any time an RTP packet is lost (as indicated by a gap in the
sequence number), the receiver may discard all packets until
Beginning-of-slice bit is set. At this point, sufficient
information is contained in the stream to allow processing by an
decoder starting at the next slice boundary (possibly
reconstruction of the GOP_header and/or Picture_Header as
above).
[1] ISO/IEC International Standard 11172; "Coding of moving
and associated audio for digital storage media up to about 1,5
Mbits/s", November 1993.
[2] ISO/IEC International Standard 13818; "Generic coding of
pictures and associated audio information", November 1994.
[3] H. Schulzrinne, S. Casner, R. Frederick, V. Jacobson
"RTP: A Transport Protocol for Real-Time Applications",
RFC 1889, January 1996.
[4] H. Schulzrinne, "RTP Profile for Audio and Video
with Minimal Control", RFC 1890, January 1996.
Hoffman, et. al. Standards Track [Page 10]
RFC 2038 RTP Payload Format for MPEG1/MPEG2 Video October 1996
Authors'
Gerard
Sun Microsystems, Inc
Mail-stop UMPK14-305
2550 Garcia
Mountain View, California 94043-1100
Phone: +1 415-786-6373
EMail: gerard.fernando@eng.sun.
Vivek
Precept Software, Inc
1072 Arastradero Rd
Palo Alto, CA 94304
Phone: +1 415-845-5200
EMail: goyal@precept.
Don
Sun Microsystems, Inc
Mail-stop UMPK14-305
2550 Garcia
Mountain View, California 94043-1100
Phone: +1 503-297-1580
EMail: don.hoffman@eng.sun.
Hoffman, et. al. Standards Track [Page 11]
if you see any problems within the linking, don't worry be happy,
this is version 0.1 of the Relevance System and you gotta expect some crappy subroutines sometimes,
just be content we did not write this in Java, which would have made this "bigger and better" HAHAHHA.
RFC documents can be found at I.E.T.F.
Relevance System Copyright © 2002 Spectrum WorldResearch
other technical nosh by ServerMasters Corporation
collaboration of BobX
