As per Relevance of the word different, we have this rfc below:
Network Working Group K.
Request for Comments: 2638 V.
Category: Informational
L.
July 1999
A Two-bit Differentiated Services Architecture for the
Status of this
This memo provides information for the Internet community. It
not specify an Internet standard of any kind. Distribution of
memo is unlimited
Copyright
Copyright (C) The Internet Society (1999). All Rights Reserved
This document was originally submitted as an internet draft
November of 1997. As one of the documents predating the formation
the IETF's Differentiated Services Working Group, many of the
presented here, in concert with Dave Clark's subsequent
to the December 1997 meeting of the IETF Integrated Services
Group, were key to the work which led to RFCs 2474 and 2475 and
section on allocation remains a timely proposal. For this reason,
to provide a reference, it is being submitted in its original form
The forwarding path portion of this document is intended as a
of where we were at in late 1997 and not as an indication of
direction
The postscript version of this document includes Clark's slides as
appendix. The postscript version of this document also includes
figures that aid greatly in its readability
1.
This document presents a differentiated services architecture for
internet. Dave Clark and Van Jacobson each presented work
differentiated services at the Munich IETF meeting [2,3].
explained how to use one bit of the IP header to deliver a new
of service to packets in the internet. These were two very
kinds of service with quite different policy assumptions.
discussion has convinced us that both service types have merit
that both service types can be implemented with a set of very
Nichols, et al. Informational [Page 1]
RFC 2638 Two-bit Differentiated Services Architecture July 1999
mechanisms. We propose an architectural framework that permits
use of both of these service types and exploits their similarities
forwarding path mechanisms. The major goals of this architecture
each shared with one or both of those two proposals: keep
forwarding path simple, push complexity to the edges of the
to the extent possible, provide a service that avoids
about the type of traffic using it, employ an allocation policy
will be compatible with both long-term and short-term provisioning
make it possible for the dominant Internet traffic model to
best-effort
The major contributions of this document are to present two
service types, a set of general mechanisms for the forwarding
that can be used to implement a range of differentiated services
to propose a flexible framework for provisioning a
services network. It is precisely this kind of architecture that
needed for expedient deployment of differentiated services: we need
framework and set of primitives that can be implemented in
short-term and provide interoperable services, yet can provide
"sandbox" for experimentation and elaboration that can lead in
to more levels of differentiation within each service as needed
At the risk of belaboring an analogy, we are motivated to
services tiers in somewhat the same fashion as the airlines do
first class, business class and coach class. The latter also
tiering built in due to the various restrictions put on the purchase
A part of the analogy we want to stress is that best effort traffic
like coach class seats on an airplane, is still expected to make
the bulk of internet traffic. Business and first class carry a
number of passengers, but are quite important to the economics of
airline industry. The various economic forces and realities
to dictate the relative allocation of the seats and to try to
the airplane. We don't expect that differentiated services
comprise all the traffic on the internet, but we do expect that
services will lead to a healthy economic and service environment
This document is organized into sections describing
architecture, mechanisms, the bandwidth allocation architecture,
this architecture might interoperate with RSVP/int-serv work,
gives recommendations for deployment
Nichols, et al. Informational [Page 2]
RFC 2638 Two-bit Differentiated Services Architecture July 1999
2.
2.1
The current internet delivers one type of service, best-effort,
all traffic. A number of proposals have been made concerning
addition of enhanced services to the Internet. We focus on
particular methods of adding a differentiated level of service to IP
each designated by one bit [1,2,3]. These services represent
radical departure from the Internet's traditional service, but
are also a radical departure from traditional "quality of service
architectures which rely on circuit-based models. Both
proposals seek to define a single common mechanism that is used
interior network routers, pushing most of the complexity and state
differentiated services to the network edges. Both use bandwidth
the resource that is being requested and allocated. Clark
Wroclawski defined an "Assured" service that follows "
capacity" usage profiles that are statistically provisioned [3].
assurance that the user of such a service receives is that
traffic is unlikely to be dropped as long as it stays within
expected capacity profile. The exact meaning of "unlikely" depends
how well provisioned the service is. An Assured service traffic
may exceed its Profile, but the excess traffic is not given the
assurance level. Jacobson defined a "Premium" service that
provisioned according to peak capacity Profiles that are strictly
oversubscribed and that is given its own high-priority queue
routers [2]. A Premium service traffic flow is shaped and hard
limited to its provisioned peak rate and shaped so that bursts
not injected into the network. Premium service presents a "
wire" where a flow's bursts may queue at the shaper at the edge
the network, but thereafter only in proportion to the indegree
each router. Despite their many similarities, these two
result in fundamentally different services. The former uses
management to provide a "better effort" service while the
creates a service with little jitter and queueing delay and no
for queue management on the Premium packets's queue
An Assured service was introduced in [3] by Clark and Wroclawski
though we have made some alterations in its specification for
architecture. Further refinements and an "Expected Capacity
framework are given in Clark and Fang [10]. This framework
focused on "providing different levels of best-effort service
times of network congestion" but also mentions that it is possible
have a separate router queue to implement a "guaranteed" level
assurance. We believe this framework and our Two-bit
are compatible but this needs further exploration. As
service has not been documented elsewhere, we describe it next
follow this with a description of the two-bit architecture
Nichols, et al. Informational [Page 3]
RFC 2638 Two-bit Differentiated Services Architecture July 1999
2.2 Premium
In [2], a Premium service was presented that is
different from the Internet's current best effort service.
service is not meant to replace best effort but primarily to meet
emerging demand for a commercial service that can share the
with best effort traffic. This is desirable economically, since
same network can be used for both kinds of traffic. It is
that Premium traffic would be allocated a small percentage of
total network capacity, but that it would be priced much higher.
use of such a service might be to create "virtual leased lines",
saving the cost of building and maintaining a separate network
Premium service, not unlike a standard telephone line, is a
which the customer expects to be there when the receiver is lifted
although it may, depending on the household, be idle a good deal
the time. Provisioning Premium traffic in this way reduces
capacity of the best effort internet by the amount of
allocated, in the worst case, thus it would have to be
accordingly. On the other hand, whenever that capacity is not
used it is available to best effort traffic. In contrast to
best effort traffic which is bursty and requires queue management
deal fairly with congestive episodes, this Premium service by
creates very regular traffic patterns and small or
queues
Premium service levels are specified as a desired peak bit-rate for
specific flow (or aggregation of flows). The user contract with
network is not to exceed the peak rate. The network contract is
the contracted bandwidth will be available when traffic is sent
First-hop routers (or other edge devices) filter the packets
the network, set the Premium bit of those that match a
service specification, and perform traffic shaping on the flow
smooths all traffic bursts before they enter the network.
approach requires no changes in hosts. A compliant router along
path needs two levels of priority queueing, sending all packets
the Premium bit set first. Best-effort traffic is unmarked and
and sent at the lower priority. This results in two "
networks": one which is identical to today's Internet with
designed to absorb traffic bursts; and one where traffic is
and shaped to a contracted peak-rate, but packets move through
network of queues where they experience almost no queueing delay
In this architecture, forwarding path decisions are made
and more simply than the setting up of the service agreements
traffic profiles. With the exception of policing and shaping
administrative or "trust" boundaries, the only actions that need
be handled in the forwarding path are to classify a packet into
of two queues on a single bit and to service the two queues
Nichols, et al. Informational [Page 4]
RFC 2638 Two-bit Differentiated Services Architecture July 1999
simple priority. Shaping must include both rate and burst parameters
the latter is expected to be small, in the one or two packet range
Policing at boundaries enforces rate compliance, and may
implemented by a simple token bucket. The admission and set-
procedures are expected to evolve, in time, to be
configurable and fairly complex while the mechanisms in
forwarding path remain simple
A Premium service built on this architecture can be deployed in
useful way once the forwarding path mechanisms are in place by
static allocations. Traffic flows can be designated for
treatment through network management configuration. Traffic
should be designated by the source, the destination, or
combination of fields in the packet header. First-hop (of leaf
routers will filter flows on all or part of the header
consisting of the source IP address, destination IP address,
identifier, source port number, and destination port number. Based
this classification, a first-hop router performs traffic shaping
sets the designated Premium bit of the precedence field. End-
are thus not required to be "differentiated services aware",
if and when end-systems become universally "aware", they might
their own shaping and first-hop routers merely police
Adherence to the subscribed rate and burst size must be enforced
the entry to the network, either by the end-system or by the first
hop router. Within an intranet, administrative domain, or "
region" the packets can then be classified and serviced solely on
Premium bit. Where packets cross a boundary, the policing function
critical. The entered region will check the prioritized packet
for conformance to a rate the two regions have agreed upon
discarding packets that exceed the rate. It is thus in the
interests of a region to ensure conformance to the agreed-upon
at the egress. This requirement means that Premium traffic is burst
free and, together with the no oversubscription rule, leads
to the observation that Premium queues can easily be sized to
the need to drop packets and thus the need for a queue
policy. At each router, the largest queue size is related to the in
degree of other routers and is thus quite small, on the order of
packets
Premium bandwidth allocations must not be oversubscribed as
represent a commitment by the network and should be
accordingly. Note that, in this architecture, Premium traffic
also experience considerably less delay variation than either
effort traffic or the Assured data traffic of [3]. Premium
might be configured on a subscription basis in the near-term, or on
demand when dynamic set-up or signaling is available
Nichols, et al. Informational [Page 5]
RFC 2638 Two-bit Differentiated Services Architecture July 1999
Figure 1 shows how a Premium packet flow is established within
particular administrative domain, Company A, and sent across
access link to Company A's ISP. Assume that the host's first-
router has been configured to match a flow from the host's IP
to a destination IP address that is reached through ISP. A
flow is configured from a host with a rate which is both smaller
the total Premium allocation Company A has from the ISP, r bytes
second, and smaller than the amount of that allocation has
assigned to other hosts in Company A. Packets are not marked in
special way when they leave the host. The first-hop router clears
Premium bit on all arriving packets, sets the Premium bit on
packets in the designated flow, shapes packets in the Premium flow
a configured rate and burst size, queues best-effort unmarked
in the low priority queue and shaped Premium packets in the
priority queue, and sends packets from those two queues at
priority. Intermediate routers internal to Company A enqueue
in one of two output queues based on the Premium bit and service
queues with simple priority. Border routers perform quite
tasks, depending on whether they are processing an egress flow or
ingress flow. An egress border router may perform some reshaping
the aggregate Premium traffic to conform to rate r, depending on
number of Premium flows aggregated. Ingress border routers only
to perform a simple policing function that can be implemented with
token bucket. In the example, the ISP accepts all Premium
from A as long as the flow does not exceed r bytes per second
Figure 1. Premium traffic flow from end-host to organization's
2.3 Two-bit differentiated services
Clark's and Jacobson's proposals are markedly similar in the
and type of functional blocks that are needed to implement them
Furthermore, they implement quite different services which are
incompatible in a network. The Premium service implements
guaranteed peak bandwidth service with negligible queueing delay
cannot starve best effort traffic and can be allocated in a
straightforward fashion. This service would seem to have a
appeal for commercial applications, video broadcasts, voice-over-IP
and VPNs. On the other hand, this service may prove both
restrictive (in its hard limits) and overdesigned (no overallocation
for some applications. The Assured service implements a service
has the same delay characteristics as (undropped) best effort
and the firmness of its guarantee depends on how well
links are provisioned for bursts of Assured packets. On the
hand, it permits traffic flows to use any additional
capacity without penalty and occasional dropped packets for
congestive periods may be acceptable to many users. This
might be what an ISP would provide to individual customers who
Nichols, et al. Informational [Page 6]
RFC 2638 Two-bit Differentiated Services Architecture July 1999
willing to pay a bit more for internet service that seems
by congestive periods. Both services are only as good as
admission control schemes, though this can be more difficult
traffic which is not peak-rate allocated
There may be some additional benefits of deploying both services.
the extent that Premium service is a conservative allocation
resources, unused bandwidth that had been allocated to Premium
provide some "headroom" for underallocated or burst periods
Assured traffic or for best effort. Network elements that deploy
services will be performing RED queue management on all non-
traffic, as suggested in [4], and the effects of mixing the
streams with best effort might serve to reduce burstiness in
latter. A strength of the Assured service is that it allows bursts
happen in their natural fashion, but this also makes
provisioning, admission control and allocation problem more
so it may take more time and experimentation before this
policy for this service is completely defined. A Premium
could be deployed that employs static allocations on peak rates
no statistical sharing
As there appear to be a number of advantages to an architecture
permits these two types of service and because, as we shall see,
can be made to share many of the same mechanisms, we
designating two bit-patterns from the IP header precedence field.
leave the explicit designation of these bit-patterns to the
process thus we use the shorthand notation of denoting each
by a bit, one we will call the Premium or P-bit, the other we
the assurance or A-bit. It is possible for a network to
only one of these services and to have network elements that
look at the one applicable bit, but we focus on the two
architecture. Further, we assume the case where no changes are
in the hosts, appropriate packet marking all being done in
network, at the first-hop, or leaf, router. We describe
forwarding path architecture in this section, assuming that
service has been allocated through mechanisms we will discuss
section 4.
In a more general sense, Premium service denotes packets that
enqueued at a higher priority than the ordinary best-effort queue
Similarly, Assured service denotes packets that are
preferentially with respect to the dropping probability within
"normal" queue. There are a number of ways to add more service
within each of these service types [7], but this document takes
position of specifying the base-level services of Premium
Assured
Nichols, et al. Informational [Page 7]
RFC 2638 Two-bit Differentiated Services Architecture July 1999
The forwarding path mechanisms can be broken down into those
happen at the input interface, before packet forwarding, and
that happen at the output interface, after packet forwarding
Intermediate routers only need to implement the post
forwarding functions, while leaf and border routers must
functions on arriving packets before forwarding. We describe
mechanisms this way for illustration; other ways of composing
functions are possible
Leaf routers are configured with a traffic profile for a
flow based on its packet header. This functionality has been
by the RSVP Working Group in RFC 2205. Figure 2 shows what happens
a packet that arrives at the leaf router, before it is passed to
forwarding engine. All arriving packets must have both the A-bit
the P-bit cleared after which packets are classified on their header
If the header does not match any configured values, it is
forwarded. Matched flows pass through individual Markers that
been configured from the usage profile for that flow: service
(Premium or Assured), rate (peak for Premium, "expected"
Assured), and permissible burst size (may be optional for Premium).
Assured flow packets emerge from the Marker with their A-bits
when the flow is in conformance to its Profile, but the flow
otherwise unchanged. For a Premium flow, the Marker will hold
when necessary to enforce their configured rate. Thus Premium
packets emerge from the Marker in a shaped flow with their P-
set. (It is possible for Premium flow packets to be dropped inside
the Marker as we describe below.) Packets are passed to
forwarding engine when they emerge from Markers. Packets that
either their P or A bits set we will refer to as Marked packets
Figure 2. Block diagram of leaf router input
Figure 3 shows the inner workings of the Marker. For both Assured
Premium packets, a token bucket "fills" at the flow rate that
specified in the usage profile. For Assured service, the token
depth is set by the Profile's burst size. For Premium service,
token bucket depth must be limited to the equivalent of only one
two packets. (We suggest a depth of one packet in early deployments.)
When a token is present, Assured flow packets have their A-bit set
one, otherwise the packet is passed to the forwarding engine.
Premium-configured Marker, arriving packets that see a token
have their P-bits set and are forwarded, but when no token
present, Premium flow packets are held until a token arrives. If
Premium flow bursts enough to overflow the holding queue, its
will be dropped. Though the flow set up data can be used to
a size limit for the holding queue (this would be the meaning of
"burst" in Premium service), it is not necessary.
holding queues should be capable of holding at least two bandwidth
Nichols, et al. Informational [Page 8]
RFC 2638 Two-bit Differentiated Services Architecture July 1999
delay products, adequate for TCP connections. A smaller value
be used to suit delay requirements of a specific application
Figure 3. Markers to implement the two different
In practice, the token bucket should be implemented in bytes and
token is considered to be present if the number of bytes in
bucket is equal or larger to the size of the packet. For Premium,
bucket can only be allowed to fill to the maximum packet size;
Assured may fill to the configured burst parameter. Premium
is held until a sufficient byte credit has accumulated and
holding buffer provides the only real queue the flow sees in
network. For Assured, traffic, we just test if the bytes in
bucket are sufficient for the packet size and set A if so. If not
the only difference is that A is not set. Assured traffic goes into
queue following this step and potentially sees a queue at every
along its path
Each output interface of a router must have two queues and
implement a test on the P-bit to select a packet's output queue.
two queues must be serviced by simple priority, Premium
first. Each output interface must implement the RED-based
mechanism described in [3] on the lower priority queue. RIO uses
thresholds for when to begin dropping packets, a lower one based
total queue occupancy for ordinary best effort traffic and one
on the number of packets enqueued that have their A-bit set.
means that any action preferential to Assured service traffic
only be taken when the queue's capacity exceeds the threshold
for ordinary best effort service. In this case, only unmarked
will be dropped (using the RED algorithm) unless the threshold
for Assured service is also reached. Keeping an accurate count of
number of A-bit packets currently in a queue requires either
the A-bit at both entry and exit of the queue or some
state in the router. Figure 4 is a block diagram of the
interface for all routers
Figure 4. Router output interface for two-bit
The packet output of a leaf router is thus a shaped stream of
with P-bits set mingled with an unshaped best effort stream
packets, some of which may have A-bits set. Premium service
cannot starve best effort traffic because it is both burst
bandwidth controlled. Assured service might rely only on
conservative allocation to prevent starvation of unmarked traffic
but bursts of Assured traffic might then close out best-
traffic at bottleneck queues during congestive periods
Nichols, et al. Informational [Page 9]
RFC 2638 Two-bit Differentiated Services Architecture July 1999
After [3], we designate the forwarding path objects that test
against their usage profiles "Profile Meters". Border routers
require Profile Meters at their input interfaces. The
agreement between adjacent administrative domains must specify a
rate on all P traffic and a rate and burst for A traffic (
possibly a start time and duration). A Profile Meter is required
the ingress of a trust region to ensure that differentiated
packet flows are in compliance with their agreed-upon rates. Non
compliant packets of Premium flows are discarded while non-
packets of Assured flows have their A-bits reset. For example,
figure 1, if the ISP has agreed to supply Company A with r bytes/
of Premium service, P-bit marked packets that enter the ISP
the link from Company A will be dropped if they exceed r. If instead
the service in figure 1 was Assured service, the packets would
be unmarked, forwarded as best effort
The simplest border router input interface is a Profile
constructed from a token bucket configured with the contracted
across that ingress link (see figure 5). Each type, Premium
Assured, and each interface must have its own profile
corresponding to a particular class across a particular boundary
(This is in contrast to models where every flow that crosses
boundary must be separately policed and/or shaped.) The
mechanisms required at a border router input interface depend on
allocation policy deployed; a more complex approach is presented
section 4.
Figure 5. Border router input interface Profile
3.
3.1 Forwarding Path
Section 2.3 introduced the forwarding path objects of Markers
Profile Meters. In this section we specify the primitive
blocks required to compose them. The primitives are:
classifier, bit-pattern classifier, bit setter, priority queues
policing token bucket and shaping token bucket. These primitives
compose a Marker (either a policing or a shaping token bucket plus
bit setter) and a Profile Meter (a policing token bucket plus
dropper or bit setter).
General Classifier: Leaf or first-hop routers must perform
transport-level signature matching based on a tuple in the
header, a functionality which is part of any RSVP-capable router.
described above, packets whose tuples match one of the
flows are conformance tested and have the appropriate service
set. This function is memory- and processing-intensive, but is
Nichols, et al. Informational [Page 10]
RFC 2638 Two-bit Differentiated Services Architecture July 1999
at the edges of the network where there are fewer flows
Bit-pattern classifier: This primitive comprises a simple two-
decision based on whether a particular bit-pattern in the IP
is set or not. As in figure 4, the P-bit is tested when a
arrives at a non-leaf router to determine whether to enqueue it
the high priority output queue or the low priority packet queue.
A-bit of packets bound for the low priority queue is tested to 1)
increment the count of Assured packets in the queue if set and 2)
determine which drop probability will be used for that packet
Packets exiting the low priority queue must also have the A-
tested so that the count of enqueued Assured packets can
decremented if necessary
Bit setter: The A-bits and P-bits must be set or cleared in
places. A functional block that sets the appropriate bits of the
header to a configured bit-pattern would be the most general
Priority queues: Every network element must include (at least)
levels of simple priority queueing. The high priority queue is
the Premium traffic and the service rule is to send packets in
queue first and to exhaustion. Recall that Premium traffic must
be oversubscribed, thus Premium traffic should see little or
queue
Shaping token bucket:This is the token bucket required at the
router for Premium traffic and shown in figure 3. As we shall see
shaping is also useful at egress points of a trust region.
arriving packet is immediately forwarded if there is a token
in the bucket, otherwise the packet is enqueued until the
contains tokens sufficient to send it. Shaping requires
mechanisms, packet memory, and some state block for each flow and
thus a memory and computation-intensive process
Policing token bucket: This is the token bucket required for
Meters and shown in figure 5. Policing token buckets never
arriving packets, but check on arrival to see if a token is
for the packet's service class. If so, the packet is
immediately. If not, the policing action is taken, dropping
Premium and reclassifying or unmarking for Assured
3.2 Passing configuration
Clearly, mechanisms are required to communicate the
about the request to the leaf router. This configuration
is the rate, burst, and whether it is a Premium or Assured type
There may also need to be a specific field to set or clear
configuration. This information can be passed in a number of ways
Nichols, et al. Informational [Page 11]
RFC 2638 Two-bit Differentiated Services Architecture July 1999
including using the semantics of RSVP, SNMP, or directly set by
network administrator in some other way. There must be
mechanisms for authenticating the sender of this information.
expect configuration to be done in a variety of ways in
deployments and a protocol and mechanism for this to be a topic
future standards work
3.3
The requirements of shapers motivate their placement at the edges
the network where the state per router can be smaller than in
middle of a network. The greatest burden of flow matching and
will be at leaf routers where the speeds and buffering
should be less than those that might be required deeper in
network. This functionality is not required at every network
on the path. Routers that are internal to a trust region will
need to shape traffic. Border routers may need or desire to shape
aggregate flow of Marked packets at their egress in order to
that they will not burst into non-compliance with the
mechanism at the ingress to the other domain (though this may not
necessary if the in-degree of the router is low). Further,
shaping would be applied to an aggregation of all the Premium
that exit the domain via that path, not to each flow individually
These mechanisms are within reach of today's technology and it
plausible to us that Premium and Assured services are all that
needed in the Internet. If, in time, these services are
insufficient, this architecture provides a migration path
delivering other kinds of service levels to traffic. The A- and P
bits would continue to be used to identify traffic that gets
service, but further filter matching could be done on packet
to differentiate service levels further. Using the bits this
reduces the number of packets that have to have further matching
on them rather than filtering every incoming packet. More
levels and more complex scheduling could be added for P-bit
and more levels of drop priority could be added for A-bit traffic
experience shows them to be necessary and processing speeds
sufficient. We propose that the services described here be
as "at least" services. Thus, a network element should at least
capable of mapping all P-bit traffic to Premium service and
mapping all A-bit traffic to be treated with one level of priority
the "best effort" queue (it appears that the single level of A-
traffic should map to a priority that is equivalent to the best
in a multi-level element that is also in the path).
On the other hand, what is the downside of deploying an
for both classes of service if later experience convinces us
only one of them is needed? The functional blocks of both
Nichols, et al. Informational [Page 12]
RFC 2638 Two-bit Differentiated Services Architecture July 1999
classes are similar and can be provided by the same mechanism
parameterized differently. If Assured service is not used,
little is lost. A RED-managed best effort queue has been
recommended in [4] and, to the extent that the deployment of
architecture pushes the deployment of RED-managed best effort queues
it is clearly a positive. If Premium service goes unused, the two
queues with simple priority service is not required and the
function of the Marker may be unused, thus these would impose
unnecessary implementation cost
4. The Architectural Framework for Marked Traffic
Thus far we have focused on the service definitions and
forwarding path mechanisms. We now turn to the problem of
the level of Marked traffic throughout the Internet. We observe
most organizations have fixed portions of their budgets,
data communications, that are determined on an annual or
basis. Some additional monies might be attached to specific
for discretionary costs that arise in the shorter term. In turn
service providers (ISPs and NSPs) must do their planning on
and quarterly bases and thus cannot be expected to
differentiated services purely "on call". Provisioning sets up
levels of Marked traffic while call set-up creates an allocation
Marked traffic for a single flow's duration. Static levels can
provisioned with time-of-day specifications, but cannot be changed
response to a dynamic message. We expect both kinds of
allocation to be important. The purchasers of Marked services
generally be expected to work on longer-term budget cycles
these services will be accounted for similarly to many
services today. A mail-order house may wish to purchase a
allocation of bandwidth in and out of its web-server to
potential customers a "fast" feel when browsing their site.
allocation might be based on hit rates of the previous quarter
some sort of industry-based averages. In addition, there needs to
a dynamic allocation capability to respond to particular events,
as a demonstration, a network broadcast by a company's CEO, or
particular network test. Furthermore, a dynamic capability may
needed in order to meet a precommitted service level when
particular source or destination is allowed to be "anywhere on
Internet". "Dynamic" covers the range from a telephoned or e-
request to a signalling type model. A strictly statically
scenario is expected to be useful in initial deployment
differentiated services and to make up a major portion of the
traffic for the forseeable future
Without a "per call" dynamic set up, the preconfiguring of
profiles can always be construed as "paying for bits you don't use
whether the type of service is Premium or Assured. We prefer to
Nichols, et al. Informational [Page 13]
RFC 2638 Two-bit Differentiated Services Architecture July 1999
of this as paying for the level of service that one expects to
available at any time, for example paying for a telephone line.
customer might pay an additional flat fee to have the privilege
calling a wide local area for no additional charge or might pay
the call. Although a customer might pay on a "per call" basis
every call made anywhere, it generally turns out not to be the
economical option for most customers. It's possible similar
structures might arise in the internet
We use Allocation to refer to the process of making Marked
commitments anywhere along this continuum from strictly
to dynamic call set-up and we require an Allocation
capable of encompassing this entire spectrum in any mix. We
observe that Allocation must follow organizational hierarchies,
is each organization must have complete responsibility for
Allocation of the Marked traffic resource within its domain. Finally
we observe that the only chance of success for incremental
lies in an Allocation architecture that is made up of
agreements, as multilateral agreements are much too complex
administer. Thus, the Allocation architecture is made up
agreements across boundaries as to the amount of Marked traffic
will be allowed to pass. This is similar to "settlement" models
today
4.1 Bandwidth Brokers: Allocating and Controlling Bandwidth
The goal of differentiated services is controlled sharing of
organization's Internet bandwidth. The control can be
independently by individuals, i.e., users set bit(s) in their
to distinguish their most important traffic, or it can be done
agents that have some knowledge of the organization's priorities
policies and allocate bandwidth with respect to those policies
Independent labeling by individuals is simple to implement
unlikely to be sufficient since it's unreasonable to expect
individuals to know all their organization's priorities and
network use and always mark their traffic accordingly. Thus
architecture is designed with agents called bandwidth brokers (BB
[2], that can be configured with organizational policies, keep
of the current allocation of marked traffic, and interpret
requests to mark traffic in light of the policies and
allocation
We note that such agents are inherent in any but the most
notions of sharing. Neither individuals nor the routers
packets transit have the information necessary to decide
packets are most important to the organization. Since these
must exist, they can be used to allocate bandwidth for end-to-
connections with far less state and simpler trust relationships
Nichols, et al. Informational [Page 14]
RFC 2638 Two-bit Differentiated Services Architecture July 1999
deploying per flow or per filter guarantees in all network
on an end-to-end path. BBs make it possible for bandwidth
to follow organizational hierarchies and, in concert with
forwarding path mechanisms discussed in section 3, reduce the
required to set up and maintain a flow over architectures
require checking the full flow header at every network element
Organizationally, the BB architecture is motivated by the
that multilateral agreements rarely work and this architecture
end-to-end services to be constructed out of purely
agreements. BBs only need to establish relationships of limited
with their peers in adjacent domains, unlike schemes that require
setting of flow specifications in routers throughout an end-to-
path. In practical technical terms, the BB architecture makes
possible to keep state on an administrative domain basis, rather
at every router and the service definitions of Premium and
service make it possible to confine per flow state to just the
routers
BBs have two responsibilities. Their primary one is to parcel
their region's Marked traffic allocations and set up the leaf
within the local domain. The other is to manage the messages that
sent across boundaries to adjacent regions' BBs. A BB is
with a particular trust region, one per domain. A BB has a
database that keeps the information on who can do what when and
method of using that database to authenticate requesters. Only a
can configure the leaf routers to deliver a particular service
flows, crucial for deploying a secure system. If the deployment
Differentiated Services has advanced to the stage where
allocated, marked flows are possible between two adjacent domains
BBs also provide the hook needed to implement this. Each domain's
establishes a secure association with its peer in the adjacent
to negotiate or configure a rate and a service class (Premium
Assured) across the shared boundary and through the peer's domain.
we shall see, it is possible for some types of service
particularly in early implementations, that this "secure association
is not automatic but accomplished through human negotiation
subsequent manual configuration of the adjacent BBs according to
negotiated agreement. This negotiated rate is a capability that a
controls for all hosts in its region
When an allocation is desired for a particular flow, a request
sent to the BB. Requests include a service type, a target rate,
maximum burst, and the time period when service is required.
request can be made manually by a network administrator or a user
it might come from another region's BB. A BB first authenticates
credentials of the requester, then verifies there exists
bandwidth sufficient to meet the request. If a request passes
tests, the available bandwidth is reduced by the requested amount
Nichols, et al. Informational [Page 15]
RFC 2638 Two-bit Differentiated Services Architecture July 1999
the flow specification is recorded. In the case where the flow has
destination outside this trust region, the request must fall
the class allocation through the "next hop" trust region that
established through a bilateral agreement of the two trust regions
The requester's BB informs the adjacent region's BB that it will
using some of this rate allocation. The BB configures the
leaf router with the information about the packet flow to be given
service at the time that the service is to commence.
configuration is "soft state" that the BB will periodically refresh
The BB in the adjacent region is responsible for configuring
border router to permit the allocated packet flow to pass and for
additional configurations and negotiations within and across
borders that will allow the flow to reach its final destination
At DMZs, there must be an unambiguous way to determine the
source of a packet. An interface's source could be determined
its MAC address which would then be used to classify packets
coming across a logical link directly from the source
corresponding to that MAC address. Thus with this understanding
can continue to use figures illustrating a single pipe between
different domains
In this way, all agreements and negotiations are performed
two adjacent domains. An initial request might cause
between BBs on several domains along a path, but each
is only between two adjacent BBs. Initially, these agreements will
prenegotiated and fairly static. Some may become more dynamic as
service evolves
4.2
This section gives examples of BB transactions in a non-trivial
multi-transit-domain Internet. The BB framework allows
points across a spectrum from "no signalling across boundaries"
"each flow set up dynamically". We might expect to move across
spectrum over time, as the necessary mechanisms are
deployed and BBs become more sophisticated, but the
allocated portions of the spectrum should always have uses.
believe the ability to support this wide spectrum of
simultaneously will be important both in incremental deployment
in allowing ISPs to make a wide range of offerings and pricings
users. The examples of this section roughly follow the spectrum
increasing sophistication. Note that we assume that domains
for some amount of Marked traffic which can be requested as
Assured or Premium in each individual flow setup transaction.
examples say "Marked" although actual transactions would have
specify either Assured or Premium
Nichols, et al. Informational [Page 16]
RFC 2638 Two-bit Differentiated Services Architecture July 1999
A statically configured example with no BB messages exchanged:
all allocations are statically preallocated through purely
agreements between users (individual TCPs, individual hosts,
networks, or whole ISPs) [6]. The allocations are in the form
usage profiles of rate, burst, and a time during which that
is to be active. Users and providers negotiate these Profiles
are then installed in the user domain BB and in the provider
BB. No BB messages cross the boundary; we assume this negotiation
done by human representatives of each domain. In this case, BBs
have to perform one of their two functions, that of allocating
Profile within their local domain. It is even possible to set all
this suballocations up in advance and then the BB only needs to
up and tear down the Profile at the proper time and to refresh
soft state in the leaf routers. From the user domain BB, the
is sent as soft state to the first hop router of the flow during
specified time. These Profiles might be set using RSVP, a variant
RSVP, SNMP, or some vendor-specific mechanism. Although this
approach can work for all Marked traffic, due to the strictly
oversubscribed requirement, it is only appropriate for
traffic as long as it is kept to a small percentage of the
path through a domain or is otherwise constrained to a well-
behavior. Similar restrictions might hold for Assured depending
the expectation associated with the service
In figure 6, we show an example of setting a Profile in a
router. A usage profile has been negotiated with the ISP for
entire domain and the BB parcels it out among individual flows
requested. The leaf router mechanism is that shown in figure 3,
the token bucket set to the parameters from the usage profile.
ISP's BB would configure its own Profile Meter at the ingress
from that customer to ensure the Profile was maintained.
mechanism was shown in figure 5. We assume that the time duration
start times for any Profile to be active are maintained in the BB
The Profile is sent to the ingress device or cleared from the
device by messages sent from the BB. In this example, we assume
van@lbl wants to talk to ddc@mit. The LBL-BB is sent a request
Van asking that premium service be assigned to a flow that
designated as having source address "V:4" and going to
address "D:8". This flow should be configured for a rate of 128kb/
and allocated from 1pm to 3pm. The request must be "signed" in
secure, verifiable manner. The request might be sent as data to
LBL-BB, an e-mail message to a network administrator, or in a
call to a network administrator. The LBL-BB receives this message
verifies that there is 128kb/sec of unused Premium service for
domain from 1-3pm, then sends a message to Leaf1 that sets up
appropriate Profile Meter. The message to Leaf1 might be an
message, or SNMP, or some proprietary method. All the domains
must have sufficient reserve capacity to meet this request
Nichols, et al. Informational [Page 17]
RFC 2638 Two-bit Differentiated Services Architecture July 1999
Figure 6. Bandwidth Broker setting Profiles in leaf
A statically configured example with BB messages exchanged: Next
present an example where all allocations are statically
but BB messages are exchanged for greater flexibility. Figure 7
an end-to-end example for Marked traffic in a statically
internet. The numbers at the trust region boundaries indicate
total statically allocated Marked packet rates that will be
across those boundaries. For example, 100kbps of Marked traffic
be sent from LBL to ESNet; a Profile Meter at the ESNet
boundary would have a token bucket set to rate 100kbps. (There MAY
a shaper set at LBL's egress to ensure that the Marked
conforms to the aggregate Profile.) The tables inside the
network "bubbles" show their policy databases and reflect the
after the transaction is complete. In Figure 7, V wants to transmit
flow from LBL to D at MIT at 10 Kbps. As in figure 6, a request
this profile is made of LBL's BB. LBL's BB authenticates the
and checks to see if there is 10kbps left in its Marked
going in that direction. There is, so the LBL-BB passes a message
the ESNet-BB saying that it would like to use 10kbps of its
allocation for this flow. ESNet authenticates the message, checks
database and sees that it has a 10kbps Marked allocation to
(the next region in that direction) that is being unused. The
is that ESNet-BB must always inform ("ask") NEARNet-BB when it
about to use part of its allocation. NEARNET-BB authenticates
message, checks its database and discovers that 20kbps of
allocation to MIT is unused and the policy at that boundary is to
inform MIT when part of the allocation is about to be used ("<50 ok
where the total allocation is 50). The dotted lines indicate
"implied" transaction, that is the transaction that would
happened if the policy hadn't said "don't ask me". Now each BB
pass an "ok" message to this request across its boundary. This
V to send to D, but not vice versa. It would also be possible for
request to originate from D
Figure 7. End-to-end example with static allocation
Consider the same example where the ESNet-BB finds all of its
allocation to NEARNet, 10 kbps, in use. With static allocations
ESNet must transmit a "no" to this request back to the LBL-BB
Presumably, the LBL-BB would record this information to complain
ESNet about the overbooking at the end of the month! One solution
this sort of "busy signal" is for ESNet to get better at
its customers needs or require long advance bookings for every flow
but it's also possible for bandwidth brokerage decisions to
dynamic
Nichols, et al. Informational [Page 18]
RFC 2638 Two-bit Differentiated Services Architecture July 1999
Figure 8. End-to-end static allocation example with no
Dynamic Allocation and additional mechanism: As we shall see,
allocation requires more complex BBs as well as more complex
policing, including the necessity to keep more state. However,
enables an important service with a small increase in state
The next set of figures (starting with figure 9) show what happens
the case of dynamic allocation. As before, V requests 10kbps to
to D at MIT. Since the allocation is dynamic, the border policers
not have a preset value, instead being set to reflect the
peak value of Marked traffic permitted to cross that boundary.
request is sent to the LBL-BB
Figure 9. First step in end-to-end dynamic allocation example
In figure 10, note that ESNet has no allocation set up to NEARNet
This system is capable of dynamic allocations in addition to static
so it asks NEARNet if it can "add 10" to its allocation from ESNet
As in the figure 7 example, MIT's policy is set to "don't ask"
this case, so the dotted lines represent "implicit transactions
where no messages were exchanged. However, NEARNet does update
table to indicate that it is now using 20kbps of the
allocation to MIT
Figure 10. Second step in end-to-end dynamic allocation
In figure 11, we see the third step where MIT's "virtual ok"
the NEARNet-BB to tell its border router to increase the
allocation across the ESNet-NEARNet boundary by 10 kbps
Figure 11. Third step in end-to-end dynamic allocation
Figure 11 shows NEARNet-BB's "ok" for that request transmitted
to ESNet-BB. This causes ESNet-BB to send its border router a
to create a 10 kbps subclass for the flow "V->D". This is required
order to ensure that the 10kpbs that has just been
allocated gets used only for that connection. Note that this
require that the per flow state be passed from LBL-BB to ESNet-BB
but this is the only boundary that needs that level of
information and this further classification will only need to be
at that one boundary router and only on packets coming from LBL.
dynamic allocation requires more complex Profile Metering than
shown in figure 5.
Figure 12. Fourth step in end-to-end dynamic allocation example
Nichols, et al. Informational [Page 19]
RFC 2638 Two-bit Differentiated Services Architecture July 1999
In figure 12, the ESNet border router gives the "ok" that a
has been created, causing the ESNet-BB to send an "ok" to the LBL-
which lets V know the request has been approved
Figure 13. Final step in end-to-end dynamic allocation
For dynamic allocation, a basic version of a CBQ scheduler [5]
have all the required functionality to set up the subclasses.
currently provides a way to move the TSpec for the flow
For multicast flows, we assume that packets that are bound for
least one egress can be carried through a domain at that level
service to all egress points. If a particular multicast branch
been subscribed to at best-effort when upstream branches are Marked
it will have its bit settings cleared before it crosses the boundary
The information required for this flow identification is used
augment the existing state that is already kept on this flow
it is a multicast flow. We note that we are already "catching"
flow, but now we must potentially clear the bit-pattern
5. RSVP/int-serv and this
Much work has been done in recent years on the definition of
integrated services for the internet and the specification of
RSVP signalling protocol. The two-bit architecture proposed in
work can easily interoperate with those specifications. In
section we first discuss how the forwarding mechanisms described
section 3 can be used to support integrated services. Second,
discuss how RSVP could interoperate with the administrative
of the BBs to provide better scaling
5.1 Providing Controlled-Load and Guaranteed
We believe that the forwarding path mechanisms described in section 3
are general enough that they can also be used to provide
Controlled-Load service [8] and a version of the Guaranteed
of Service [9], as developed by the int-serv WG. First note
Premium service can be thought of as a constrained case
Controlled-Load service where the burst size is limited to one
and where non-conforming packets are dropped. A network element
has implemented the mechanisms to support premium service can
support the more general controlled-load service by making one
more minor parameter adjustments, e.g. by lifting the constraint
the token bucket size, or configuring the Premium service rate
the peak traffic rate parameter in the Controlled-Load specification
and by changing the policing action on out-of-profile packets
dropping to sending the packets to the Best-effort queue
Nichols, et al. Informational [Page 20]
RFC 2638 Two-bit Differentiated Services Architecture July 1999
It is also possible to implement Guaranteed Quality of Service
the mechanisms of Premium service. From RFC 2212 [9]: "The
of guaranteed service relies on the result that the fluid delay of
flow obeying a token bucket (r, b) and being served by a line
bandwidth R is bounded by b/R as long as R is no less than r
Guaranteed service with a service rate R, where now R is a share
bandwidth rather than the bandwidth of a dedicated line
this behavior." The service model of Premium clearly fits this model
RFC 2212 states that "Non-conforming datagrams SHOULD be treated
best-effort datagrams." Thus, a policing Profile Meter that
non-conforming datagrams would be acceptable, but it's also
to change the action for non-compliant packets from a drop to
to the best-effort queue
5.2 RSVP and
In this section we discuss how RSVP signaling can be used
conjunction with the BBs described in section 4 to deliver a
scalable end-to-end resource set up for Integrated Services. First
note that the BB architecture has three major differences with
original RSVP resource set up model
1. There exist apriori bilateral business relations between BBs
adjacent trust regions before one can set up end-to-end
allocation; real-time signaling is used only to activate/confirm
availability of pre-negotiated Marked bandwidth, and to
readjust the allocation amount when necessary. We note that
real-time signaling across domains is not required, but depends
the nature of the bilateral agreement (e.g., the agreement
state "I'll tell you whenever I'm going to use some of my allocation
or not).
2. A few bits in the packet header, i.e. the P-bit and A-bit,
used to mark the service class of each packet, therefore a
packet classification (by checking all relevant fields in the header
need be done only once at the leaf router; after that packets will
served according to their class bit settings
3. RSVP resource set up assumes that resources will be reserved hop
by-hop at each router along the entire end-to-end path
RSVP messages sent to leaf routers by hosts can be intercepted
sent to the local domain's BB. The BB processes the message and,
the request is approved, forwards a message to the leaf router
sets up appropriate per-flow packet classification. A message
also be sent to the egress border router to add to the
Marked traffic allocation for packet shaping by the Profile Meter
outbound traffic. (Its possible that this is always set to the
Nichols, et al. Informational [Page 21]
RFC 2638 Two-bit Differentiated Services Architecture July 1999
allocation.) An RSVP message must be sent across the boundary
adjacent ISP's border router, either from the local domain's
router or from the local domain's BB. If the ISP is also
the RSVP with a BB and diff-serv framework, its border
forwards the message to the ISP's local BB. A similar process (
what happened in the first domain) can be carried out in the
domain, then an RSVP message gets forwarded to the next ISP along
path. Inside a domain, packets are served solely according to
Marked bits. The local BB knows exactly how much Premium traffic
permitted to enter at each border router and from which border
packets exit
6.
This document has presented a reference architecture
differentiated services. Several variations can be envisioned
particularly for early and partial deployments, but we do
enumerate all of these variations here. There has been a great
demand for differentiated services lately. As one of the many
to meet that demand this memo sketches out the framework of
flexible architecture for offering differential services, and
particular defines a simple set of packet forwarding path
to support two basic types of differential services. Although
remain a number of issues and parameters that need
exploration and refinement, we believe it is both possible
feasible at this time to start deployment of differentiated
incrementally. First, given that the basic mechanisms required in
packet forwarding path are clearly understood, both Assured
Premium services can be implemented today with manually
BBs and static resource allocation. Initially we
conservative choices on the amount of Marked traffic that is
into the network. Second, we plan to continue the effort started
this memo and the experimental work of the authors to define
deploy increasingly sophisticated BBs. We hope to turn the
gained from in-progress trial implementations on ESNet and CAIRN
future proposals to the IETF
Future revisions of this memo will present the receiver-based
multicast flow allocations in detail. After this step is finished
we believe the basic picture of an scalable, robust, secure
management and allocation system will be completed. In this memo,
described how the proposed architecture supports two services
seem to us to provide at least a good starting point for
deployment of differentiated services. Our main intent is to
an architecture with three services, Premium, Assured, and
effort, that can be determined by specific bit- patterns, but not
preclude additional levels of differentiation within each service.
seems that more experimentation and experience is required before
Nichols, et al. Informational [Page 22]
RFC 2638 Two-bit Differentiated Services Architecture July 1999
could standardize more than one level per service class. Our base
level approach says that everyone has to provide "at least"
service and Assured service as documented. We feel rather
about both 1) that we should not try to define, at this time
something beyond the minimalist two service approach and 2) that
architecture we define must be open-ended so that more levels
differentiation might be standardized in the future. We believe
architecture is completely compatible with approaches that
define more levels of differentiation within a particular service,
the benefits of doing so become well understood
7.
The authors have benefited from many discussions, both in person
electronically and wish to particularly thank Dave Clark who has
responsible for the genesis of many of the ideas presented here
though he does not agree with all of the content this document.
also thank Sally Floyd for comments on an earlier draft. A
from Jon Crowcroft was partially responsible for our
section 5. Comments from Fred Baker made us try to make it
that we are defining two base-level services, irrespective of the
patterns used to encode them
8. Security
There are no security considerations associated with this document
9.
[1] D. Clark, "Adding Service Discrimination to the Internet",
Proceedings of the 23rd Annual Telecommunications Policy
Conference (TPRC), Solomons, MD, October 1995.
[2] V. Jacobson, "Differentiated Services Architecture", talk in
Int-Serv WG at the Munich IETF, August, 1997.
[3] Clark, D. and J. Wroclawski, "An Approach to Service
in the Internet", Work in Progress, also talk by D. Clark in
Int-Serv WG at the Munich IETF, August, 1997.
[4] Braden, et al., "Recommendations on Queue Management
Congestion Avoidance in the Internet", RFC 2309, April 1998.
[4] Braden, R., Zhang, L., Berson, S., Herzog, S. and S. Jamin
"Resource Reservation Protocol (RSVP) - Version 1
Specification", RFC 2205, September 1997.
Nichols, et al. Informational [Page 23]
RFC 2638 Two-bit Differentiated Services Architecture July 1999
[5] S. Floyd and V. Jacobson, "Link-sharing and Resource
Models for Packet Networks", IEEE/ACM Transactions on Networking
pp 365-386, August 1995.
[6] D. Clark, private communication, October 26, 1997.
[7] "Advanced QoS Services for the Intelligent Internet",
Systems White Paper, 1997.
[8] Wroclawski, J., "Specification of the Controlled-Load
Element Service", RFC 2211, September 1997.
[9] Shenker, S., Partirdge, C. and R. Guerin, "Specification
Guaranteed Quality of Service", RFC 2212, September 1997.
[10] D. Clark and W. Fang, "Explicit Allocation of Best Effort
Delivery Service", IEEE/ACM Transactions on Networking, August
1998, Vol6, No 4, pp. 362-373. also at: http://
diffserv.lcs.mit.edu/Papers/exp-alloc-ddc-wf.
Authors'
Kathleen
Cisco Systems, Inc
170 West Tasman
San Jose, CA 95134-1706
Phone: 408-525-4857
EMail: kmn@cisco.
Van
Cisco Systems, Inc
170 West Tasman
San Jose, CA 95134-1706
EMail: van@cisco.
Lixia
4531G Boelter
Los Angeles, CA 90095
Phone: 310-825-2695
EMail: lixia@cs.ucla.
Nichols, et al. Informational [Page 24]
RFC 2638 Two-bit Differentiated Services Architecture July 1999
Appendix: A Combined Approach to Differential Service in the Internet
David D.
After the draft-nichols-diff-svc-00 was submitted, the co-authors
a discussion with Dave Clark and John Wroclawski which resulted
Clark's using the presentation slot for the draft at the
1997 IETF Integrated Services Working Group meeting. A reading of
slides shows that it was Clark's proposal on "mechanisms",
"services", and "rules" and how to proceed in the standards
that has guided much of the process in the subsequently formed
Differentiated Services Working Group. We believe Dave Clark's
gave us a solid approach for bringing quality of service to
Internet in a manner that is compatible with its strengths
The slides presented at the December 1997 IETF Integrated
Working Group are included with the Postscript version
Nichols, et al. Informational [Page 25]
RFC 2638 Two-bit Dif