As per Relevance of the word experience, we have this rfc below:
Network Working Group Y. Rekhter,
Request for Comments: 1266 T.J. Watson Research Center, IBM Corp
October 1991
Experience with the BGP
1. Status of this Memo
This memo provides information for the Internet community. It
not specify an Internet standard. Distribution of this memo
unlimited
2. Introduction
The purpose of this memo is to document how the requirements
advancing a routing protocol to Draft Standard have been satisfied
Border Gateway Protocol (BGP). This report documents experience
BGP. This is the second of two reports on the BGP protocol.
required by the Internet Activities Board (IAB) and the
Engineering Steering Group (IESG), the first report will present
performance analysis of the BGP protocol
The remaining sections of this memo document how BGP
General Requirements specified in Section 3.0, as well
Requirements for Draft Standard specified in Section 5.0 of
"Internet Routing Protocol Standardization Criteria" document [1].
This report is based on the work of Dennis Ferguson (University
Toronto), Susan Hares (MERIT/NSFNET), and Jessica Yu (MERIT/NSFNET).
Details of their work were presented at the Twentieth IETF
(March 11-15, 1991, St. Louis) and are available from the
Proceedings
Please send comments to iwg@rice.edu
3. Acknowledgements
The BGP protocol has been developed by the IWG/BGP Working Group
the Internet Engineering Task Force. We would like to express
deepest thanks to Guy Almes (Rice University) who was the
chairman of the IWG Working Group. We also like to explicitly
Bob Hinden (BBN) for the review of this document as well as
constructive and valuable comments
BGP Working Group [Page 1]
RFC 1266 Experience with the BGP Protocol October 1991
4. Documentation
BGP is an inter-autonomous system routing protocol designed for
TCP/IP internets. Version 1 of the BGP protocol was published in
1105. Since then BGP Versions 2 and 3 have been developed. Version 2
was documented in RFC 1163. Version 3 is documented in [3].
changes between versions 1, 2 and 3 are explained in Appendix 3
[3]. Most of the functionality that was present in the Version 1
present in the Version 2 and 3. Changes between Version 1
Version 2 affect mostly the format of the BGP messages.
between Version 2 and Version 3 are quite minor
BGP Version 2 removed from the protocol the concept of "up", "down",
and "horizontal" relations between autonomous systems that
present in the Version 1. BGP Version 2 introduced the concept
path attributes. In addition, BGP Version 2 clarified parts of
protocol that were "underspecified". BGP Version 3 lifted some
the restrictions on the use of the NEXT_HOP path attribute, and
the BGP Identifier field to the BGP OPEN message. It also
the procedure for distributing BGP routes between the BGP
within an autonomous system. Possible applications of BGP in
Internet are documented in [2].
The BGP protocol was developed by the IWG/BGP Working Group of
Internet Engineering Task Force. This Working Group has a
list, iwg@rice.edu, where discussions of protocol features
operation are held. The IWG/BGP Working Group meets regularly
the quarterly Internet Engineering Task Force conferences. Reports
these meetings are published in the IETF's Proceedings
5.
A BGP Management Information Base has been published [4]. The
was written by Steve Willis (swillis@wellfleet.com) and John
(jburruss@wellfleet.com).
Apart from a few system variables, the BGP MIB is broken into
tables: the BGP Peer Table and the BGP Received Path Attribute Table
The Peer Table reflects information about BGP peer connections,
as their state and current activity. The Received Path
Table contains all attributes received from all peers before
routing policy has been applied. The actual attributes used
determining a route are a subset of the received attribute table
The BGP MIB is quite small. It contains total of 27 objects
BGP Working Group [Page 2]
RFC 1266 Experience with the BGP Protocol October 1991
6. Security architecture
BGP provides flexible and extendible mechanism for authentication
security. The mechanism allows to support schemes with various
of complexity. All BGP sessions are authenticated based on the
Identifier of a peer. In addition, all BGP sessions are
based on the autonomous system number advertised by a peer. As
of the BGP authentication mechanism, the protocol allows to
encrypted digital signature in every BGP message. All
failures result in sending the NOTIFICATION messages and
termination of the BGP connection
Since BGP runs over TCP and IP, BGP's authentication scheme may
augmented by any authentication or security mechanism provided
either TCP or IP
7. Implementations
There are multiple interoperable implementations of BGP
available. This section gives a brief overview of the
completely independent implementations that are currently used in
operational Internet. They are
- cisco. This implementation was wholly developed by cisco
It runs on the proprietary operating system used by
cisco routers. Consult Kirk Lougheed (lougheed@cisco.com
for more details
- "gated". This implementation was developed wholly by
Honig (jch@risci.cit.cornell.edu) and Dennis
(dennis@CAnet.CA). It runs on a variety of operating
(4.3 BSD, AIX, etc...). It is the only available public
code for BGP. Consult Jeff Honig or Dennis Ferguson for
details
- NSFNET. This implementation was developed wholly by
Rekhter (yakov@watson.ibm.com). It runs on the T1
Backbone and T3 NSFNET Backbone. Consult Yakov Rekhter
more details
To facilitate efficient BGP implementations, and avoid commonly
mistakes, the implementation experience with BGP in "gated"
documented as part of RFC 1164. Implementors are strongly
to follow the implementation suggestions outlined in that document
Experience with implementing BGP showed that the protocol
relatively simple to implement. On the average BGP
takes about 1 man/month effort
BGP Working Group [Page 3]
RFC 1266 Experience with the BGP Protocol October 1991
Note that, as required by the IAB/IESG for Draft Standard status
there are multiple interoperable completely
implementations, namely those from cisco, "gated", and IBM
8. Operational experience
This section discusses operational experience with BGP
BGP has been used in the production environment since 1989. This
involves all three implementations listed above. Production use
BGP includes utilization of all significant features of the protocol
The present production environment, where BGP is used as the inter
autonomous system routing protocol, is highly heterogeneous.
terms of the link bandwidth it varies from 56 Kbits/sec to 45
Mbits/sec. In terms of the actual routes that run BGP it ranges
a relatively slow performance PC/RT to a very high
RS/6000, and includes both the special purpose routers (cisco)
the general purpose workstations running UNIX. In terms of the
topologies it varies from a very sparse (spanning tree or a ring
CA*Net) to a quite dense (T1 or T3 NSFNET Backbones).
At the time of this writing BGP is used as an inter-autonomous
routing protocol between the following autonomous systems: CA*Net, T
NSFNET Backbone, T3 NSFNET Backbone, T3 NSFNET Test Network, CICNET
MERIT, and PSC. Within CA*Net there are 10 border
participating in BGP. Within T1 NSFNET Backbone there are 20
routers participating in BGP. Within T3 NSFNET Backbone there are 15
border routers participating in BGP. Within T3 NSFNET Test
there are 7 border routers participating in BGP. Within CICNET
are 2 border routers participating in BGP. Within MERIT there is 1
border router participating in BGP. Within PSC there is 1
participating in BGP. All together there are 56 border
spanning 7 autonomous systems that are running BGP. Out of these, 49
border routers that span 6 autonomous systems are part of
operational Internet
BGP is used both for the exchange of routing information between
transit and a stub autonomous system, and for the exchange of
information between multiple transit autonomous systems. It
both the Backbones (CA*Net, T1 NSFNET Backbone, T3 NSFNET Backbone),
and the Regional Networks (PSC, MERIT).
Within CA*Net, T3 NSFNET Backbone, and T3 NSFNET Test Network BGP
used as the exclusive carrier of the exterior routing
both between the autonomous systems that correspond to the
networks, and with the autonomous system of each network. At the
of this writing within the T1 NSFNET Backbone BGP is used
with the NSFNET Backbone Interior Routing Protocol to carry
BGP Working Group [Page 4]
RFC 1266 Experience with the BGP Protocol October 1991
exterior routing information. T1 NSFNET Backbone is in the process
moving toward carrying the exterior routing information
by BGP. The full set of exterior routes that is carried by BGP
well over 2,000 networks
Operational experience described above involved multi-
deployment (cisco, "gated", and NSFNET).
Specific details of the operational experience with BGP in the
were presented at the Twentieth IETF meeting (March 11-15, 1991, St
Louis) by Susan Hares (MERIT/NSFNET). Specific details of
operational experience with BGP in the CA*Net were presented at
Twentieth IETF meeting (March 11-15, 1991, St. Louis) by
Ferguson (University of Toronto). Both of these presentations
available in the IETF Proceedings
Operational experience with BGP exercised all basic features of
protocol, including the authentication and routing loop suppression
Bandwidth consumed by BGP has been measured at the
points between CA*Net and T1 NSFNET Backbone. The results of
measurements were presented by Dennis Ferguson during the last IETF
and are available from the IETF Proceedings. These results
clear superiority of BGP as compared with EGP in the area
bandwidth consumed by the protocol. Observations on the CA*Net
Dennis Ferguson, and on the T1 NSFNET Backbone by Susan
confirmed clear superiority of BGP as compared with EGP in the
of CPU requirements
9. Using TCP as a transport for BGP
9.1. Introduction
On multiple occasions some members of IETF expressed concern
using TCP as a transport protocol for BGP. In this section we
the use of TCP for BGP in terms of
- real versus perceived
- offer potential solutions to real
- perspective on the convergence
-
BGP is based on the incremental updates. This is done
to conserve the CPU and bandwidth requirements. Extensive
experience with BGP in the Internet showed that indeed the use of
incremental updates allows significant saving both in terms of
CPU utilization and bandwidth consumption. However, to
correctly the incremental updates must be exchanged over a
BGP Working Group [Page 5]
RFC 1266 Experience with the BGP Protocol October 1991
transport. BGP uses TCP as such transport. It had been
that another transport protocol would be more suitable for BGP
9.2. Examination of Problems - Real and "perceived".
Extensive operational experience with BGP in the Internet showed
the only real problem that was attributed to BGP in general, and
use of TCP as the transport for BGP in particular, was its
convergence in presence of congestion. This problem was
in CA*Net. As we mentioned before, CA*Net is composed of 10
that form a ring. The routers are connected by 56 Kbits/sec links
All links are heavily utilized and are often congested.
with BGP in CA*Net showed that unless special measures are taken,
protocol may exhibit slow convergence when BGP information is
over the slow speed (56 Kbits/sec) congested links. This is because
large percentage of packets carrying BGP information are
dropped due to congestion. Therefore, there are three inter-
problems: congestion, packet drops, and the resulting
convergence of routing under congestion and packet drops
Observe, that any transport protocol used by BGP would
difficulty preventing packets from being dropped under congestion
since it has no direct control over the routers that drop
packets, and the congestion has nothing to do with the BGP traffic
Therefore, since BGP is not the cause of congestion, and
directly influence dropping at the routers, replacing TCP (as the
transport) with another transport protocol would have no effect
packets being dropped due to congestion. We think that once a
is congested, packets will be dropped (regardless of whether
packets carry BGP or any other information), unless special
outside of BGP in general, and the transport protocol used by BGP
particular, are taken
If packets carrying routing information are lost, any
routing protocol will exhibit slow convergence. If quick
is viewed as important for a routing within a network,
measures to minimize the loss of packets that carry
information must be taken. The next section suggests some
methods
9.3. Solutions to the problem
Two possible measures could be taken to reduce the drop of
packets which slows convergence of routing
1) alleviate the
2) reduce the percentage of BGP packets that are dropped
BGP Working Group [Page 6]
RFC 1266 Experience with the BGP Protocol October 1991
to congestion by marking BGP packets and setting policies
routers to try not to drop BGP
Alleviating the network congestion is a subject outside the
of BGP, and will not be discussed in this paper
Operational experience with BGP in CA*Net shows that reducing
percentage of BGP packets dropped due to congestion by marking them
and setting policies to routers to try not to drop BGP
completely solves the problem of slow convergence in presence
congestion
The BGP packets can be marked (explicitly or implicitly) by
following three methods
a) by means of IP precedence (Internetwork Control
b) by using a well-known TCP port
c) by identifying packets by just source or destination
address
Appendix 4 of the BGP protocol specification, RFC 1163,
the use of IP precedence (Internetwork Control) because
precedence provides a well-defined mechanism to mark BGP packets
The method of a well-known TCP port number to identify packets
similar to the one that was used by Dave Mills in the NSFNET Phase I
Dave Mills identified Telnet traffic by a well known TCP port number
and gave it priority over the rest of the traffic. CA*Net
BGP traffic based on it's source and destination IP address.
receive a priority if either the source or the destination IP
belongs to CA*Net
If packets that carry the routing information are being
(because of congestion), one also may ask about how does a
routing protocol react to such an event. In the case of BGP
packets are retransmitted using the TCP retransmission mechanism.
seems plausible that being more aggressive in terms of
retransmission should have positive effect on the convergence.
can be done completely within TCP by adjusting the TCP
timers. However, we would like to point out that the change in
retransmission strategy should not be viewed as a cure for
problem, since the root of the problem lies in the way how
that carry the BGP information are handled within a
network, and not in how frequently the lost packets
retransmitted
It should also be pointed out that the local system can control
BGP Working Group [Page 7]
RFC 1266 Experience with the BGP Protocol October 1991
amount of data to be retransmitted (in case of a congestion
losses) by adjusting the TCP Window size. That allows to control
amount of potentially obsolete data that has to be retransmitted
9.4. Perspective on the Convergence Problem
To put the convergence problem in a proper perspective, we'd like
point out that much of the Internet now uses EGP at AS borders
ensuring that routing changes cannot be guaranteed to
between ASes in less than a few minutes. It would take huge amount
congestion to slow BGP to this pace. Additionally, the problems
EGP in the face of packet loss are well known and far exceed
imaginable problem BGP/TCP might ever suffer. Therefore, the
case behavior of BGP is about the same as the steady case behavior
EGP
Within an AS the speed of convergence of the AS's IGP in the face
congestion is of far greater concern than the propagation speed
BGP, and indeed avoiding loss of packets carrying IGP, and a
aggressive transport is similarly of much greater importance for
IGP than for BGP
The issue of BGP convergence is of exaggerated importance to CA*
since CA*Net carries no information about external routes in its IGP
CA*Net uses BGP to transfer external routes for use in
internal routes through the CA*Net network. The reason CA*Net
this has nothing to do with BGP. Under more ordinary circumstances
IGP carries external routing information for use in
internal routes. CA*Net shows that BGP can work under extreme stress
However, it's results should not be taken as the norm since
networks will use BGP in a different (and less stressful
configuration, where information about external routes will
carried by an IGP
9.5. Conclusion
The extensive operational experience with BGP showed that the
problem attributed to BGP was the slow convergence problem
presence of congestion. We demonstrated that this problem
nothing to do with BGP in general, or with TCP as the BGP
in particular, but is directly related to the way how packets
carry routing information are handled within a congested network.
document suggests possible ways of solving the problem. We
like to point out that the issue of convergence in presence
congested network is important to all distributed routing protocol
and not just to BGP. Therefore, we recommend that every
protocol (whether it is intra-autonomous system or inter-
system) should clearly specify how its behavior is affected by
BGP Working Group [Page 8]
RFC 1266 Experience with the BGP Protocol October 1991
congestion in the networks, and what are the possible mechanisms
avoid the negative effect of congestion (if any).
10. Bibliography
[1] Hinden, B., "Internet Routing Protocol Standardization Criteria",
RFC 1264, BBN, October 1991.
[2] Rekhter, Y., and P. Gross, "Application of the Border
Protocol in the Internet", RFC 1268, T.J. Watson Research Center
IBM Corp., ANS, October 1991.
[3] Lougheed, K., and Y. Rekhter, "A Border Gateway Protocol 3 (BGP
3)", RFC 1267, cisco Systems, T.J. Watson Research Center,
Corp., October 1991.
[4] Willis, S., and J. Burruss, "Definitions of Managed Objects
the Border Gateway Protocol (Version 3)", RFC 1269,
Communications Inc., October 1991.
Security
Security issues are discussed in section 6.
Author's
Yakov
T.J. Watson Research Center IBM
P.O. Box 218
Yorktown Heights, NY 10598
Phone: (914) 945-3896
EMail: yakov@watson.ibm.
IETF BGP WG mailing list: iwg@rice.
To be added: iwg-request@rice.
BGP Working Group [Page 9]
if you see any problems within the linking, don't worry be happy,
this is version 0.1 of the Relevance System and you gotta expect some crappy subroutines sometimes,
just be content we did not write this in Java, which would have made this "bigger and better" HAHAHHA.
RFC documents can be found at I.E.T.F.
Relevance System Copyright © 2002 Spectrum WorldResearch
other technical nosh by ServerMasters Corporation
collaboration of BobX
