As per Relevance of the word indicate, we have this rfc below:
Network Working Group J.
Request for Comments: 3116 C.
Category: Informational ANC, Inc
June 2001
Methodology for ATM
Status of this
This memo provides information for the Internet community. It
not specify an Internet standard of any kind. Distribution of
memo is unlimited
Copyright
Copyright (C) The Internet Society (2001). All Rights Reserved
This document discusses and defines a number of tests that may
used to describe the performance characteristics of ATM (
Transfer Mode) based switching devices. In addition to defining
tests this document also describes specific formats for reporting
results of the tests
This memo is a product of the Benchmarking Methodology Working
(BMWG) of the Internet Engineering Task Force (IETF).
Table of
1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . . 4
2. Background . . . . . . . . . . . . . . . . . . . . . . . . . . 5
2.1. Test Device Requirements . . . . . . . . . . . . . . . . . . 5
2.2. Systems Under Test (SUTs). . . . . . . . . . . . . . . . . . 5
2.3. Test Result Evaluation . . . . . . . . . . . . . . . . . . . 5
2.4. Requirements . . . . . . . . . . . . . . . . . . . . . . . . 5
2.5. Test Configurations for SONET. . . . . . . . . . . . . . . . 6
2.6. SUT Configuration. . . . . . . . . . . . . . . . . . . . . . 7
2.7. Frame Formats. . . . . . . . . . . . . . . . . . . . . . . . 8
2.8. Frame Sizes. . . . . . . . . . . . . . . . . . . . . . . . . 8
2.9. Verifying Received IP PDU's. . . . . . . . . . . . . . . . . 9
2.10. Modifiers . . . . . . . . . . . . . . . . . . . . . . . . . 9
2.10.1. Management IP PDU's . . . . . . . . . . . . . . . . . . . 9
2.10.2. Routing Update IP PDU's . . . . . . . . . . . . . . . . . 10
2.11. Filters . . . . . . . . . . . . . . . . . . . . . . . . . . 10
2.11.1. Filter Addresses. . . . . . . . . . . . . . . . . . . . . 11
2.12. Protocol Addresses. . . . . . . . . . . . . . . . . . . . . 12
Dunn & Martin Informational [Page 1]
RFC 3116 Methodology for ATM Benchmarking June 2001
2.13. Route Set Up. . . . . . . . . . . . . . . . . . . . . . . . 12
2.14. Bidirectional Traffic . . . . . . . . . . . . . . . . . . . 12
2.15. Single Stream Path. . . . . . . . . . . . . . . . . . . . . 12
2.16. Multi-port. . . . . . . . . . . . . . . . . . . . . . . . . 13
2.17. Multiple Protocols. . . . . . . . . . . . . . . . . . . . . 14
2.18. Multiple IP PDU Sizes . . . . . . . . . . . . . . . . . . . 14
2.19. Testing Beyond a Single SUT . . . . . . . . . . . . . . . . 14
2.20. Maximum IP PDU Rate . . . . . . . . . . . . . . . . . . . . 15
2.21. Busty Traffic . . . . . . . . . . . . . . . . . . . . . . . 15
2.22. Trial Description . . . . . . . . . . . . . . . . . . . . . 16
2.23. Trial Duration. . . . . . . . . . . . . . . . . . . . . . . 16
2.24. Address Resolution. . . . . . . . . . . . . . . . . . . . . 16
2.25. Synchronized Payload Bit Pattern. . . . . . . . . . . . . . 16
2.26. Burst Traffic Descriptors . . . . . . . . . . . . . . . . . 17
3. Performance Metrics. . . . . . . . . . . . . . . . . . . . . . 17
3.1. Physical Layer-SONET . . . . . . . . . . . . . . . . . . . . 17
3.1.1. Pointer Movements. . . . . . . . . . . . . . . . . . . . . 17
3.1.1.1. Pointer Movement Propagation . . . . . . . . . . . . . . 17
3.1.1.2. Cell Loss due to Pointer Movement. . . . . . . . . . . . 19
3.1.1.3. IP Packet Loss due to Pointer Movement . . . . . . . . . 20
3.1.2. Transport Overhead (TOH) Error Count . . . . . . . . . . . 21
3.1.2.1. TOH Error Propagation. . . . . . . . . . . . . . . . . . 21
3.1.2.2. Cell Loss due to TOH Error . . . . . . . . . . . . . . . 22
3.1.2.3. IP Packet Loss due to TOH Error. . . . . . . . . . . . . 23
3.1.3. Path Overhead (POH) Error Count. . . . . . . . . . . . . . 24
3.1.3.1. POH Error Propagation. . . . . . . . . . . . . . . . . . 24
3.1.3.2. Cell Loss due to POH Error . . . . . . . . . . . . . . . 25
3.1.3.3. IP Packet Loss due to POH Error. . . . . . . . . . . . . 26
3.2. ATM Layer. . . . . . . . . . . . . . . . . . . . . . . . . . 27
3.2.1. Two-Point Cell Delay Variation (CDV) . . . . . . . . . . . 27
3.2.1.1. Test Setup . . . . . . . . . . . . . . . . . . . . . . . 27
3.2.1.2. Two-point CDV/Steady Load/One VCC. . . . . . . . . . . . 27
3.2.1.3. Two-point CDV/Steady Load/Twelve VCCs. . . . . . . . . . 28
3.2.1.4. Two-point CDV/Steady Load/Maximum VCCs . . . . . . . . . 30
3.2.1.5. Two-point CDV/Bursty VBR Load/One VCC. . . . . . . . . . 31
3.2.1.6. Two-point CDV/Bursty VBR Load/Twelve VCCs. . . . . . . . 32
3.2.1.7. Two-point CDV/Bursty VBR Load/Maximum VCCs . . . . . . . 34
3.2.1.8. Two-point CDV/Mixed Load/Three VCC's . . . . . . . . . . 35
3.2.1.9. Two-point CDV/Mixed Load/Twelve VCCs . . . . . . . . . . 36
3.2.1.10. Two-point CDV/Mixed Load/Maximum VCCs . . . . . . . . . 38
3.2.2. Cell Error Ratio (CER) . . . . . . . . . . . . . . . . . . 39
3.2.2.1. Test Setup . . . . . . . . . . . . . . . . . . . . . . . 39
3.2.2.2. CER/Steady Load/One VCC. . . . . . . . . . . . . . . . . 40
3.2.2.3. CER/Steady Load/Twelve VCCs. . . . . . . . . . . . . . . 41
3.2.2.4. CER/Steady Load/Maximum VCCs . . . . . . . . . . . . . . 42
3.2.2.5. CER/Bursty VBR Load/One VCC. . . . . . . . . . . . . . . 43
3.2.2.6. CER/Bursty VBR Load/Twelve VCCs. . . . . . . . . . . . . 44
3.2.2.7. CER/Bursty VBR Load/Maximum VCCs . . . . . . . . . . . . 46
Dunn & Martin Informational [Page 2]
RFC 3116 Methodology for ATM Benchmarking June 2001
3.2.3. Cell Loss Ratio (CLR). . . . . . . . . . . . . . . . . . . 47
3.2.3.1. CLR/Steady Load/One VCC. . . . . . . . . . . . . . . . . 47
3.2.3.2. CLR/Steady Load/Twelve VCCs. . . . . . . . . . . . . . . 48
3.2.3.3. CLR/Steady Load/Maximum VCCs . . . . . . . . . . . . . . 49
3.2.3.4. CLR/Bursty VBR Load/One VCC. . . . . . . . . . . . . . . 51
3.2.3.5. CLR/Bursty VBR Load/Twelve VCCs. . . . . . . . . . . . . 52
3.2.3.6. CLR/Bursty VBR Load/Maximum VCCs . . . . . . . . . . . . 53
3.2.4. Cell Misinsertion Rate (CMR) . . . . . . . . . . . . . . . 54
3.2.4.1. CMR/Steady Load/One VCC. . . . . . . . . . . . . . . . . 54
3.2.4.2. CMR/Steady Load/Twelve VCCs. . . . . . . . . . . . . . . 55
3.2.4.3. CMR/Steady Load/Maximum VCCs . . . . . . . . . . . . . . 57
3.2.4.4. CMR/Bursty VBR Load/One VCC. . . . . . . . . . . . . . . 58
3.2.4.5. CMR/Bursty VBR Load/Twelve VCCs. . . . . . . . . . . . . 59
3.2.4.6. CMR/Bursty VBR Load/Maximum VCCs . . . . . . . . . . . . 60
3.2.5. CRC Error Ratio (CRC-ER) . . . . . . . . . . . . . . . . . 62
3.2.5.1. CRC-ER/Steady Load/One VCC . . . . . . . . . . . . . . . 62
3.2.5.2. CRC-ER/Steady Load/Twelve VCCs . . . . . . . . . . . . . 63
3.2.5.3. CRC-ER/Steady Load/Maximum VCCs. . . . . . . . . . . . . 64
3.2.5.4. CRC-ER/Bursty VBR Load/One VCC . . . . . . . . . . . . . 65
3.2.5.5. CRC-ER/Bursty VBR Load/Twelve VCCs . . . . . . . . . . . 66
3.2.5.6. CRC-ER/Bursty VBR Load/Maximum VCCs. . . . . . . . . . . 68
3.2.5.7. CRC-ER/Bursty UBR Load/One VCC . . . . . . . . . . . . . 69
3.2.5.8. CRC-ER/Bursty UBR Load/Twelve VCCs . . . . . . . . . . . 70
3.2.5.9. CRC-ER/Bursty UBR Load/Maximum VCCs. . . . . . . . . . . 71
3.2.5.10. CRC-ER/Bursty Mixed Load/Three VCC. . . . . . . . . . . 73
3.2.5.11. CRC-ER/Bursty Mixed Load/Twelve VCCs. . . . . . . . . . 74
3.2.5.12. CRC-ER/Bursty Mixed Load/Maximum VCCs . . . . . . . . . 75
3.2.6. Cell Transfer Delay (CTD). . . . . . . . . . . . . . . . . 76
3.2.6.1. Test Setup . . . . . . . . . . . . . . . . . . . . . . . 76
3.2.6.2. CTD/Steady Load/One VCC. . . . . . . . . . . . . . . . . 77
3.2.6.3. CTD/Steady Load/Twelve VCCs. . . . . . . . . . . . . . . 78
3.2.6.4. CTD/Steady Load/Maximum VCCs . . . . . . . . . . . . . . 79
3.2.6.5. CTD/Bursty VBR Load/One VCC. . . . . . . . . . . . . . . 81
3.2.6.6. CTD/Bursty VBR Load/Twelve VCCs. . . . . . . . . . . . . 82
3.2.6.7. CTD/Bursty VBR Load/Maximum VCCs . . . . . . . . . . . . 83
3.2.6.8. CTD/Bursty UBR Load/One VCC. . . . . . . . . . . . . . . 85
3.2.6.9. CTD/Bursty UBR Load/Twelve VCCs. . . . . . . . . . . . . 86
3.2.6.10. CTD/Bursty UBR Load/Maximum VCCs. . . . . . . . . . . . 87
3.2.6.11. CTD/Mixed Load/Three VCC's. . . . . . . . . . . . . . . 88
3.2.6.12. CTD/Mixed Load/Twelve VCCs. . . . . . . . . . . . . . . 90
3.2.6.13. CTD/Mixed Load/Maximum VCCs . . . . . . . . . . . . . . 91
3.3. ATM Adaptation Layer (AAL) Type 5 (AAL5) . . . . . . . . . . 93
3.3.1. IP Packet Loss due to AAL5 Re-assembly Errors. . . . . . . 93
3.3.2. AAL5 Re-assembly Time. . . . . . . . . . . . . . . . . . . 94
3.3.3. AAL5 CRC Error Ratio . . . . . . . . . . . . . . . . . . . 95
3.3.3.1. Test Setup . . . . . . . . . . . . . . . . . . . . . . . 95
3.3.3.2. AAL5-CRC-ER/Steady Load/One VCC. . . . . . . . . . . . . 95
3.3.3.3. AAL5-CRC-ER/Steady Load/Twelve VCCs. . . . . . . . . . . 96
Dunn & Martin Informational [Page 3]
RFC 3116 Methodology for ATM Benchmarking June 2001
3.3.3.4. AAL5-CRC-ER/Steady Load/Maximum VCCs . . . . . . . . . . 97
3.3.3.5. AAL5-CRC-ER/Bursty VBR Load/One VCC. . . . . . . . . . . 99
3.3.3.6. AAL5-CRC-ER/Bursty VBR Load/Twelve VCCs. . . . . . . . .100
3.3.3.7. AAL5-CRC-ER/Bursty VBR Load/Maximum VCCs . . . . . . . .101
3.3.3.8. AAL5-CRC-ER/Mixed Load/Three VCC's . . . . . . . . . . .102
3.3.3.9. AAL5-CRC-ER/Mixed Load/Twelve VCCs . . . . . . . . . . .104
3.3.3.10. AAL5-CRC-ER/Mixed Load/Maximum VCCs . . . . . . . . . .105
3.4. ATM Service: Signaling . . . . . . . . . . . . . . . . . . .106
3.4.1. CAC Denial Time and Connection Establishment Time. . . . .106
3.4.2. Connection Teardown Time . . . . . . . . . . . . . . . . .107
3.4.3. Crankback Time . . . . . . . . . . . . . . . . . . . . . .108
3.4.4. Route Update Response Time . . . . . . . . . . . . . . . .109
3.5. ATM Service: ILMI. . . . . . . . . . . . . . . . . . . . . .110
3.5.1. MIB Alignment Time . . . . . . . . . . . . . . . . . . . .110
3.5.2. Address Registration Time. . . . . . . . . . . . . . . . .111
4. Security Considerations . . . . . . . . . . . . . . . . . . .112
5. Notices. . . . . . . . . . . . . . . . . . . . . . . . . . . .112
6. References . . . . . . . . . . . . . . . . . . . . . . . . . .113
7. Authors' Addresses . . . . . . . . . . . . . . . . . . . . . .113
APPENDIX A . . . . . . . . . . . . . . . . . . . . . . . . . . .114
APPENDIX B . . . . . . . . . . . . . . . . . . . . . . . . . . .114
APPENDIX C . . . . . . . . . . . . . . . . . . . . . . . . . . .116
Full Copyright Statement . . . . . . . . . . . . . . . . . . . .127
1.
This document defines a specific set of tests that vendors can use
measure and report the performance characteristics of ATM
devices. The results of these tests will provide the user
data from different vendors with which to evaluate these devices
The methods defined in this memo are based on RFC 2544 "
Methodology for Network Interconnect Devices".
The document "Terminology for ATM Benchmarking" (RFC 2761),
many of the terms that are used in this document. The
document should be consulted before attempting to make use of
document
The BMWG produces two major classes of documents:
Terminology documents and Benchmarking Methodology documents.
Terminology documents present the benchmarks and other related terms
The Methodology documents define the procedures required to
the benchmarks cited in the corresponding Terminology documents
Dunn & Martin Informational [Page 4]
RFC 3116 Methodology for ATM Benchmarking June 2001
2.
2.1. Test Device
This document is based on the requirement that a test device
available. The test device can either be off the shelf or can
easily built with current technologies. The test device must have
transmitting and receiving port for the interface type under test
The test device must be configured to transmit test PDUs and
analyze received PDUs. The test device should be able to
and analyze received data at the same time
2.2. Systems Under Test (SUTs
There are a number of tests described in this document that do
apply to each SUT. Vendors should perform all of the tests that
be supported by a specific product type. It will take some time
perform all of the recommended tests under all of the
conditions
2.3. Test Result
Performing all of the tests in this document will result in a
deal of data. The applicability of this data to the evaluation of
particular SUT will depend on its expected use and the
of the network in which it will be used. For example, the
required by a switch to provide ILMI services will not be a
measurement in a network that does not use the ILMI protocol, such
an ATM WAN. Evaluating data relevant to a particular
installation may require considerable experience, which may not
readily available. Finally, test selection and evaluation of
results must be done with an understanding of generally
testing practices regarding repeatability, variance and
statistical significance of a small numbers of trials
2.4.
In this document, the words that are used to define the
of each particular requirement are capitalized. These words are
* "MUST" This word, or the words "REQUIRED" and "SHALL" mean
the item is an absolute requirement of the specification
* "SHOULD" This word or the adjective "RECOMMENDED" means that
may exist valid reasons in particular circumstances to ignore
item, but the full implications should be understood and the
carefully weighed before choosing a different course
Dunn & Martin Informational [Page 5]
RFC 3116 Methodology for ATM Benchmarking June 2001
* "MAY" This word or the adjective "OPTIONAL" means that this
is truly optional. One vendor may choose to include the
because a particular marketplace requires it or because
enhances the product, for example; another vendor may omit
same item
An implementation is not compliant if it fails to satisfy one or
of the MUST requirements for the protocols it implements.
implementation that satisfies all the MUST and all the
requirements for its protocols is said to be "
compliant"; one that satisfies all the MUST requirements but not
the SHOULD requirements for its protocols is said to
"conditionally compliant".
2.5. Test Configurations for
The test device can be connected to the SUT in a variety
configurations depending on the test point. The
configurations will be used for the tests described in this document
1) Uni-directional connection: The test devices transmit
(labeled Tx) is connected to the SUT receive port (labeled Rx).
The SUTs transmit port is connected to the test device
port (see Figure 1). In this configuration, the test device
verify that all transmitted packets are acknowledged correctly
Note that this configuration does not verify internal
functions, but verifies one port on the SUT
+-------------+ +-------------+
| Tx|-------------->|Rx |
| Test Rx|<--------------|Tx SUT |
| Device | | |
+-------------+ +-------------+
Figure 1
2) Bi-directional connection: The test devices first transmit port
connected to the SUTs first receive port. The SUTs first
port is connected to the test devices first receive port.
test devices second transmit port is connected to the SUTs
receive port. The SUTs second transmit port is connected to
test devices second receive port (see Figure 2). In
configuration, the test device can determine if all of
transmitted packets were received and forwarded correctly.
that this configuration does verify internal system functions
since it verifies two ports on the SUT
Dunn & Martin Informational [Page 6]
RFC 3116 Methodology for ATM Benchmarking June 2001
+-------------+ +-------------+
| Test Tx|-------------->|Rx |
| Device Rx|<--------------|Tx SUT |
| Tx Rx | | Tx Rx |
+-------------+ +-------------+
| ^ | ^
| | | |
| +------------------------+ |
| |
|---------------------------------|
Figure 2
3) Uni-directional passthrough connection: The test devices
transmit port is connected to the SUT1 receive port. The SUT
transmit port is connected to the test devices first receive port
The test devices second transmit port is connected to the SUT
receive port. The SUT2 transmit port is connected to the
devices second receive port (see Figure 3). In
configuration, the test device can determine if all of the
transmitted by SUT1 were correctly acknowledged by SUT2.
that this configuration does not verify internal system functions
but verifies one port on each SUT
+-------------+ +-------------+ +-------------+
| Tx|---------->|Rx Tx|---------->|Rx |
| SUT1 Rx|<----------|Tx Test Rx|<----------|Tx SUT2 |
| | | Device | | |
+-------------+ +-------------+ +-------------+
Figure 3
2.6. SUT
The SUT MUST be configured as described in the SUT users guide
Specifically, it is expected that all of the supported protocols
be configured and enabled. It is expected that all of the tests
be run without changing the configuration or setup of the SUT in
way other than that required to do the specific test. For example
it is not acceptable to disable all but one transport protocol
testing the throughput of that protocol. If PNNI or BISUP is used
initiate switched virtual connections (SVCs), the SUT
SHOULD include the normally recommended routing update intervals
keep alive frequency. The specific version of the software and
exact SUT configuration, including what functions are disabled
used during the tests MUST be included as part of the report of
results
Dunn & Martin Informational [Page 7]
RFC 3116 Methodology for ATM Benchmarking June 2001
2.7. Frame
The formats of the test IP PDUs to use for TCP/IP and UPC/IP over
are shown in Appendix C: Test Frame Formats. Note that these IP
are in accordance with RFC 2225. These exact IP PDU formats
be used in the tests described in this document for
protocol/media combination. These IP PDUs will be used as a
for testing other protocol/media combinations. The specific
that are used to define the test IP PDUs for a particular test
MUST be included in the report of the results
2.8. Frame
All of the described tests SHOULD be performed using a number of
PDU sizes. Specifically, the sizes SHOULD include the maximum
minimum legitimate sizes for the protocol under test on the
under test and enough sizes in between to be able to get a
characterization of the SUT performance. Except where noted,
least five IP PDU sizes SHOULD be tested for each test condition
Theoretically the minimum size UDP Echo request IP PDU would
of an IP header (minimum length 20 octets), a UDP header (8 octets),
AAL5 trailer (8 octets) and an LLC/SNAP code point header (8 octets);
therefore, the minimum size PDU will fit into one ATM cell.
theoretical maximum IP PDU size is determined by the size of
length field in the IP header. In almost all cases the
maximum and minimum sizes are determined by the limitations of
media. In the case of ATM, the maximum IP PDU size SHOULD be the
MTU size, which is 9180 octets
In theory it would be ideal to distribute the IP PDU sizes in a
that would evenly distribute the theoretical IP PDU rates.
recommendations incorporate this theory but specify IP PDU sizes
which are easy to understand and remember. In addition, many of
same IP PDU sizes are specified on each of the media types to
for easy performance comparisons
Note: The inclusion of an unrealistically small IP PDU size on
of the media types (i.e., with little or no space for data) is
help characterize the per-IP PDU processing overhead of the SUT
The IP PDU sizes that will be used are
44, 64, 128, 256, 1024, 1518, 2048, 4472, 9180
The minimum size IP PDU for UDP on ATM is 44 octets, the minimum
of 44 is recommended to allow direct comparison to token
performance. The IP PDU size of 4472 is recommended instead of
Dunn & Martin Informational [Page 8]
RFC 3116 Methodology for ATM Benchmarking June 2001
theoretical FDDI maximum size of 4500 octets in order to permit
same type of comparison. An IP (i.e., not UDP) IP PDU may be used
addition if a higher data rate is desired, in which case the
IP PDU size is 28 octets
2.9. Verifying received IP
The test equipment SHOULD discard any IP PDUs received during a
run that are not actual forwarded test IP PDUs. For example, keep
alive and routing update IP PDUs SHOULD NOT be included in the
of received IP PDUs. In any case, the test equipment SHOULD
the length of the received IP PDUs and check that they match
expected length
Preferably, the test equipment SHOULD include sequence numbers in
transmitted IP PDUs and check for these numbers on the received
PDUs. If this is done, the reported results SHOULD include,
addition to the number of IP PDUs dropped, the number of IP PDUs
were received out of order, the number of duplicate IP PDUs
and the number of gaps in the received IP PDU numbering sequence
This functionality is required for some of the described tests
2.10.
It is useful to characterize the SUTs performance under a number
conditions. Some of these conditions are noted below. The
results SHOULD include as many of these conditions as the
equipment is able to generate. The suite of tests SHOULD be
first without any modifying conditions, then repeated under each
the modifying conditions separately. To preserve the ability
compare the results of these tests, any IP PDUs that are required
generate the modifying conditions (excluding management queries)
be included in the same data stream as that of the normal test
PDUs and in place of one of the test IP PDUs. They MUST not
supplied to the SUT on a separate network port
2.10.1. Management IP
Most ATM data networks now make use of ILMI, signaling and OAM.
many environments, there can be a number of management
sending queries to the same SUT at the same time
Management queries MUST be made in accordance with the
specification, e.g., ILMI sysUpTime getNext requests will be made
accordance with ILMI 4.0. The response to the query MUST be
by the test equipment. Note that, for each management protocol
Dunn & Martin Informational [Page 9]
RFC 3116 Methodology for ATM Benchmarking June 2001
use, this requires that the test equipment implement the
protocol state machine. One example of the specific query IP
(ICMP) that should be used is shown in Appendix C
2.10.2. Routing update IP
The processing of PNNI updates could have a significant impact on
ability of a switch to forward cells and complete calls. If PNNI
configured on the SUT, one routing update MUST be transmitted
the first test IP PDU is transmitted during the trial. The
SHOULD verify that the SUT has properly processed the routing update
PNNI routing update IP PDUs SHOULD be sent at the rate specified
Appendix B. Appendix C defines one routing update PDU for the TCP/
over ATM example. The routing updates are designed to change
routing on a number of networks that are not involved in
forwarding of the test data. The first IP PDU sets the routing
state to "A", the second one changes the state to "B". The IP
MUST be alternated during the trial. The test SHOULD verify that
SUT has properly processed the routing update
2.11.
Filters are added to switches to selectively inhibit the
of cells that would normally be forwarded. This is usually done
implement security controls on the data that is accepted between
area and another. Different products have different capabilities
implement filters. Filters are applicable only if the SUT
the filtering feature
The SUT SHOULD be first configured to add one filter condition
the tests performed. This filter SHOULD permit the forwarding of
test data stream. This filter SHOULD be of the form as described
the SUT Users Guide
The SUT SHOULD be then reconfigured to implement a total of 25
filters. The first 24 of these filters SHOULD be based on 24
separate ATM NSAP Network Prefix addresses
The 24 ATM NSAP Network Prefix addresses SHOULD not be any that
represented in the test data stream. The last filter SHOULD
the forwarding of the test data stream. By "first" and "last"
mean to ensure that in the second case, 25 conditions must be
before the data IP over ATM PDUs will match the conditions
permit the forwarding of the IP PDU. Of course, if the SUT
the filters or does not use a linear scan of the filter rules
effect of the sequence in which the filters are input is
lost
Dunn & Martin Informational [Page 10]
RFC 3116 Methodology for ATM Benchmarking June 2001
The exact filters configuration command lines used SHOULD be
with the report of the results
2.11.1. Filter
Two sets of filter addresses are required, one for the single
case and one for the 25 filter case
The single filter case should permit traffic from ATM address [
Network Prefix] 00 00 00 00 00 01 00 to ATM address [Switch
Prefix] 00 00 00 00 00 02 00 and deny all other traffic. Note
the 13 octet Switch Network Prefix MUST be configured before
test can be run
The 25 filter case should follow the following sequence
deny [Switch Network Prefix] 00 00 00 00 00 01 00
to [Switch Network Prefix] 00 00 00 00 00 03 00
deny [Switch Network Prefix] 00 00 00 00 00 01 00
to [Switch Network Prefix] 00 00 00 00 00 04 00
deny [Switch Network Prefix] 00 00 00 00 00 01 00
to [Switch Network Prefix] 00 00 00 00 00 05 00
...
deny [Switch Network Prefix] 00 00 00 00 00 01 00
to [Switch Network Prefix] 00 00 00 00 00 0C 00
deny [Switch Network Prefix] 00 00 00 00 00 01 00
to [Switch Network Prefix] 00 00 00 00 00 0D 00
allow [Switch Network Prefix] 00 00 00 00 00 01 00
to [Switch Network Prefix] 00 00 00 00 00 02 00
deny [Switch Network Prefix] 00 00 00 00 00 01 00
to [Switch Network Prefix] 00 00 00 00 00 0E 00
deny [Switch Network Prefix] 00 00 00 00 00 01 00
to [Switch Network Prefix] 00 00 00 00 00 0F 00
...
deny [Switch Network Prefix] 00 00 00 00 00 01 00
to [Switch Network Prefix] 00 00 00 00 00 18 00
deny all
All previous filter conditions should be cleared from the
before this sequence is entered. The sequence is selected to test
see if the switch sorts the filter conditions or accepts them in
order that they were entered. Both of these procedures will
in a greater impact on performance than will some form of
coding
Dunn & Martin Informational [Page 11]
RFC 3116 Methodology for ATM Benchmarking June 2001
2.12. Protocol
It is easier to implement these tests using a single logical
of data, with one source ATM address and one destination ATM address
and for some conditions like the filters described above, a
requirement. Networks in the real world are not limited to
streams of data. The test suite SHOULD be first run with a
ATM source and destination address pair. The tests SHOULD then
repeated with using a random destination address. In the case
testing single switches, the addresses SHOULD be random and
distributed over a range of 256 seven octet user parts. In the
of testing multiple interconnected switches, the addresses SHOULD
random and uniformly distributed over the 256 network prefixes,
of which should support 256 seven octet user parts. The
address ranges to use for ATM are shown in Appendix A. IP to
address mapping MUST be accomplished as described in RFC 2225.
2.13. Route Set
It is not reasonable that all of the routing information necessary
forward the test stream, especially in the multiple address case
will be manually set up. If PNNI and/or ILMI are running, at
start of each trial a routing update MUST be sent to the SUT.
routing update MUST include all of the ATM addresses that will
required for the trial. This routing update will have to be
at the interval required by PNNI or ILMI. An example of the
and repetition interval of the update IP PDUs is given in Appendix
(interval and size) and Appendix C (format).
2.14. Bidirectional
Bidirectional performance tests SHOULD be run with the same data
being offered from each direction. The sum of the data rates
not exceed the theoretical limit for the media
2.15. Single stream
The full suite of tests SHOULD be run with the appropriate
for a single receive and transmit port on the SUT. If the
design of the SUT has multiple distinct pathways, for example
multiple interface cards each with multiple network ports, then
possible permutations of pathways SHOULD be tested separately.
multiple interconnected switches are tested, the test MUST
routes, which allow only one path between source and destination
addresses
Dunn & Martin Informational [Page 12]
RFC 3116 Methodology for ATM Benchmarking June 2001
2.16. Multi-
Many switch products provide several network ports on the
interface module. Each port on an interface module SHOULD
stimulated in an identical manner. Specifically, half of the
on each module SHOULD be receive ports and half SHOULD be
ports. For example if a SUT has two interface module each of
has four ports, two ports on each interface module be receive
and two will be transmit ports. Each receive port MUST be
the same data rate. The addresses in the input data streams
be set so that an IP PDU will be directed to each of the
ports in sequence. That is, all transmit ports will receive
identical distribution of IP PDUs from a particular receive port
Consider the following 6 port SUT
--------------
---------| Rx A Tx X|--------
---------| Rx B Tx Y|--------
---------| Rx C Tx Z|--------
--------------
The addressing of the data streams for each of the inputs SHOULD be
stream sent to Rx A
IP PDU to Tx X, IP PDU to Tx Y, IP PDU to Tx
stream sent to Rx B
IP PDU to Tx X, IP PDU to Tx Y, IP PDU to Tx
stream sent to Rx
IP PDU to Tx X, IP PDU to Tx Y, IP PDU to Tx
Note: Each stream contains the same sequence of IP
addresses; therefore, each transmit port will receive 3 IP
simultaneously. This procedure ensures that the SUT will have
process multiple IP PDUs addressed to the same transmit
simultaneously
The same configuration MAY be used to perform a bi-
multi-stream test. In this case all of the ports are considered
receive and transmit ports. Each data stream MUST consist of IP
whose addresses correspond to the ATM addresses all of the
ports
Dunn & Martin Informational [Page 13]
RFC 3116 Methodology for ATM Benchmarking June 2001
2.17. Multiple
This document does not address the issue of testing the effects of
mixed protocol environment other than to suggest that if such
are wanted then PDUs SHOULD be distributed between all of the
protocols. The distribution MAY approximate the conditions on
network in which the SUT would be used
2.18. Multiple IP PDU
This document does not address the issue of testing the effects of
mixed IP PDU size environment other than to suggest that, if
tests are required, then IP PDU size SHOULD be evenly
among all of the PDU sizes listed in this document. The
MAY approximate the conditions on the network in which the SUT
be used
2.19. Testing beyond a single
In the performance testing of a single SUT, the paradigm can
described as applying some input to a SUT and monitoring the output
The results of which can be used to form a basis of
of that device under those test conditions
This model is useful when the test input and output are
(e.g., 64-byte IP, AAL5 PDUs into the SUT; 64 byte IP, AAL5
out).
By extending the single SUT test model, reasonable
regarding multiple SUTs or heterogeneous environments may
collected. In this extension, the single SUT is replaced by a
of interconnected network SUTs. This test methodology would
the benchmarking of a variety of device/media/service/
combinations. For example, a configuration for a LAN-to-WAN-to-
test might be
(1) ATM UNI -> SUT 1 -> BISUP -> SUT 2 -> ATM
Or an extended LAN configuration might be
(2) ATM UNI -> SUT 1 -> PNNI Network -> SUT 2 -> ATM
In both examples 1 and 2, end-to-end benchmarks of each system
be empirically ascertained. Other behavior may be
through the use of intermediate devices. In example 2,
configuration may be used to give an indication of the effect of
routing on IP throughput
Dunn & Martin Informational [Page 14]
RFC 3116 Methodology for ATM Benchmarking June 2001
Because multiple SUTs are treated as a single system, there
limitations to this methodology. For instance, this methodology
yield an aggregate benchmark for a tested system. That
alone, however, may not necessarily reflect asymmetries in
between the SUTs, latencies introduced by other apparatus (e.g.,
CSUs/DSUs, switches), etc
Further, care must be used when comparing benchmarks of
systems by ensuring that the SUTs' features and configuration of
tested systems have the appropriate common denominators to
comparison
2.20. Maximum IP PDU
The maximum IP PDU rates that should be used when testing
connections SHOULD be the listed theoretical maximum rate for the
PDU size on the media
The maximum IP PDU rate that should be used when testing
connections SHOULD be greater than the listed theoretical
rate for the IP PDU size on that speed connection. The higher
for WAN tests is to compensate for the fact that some vendors
various forms of header compression
A list of maximum IP PDU rates for LAN connections is included
Appendix B
2.21. Bursty
It is convenient to measure the SUT performance under steady
load; however, this is an unrealistic way to gauge the functioning
a SUT. Actual network traffic normally consists of bursts of
PDUs
Some of the tests described below SHOULD be performed with
constant bit rate, bursty Unspecified Bit Rate (UBR) Best
[AF-TM4.1] and Variable Bit Rate Non-real Time (VBR-nrt) Best
[AF-TM4.1]. The IP PDUs within a burst are transmitted with
minimum legitimate inter-IP PDU gap
The objective of the test is to determine the minimum
between bursts that the SUT can process with no IP PDU loss.
SHOULD be run with burst sizes of 10% of Maximum Burst Size (MBS),
20% of MBS, 50% of MBS and 100% MBS. Note that the number of IP
in each burst will depend on the PDU size. For UBR, the MBS
to the associated VBR traffic parameters
Dunn & Martin Informational [Page 15]
RFC 3116 Methodology for ATM Benchmarking June 2001
2.22. Trial
A particular test consists of multiple trials. Each trial
one piece of information, for example the loss rate at a
input IP PDU rate. Each trial consists of five of phases
a) If the SUT is a switch supporting PNNI, send the routing update
the SUT receive port and wait two seconds to be sure that
routing has settled
b) Send an ATM ARP PDU to determine the ATM address corresponding
the destination IP address. The formats of the ATM ARP PDU
should be used are shown in the Test Frame Formats document
MUST be in accordance with RFC 2225.
c) Stimulate SUT with traffic load
d) Wait for two seconds for any residual IP PDUs to be received
e) Wait for at least five seconds for the SUT to restabilize
2.23. Trial
The objective of the tests defined in this document is to
characterize the behavior of a particular piece of network
under varying traffic loads. The choice of test duration must be
compromise between this objective and keeping the duration of
benchmarking test suite within reasonable bounds. The SUT SHOULD
stimulated for at least 60 seconds. If this time period results in
high variance in the test results, the SUT SHOULD be stimulated
at least 300 seconds
2.24. Address
The SUT MUST be able to respond to address resolution requests
by another SUT, an ATM ARP server or the test equipment in
with RFC 2225.
2.25. Synchronized Payload Bit Pattern
Some measurements assume that both the transmitter and
payload information is synchronized. Synchronization MUST
achieved by supplying a known bit pattern to both the transmitter
receiver. This bit pattern MUST be one of the following: PRBS-15,
PRBS-23, 0xFF00, or 0xAA55.
Dunn & Martin Informational [Page 16]
RFC 3116 Methodology for ATM Benchmarking June 2001
2.26. Burst Traffic Descriptors
Some measurements require busty traffic patterns. These
MUST conform to one of the following traffic descriptors
1) PCR=100% allotted line rate, SCR=50% allotted line rate, and MBS=8192
2) PCR=100% allotted line rate, SCR=50% allotted line rate, and MBS=4096
3) PCR=90% allotted line rate, SCR=50% allotted line rate, and MBS=8192
4) PCR=90% allotted line rate, SCR=50% allotted line rate, and MBS=4096
5) PCR=90% allotted line rate, SCR=45% allotted line rate, and MBS=8192
6) PCR=90% allotted line rate, SCR=45% allotted line rate, and MBS=4096
7) PCR=80% allotted line rate, SCR=40% allotted line rate, and MBS=65536
8) PCR=80% allotted line rate, SCR=40% allotted line rate, and MBS=32768
The allotted line rate refers to the total available line
divided by the number of VCCs in use
3. Performance
3.1. Physical Layer-
3.1.1. Pointer
3.1.1.1. Pointer Movement Propagation
Objective: To determine that the SUT does not propagate
movements as defined in RFC 2761 "Terminology for ATM Benchmarking".
Procedure
1) Set up the SUT and test device using the uni-
configuration
2) Send a specific number of IP PDUs at a specific rate through
SUT. Since this test is not a throughput test, the rate
not be greater than 90% of line rate. The cell payload
contain valid IP PDUs. The IP PDUs MUST be encapsulated in AAL5.
Dunn & Martin Informational [Page 17]
RFC 3116 Methodology for ATM Benchmarking June 2001
3) Count the IP PDUs that are transmitted by the SUT to
connectivity and load. If the count on the test device is
same on the SUT, continue the test, else lower the test
traffic rate until the counts are the same
4) Inject one forward payload pointer movement. Verify that the
does not change the pointer
5) Inject one forward payload pointer movement every 1 second
Verify that the SUT does not change the pointer
6) Discontinue the payload pointer movement
7) Inject five forward payload pointer movements every 1 second
Verify that the SUT does not change the pointer
8) Discontinue the payload pointer movement
9) Inject one backward payload pointer movement. Verify that
SUT does not change the pointer
10) Inject one backward payload pointer movement every 1 second
Verify that the SUT does not change the pointer
11) Discontinue the payload pointer movement
12) Inject five backward payload pointer movements every 1 second
Verify that the SUT does not change the pointer
13) Discontinue the payload pointer movement
Reporting Format
The results of the pointer movement propagation test SHOULD
reported in a form of a table. The rows SHOULD be labeled
pointer movement, one pointer movement per second, and
pointer movements per second. The columns SHOULD be
pointer movement and loss of pointer. The elements of the
SHOULD be either True or False, indicating whether the
condition was observed for each test
The table MUST also indicate the IP PDU size in octets and
rate in IP PDUs per second as generated by the test device
Dunn & Martin Informational [Page 18]
RFC 3116 Methodology for ATM Benchmarking June 2001
3.1.1.2. Cell Loss due to Pointer Movement
Objective: To determine if the SUT will drop cells due to
movements as defined in RFC 2761 "Terminology for ATM Benchmarking".
Procedure
1) Set up the SUT and test device using the uni-
configuration
2) Send a specific number of cells at a specific rate through
SUT. Since this test is not a throughput test, the rate
not be greater than 90% of line rate. The cell payload
contain valid IP PDUs. The IP PDUs MUST be encapsulated in AAL5.
3) Count the cells that are transmitted by the SUT to
connectivity and load. If the count on the test device is
same on the SUT, continue the test; else lower the test
traffic rate until the counts are the same
4) Inject one forward payload pointer movement. Verify that the
does not drop any cells
5) Inject one forward payload pointer movement every 1 second
Verify that the SUT does not drop any cells
6) Discontinue the payload pointer movement
7) Inject five forward payload pointer movements every 1 second
Verify that the SUT does not drop any cells
8) Discontinue the payload pointer movement
9) Inject one backward payload pointer movement. Verify that
SUT does not drop any cells
10) Inject one backward payload pointer movement every 1 second
Verify that the SUT does not drop any cells
11) Discontinue the payload pointer movement
12) Inject five backward payload pointer movements every 1 second
Verify that the SUT does not drop any cells
13) Discontinue the payload pointer movement
Dunn & Martin Informational [Page 19]
RFC 3116 Methodology for ATM Benchmarking June 2001
Reporting Format
The results of the cell loss due to pointer movement test
be reported in a form of a table. The rows SHOULD be
single pointer movement, one pointer movement per second, and
pointer movements per second. The columns SHOULD be labeled
loss and number of cells lost. The elements of column 1 SHOULD
either True or False, indicating whether the particular
was observed for each test. The elements of column 2 SHOULD
non-negative integers
The table MUST also indicate the traffic rate in IP PDUs
second as generated by the test device
3.1.1.3. IP Packet Loss due to Pointer Movement
Objective: To determine if the SUT will drop IP packets due
pointer movements as defined in RFC 2761 "Terminology for
Benchmarking".
Procedure
1) Set up the SUT and test device using the uni-
configuration
2) Send a specific number of IP packets at a specific rate
the SUT. Since this test is not a throughput test, the
should not be greater than 90% of line rate. The IP PDUs MUST
encapsulated in AAL5.
3) Count the IP packets that are transmitted by the SUT to
connectivity and load. If the count on the test device is
same on the SUT, continue the test; else lower the test
traffic rate until the counts are the same
4) Inject one forward payload pointer movement. Verify that the
does not drop any packets
5) Inject one forward payload pointer movement every 1 second
Verify that the SUT does not drop any packets
6) Discontinue the payload pointer movement
7) Inject five forward payload pointer movements every 1 second
Verify that the SUT does not drop any packets
8) Discontinue the payload pointer movement
Dunn & Martin Informational [Page 20]
RFC 3116 Methodology for ATM Benchmarking June 2001
9) Inject one backward payload pointer movement. Verify that
SUT does not drop any packets
10) Inject one backward payload pointer movement every 1 second
Verify that the SUT does not drop any packets
11) Discontinue the payload pointer movement
12) Inject five backward payload pointer movements every 1 second
Verify that the SUT does not drop any packets
13) Discontinue the payload pointer movement
Reporting Format
The results of the IP packet loss due to pointer movement
SHOULD be reported in a form of a table. The rows SHOULD
labeled single pointer movement, one pointer movement per second
and five pointer movements per second. The columns SHOULD
labeled packet loss and number of packets lost. The elements
column 1 SHOULD be either True or False, indicating whether
particular condition was observed for each test. The elements
column 2 SHOULD be non-negative integers
The table MUST also indicate the packet size in octets and
rate in packets per second as generated by the test device
3.1.2. Transport Overhead (TOH) Error
3.1.2.1. TOH Error Propagation
Objective: To determine that the SUT does not propagate TOH errors
defined in RFC 2761 "Terminology for ATM Benchmarking".
Procedure
1) Set up the SUT and test device using the uni-
configuration
2) Send a specific number of IP PDUs at a specific rate through
SUT. Since this test is not a throughput test, the rate
not be greater than 90% of line rate. The cell payload
contain valid IP PDUs. The IP PDUs MUST be encapsulated in AAL5.
3) Count the IP PDUs that are transmitted by the SUT to
connectivity and load. If the count on the test device is
same on the SUT, continue the test, else lower the test
traffic rate until the counts are the same
Dunn & Martin Informational [Page 21]
RFC 3116 Methodology for ATM Benchmarking June 2001
4) Inject one error in the first bit of the A1 and A2 Frameword
Verify that the SUT does not propagate the error
5) Inject one error in the first bit of the A1 and A2
every 1 second. Verify that the SUT does not propagate
error
6) Discontinue the Frameword error
7) Inject one error in the first bit of the A1 and A2 Frameword
4 consecutive IP PDUs in every 6 IP PDUs. Verify that the
indicates Loss of Frame
8) Discontinue the Frameword error
Reporting Format
The results of the TOH error propagation test SHOULD be
in a form of a table. The rows SHOULD be labeled single error
one error per second, and four consecutive errors every 6 IP PDUs
The columns SHOULD be labeled error propagated and loss of IP PDU
The elements of the table SHOULD be either True or False
indicating whether the particular condition was observed for
test
The table MUST also indicate the IP PDU size in octets and
rate in IP PDUs per second as generated by the test device
3.1.2.2. c TOH Error
Objective: To determine if the SUT will drop cells due TOH Errors
defined in RFC 2761 "Terminology for ATM Benchmarking".
Procedure
1) Set up the SUT and test device using the uni-
configuration
2) Send a specific number of cells at a specific rate through
SUT. Since this test is not a throughput test, the rate
not be greater than 90% of line rate. The cell payload
contain valid IP PDUs. The IP PDUs MUST be encapsulated in AAL5.
3) Count the cells that are transmitted by the SUT to
connectivity and load. If the count on the test device is
same on the SUT, continue the test; else lower the test
traffic rate until the counts are the same
Dunn & Martin Informational [Page 22]
RFC 3116 Methodology for ATM Benchmarking June 2001
4) Inject one error in the first bit of the A1 and A2 Frameword
Verify that the SUT does not drop any cells
5) Inject one error in the first bit of the A1 and A2
every 1 second. Verify that the SUT does not drop any cells
6) Discontinue the Frameword error
7) Inject one error in the first bit of the A1 and A2 Frameword
4 consecutive IP PDUs in every 6 IP PDUs. Verify that the
does drop cells
8) Discontinue the Frameword error
Reporting Format
The results of the Cell Loss due to TOH errors test SHOULD
reported in a form of a table. The rows SHOULD be labeled
error, one error per second, and four consecutive errors every 6
IP PDUs. The columns SHOULD be labeled cell loss and number
cells lost. The elements of column 1 SHOULD be either True
False, indicating whether the particular condition was
for each test. The elements of column 2 SHOULD be non-
integers
The table MUST also indicate the traffic rate in IP PDUs
second as generated by the test device
3.1.2.3. IP Packet Loss due to TOH Error
Objective: To determine if the SUT will drop IP packets due to
errors as defined in RFC 2761 "Terminology for ATM Benchmarking".
Procedure
1) Set up the SUT and test device using the uni-
configuration
2) Send a specific number of IP packets at a specific rate
the SUT. Since this test is not a throughput test, the
should not be greater than 90% of line rate. The IP PDUs MUST
encapsulated in AAL5.
3) Count the IP packets that are transmitted by the SUT to
connectivity and load. If the count on the test device is
same on the SUT, continue the test; else lower the test
traffic rate until the counts are the same
Dunn & Martin Informational [Page 23]
RFC 3116 Methodology for ATM Benchmarking June 2001
4) Inject one error in the first bit of the A1 and A2 Frameword
Verify that the SUT does not drop any packets
5) Inject one error in the first bit of the A1 and A2
every 1 second. Verify that the SUT does not drop any packets
6) Discontinue the Frameword error
7) Inject one error in the first bit of the A1 and A2 Frameword
4 consecutive IP PDUs in every 6 IP PDUs. Verify that the
does drop packets
8) Discontinue the Frameword error
Reporting Format
The results of the IP packet loss due to TOH errors test SHOULD
reported in a form of a table. The rows SHOULD be labeled
error, one error per second, and four consecutive errors every 6
IP PDUs. The columns SHOULD be labeled packet loss and number
packets lost. The elements of column 1 SHOULD be either True
False, indicating whether the particular condition was
for each test. The elements of column 2 SHOULD be non-
integers
The table MUST also indicate the packet size in octets and
rate in packets per second as generated by the test device
3.1.3. Path Overhead (POH) Error
3.1.3.1. POH Error Propagation
Objective: To determine that the SUT does not propagate POH errors
defined in RFC 2761 "Terminology for ATM Benchmarking".
Procedure
1) Set up the SUT and test device using the uni-
configuration
2) Send a specific number of IP PDUs at a specific rate through
SUT. Since this test is not a throughput test, the rate
not be greater than 90% of line rate. The cell payload
contain valid IP PDUs. The IP PDUs MUST be encapsulated in AAL5.
Dunn & Martin Informational [Page 24]
RFC 3116 Methodology for ATM Benchmarking June 2001
3) Count the IP PDUs that are transmitted by the SUT to
connectivity and load. If the count on the test device is
same on the SUT, continue the test, else lower the test
traffic rate until the counts are the same
4) Inject one error in the B3 (Path BIP8) byte. Verify that the
does not propagate the error
5) Inject one error in the B3 byte every 1 second. Verify that
SUT does not propagate the error
6) Discontinue the POH error
Reporting Format
The results of the POH error propagation test SHOULD be
in a form of a table. The rows SHOULD be labeled single
and one error per second. The columns SHOULD be labeled
propagated and loss of IP PDU. The elements of the table
be either True or False, indicating whether the
condition was observed for each test
The table MUST also indicate the IP PDU size in octets
traffic rate in IP PDUs per second as generated by the
device
3.1.3.2. Cell Loss due to POH Error
Objective: To determine if the SUT will drop cells due POH Errors
defined in RFC 2761 "Terminology for ATM Benchmarking".
Procedure
1) Set up the SUT and test device using the uni-
configuration
2) Send a specific number of cells at a specific rate through
SUT. Since this test is not a throughput test, the rate
not be greater than 90% of line rate. The cell payload
contain valid IP PDUs. The IP PDUs MUST be encapsulated in AAL5.
3) Count the cells that are transmitted by the SUT to
connectivity and load. If the count on the test device is
same on the SUT, continue the test; else lower the test
traffic rate until the counts are the same
4) Inject one error in the B3 (Path BIP8) byte. Verify that the
does not drop any cells
Dunn & Martin Informational [Page 25]
RFC 3116 Methodology for ATM Benchmarking June 2001
5) Inject one error in the B3 byte every 1 second. Verify that
SUT does not drop any cells
6) Discontinue the POH error
Reporting Format
The results of the Cell Loss due to POH errors test SHOULD
reported in a form of a table. The rows SHOULD be labeled
error and one error per second. The columns SHOULD be
cell loss and number of cells lost. The elements of column 1
SHOULD be either True or False, indicating whether the
condition was observed for each test. The elements of column 2
SHOULD be non-negative integers
The table MUST also indicate the traffic rate in IP PDUs
second as generated by the test device
3.1.3.3. IP Packet Loss due to POH Error
Objective: To determine if the SUT will drop IP packets due to
errors as defined in RFC 2761 "Terminology for ATM Benchmarking".
Procedure
1) Set up the SUT and test device using the uni-
configuration
2) Send a specific number of IP packets at a specific rate
the SUT. Since this test is not a throughput test, the
should not be greater than 90% of line rate. The IP PDUs MUST
encapsulated in AAL5.
3) Count the IP packets that are transmitted by the SUT to
connectivity and load. If the count on the test device is
same on the SUT, continue the test; else lower the test
traffic rate until the counts are the same
4) Inject one error in the B3 (Path BIP8) byte. Verify that the
does not drop any packets
5) Inject one error in the B3 byte every 1 second. Verify that
SUT does not drop any packets
6) Discontinue the POH error
Dunn & Martin Informational [Page 26]
RFC 3116 Methodology for ATM Benchmarking June 2001
Reporting Format
The results of the IP packet loss due to POH errors test SHOULD
reported in a form of a table. The rows SHOULD be labeled
error and one error per second. The columns SHOULD be
packet loss and number of packets lost. The elements of column 1
SHOULD be either True or False, indicating whether the
condition was observed for each test. The elements of column 2
SHOULD be non-negative integers
The table MUST also indicate the packet size in octets and
rate in packets per second as generated by the test device
3.2. ATM
3.2.1. Two-Point Cell Delay Variation (CDV
3.2.1.1. Test
The cell delay measurements assume that both the transmitter
receiver timestamp information is synchronized.
SHOULD be achieved by supplying a common clock signal (minimum of 100
Mhz or 10 ns resolution) to both the transmitter and receiver.
maximum timestamp values MUST be recorded to ensure
in the case of counter rollover. The cell delay measurements
utilize the O.191 cell (ITUT-O.191) encapsulated in a valid
packet. If the O.191 cell is not available, a test cell
in a valid IP packet MAY be used. The test cell MUST contain
transmit timestamp which can be correlated with a receive timestamp
A description of the test cell MUST be included in the test results
The description MUST include the timestamp length (in bits),
rollover value, and the timestamp accuracy (in ns).
3.2.1.2. Two-point CDV/Steady Load/One
Objective: To determine the SUT variation in cell transfer delay
one VCC as defined in RFC 2761 "Terminology for ATM Benchmarking".
Procedure
1) Set up the SUT and test device using the bi-
configuration
2) Configure the SUT and test device with one VCC. The VCC
contain one VPI/VCI. The VCC MUST be configured as either a CBR
VBR, or UBR connection. The VPI/VCI MUST not be one of
reserved ATM signaling channels (e.g., [0,5], [0,16]).
Dunn & Martin Informational [Page 27]
RFC 3116 Methodology for ATM Benchmarking June 2001
3) Send a specific number of IP packets containing timestamps at
specific constant rate through the SUT via the defined test VCC
Since this test is not a throughput test, the rate should not
greater than 90% of line rate. The IP PDUs MUST be
in AAL5.
4) Count the IP packets that are transmitted by the SUT to
connectivity and load. If the count on the test device is
same on the SUT, continue the test; else lower the test
traffic rate until the counts are the same
5) Record the packets timestamps at the transmitter and
ends of the test device
Reporting Format
The results of the Two-point CDV/Steady Load/One VCC test
be reported in a form of text, graph, and histogram
The text results SHOULD display the numerical values of the CDV
The values given SHOULD include: time period of test in s,
VPI/VCI value, total number of cells transmitted and received
the given VPI/VCI during the test in positive integers,
and minimum CDV during the test in us, and peak-to-peak CDV in us
The graph results SHOULD display the cell delay values. The x
coordinate SHOULD be the test run time in either seconds,
or days depending on the total length of the test. The x
coordinate time SHOULD be configurable. The y-coordinate
be the cell delay in us. The integration time per point MUST
indicated
The histogram results SHOULD display the peak-to-peak cell delay
The x-coordinate SHOULD be the cell delay in us with at least 256
bins. The y-coordinate SHOULD be the number of cells observed
each bin
The results MUST also indicate the packet size in octets,
rate in packets per second, and bearer class as generated by
test device. The VCC and VPI/VCI values MUST be indicated.
bearer class of the created VCC MUST also be indicated
3.2.1.3. Two-point CDV/Steady Load/Twelve
Objective: To determine the SUT variation in cell transfer delay
twelve VCCs as defined in RFC 2761 "Terminology for
Benchmarking".
Dunn & Martin Informational [Page 28]
RFC 3116 Methodology for ATM Benchmarking June 2001
Procedure
1) Set up the SUT and test device using the bi-
configuration
2) Configure the SUT and test device with twelve VCCs, using 1
and 12 VCIs. The VCC's MUST be configured as either a CBR, VBR
or UBR connection. The VPI/VCIs MUST not be one of the
ATM signaling channels (e.g., [0,5], [0,16]).
3) Send a specific number of IP packets containing timestamps at
specific constant rate through the SUT via the defined test VCCs
All of the VPI/VCI pairs will generate traffic at the
traffic rate. Since this test is not a throughput test, the
should not be greater than 90% of line rate. The IP PDUs MUST
encapsulated in AAL5.
4) Count the IP packets that are transmitted by the SUT on all
to verify connectivity and load. If the count on the test
is the same on the SUT, continue the test; else lower the
device traffic rate until the counts are the same
5) Record the packets timestamps at the transmitter and
ends of the test device for all VCCs
Reporting Format
The results of the Two-point CDV/Steady Load/Twelve VCCs
SHOULD be reported in a form of text, graph, and histograms
The text results SHOULD display the numerical values of the CDV
The values given SHOULD include: time period of test in s,
VPI/VCI values, total number of cells transmitted and received
each VCC during the test in positive integers, maximum and
CDV on each VCC during the test in us, and peak-to-peak CDV
each VCC in us
The graph results SHOULD display the cell delay values. The x
coordinate SHOULD be the test run time in either seconds,
or days depending on the total length of the test. The x
coordinate time SHOULD be configurable. The y-coordinate
be the cell delay for each VCC in ms. There SHOULD be 12
on the graph, one curves indicated and labeled for each VCC.
integration time per point MUST be indicated
Dunn & Martin Informational [Page 29]
RFC 3116 Methodology for ATM Benchmarking June 2001
The histograms SHOULD display the peak-to-peak cell delay.
will be one histogram for each VCC. The x-coordinate SHOULD
the cell delay in us with at least 256 bins. The y-
SHOULD be the number of cells observed in each bin
The results MUST also indicate the packet size in octets,
rate in packets per second, and bearer class as generated by
test device. The VCC and VPI/VCI values MUST be indicated.
bearer class of the created VCC MUST also be indicated
3.2.1.4. Two-point CDV/Steady Load/Maximum
Objective: To determine the SUT variation in cell transfer delay
the maximum number VCCs supported on the SUT as defined in RFC 2761
"Terminology for ATM Benchmarking".
Procedure
1) Set up the SUT and test device using the bi-
configuration
2) Configure the SUT and test device with the maximum number of
supported on the SUT. For example, if the maximum number of
supported on the SUT is 1024, define 256 VPIs with 4 VCIs
VPI. The VCC's MUST be configured as either a CBR, VBR, or
connection. The VPI/VCIs MUST not be one of the reserved
signaling channels (e.g., [0,5], [0,16]).
3) Send a specific number of IP packets containing timestamps at
specific constant rate through the SUT via the defined test VCCs
All of the VPI/VCI pairs will generate traffic at the
traffic rate. Since this test is not a throughput test, the
should not be greater than 90% of line rate. The IP PDUs MUST
encapsulated in AAL5.
4) Count the IP packets that are transmitted by the SUT on all
to verify connectivity and load. If the count on the test
is the same on the SUT, continue the test; else lower the
device traffic rate until the counts are the same
5) Record the packets timestamps at the transmitter and
ends of the test device for all VCCs
Reporting Format
The results of the Two-point CDV/Steady Load/Maximum VCCs
SHOULD be reported in a form of text, graphs, and histograms
Dunn & Martin Informational [Page 30]
RFC 3116 Methodology for ATM Benchmarking June 2001
The text results SHOULD display the numerical values of the CDV
The values given SHOULD include: time period of test in s,
VPI/VCI values, total number of cells transmitted and received
each VCC during the test in positive integers, maximum and
CDV on each VCC during the test in us, and peak-to-peak CDV
each VCC in us
The graph results SHOULD display the cell delay values.
will be (Max number of VCCs/10) graphs, with 10 VCCs indicated
each graph. The x-coordinate SHOULD be the test run time
either seconds, minutes or days depending on the total length
the test. The x-coordinate time SHOULD be configurable. The y
coordinate SHOULD be the cell delay for each VCC in us.
SHOULD be no more than 10 curves on each graph, one
indicated and labeled for each VCC. The integration time
point MUST be indicated
The histograms SHOULD display the peak-to-peak cell delay.
will be one histogram for each VCC. The x-coordinate SHOULD
the cell delay in us with at least 256 bins. The y-
SHOULD be the number of cells observed in each bin
The results MUST also indicate the packet size in octets,
rate in packets per second, and bearer class as generated by
test device. The VCC and VPI/VCI values MUST be indicated.
bearer class of the created VCC MUST also be indicated
3.2.1.5. Two-point CDV/Bursty VBR Load/One
Objective: To determine the SUT variation in cell transfer delay
one VCC as defined in RFC 2761 "Terminology for ATM Benchmarking".
Procedure
1) Set up the SUT and test device using the bi-
configuration
2) Configure the SUT and test device with one VCC. The VCC
contain one VPI/VCI. The VCC MUST be configured as either a
or VBR connection. The VPI/VCI MUST not be one of the
ATM signaling channels (e.g., [0,5], [0,16]).
3) Send a specific number of IP packets containing timestamps at
specific VBR through the SUT via the defined test VCC.
this test is not a throughput test, the rate should not
greater than 90% of line rate. The IP PDUs MUST be
in AAL5.
Dunn & Martin Informational [Page 31]
RFC 3116 Methodology for ATM Benchmarking June 2001
4) Count the IP packets that are transmitted by the SUT to
connectivity and load. If the count on the test device is
same on the SUT, continue the test; else lower the test
traffic rate until the counts are the same
5) Record the packets timestamps at the transmitter and
ends of the test device
Reporting Format
The results of the Two-point CDV/Bursty VBR Load/One VCC
SHOULD be reported in a form of text, graph, and histogram
The text results SHOULD display the numerical values of the CDV
The values given SHOULD include: time period of test in s,
VPI/VCI value, total number of cells transmitted and received
the given VPI/VCI during the test in positive integers,
and minimum CDV during the test in us, and peak-to-peak CDV in us
The graph results SHOULD display the cell delay values. The x
coordinate SHOULD be the test run time in either seconds,
or days depending on the total length of the test. The x
coordinate time SHOULD be configurable. The y-coordinate
be the cell delay in us. The integration time per point MUST
indicated
The histogram results SHOULD display the peak-to-peak cell delay
The x-coordinate SHOULD be the cell delay in us with at least 256
bins. The y-coordinate SHOULD be the number of cells observed
each bin
The results MUST also indicate the packet size in octets,
rate in packets per second, and bearer class as generated by
test device. The VCC and VPI/VCI values MUST be indicated.
PCR, SCR, and MBS MUST be indicated. The bearer class of
created VCC MUST also be indicated
3.2.1.6. Two-point CDV/Bursty VBR Load/Twelve
Objective: To determine the SUT variation in cell transfer delay
twelve VCCs as defined in RFC 2761 "Terminology for
Benchmarking".
Procedure
1) Set up the SUT and test device using the bi-
configuration
Dunn & Martin Informational [Page 32]
RFC 3116 Methodology for ATM Benchmarking June 2001
2) Configure the SUT and test device with twelve VCCs, using 1
and 12 VCIs. The VCC's MUST be configured as either a CBR or
connection. The VPI/VCIs MUST not be one of the reserved
signaling channels (e.g., [0,5], [0,16]).
3) Send a specific number of IP packets containing timestamps at
specific VBR through the SUT via the defined test VCCs. All
the VPI/VCI pairs will generate traffic at the same traffic rate
Since this test is not a throughput test, the rate should not
greater than 90% of line rate. The IP PDUs MUST be
in AAL5.
4) Count the IP packets that are transmitted by the SUT on all
to verify connectivity and load. If the count on the test
is the same on the SUT, continue the test; else lower the
device traffic rate until the counts are the same
5) Record the packets timestamps at the transmitter and
ends of the test device for all VCCs
Reporting Format
The results of the Two-point CDV/Bursty VBR Load/Twelve VCCs
SHOULD be 