As per Relevance of the word terminology, we have this rfc below:
Network Working Group S. Bradner,
Request for Comments: 1242 Harvard
July 1991
Benchmarking Terminology for Network Interconnection
Status of this
This memo provides information for the Internet community. It
not specify an Internet standard. Distribution of this memo
unlimited
This memo discusses and defines a number of terms that are used
describing performance benchmarking tests and the results of
tests. The terms defined in this memo will be used in
memos to define specific benchmarking tests and the suggested
to be used in reporting the results of each of the tests. This
is a product of the Benchmarking Methodology Working Group (BMWG)
the Internet Engineering Task Force (IETF).
1.
Vendors often engage in "specsmanship" in an attempt to give
products a better position in the marketplace. This usually
much "smoke & mirrors" used to confuse the user. This memo
follow-up memos attempt to define a specific set of terminology
tests that vendors can use to measure and report the
characteristics of network devices. This will provide the
comparable data from different vendors with which to evaluate
devices
2. Definition
Term to be defined. (e.g., Latency
Definition
The specific definition for the term
Discussion
A brief discussion about the term, it's
and any restrictions on measurement procedures
Measurement units
The units used to report measurements of
term, if applicable
Benchmarking Methodology Working Group [Page 1]
RFC 1242 Benchmarking Terminology July 1991
Issues
List of issues or conditions that effect this term
See Also
List of other terms that are relevant to the
of this term
3. Term
3.1 Back-to-
Definition
Fixed length frames presented at a rate such that
is the minimum legal separation for a given
between frames over a short to medium period of time
starting from an idle state
Discussion
A growing number of devices on a network can
bursts of back-to-back frames. Remote disk
using protocols like NFS, remote disk backup
like rdump, and remote tape access systems can
configured such that a single request can result
a block of data being returned of as much as 64K octets
Over networks like ethernet with a relatively small
this results in many fragments to be transmitted.
fragment reassembly will only be attempted if
fragments have been received, the loss of even
fragment because of the failure of some
network device to process enough continuous frames
cause an endless loop as the sender
attempts to send its large data block
With the increasing size of the Internet,
updates can span many frames, with modern routers
to transmit very quickly. Missing frames of
information can produce false indications
unreachability. Tests of this parameter are
to determine the extent of data buffering in
device
Measurement units
Number of N-octet frames in burst
Issues
See Also
Benchmarking Methodology Working Group [Page 2]
RFC 1242 Benchmarking Terminology July 1991
3.2
Definition
A system which forwards data frames based on
in the data link layer
Discussion
Measurement units
n/
Issues
See Also
bridge/router (3.3)
router (3.15)
3.3 bridge/
Definition
A bridge/router is a network device that can
function as a router and/or a bridge based on
protocol of a specific frame
Discussion
Measurement units
n/
Issues
See Also
bridge (3.2)
router (3.15)
3.4 Constant
Definition
Fixed length frames at a fixed interval time
Discussion
Although it is rare, to say the least, to
a steady state load on a network device in the
world, measurement of steady state performance
be useful in evaluating competing devices.
frame size is specified and constant. All
parameters are constant. When there is a
in the frame, it must be verified
Benchmarking Methodology Working Group [Page 3]
RFC 1242 Benchmarking Terminology July 1991
Measurement units
n/
Issues
unidirectional vs.
See Also
3.5 Data link frame
Definition
The number of octets in the frame from the first
following the preamble to the end of the FCS,
present, or to the last octet of the data if
is no FCS
Discussion
There is much confusion in reporting the
sizes used in testing network devices or
measurement. Some authors include the checksum
some do not. This is a specific definition for
in this and subsequent memos
Measurement units
Issues
See Also
3.6 Frame Loss
Definition
Percentage of frames that should have been
by a network device under steady state (constant
load that were not forwarded due to lack
resources
Discussion
This measurement can be used in reporting
performance of a network device in an
state. This can be a useful indication of how
device would perform under pathological
conditions such as broadcast storms
Measurement units
Percentage of N-octet offered frames that are dropped
To be reported as a graph of offered load vs frame loss
Benchmarking Methodology Working Group [Page 4]
RFC 1242 Benchmarking Terminology July 1991
Issues
See Also
overhead behavior (3.11)
policy based filtering (3.13)
MTU mismatch behavior (3.10)
3.7 Inter Frame
Definition
The delay from the end of a data link frame as
in section 3.5, to the start of the preamble of
next data link frame
Discussion
There is much confusion in reporting the
frame time used in testing network devices.
is a specific definition for use in this and
memos
Measurement units
Time with fine enough units to distinguish
2 events
Issues
Link data rate
See Also
3.8
Definition
For store and forward devices
The time interval starting when the last bit of
input frame reaches the input port and ending
the first bit of the output frame is seen on
output port
For bit forwarding devices
The time interval starting when the end of the
bit of the input frame reaches the input port
ending when the start of the first bit of the
frame is seen on the output port
Discussion
Variability of latency can be a problem
Some protocols are timing dependent (e.g., LAT and IPX).
Future applications are likely to be sensitive
Benchmarking Methodology Working Group [Page 5]
RFC 1242 Benchmarking Terminology July 1991
network latency. Increased device delay can
the useful diameter of net. It is desired
eliminate the effect of the data rate on the
measurement. This measurement should only reflect
actual within device latency. Measurements should
taken for a spectrum of frame sizes without
the device setup
Ideally, the measurements for all devices would be
the first actual bit of the frame after the preamble
Theoretically a vendor could design a device
normally would be considered a store and
device, a bridge for example, that begins
a frame before it is fully received. This type
device is known as a "cut through" device.
assumption is that the device would somehow
the partially transmitted frame if in receiving
remainder of the input frame, something came up
the frame or this specific forwarding of it was
error. For example, a bad checksum. In this case
the device would still be considered a store
forward device and the latency would still
from last bit in to first bit out, even though
value would be negative. The intent is to
the device as a unit without regard to the
structure
Measurement units
Time with fine enough units to distinguish
2 events
Issues
See Also
link speed mismatch (3.9)
constant load (3.4)
back-to-back (3.1)
policy based filtering (3.13)
single frame behavior (3.16)
3.9 Link Speed
Definition
Speed mismatch between input and output data rates
Discussion
This does not refer to frame rate per se, it refers
the actual data rate of the data path. For example
Benchmarking Methodology Working Group [Page 6]
RFC 1242 Benchmarking Terminology July 1991
an Ethernet on one side and a 56KB serial link on
other. This is has also been referred to as the "
hose effect". Networks that make use of serial
between local high speed networks will usually
link speed mismatch at each end of the serial links
Measurement units
Ratio of input and output data rates
Issues
See Also
constant load (3.4)
back-to-back (3.1)
3.10 MTU-mismatch
Definition
The network MTU (Maximum Transmission Unit) of
output network is smaller than the MTU of the
network, this results in fragmentation
Discussion
The performance of network devices can be
affected by having to fragment frames
Measurement units
Description of behavior
Issues
See Also
3.11 Overhead
Definition
Processing done other than that for normal data frames
Discussion
Network devices perform many functions in
to forwarding frames. These tasks range from
hardware testing to the processing of
information and responding to network
requests. It is useful to know what the effect
these sorts of tasks is on the device performance
An example would be if a router were to
forwarding or accepting frames during the
of large routing update for a complex protocol
Benchmarking Methodology Working Group [Page 7]
RFC 1242 Benchmarking Terminology July 1991
OSPF. It would be good to know of this sort
behavior
Measurement units
Any quantitative understanding of this behavior is
the determination of its effect on other measurements
Issues
bridging and routing
control
ip options
error
event logging/statistics
See Also
policy based filtering (3.13)
3.12 Overloaded
Definition
When demand exceeds available system resources
Discussion
Devices in an overloaded state will lose frames.
device might lose frames that contain routing
configuration information. An overloaded state
assumed when there is any frame loss
Measurement units
Description of behavior of device in any
states for both input and output overload conditions
Issues
How well does the device recover from overloaded state
How does source quench production effect device
What does device do when its resources are exhausted
What is response to system management in
state
See Also
3.13 Policy based
Definition
Filtering is the process of discarding
Benchmarking Methodology Working Group [Page 8]
RFC 1242 Benchmarking Terminology July 1991
frames by administrative decision where
operation would be to forward them
Discussion
Many network devices have the ability to
configured to discard frames based on a
of criteria. These criteria can range from
source or destination addresses to
specific fields in the data frame itself
Configuring many network devices to
filtering operations impacts the
of the device
Measurement units
n/
Issues
flexibility of filter
number of filter
See Also
3.14 Restart
Definition
Reinitialization of system causing data loss
Discussion
During a period of time after a power up
reset, network devices do not accept and
frames. The duration of this period of
can be useful in evaluating devices. In addition
some network devices require some form of
when specific setup variables are modified. If
reset period were long it might discourage
managers from modifying these variables on
networks
Measurement units
Description of device behavior under various
conditions
Issues
Types
power
reload software
flush port, reset
restart current code image, without
Benchmarking Methodology Working Group [Page 9]
RFC 1242 Benchmarking Terminology July 1991
Under what conditions is a restart required
Does the device know when restart needed (i.e.,
state timeout)?
Does the device recognize condition of too
auto-restart
Does the device run diagnostics on all or some resets
How may restart be initiated
physical
remote via terminal line or login over
See Also
3.15
Definition
A system which forwards data frames based
information in the network layer
Discussion
This implies "running" the network level
routing algorithm and performing whatever
that the protocol requires. For example,
the TTL field in the TCP/IP header
Measurement units
n/
Issues
See Also
bridge (3.2)
bridge/router (3.3)
3.16 Single frame
Definition
One frame received on the input to a device
Discussion
A data "stream" consisting of a single frame
require a network device to do a lot of processing
Figuring routes, performing ARPs,
permissions etc., in general, setting up cache entries
Devices will often take much more time to process
single frame presented in isolation than it would
the same frame were part of a steady stream.
is a worry that some devices would even discard a
frame as part of the cache setup procedure under
Benchmarking Methodology Working Group [Page 10]
RFC 1242 Benchmarking Terminology July 1991
assumption that the frame is only the first of many
Measurement units
Description of the behavior of the device
Issues
See Also
policy based filtering (3.13)
3.17
Definition
The maximum rate at which none of the offered
are dropped by the device
Discussion
The throughput figure allows vendors to report
single value which has proven to have use in
marketplace. Since even the loss of one frame in
data stream can cause significant delays
waiting for the higher level protocols to time out
it is useful to know the actual maximum
rate that the device can support. Measurements
be taken over a assortment of frame sizes.
measurements for routed and bridged data in
devices that can support both. If there is a
in the received frame, full checksum processing
be done
Measurement units
N-octet input frames per
input bits per
Issues
single path vs.
unidirectional vs
checksum processing required on some
See Also
frame loss rate (3.6)
constant load (3.4)
back-to-back (3.1)
Benchmarking Methodology Working Group [Page 11]
RFC 1242 Benchmarking Terminology July 1991
4.
This memo is a product of the IETF BMWG working group
Chet Birger, Coral
Scott Bradner, Harvard University (chair
Steve Butterfield, independant
Frank Chui,
Phill Gross,
Stev Knowles, FTP Software, Inc
Mat Lew,
Gary Malkin, FTP Software, Inc
K.K. Ramakrishnan, Digital Equipment Corp
Mick Scully, Ungerman
William M. Seifert, Wellfleet Communications Corp
John Shriver, Proteon, Inc
Dick Sterry,
Geof Stone, Network Systems Corp
Geoff Thompson,
Mary Youssef,
Security
Security issues are not discussed in this memo
Author's
Scott
Harvard
William James Hall 1232
33 Kirkland
Cambridge, MA 02138
Phone: (617) 495-3864
EMail: SOB@HARVARD.HARVARD.
Or, send comments to: bmwg@harvisr.harvard.edu
Benchmarking Methodology Working Group [Page 12]
if you see any problems within the linking, don't worry be happy,
this is version 0.1 of the Relevance System and you gotta expect some crappy subroutines sometimes,
just be content we did not write this in Java, which would have made this "bigger and better" HAHAHHA.
RFC documents can be found at I.E.T.F.
Relevance System Copyright © 2002 Spectrum WorldResearch
other technical nosh by ServerMasters Corporation
collaboration of BobX
