As per Relevance of the word experiment, we have this rfc below:
Network Working Group D.L.
Request for Comments: 889 December 1983
Internet Delay
This memo reports on some measurement experiments and suggests some
improvements to the TCP retransmission timeout calculation. This memo
both a status report on the measurements and advice to implementers of TCP
1.
This memorandum describes two series of experiments designed to
the transmission characteristics of the Internet system. One series
experiments was designed to determine the network delays with respect
packet length, while the other was designed to assess the effectiveness of
TCP retransmission-timeout algorithm specified in the standards documents
Both sets of experiments were conducted during the October - November 1983
time frame and used many hosts distributed throughout the Internet system
The objectives of these experiments were first to accumulate
data on actual network paths that could be used as a benchmark of
system performance, and second to apply these data to refine individual
implementations and improve their performance
The experiments were done using a specially instrumented measurement
called a Fuzzball, which consists of an LSI-11 running IP/TCP and
application-layer protocols including TELNET, FTP and SMTP mail. Among
various measurement packages is the original PING (Packet InterNet Groper
program used over the last six years for numerous tests and measurements
the Internet system and its client nets. This program contains facilities
send various kinds of probe packets, including ICMP Echo messages, process
reply and record elapsed times and other information in a data file, as
as produce real-time snapshot histograms and traces
Following an experiment run, the data collected in the file were
by another set of programs and plotted on a Peritek bit-map display with
monitor. The plots have been found invaluable in the indentification
understanding of the causes of netword glitches and other "zoo" phenomena
Finally, summary data were extracted and presented in this memorandum.
raw data files, including bit-map image files of the various plots,
available to other experimenters upon request
The Fuzzballs and their local-net architecture, called DCN, have
two-dozen clones scattered worldwide, including one (DCN1) at the
Corporation offices in McLean, Virginia, and another at the
Telecommunications Adminstration (NTA) near Oslo, Norway. The DCN1
is connected to the ARPANET at the Mitre IMP by means of 1822 Error
Units operating over a 56-Kbps line. The NTA Fuzzball is connected to
NTARE Gateway by an 1822 interface and then via VDH/HAP operating over
9.6-Kbps line to SATNET at the Tanum (Sweden) SIMP. For most
described below, these details of the local connectivity can be ignored,
only relatively small delays are involved
Internet Delay Experiments Page 2
D.L.
The remote test hosts were selected to represent canonical paths in
Internet system and were scattered all over the world. They included some
the ARPANET, MILNET, MINET, SATNET, TELENET and numerous local nets
via these long-haul nets. As an example of the richness of the
system connectivity and the experimental data base, data are included
three different paths from the ARPANET-based measurement host to London hosts
two via different satellite links and one via an undersea cable
2. Packet Length Versus
This set of experiments was designed to determine whether delays
the Internet are significantly influenced by packet length. In cases
the intrinsic propagation delays are high relative to the time to transmit
individual packet, one would expect that delays would not be strongly
by packet length. This is the case with satellite nets, including SATNET
WIDEBAND, but also with terrestrial nets where the degree of
aggregation is high, so that the measured traffic is a small proportion of
total traffic on the path. However, in cases where the intrinsic
delays are low and the measured traffic represents the bulk of the traffic
the path, quite the opposite would be expected
The objective of the experiments was to assess the degree to which
performance could be improved by refining the retransmission-timeout
to include a dependency on packet length. Another objective was to
the nature of the delay characteristic versus packet length on tandem
spanning networks of widely varying architectures, including local-nets
terrestrial long-haul nets and satellite nets
2.1. Experiment
There were two sets of experiments to measure delays as a function
packet length. One of these was based at DCN1, while the other was based
NTA. All experiments used ICMP Echo/Reply messages with embedded timestamps
A cycle consisted of sending an ICMP Echo message of specified length,
for the corresponding ICMP Reply message to come back and recording
elapsed time (normalized to one-way delay). An experiment run, resulting
one line of the table below, consisted of 512 of these volleys
The length of each ICMP message was determined by a random-
generator uniformly distributed between zero and 256. Lengths less than 40
were rounded up to 40, which is the minimum datagram size for an ICMP
containing timestamps and just happens to also be the minimum TCP
size. The maximum length was chosen to avoid complications due
fragmentation and reassembly, since ICMP messages are not
fragmented or reassembled by the gateways
The data collected were first plotted as a scatter diagram on a
bit-map display. For all paths involving the ARPANET, this
revealed two distinct characteristics, one for short (single-packet)
less than 126 octets in length and the other for long (multi-packet)
Internet Delay Experiments Page 3
D.L.
longer than this. Linear regression lines were then fitted to
characteristic with the results shown in the following table. (Only
characteristic was assumed for ARPANET-exclusive paths.) The table shows
each host the delays, in milliseconds, for each type of message along with
rate computed on the basis of these delays. The "Host ID" column
the host at the remote end of the path, with a letter suffix used
necessary to identify a particular run
Internet Delay Experiments Page 4
D.L.
Host Single-packet Rate Multi-packet Rate
ID 40 125 (bps) 125 256 (bps
---------------------------------------------------------------------------
DCN1 to nearby local-net hosts (calibration
DCN5 9 13 366422 DMA 1822
DCN8 14 20 268017
IMP17 22 60 45228 56K 1822/
FORD1 93 274 9540 9600 DDCMP
UMD1 102 473 4663 4800
DCN6 188 550 4782 4800
FACC 243 770 3282 9600/4800
FOE 608 1917 1320 9600/14.4K stat
DCN1 to ARPANET hosts and local
MILARP 61 105 15358 133 171 27769 MILNET
ISID-L 166 263 6989 403 472 15029 low-traffic
SCORE 184 318 5088 541 608 15745 low-traffic
RVAX 231 398 4061 651 740 11781 Purdue local
AJAX 322 578 2664 944 1081 7681 MIT local
ISID-H 333 520 3643 715 889 6029 high-traffic
BERK 336 967 1078 1188 1403 4879 UC
WASH 498 776 2441 1256 1348 11379 U
DCN1 to MILNET/MINET hosts and local
ISIA-L 460 563 6633 1049 1140 11489 low-traffic
ISIA-H 564 841 2447 1275 1635 2910 high-traffic
BRL 560 973 1645 1605 1825 4768 BRL local
LON 585 835 2724 1775 1998 4696 MINET host (London
HAWAII 679 980 2257 1817 1931 9238 a long way
OFFICE3 762 1249 1396 2283 2414 8004 heavily loaded
KOREA 897 1294 1712 2717 2770 19652 a long, long way
DCN1 to TELENET hosts via
RICE 1456 2358 754 3086 3543 2297 via VAN
DCN1 to SATNET hosts and local nets via
UCL 1089 1240 4514 1426 1548 8558 UCL zoo
NTA-L 1132 1417 2382 1524 1838 3339 low-traffic
NTA-H 1247 1504 2640 1681 1811 8078 high-traffic
NTA to SATNET
TANUM 107 368 6625 9600 bps Tanum
ETAM 964 1274 5576 Etam channel
GOONY 972 1256 6082 Goonhilly channel
Internet Delay Experiments Page 5
D.L.
2.2 Analysis of
The data clearly show a strong correlation between delay and length,
the longest packets showing delays two to three times the shortest. On
via ARPANET clones the delay characteristic shows a stonger correlation
length for single-packet messages than for multi-packet messages, which
consistent with a design which favors low delays for short messages and
throughputs for longer ones
Most of the runs were made during off-peak hours. In the few cases
runs were made for a particular host during both on-peak and off-peak hours
comparison shows a greater dependency on packet length than on traffic shift
TCP implementors should be advised that some dependency on packet
may have to be built into the retransmission-timeout estimation algorithm
insure good performance over lossy nets like SATNET. They should also
advised that some Internet paths may require stupendous timeout
ranging to many seconds for the net alone, not to mention additional delays
host-system queues
I call to your attention the fact that the delays (at least for
larger packets) from ARPANET hosts (e.g. DCN1) to MILNET hosts (e.g. ISIA
are in the same ballpark as the delays to SATNET hosts (e.g. UCL)! I
also observed that the packet-loss rates on the MILNET path are at present
neglible (18 in 512 for ISIA-2). Presumably, the loss is in the gateways
however, there may well be a host or two out there swamping the gateways
retransmitted data and which have a funny idea of the "normal"
interval. The recent discovery of a bug in the TOPS-20 TCP implementation
where spurious ACKs were generated at an alarming rate, would seem to
that suspicion
3. Retransmission-Timeout
One of the basic features of TCP which allow it to be used on
spanning many nets of widely varying delay and packet-loss characteristics
the retranansmission-timeout algorithm, sometimes known as the "
Algorithm" for the original designers. The algorithm operates by
the time and initial sequence number when a segment is transmitted,
computing the elapsed time for that sequence number to be acknowledged.
are various degrees of sophistication in the implementation of the algorithm
ranging from allowing only one such computation to be in progress at a time
allowing one for each segment outstanding at a time on the connection
The retransmission-timeout algorithm is basically an estimation process
It maintains an extimate of the current roundtrip delay time and updates it
new delay samples are computed. The algorithm smooths these samples and
establishes a timeout, which if exceeded causes a retransmission.
selection of the parameters of this algorithm are vitally important in
to provide effective data transmission and avoid abuse of the Internet
by excessive retransmissions. I have long been suspicious of the
Internet Delay Experiments Page 6
D.L.
suggested in the specification and used in some implementations, especially
cases involving long-delay paths involving lossy nets. The experiment
designed to simulate the operation of the algorithm using data collected
real paths involving some pretty leaky Internet plumbing
3.1. Experiment
The experiment data base was constructed of well over a hundred
using ICMP Echo/Reply messages bounced off hosts scattered all over the world
Most runs, including all those summarized here, consisted of 512 echo/
cycles lasting from several seconds to twenty minutes or so. Other
designed to detect network glitches lasted several hours. Some runs
packets of constant length, while others used different lengths
from 40 to 256 octets. The maximum length was chosen to avoid
fragmented or reassembled by the gateways
The object of the experiment was to simulate the packet
distribution seen by TCP over the paths measured. Only the network delay
of interest here, not the queueing delays within the hosts themselves,
can be considerable. Also, only a single packet was allowed in flight,
that stress on the network itself was minimal. Some tests were
during busy periods of network activity, while others were conducted
quiet hours
The 512 data points collected during each run were processed by a
which plotted on a color bit-map display each data point (x,y), where
represents the time since initiation of the experiment the and y the
delay, normalized to the one-way delay. Then, the
retransmission-timeout algorithm was run on these data and its
timeout plotted in the same way. The display immediately reveals how
algorithm behaves in the face of varying traffic loads, network glitches,
packets and superfluous retransmissions
Each experiment run also produced summary statistics, which
summarized in the table below. Each line includes the Host ID,
identifies the run. The suffix -1 indicates 40-octet packets, -2
256-octet packets and no suffix indicates uniformly distributed
between 40 and 256. The Lost Packets columns refer to instances when no
Reply message was received for thirty seconds after transmission of the
Echo message, indicating probable loss of one or both messages. The
Packets columns refer to instances when the computed timeout is less than
measured delay, which would result in a superfluous retransmission. For
of these two types of packets the column indicates the number of
and the Time column indicates the total accumulated time required for
recovery action
For reference purposes, the Mean column indicates the computed mean
of the echo/reply cycles, excluding those cycles involving packet loss,
the CoV column indicates the coefficient of variation. Finally, the
Internet Delay Experiments Page 7
D.L.
column indicates the efficiency, computed as the ratio of the total
accumulated while sending good data to this time plus the lost-packet
rtx-packet time
Complete sets of runs were made for each of the hosts in the table
for each of several selections of algorithm parameters. The table
reflects values, selected as described later, believed to be a good
for use on existing paths in the Internet system
Internet Delay Experiments Page 8
D.L.
Host Total Lost Packets RTX Packets Mean CoV
ID Time Time Time
-------------------------------------------------------------------
DCN1 to nearby local-net hosts (calibration
DCN5 5 0 0 0 0 11 .15 1
DCN8 8 0 0 0 0 16 .13 1
IMP17 19 0 0 0 0 38 .33 1
FORD1 86 0 0 1 .2 167 .33 .99
UMD1 135 0 0 2 .5 263 .45 .99
DCN6 177 0 0 0 0 347 .34 1
FACC 368 196 222.1 6 9.2 267 1.1 .37
FOE 670 3 7.5 21 73.3 1150 .69 .87
FOE-1 374 0 0 26 61.9 610 .75 .83
FOE-2 1016 3 16.7 10 47.2 1859 .41 .93
DCN1 to ARPANET hosts and local
MILARP 59 0 0 2 .5 115 .39 .99
ISID 163 0 0 1 1.8 316 .47 .98
ISID-1 84 0 0 2 1 163 .18 .98
ISID-2 281 0 0 3 17 516 .91 .93
ISID * 329 0 0 5 12.9 619 .81 .96
SCORE 208 0 0 1 .8 405 .46 .99
RVAX 256 1 1.3 0 0 499 .42 .99
AJAX 365 0 0 0 0 713 .44 1
WASH 494 0 0 2 2.8 960 .39 .99
WASH-1 271 0 0 5 8 514 .34 .97
WASH-2 749 1 9.8 2 17.5 1411 .4 .96
BERK 528 20 50.1 4 35 865 1.13 .83
DCN1 to MILNET/MINET hosts and local
ISIA 436 4 7.4 2 15.7 807 .68 .94
ISIA-1 197 0 0 0 0 385 .27 1
ISIA-2 615 0 0 2 15 1172 .36 .97
ISIA * 595 18 54.1 6 33.3 992 .77 .85
BRL 644 1 3 1 1.9 1249 .43 .99
BRL-1 318 0 0 4 13.6 596 .68 .95
BRL-2 962 2 8.4 0 0 1864 .12 .99
LON 677 0 0 3 11.7 1300 .51 .98
LON-1 302 0 0 0 0 589 .06 1
LON-2 1047 0 0 0 0 2044 .03 1
HAWAII 709 4 12.9 3 18.5 1325 .55 .95
OFFICE3 856 3 12.9 3 10.3 1627 .54 .97
OFF3-1 432 2 4.2 2 6.9 823 .31 .97
OFF3-2 1277 7 39 3 41.5 2336 .44 .93
KOREA 1048 3 14.5 2 18.7 1982 .48 .96
KOREA-1 506 4 8.6 1 2.2 967 .18 .97
KOREA-2 1493 6 35.5 2 19.3 2810 .19 .96
DCN1 to TELENET hosts via
RICE 677 2 6.8 3 12.1 1286 .41 .97
Internet Delay Experiments Page 9
D.L.
RICE-1 368 1 .1 3 2.3 715 .11 .99
RICE-2 1002 1 4.4 1 9.5 1930 .19 .98
DCN1 to SATNET hosts and local nets via
UCL 689 9 26.8 0 0 1294 .21 .96
UCL-1 623 39 92.8 2 5.3 1025 .32 .84
UCL-2 818 4 13.5 0 0 1571 .15 .98
NTA 779 12 38.7 1 3.7 1438 .24 .94
NTA-1 616 24 56.6 2 5.3 1083 .25 .89
NTA-2 971 19 71.1 0 0 1757 .2 .92
NTA to SATNET hosts and local
TANUM 110 3 1.6 0 0 213 .41 .98
GOONY 587 19 44.2 1 2.9 1056 .23 .91
ETAM 608 32 76.3 1 3.1 1032 .29 .86
UCL 612 5 12.6 2 8.5 1154 .24 .96
Note: * indicates randomly distributed packets during periods of high
activity. The same entry without the * indicates randomly distributed
during periods of low ARPANET activity
Internet Delay Experiments Page 10
D.L.
3.2 Discussion of
It is immediately obvious from visual inspection of the bit-map
that the delay distribution is more-or-less Poissonly distributed about
relatively narrow range with important exceptions. The exceptions
characterized by occasional spasms where one or more packets can be
many times the typical value. Such glitches have been commonly noted
on paths involving ARPANET and SATNET, but the true impact of their
on the timeout algorithm is much greater than I expected. What
happens is that the algorithm, when confronted with a short burst
long-delay packets after a relatively long interval of well-mannered behavior
takes much too long to adapt to the spasm, thus inviting many
retransmissions and leading to congestion
The incidence of long-delay bursts, or glitches, varied widely during
experiments. Some of them were glitch-free, but most had at least one
in 512 echo/reply volleys. Glitches did not seem to correlate well
increases in baseline delay, which occurs as the result of traffic surges,
did they correlate well with instances of packet loss. I did not notice
particular periodicity, such as might be expected with regular pinging,
example; however, I did not process the data specially for that
There was no correction for packet length used in any of
experiments, in spite of the results of the first set of experiments
previously. This may be done in a future set of experiments. The
does cope well in the case of constant-length packets and in the case
randomly distributed packet lengths between 40 and 256 octets, as indicated
the table. Future experiments may involve bursts of short packets followed
bursts of longer ones, so that the speed of adaptation of the algorithm can
directly deterimend
One particularily interesting experiment involved the FOE
(FORD-FOE), which is located in London and reached via a 14.4-Kbps
cable and statistical multiplexor. The multiplexor introduces a moderate
delay, but with an extremely large delay dispersion. The
retransmission-timeout algorithm had a hard time with this circuit, as
be expected; however, with the improvments described below, TCP
was acceptable. It is unlikely that many instances of such ornery
will occur in the Internet system, but it is comforting to know that
algorithm can deal effectively with them
3.3. Improvments to the
The specified retransmission-timeout algorithm, really a first-
linear recursive filter, is characterized by two parameters, a
factor F and a threshold factor G. For each measured delay sample R the
estimator E is updated
E = F*E + (1 - F)*R .
Internet Delay Experiments Page 11
D.L.
Then, if an interval equal to G*E expires after transmitting a packet,
packet is retransmitted. The current TCP specification suggests values in
range 0.8 to 0.9 for F and 1.5 to 2.0 for G. These values have been
reasonable up to now over ARPANET and SATNET paths
I found that a simple change to the algorithm made a worthwhile change
the efficiency. The change amounts to using two values of F, one (F1) when
< E in the expression above and the other (F2) when R >= E, with F1 > F2.
effect is to make the algorithm more responsive to upward-going trends
delay and less respnsive to downward-going trends. After a number of trials
concluded that values of F1 = 15/16 and F2 = 3/4 (with G = 2) gave the
all-around performance. The results on some paths (FOE, ISID, ISIA)
better by some ten percent in efficiency, as compared to the values now
in typical implementations where F = 7/8 and G = 2. The results on most
were better by five percent, while on a couple (FACC, UCL) the results
worse by a few percent
There was no clear-cut gain in fiddling with G. The value G = 2
to represent the best overall compromise. Note that increasing G
superfluous retransmissions less likely, but increases the total delay
packets are lost. Also, note that increasing F2 too much tends to
overshoot in the case of network glitches and leads to the same result.
table above was constructed using F1 = 15/16, F2 = 3/4 and G = 2.
Readers familiar with signal-detection theory will recognize
suggestion as analogous to an ordinary peak-detector circuit. F1
the discharge time-constant, while F2 represents the charge time-constant.
represents a "squelch" threshold, as used in voice-operated switches,
example. Some wag may be even go on to suggest a network glitch should
called a netspurt
Internet Delay Experiments Page 12
D.L.
Appendix. Index of Test
Name Address NIC Host
-------------------------------------
DCN1 to nearby local-net hosts (calibration
DCN5 128.4.0.5 DCN
DCN8 128.4.0.8 DCN
IMP17 10.3.0.17 DCN-
FORD1 128.5.0.1 FORD
UMD1 128.8.0.1 UMD
DCN6 128.4.0.6 DCN
FACC 128.5.32.1 FORD-WDL
FOE 128.5.0.15 FORD-
DCN1 to ARPANET hosts and local
MILARP 10.2.0.28 ARPA-MILNET-
ISID 10.0.0.27 USC-
SCORE 10.3.0.11 SU-
RVAX 128.10.0.2 PURDUE-
AJAX 18.10.0.64 MIT-
WASH 10.0.0.91
BERK 10.2.0.78 UCB-
DCN1 to MILNET/MINET hosts and local
ISIA 26.3.0.103 USC-
BRL 192.5.21.6 BRL-
LON 24.0.0.7 MINET-LON-
HAWAII 26.1.0.36 HAWAII-
OFFICE3 26.2.0.43 OFFICE-3
KOREA 26.0.0.117 KOREA-
DCN1 to TELENET hosts via
RICE 14.0.0.12
DCN1 to SATNET hosts and local nets via
UCL 128.16.9.0 UCL-
NTA 128.39.0.2 NTARE
NTA to SATNET hosts and local
TANUM 4.0.0.64 TANUM-
GOONY 4.0.0.63 GOONHILLY-
ETAM 4.0.0.62 ETAM-
if you see any problems within the linking, don't worry be happy,
this is version 0.1 of the Relevance System and you gotta expect some crappy subroutines sometimes,
just be content we did not write this in Java, which would have made this "bigger and better" HAHAHHA.
RFC documents can be found at I.E.T.F.
Relevance System Copyright © 2002 Spectrum WorldResearch
other technical nosh by ServerMasters Corporation
collaboration of BobX
