As per Relevance of the word receiving, we have this rfc below:
Network Working Group David D.
Request for Comments: 998 Mark L.
Obsoletes: RFC 969 Lixia
March 1987
NETBLT: A Bulk Data Transfer
1.
This document is a description of, and a specification for,
NETBLT protocol. It is a revision of the specification published
NIC RFC-969. The protocol has been revised after extensive
into NETBLT's performance over long-delay, high-bandwidth
channels. Most of the changes in the protocol specification have
do with the computation and use of data timers in a
buffering data transfer model
This document is published for discussion and comment, and does
constitute a standard. The proposal may change and certain parts
the protocol have not yet been specified; implementation of
document is therefore not advised
2.
NETBLT (NETwork BLock Transfer) is a transport level
intended for the rapid transfer of a large quantity of data
computers. It provides a transfer that is reliable and
controlled, and is designed to provide maximum throughput over a
variety of networks. Although NETBLT currently runs on top of
Internet Protocol (IP), it should be able to operate on top of
datagram protocol similar in function to IP
NETBLT's motivation is to achieve higher throughput than
protocols might offer. The protocol achieves this goal by trying
minimize the effect of several network-related problems:
congestion, delays over satellite links, and packet loss
Its transmission rate-control algorithms deal well with
congestion; its multiple-buffering capability allows high
over long-delay satellite channels, and its
timeout/retransmit algorithms minimize the effect of packet
during a transfer. Most importantly, NETBLT's features give it
performance over long-delay channels without impairing
over high-speed LANs
Clark, Lambert, & Zhang [Page 1]
RFC 998 March 1987
The protocol works by opening a connection between two "clients" (
"sender" and the "receiver"), transferring the data in a series
large data aggregates called "buffers", and then closing
connection. Because the amount of data to be transferred can be
large, the client is not required to provide at once all the data
the protocol module. Instead, the data is provided by the client
buffers. The NETBLT layer transfers each buffer as a sequence
packets; since each buffer is composed of a large number of packets
the per-buffer interaction between NETBLT and its client is far
efficient than a per-packet interaction would be
In its simplest form, a NETBLT transfer works as follows:
sending client loads a buffer of data and calls down to the
layer to transfer it. The NETBLT layer breaks the buffer up
packets and sends these packets across the network in
datagrams. The receiving NETBLT layer loads these packets into
matching buffer provided by the receiving client. When the
packet in the buffer has arrived, the receiving NETBLT checks to
that all packets in that buffer have been correctly received.
some packets are missing, the receiving NETBLT requests that they
resent. When the buffer has been completely transmitted,
receiving client is notified by its NETBLT layer. The
client disposes of the buffer and provides a new buffer to
more data. The receiving NETBLT notifies the sender that the
buffer is ready, and the sender prepares and sends the next buffer
the same manner. This continues until all the data has been sent;
that time the sender notifies the receiver that the transmission
been completed. The connection is then closed
As described above, the NETBLT protocol is "lock-step". Action
after a buffer is transmitted, and begins again after confirmation
received from the receiver of data. NETBLT provides for
buffering, a transfer model in which the sending NETBLT can
new buffers while earlier buffers are waiting for confirmation
the receiving NETBLT. Multiple buffering makes packet
essentially continuous and markedly improves performance
The remainder of this document describes NETBLT in detail. The
sections describe the philosophy behind a number of
features: packetization, flow control, transfer reliability,
connection management. The final sections describe NETBLT's
formats
3. Buffers and
NETBLT is designed to permit transfer of a very large amounts of
between two clients. During connection setup the sending NETBLT
inform the receiving NETBLT of the transfer size; the
transfer length is 2**32 bytes. This limit should permit
practical application. The transfer size parameter is for the use
the receiving client; the receiving NETBLT makes no use of it.
Clark, Lambert, & Zhang [Page 2]
RFC 998 March 1987
NETBLT receiver accepts data until told by the sender that
transfer is complete
The data to be sent must be broken up into buffers by the client
Each buffer must be the same size, save for the last buffer.
connection setup, the sending and receiving NETBLTs negotiate
buffer size, based on limits provided by the clients. Buffer
are in bytes only; the client is responsible for placing data
buffers on byte boundaries
NETBLT has been designed and should be implemented to work
buffers of any size. The only fundamental limitation on buffer
should be the amount of memory available to the client.
should be as large as possible since this minimizes the number
buffer transmissions and therefore improves performance
NETBLT is designed to require a minimum amount of memory,
the client to allocate as much memory as possible for buffer storage
In particular, NETBLT does not keep buffer copies for
purposes. Instead, data to be retransmitted is recopied
from the client buffer. This means that the client cannot
buffer storage piece by piece as the buffer is sent, but this has
been a problem in preliminary NETBLT implementations
Buffers are broken down by the NETBLT layer into sequences of
packets. As with the buffer size, the DATA packet size is
between the sending and receiving NETBLTs during connection setup
Unlike buffer size, DATA packet size is visible only to the
layer
All DATA packets save the last packet in a buffer must be the
size. Packets should be as large as possible, since NETBLT'
performance is directly related to packet size. At the same time
the packets should not be so large as to cause
fragmentation, since this normally causes performance degradation
All buffers save the last buffer must be the same size; the
buffer can be any size required to complete the transfer. Since
receiving NETBLT does not know the transfer size in advance, it
some way of identifying the last packet in each buffer. For
reason, the last packet of every buffer is not a DATA packet
rather an LDATA packet. DATA and LDATA packets are identical
for the packet type
4. Flow
NETBLT uses two strategies for flow control, one internal and one
the client level
The sending and receiving NETBLTs transmit data in buffers;
flow control is therefore at a buffer level. Before a buffer can
Clark, Lambert, & Zhang [Page 3]
RFC 998 March 1987
transmitted, NETBLT confirms that both clients have set up
buffers, that one is ready to send data, and that the other is
to receive data. Either client can therefore control the flow
data by not providing a new buffer. Clients cannot stop a
transfer once it is in progress
Since buffers can be quite large, there has to be another method
flow control that is used during a buffer transfer. The NETBLT
provides this form of flow control
There are several flow control problems that could arise while
buffer is being transmitted. If the sending NETBLT is
data faster than the receiving NETBLT can process it, the receiver'
ability to buffer unprocessed packets could be overflowed,
packet loss. Similarly, a slow gateway or intermediate network
cause packets to collect and overflow network packet buffer space
Packets will then be lost within the network. This problem
particularly acute for NETBLT because NETBLT buffers will
be quite large, and therefore composed of many packets
A traditional solution to packet flow control is a window system,
which the sending end is permitted to send only a certain number
packets at a time. Unfortunately, flow control using windows
to result in low throughput. Windows must be kept small in order
avoid overflowing hosts and gateways, and cannot easily be updated
since an end-to-end exchange is required for each window change
To permit high throughput over a variety of networks and gateways
NETBLT uses a novel flow control method: rate control.
transmission rate is negotiated by the sending and receiving
during connection setup and after each buffer transmission.
sender uses timers, rather than messages from the receiver,
maintain the negotiated rate
In its simplest form, rate control specifies a minimum time
per packet transmission. This can cause performance problems
several reasons. First, the transmission time for a single packet
very small, frequently smaller than the granularity of the
mechanism. Also, the overhead required to maintain timing
on a per packet basis is relatively high and lowers performance
The solution is to control the transmission rate of groups
packets, rather than single packets. The sender transmits a burst
packets over a negotiated time interval, then sends another burst
In this way, the overhead decreases by a factor of the burst size
and the per-burst transmission time is long enough that
mechanisms will work properly. NETBLT's rate control therefore
two parts, a burst size and a burst rate, with (burst size)/(
rate) equal to the average transmission time per packet
Clark, Lambert, & Zhang [Page 4]
RFC 998 March 1987
The burst size and burst rate should be based not only on the
transmission and processing speed which each end can handle, but
on the capacities of any intermediate gateways or networks
Following are some intuitive values for packet size, buffer size
burst size, and burst rate
Packet sizes can be as small as 128 bytes. Performance with
this small is almost always bad, because of the high per-
processing overhead. Even the default Internet Protocol packet
of 576 bytes is barely big enough for adequate performance.
networks do not support packet sizes much larger than one or
thousand bytes, and packets of this size can also get fragmented
traveling over intermediate networks, lowering performance
The size of a NETBLT buffer is limited only by the amount of
available to a client. Theoretically, buffers of 100 Kbytes or
are possible. This would mean the transmission of 50 to 100
per buffer
The burst size and burst rate are obviously very machine dependent
There is a certain amount of transmission overhead in the sending
receiving machines associated with maintaining timers and
processes. This overhead can be minimized by sending packets
large bursts. There are also limitations imposed on the burst
by the number of available packet buffers in the operating
kernel. On most modern operating systems, a burst size of
five and ten packets should reduce the overhead to an
level. A preliminary NETBLT implementation for the IBM PC/AT
packets in bursts of five. It could send more, but is limited by
available memory
The burst rate is in part determined by the granularity of
sender's timing mechanism, and in part by the processing speed of
receiver and any intermediate gateways. It is also directly
to the burst size. Burst rates from 20 to 45 milliseconds per 5-
packet burst have been tried on the IBM PC/AT and Symbolics 3600
NETBLT implementations with good results within a single local-
network. This value clearly depends on the network bandwidth
packet buffering available
All NETBLT flow control parameters (packet size, buffer size,
size, and burst rate) are negotiated during connection setup.
negotiation process is the same for all parameters. The
initiating the connection (the active end) proposes and sends a
of values for each parameter in its connection request. The
client (the passive end) compares these values with the highest
performance values it can support. The passive end can then
any of the parameters, but only by making them more restrictive.
modified parameters are then sent back to the active end in
response message
Clark, Lambert, & Zhang [Page 5]
RFC 998 March 1987
The burst size and burst rate can also be re-negotiated after
buffer transmission to adjust the transfer rate according to
performance observed from transferring the previous buffer.
receiving end sends burst size and burst rate values in its
messages (described later). The sender compares these values
the values it can support. Again, it may then modify any of
parameters, but only by making them more restrictive. The
parameters are then communicated to the receiver in a NULL-
packet, described later
Obviously each of the parameters depend on many factors --
and host processing speeds, available memory, timer granularity --
some of which cannot be checked by either client. Each client
therefore try to make as best a guess as it can, tuning
performance on subsequent transfers
5. The NETBLT Transfer
Each NETBLT transfer has three stages, connection setup,
transfer, and connection close. The stages are described in
below, along with methods for insuring that each stage
reliably
5.1. Connection
A NETBLT connection is set up by an exchange of two packets
the active NETBLT and the passive NETBLT. Note that either
can send or receive data; the words "active" and "passive" are
used to differentiate the end making the connection request from
end responding to the connection request. The active end sends
OPEN packet; the passive end acknowledges the OPEN packet in one
two ways. It can either send a REFUSED packet, indicating that
connection cannot be completed for some reason, or it can
the connection setup by sending a RESPONSE packet. At this point
transfer can begin
As discussed in the previous section, the OPEN and RESPONSE
are used to negotiate flow control parameters. Other parameters
in the data transfer are also negotiated. These parameters are (1)
the maximum number of buffers that can be sending at any one time
and (2) whether or not DATA packet data will be checksummed.
automatically checksums all non-DATA/LDATA packets. If
negotiated checksum flag is set to TRUE (1), both the header and
data of a DATA/LDATA packet are checksummed; if set to FALSE (0),
only the header is checksummed. The checksum value is the
negation of the ones-complement sum of the 16-bit words
checksummed
Finally, each end transmits its death-timeout value in seconds
either the OPEN or the RESPONSE packet. The death-timeout value
be used to determine the frequency with which to send
Clark, Lambert, & Zhang [Page 6]
RFC 998 March 1987
packets during idle periods of an opened connection (death timers
KEEPALIVE packets are described in the following section).
The active end specifies a passive client through a client-
"well-known" 16 bit port number on which the passive end listens
The active end identifies itself through a 32 bit Internet
and a unique 16 bit port number
In order to allow the active and passive ends to
miscellaneous useful information, an unstructured, variable-
field is provided in OPEN and RESPONSE packets for any client
specific information that may be required. In addition, a "
for refusal" field is provided in REFUSED packets
Recovery for lost OPEN and RESPONSE packets is provided by the use
timers. The active end sets a timer when it sends an OPEN packet
When the timer expires, another OPEN packet is sent, until
predetermined maximum number of OPEN packets have been sent.
timer is cleared upon receipt of a RESPONSE packet
To prevent duplication of OPEN and RESPONSE packets, the OPEN
contains a 32 bit connection unique ID that must be returned in
RESPONSE packet. This prevents the initiator from confusing
response to the current request with the response to an
connection request (there can only be one connection between any
ports). Any OPEN or RESPONSE packet with a destination port
that of an open connection has its unique ID checked. If the
ID of the packet matches the unique ID of the connection, then
packet type is checked. If it is a RESPONSE packet, it is treated
a duplicate and ignored. If it is an OPEN packet, the passive
sends another RESPONSE (assuming that a previous RESPONSE packet
sent and lost, causing the initiating NETBLT to retransmit its
packet). A non-matching unique ID must be treated as an attempt
open a second connection between the same port pair and is
by sending an ABORT message
5.2. Data
The simplest model of data transfer proceeds as follows. The
client sets up a buffer full of data. The receiving NETBLT sends
GO message inside a CONTROL packet to the sender, signifying that
too has set up a buffer and is ready to receive data. Once the
message is received, the sender transmits the buffer as a series
DATA packets followed by an LDATA packet. When the last packet
the buffer has been received, the receiver sends a RESEND
inside a CONTROL packet containing a list of packets that were
received. The sender resends these packets. This process
until there are no missing packets. At that time the receiver
an OK message inside a CONTROL packet, sets up another buffer
receive data, and sends another GO message. The sender,
received the OK message, sets up another buffer, waits for the
Clark, Lambert, & Zhang [Page 7]
RFC 998 March 1987
message, and repeats the process
The above data transfer model is effectively a lock-step protocol
and causes time to be wasted while the sending NETBLT waits
permission to send a new buffer. A more efficient transfer
uses multiple buffering to increase performance. Multiple
is a technique in which the sender and receiver allocate and
buffers in a manner that allows error recovery or
transmission confirmation of previous buffers to be concurrent
transmission of the current buffer
During the connection setup phase, one of the negotiated
is the number of concurrent buffers permitted during the transfer
If there is more than one buffer available, transfer of the
buffer may start right after the current buffer finishes. This
illustrated in the following example
Assume two buffers A and B in a multiple-buffer transfer, with
preceding B. When A has been transferred and the sending NETBLT
waiting for either an OK or a RESEND message for it, the
NETBLT can start sending B immediately, keeping data flowing at
stable rate. If the receiver of data sends an OK for A, all is well
if it receives a RESEND, the missing packets specified in the
message are retransmitted
In the multiple-buffer transfer model, all packets to be sent
re-ordered by buffer number (lowest number first), with the
rate specified by the burst size and burst rate. Since
numbers increase monotonically, packets from an earlier buffer
always precede packets from a later buffer
Having several buffers transmitting concurrently is actually not
much more complicated than transmitting a single buffer at a time
The key is to visualize each buffer as a finite state machine
several buffers are merely a group of finite state machines, each
one of several states. The transfer process consists of
buffers through various states until the entire transmission
completed
There are several obvious flaws in the data transfer model
described above. First, what if the GO, OK, or RESEND messages
lost? The sender cannot act on a packet it has not received, so
protocol will hang. Second, if an LDATA packet is lost, how does
receiver know when the buffer has been transmitted? Solutions
each of these problems are presented below
5.2.1. Recovering from Lost Control
NETBLT solves the problem of lost OK, GO, and RESEND messages in
ways. First, it makes use of a control timer. The receiver can
one or more control messages (OK, GO, or RESEND) within a
Clark, Lambert, & Zhang [Page 8]
RFC 998 March 1987
CONTROL packet. Whenever the receiver sends a control packet,
sets a control timer. This timer is either "reset" (set again)
"cleared" (deactivated), under the following conditions
When the control timer expires, the receiving NETBLT resends
control packet and resets the timer. The receiving NETBLT
to resend control packets in response to control timer's
until either the control timer is cleared or the receiving NETBLT'
death timer (described later) expires (at which time it shuts
the connection).
Each control message includes a sequence number which starts at
and increases by one for each control message sent. The
NETBLT checks the sequence number of every incoming control
against all other sequence numbers it has received. It stores
highest sequence number below which all other received
numbers are consecutive (in following paragraphs this is called
high-acknowledged-sequence-number) and returns this number in
packet flowing back to the receiver. The receiver is permitted
clear its control timer when it receives a packet from the
with a high-acknowledged-sequence-number greater than or equal to
highest sequence number in the control packet just sent
Ideally, a NETBLT implementation should be able to cope with out-of
sequence control messages, perhaps collecting them for
processing, or even processing them immediately. If an
control message "fills" a "hole" in a group of message
numbers, the implementation could even be clever enough to
this and adjust its outgoing sequence value accordingly
The sending NETBLT, upon receiving a CONTROL packet, should act
the packet as quickly as possible. It either sets up a new
(upon receipt of an OK message for a previous buffer), marks data
resending (upon receipt of a RESEND message), or prepares a
for sending (upon receipt of a GO message). If the sending NETBLT
not in a position to send data, it should send a NULL-ACK packet
which contains its high-acknowledged-sequence-number (this
the receiving NETBLT to acknowledge any outstanding
messages), and wait until it can send more data. In all of
cases, the system overhead for a response to the incoming
message should be small and relatively constant
The small amount of message-processing overhead allows
control timers to be set for all types of control messages with
single, simple algorithm -- the network round-trip transit time,
a variance factor. This is more efficient than schemes used by
protocols, where timer value calculation has been a problem
the processing time for a particular packet can vary
depending on the packet type
Control timer value estimation is extremely important in a high
Clark, Lambert, & Zhang [Page 9]
RFC 998 March 1987
performance protocol like NETBLT. A long control timer causes
receiving NETBLT to wait for long periods of time
retransmitting unacknowledged messages. A short control timer
causes the sending NETBLT to receive many duplicate control
(which it can reject, but which takes time).
In addition to the use of control timers, NETBLT reduces lost
messages by using a single long-lived control packet; the packet
treated like a FIFO queue, with new control messages added on at
end and acknowledged control messages removed from the front.
implementation places control messages in the control packet
transmits the entire control packet, consisting of any
control messages plus new messages just added. The entire
packet is also transmitted whenever the control timer expires.
control packet transmissions are fairly frequent,
messages may be transmitted several times before they are
acknowledged. This redundant transmission of control
provides automatic recovery for most control message losses over
noisy channel
This scheme places some burdens on the receiver of the
messages. It must be able to quickly reject duplicate
messages, since a given message may be retransmitted several
before its acknowledgement is received and it is removed from
control packet. Typically this is fairly easy to do; the sender
data merely throws away any control messages with sequence
lower than its high-acknowledged-sequence-number
Another problem with this scheme is that the control packet
become larger than the maximum allowable packet size if too
control messages are placed into it. This has not been a problem
the current NETBLT implementations: a typical control packet size
1000 bytes; RESEND control messages average about 20 bytes in length
GO messages are 8 bytes long, and OK messages are 16 bytes long
This allows 50-80 control messages to be placed in the
packet, more than enough for reasonable transfers.
implementations can provide for multiple control packets if a
control packet may not be sufficient
The control timer value must be carefully estimated. It can have
its initial value an arbitrary number. Subsequent control
should have their timer values based on the network round-
transit time (i.e. the time between sending the control packet
receiving the acknowledgment of all messages in the control packet
plus a variance factor. The timer value should be
updated, based on a smoothed average of collected round-trip
times
Clark, Lambert, & Zhang [Page 10]
RFC 998 March 1987
5.2.2. Recovering from Lost LDATA
NETBLT solves the problem of LDATA packet loss by using a data
for each buffer at the receiving end. The simplest data timer
has a data timer set when a buffer is ready to be received; if
data timer expires, the receiving NETBLT assumes a lost LDATA
and sends a RESEND message requesting all missing DATA packets in
buffer. When all packets have been received, the timer is cleared
Data timer values are not based on network round-trip transit time
instead they are based on the amount of time taken to transfer
buffer (as determined by the number of DATA packet bursts in
buffer times the burst rate) plus a variance factor <1>.
Obviously an accurate estimation of the data timer value is
important. A short data timer value causes the receiving NETBLT
send unnecessary RESEND packets. This causes serious
degradation since the sending NETBLT has to stop what it is doing
resend a number of DATA packets
Data timer setting and clearing turns out to be fairly complicated
particularly in a multiple-buffering transfer model.
understanding how and when data timers are set and cleared, it
helpful to visualize each buffer as a finite-state machine and take
look at the various states
The state sequence for a sending buffer is simple. When a GO
for the buffer is received, the buffer is created, filled with data
and placed in a SENDING state. When an OK for that buffer has
received, it goes into a SENT state and is disposed of
The state sequence for a receiving buffer is a little
complicated. Assume existence of a buffer A. When a control
for A is sent, the buffer moves into state ACK-WAIT (it is
for acknowledgement of the control message).
As soon as the control message has been acknowledged, buffer A
from the ACK-WAIT state into the ACKED state (it is now waiting
DATA packets to arrive). At this point, A's data timer is set
the control message removed from the control packet. Estimation
the data timer value at this point is quite difficult. In
multiple-buffer transfer model, the receiving NETBLT can send
GO messages at once. A single DATA packet from the sending
could acknowledge all the GO messages, causing several buffers
start up data timers. Clearly each of the data timers must be set
a manner that takes into account each buffer's place in the order
transmission. Packets for a buffer A - 1 will always be
before packets in A, so A's data timer must take into account
arrival of all of A - 1's DATA packets as well as arrival of its
DATA packets. This means that the timer values become
less accurate for higher-numbered buffers. Because this data
Clark, Lambert, & Zhang [Page 11]
RFC 998 March 1987
value can be quite inaccurate, it is called a "loose" data timer
The loose data timer value is recalculated later (using the
algorithm, but with updated information), giving a "tight" timer,
described below
When the first DATA packet for A arrives, A moves from the
state to the RECEIVING state and its data timer is set to a
"tight" value. The tight timer value is calculated in the
manner as the loose timer, but it is more accurate since we
moved forward in time and those buffers numbered lower than A
presumably been dealt with (or their packets would have
before A's), leaving fewer packets to arrive between the setting
the data timer and the arrival of the last DATA packet in A
The receiving NETBLT also sets the tight data timers of any
numbered lower than A that are also in the ACKED state. This is
as an optimization: we know that buffers are processed in order
lowest number first. If a buffer B numbered lower than A is in
ACKED state, its DATA packets should arrive before A's. Since A'
have arrived first, B's must have gotten lost. Since B's loose
timer has not expired (it would then have sent a RESEND message
be in the ACK-WAIT state), we set the tight timer, allowing
missing packets to be detected earlier. An immediate RESEND is
sent because it is possible that A's packet was re-ordered before B'
by the network, and that B's packets may arrive shortly
When all DATA packets for A have been received, it moves from
RECEIVING state to the RECEIVED state and is disposed of. Had
packets been missing, A's data timer would have expired and A
have moved into the ACK-WAIT state after sending a RESEND message
The state progression would then move as in the above example
The control and data timer system can be summarized as follows
normally, the receiving NETBLT is working under one of two types
timers, a control timer or a data timer. There is one data timer
buffer transmission and one control timer per control packet.
data timer is active while its buffer is in either the ACKED (
data timer value is used) or the RECEIVING (tight data timer value
used) states; a control timer is active whenever the receiving
has any unacknowledged control messages in its control packet
5.2.3. Death Timers and Keepalive
The above system still leaves a few problems. If the sending
is not ready to send, it sends a single NULL-ACK packet to clear
outstanding control timers at the receiving end. After this
receiver will wait. The sending NETBLT could die and the receiver
with its control timer cleared, would hang. Also, the above
puts timers only on the receiving NETBLT. The sending NETBLT has
timers; if the receiving NETBLT dies, the sending NETBLT will
while waiting for control messages to arrive
Clark, Lambert, & Zhang [Page 12]
RFC 998 March 1987
The solution to the above two problems is the use of a death
and a keepalive packet for both the sending and receiving NETBLTs
As soon as the connection is opened, each end sets a death timer
this timer is reset every time a packet is received. When a NETBLT'
death timer expires, it can assume the other end has died and
close the connection
It is possible that the sending or receiving NETBLTs will have
wait for long periods while their respective clients get buffer
and load their buffers with data. Since a NETBLT waiting for
space is in a perfectly valid state, the protocol must have
method for preventing the other end's death timer from expiring.
solution is to use a KEEPALIVE packet, which is sent repeatedly
fixed intervals when a NETBLT cannot send other packets. Since
death timer is reset whenever a packet is received, it will
expire as long as the other end sends packets
The frequency with which KEEPALIVE packets are transmitted
computed as follows: At connection startup, each NETBLT chooses
death-timer value and sends it to the other end in either the OPEN
the RESPONSE packet. The other end takes the death-timeout value
uses it to compute a frequency with which to send KEEPALIVE packets
The KEEPALIVE frequency should be high enough that several
packets can be lost before the other end's death timer expires (e.g
death timer value divided by four).
The death timer value is relatively easy to estimate. Since it
continually reset, it need not be based on the transfer size
Instead, it should be based at least in part on the type
application using NETBLT. User applications should have
death timeout values to avoid forcing humans to wait long periods
time for a death timeout to occur. Machine applications can
longer timeout values
5.3. Closing the
There are three ways to close a connection: a connection close,
"quit", or an "abort".
5.3.1. Successful
After a successful data transfer, NETBLT closes the connection.
the sender is transmitting the last buffer of data, it sets a "last
buffer" flag on every DATA packet in the buffer. This means that
NEW data will be transmitted. The receiver knows the transfer
completed successfully when all of the following are true: (1) it
received DATA packets with a "last-buffer" flag set, (2) all
control messages have been acknowledged, and (3) it has
outstanding buffers with missing packets. At that point,
receiver is permitted to close its half of the connection.
sender knows the transfer has completed when the following are true
Clark, Lambert, & Zhang [Page 13]
RFC 998 March 1987
(1) it has transmitted DATA packets with a "last-buffer" flag set
(2) it has received OK messages for all its buffers. At that point
it "dallies" for a predetermined period of time before closing
half of the connection. If the NULL-ACK packet acknowledging
receiver's last OK message was lost, the receiver has time
retransmit the OK message, receive a new NULL-ACK, and recognize
successful transfer. The dally timer value MUST be based on
receiver's control timer value; it must be long enough to allow
receiver's control timer to expire so that the OK message can be re
sent. For this reason, all OK messages contain (in addition to
burst size and burst rate values), the receiver's current
timer value in milliseconds. The sender uses this value to
its dally timer value
Since the dally timer value may be quite large, the receiving
is permitted to "short-circuit" the sending NETBLT's dally timer
transmitting a DONE packet. The DONE packet is transmitted when
receiver knows the transfer has been successfully completed.
the sender receives a DONE packet, it is allowed to clear its
timer and close its half of the connection immediately. The
packet is not reliably transmitted, since failure to receive it
means that the sending NETBLT will take longer time to close its
of the connection (as it waits for its dally timer to clear
5.3.2. Client
During a NETBLT transfer, one client may send a QUIT packet to
other if it thinks that the other client is malfunctioning.
the QUIT occurs at a client level, the QUIT transmission can
occur between buffer transmissions. The NETBLT receiving the
packet can take no action other than immediately notifying its
and transmitting a QUITACK packet. The QUIT sender must time out
retransmit until a QUITACK has been received or its death
expires. The sender of the QUITACK dallies before quitting, so
it can respond to a retransmitted QUIT
5.3.3. NETBLT
An ABORT takes place when a NETBLT layer thinks that it or
opposite is malfunctioning. Since the ABORT originates in the
layer, it can be sent at any time. The ABORT implies that the
layer is malfunctioning, so no transmit reliability is expected,
the sender can immediately close it connection
6. Protocol Layering
NETBLT is implemented directly on top of the Internet Protocol (IP).
It has been assigned an official protocol number of 30 (decimal).
Clark, Lambert, & Zhang [Page 14]
RFC 998 March 1987
7. Planned
As currently specified, NETBLT has no algorithm for determining
rate-control parameters (burst rate, burst size, etc.). In
performance testing, these parameters have been set by the
performing the test. We are now exploring ways to have NETBLT
and adjust its rate-control parameters automatically
8. Packet
NETBLT packets are divided into three categories, all of which
a common packet header. First, there are those packets that
only from data sender to receiver; these contain the high
acknowledged-sequence-numbers which the receiver uses for
message transmission reliability. These packets are the NULL-ACK
DATA, and LDATA packets. Second, there is a packet that travels
from receiver to sender. This is the CONTROL packet; each
packet can contain an arbitrary number of control messages (GO, OK
or RESEND), each with its own sequence number. Finally, there
those packets which either have special ways of insuring reliability
or are not reliably transmitted. These are the OPEN, RESPONSE
REFUSED, QUIT, QUITACK, DONE, KEEPALIVE, and ABORT packets.
these, all save the DONE packet can be sent by both sending
receiving NETBLTs
All packets are "longword-aligned", i.e. all packets are a
of 4 bytes in length and all 4-byte fields start on a
boundary. All arbitrary-length string fields are terminated with
least one null byte, with extra null bytes added at the end to
a field that is a multiple of 4 bytes long
Clark, Lambert, & Zhang [Page 15]
RFC 998 March 1987
Packet Formats for
OPEN (type 0) and RESPONSE (type 1):
1 2 3
1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2
+---------------+---------------+---------------+---------------+
| Checksum | Version | Type |
+---------------+---------------+---------------+---------------+
| Length | Local Port |
+---------------+---------------+---------------+---------------+
| Foreign Port | Longword Alignment Padding |
+---------------+---------------+---------------+---------------+
| Connection Unique ID |
+---------------+---------------+---------------+---------------+
| Buffer Size |
+---------------+---------------+---------------+---------------+
| Transfer Size |
+---------------+---------------+---------------+---------------+
| DATA packet size | Burst Size |
+---------------+---------------+---------------+---------------+
| Burst Rate | Death Timer Value |
+---------------+---------------+---------------+---------------+
| Reserved (MBZ) |C|M| Maximum # Outstanding Buffers |
+---------------+---------------+---------------+---------------+
| Client String ...
+---------------+---------------+---------------
Longword Alignment Padding |
---------------+-------------------------------+
Checksum: packet checksum (algorithm is described in the
"Connection Setup")
Version: the NETBLT protocol version
Type: the NETBLT packet type number (OPEN = 0, RESPONSE = 1,
etc.)
Length: the total length (NETBLT header plus data, if present
of the NETBLT packet in
Local Port: the local NETBLT's 16-bit port
Foreign Port: the foreign NETBLT's 16-bit port
Connection UID: the 32 bit connection UID specified in
section "Connection Setup".
Buffer size: the size in bytes of each NETBLT buffer (save
last
Clark, Lambert, & Zhang [Page 16]
RFC 998 March 1987
Transfer size: (optional) the size in bytes of the transfer
This is for client information only; the receiving NETBLT
NOT make use of it
Data packet size: length of each DATA packet in
Burst Size: Number of DATA packets in a
Burst Rate: Transmit time in milliseconds of a single
Death timer: Packet sender's death timer value in
"M": the transfer mode (0 = READ, 1 = WRITE
"C": the DATA packet data checksum flag (0 = do not
DATA packet data, 1 = do
Maximum Outstanding Buffers: maximum number of buffers that
be transferred before waiting for an OK message from
receiving NETBLT
Client string: an arbitrary, null-terminated, longword-
string for use by NETBLT clients
KEEPALIVE (type 2), QUITACK (type 4), and DONE (type 11)
1 2 3
1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2
+---------------+---------------+---------------+---------------+
| Checksum | Version | Type |
+---------------+---------------+---------------+---------------+
| Length | Local Port |
+---------------+---------------+---------------+---------------+
| Foreign Port | Longword Alignment Padding |
+---------------+---------------+---------------+---------------+
Clark, Lambert, & Zhang [Page 17]
RFC 998 March 1987
QUIT (type 3), ABORT (type 5), and REFUSED (type 10)
1 2 3
1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2
+---------------+---------------+---------------+---------------+
| Checksum | Version | Type |
+---------------+---------------+---------------+---------------+
| Length | Local Port |
+---------------+---------------+---------------+---------------+
| Foreign Port | Longword Alignment Padding |
+---------------+---------------+---------------+---------------+
| Reason for QUIT/ABORT/REFUSE...
+---------------+---------------+---------------
Longword Alignment Padding |
---------------+-------------------------------+
DATA (type 6) and LDATA (type 7):
1 2 3
1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2
+---------------+---------------+---------------+---------------+
| Checksum | Version | Type |
+---------------+---------------+---------------+---------------+
| Length | Local Port |
+---------------+---------------+---------------+---------------+
| Foreign Port | Longword Alignment Padding |
+---------------+---------------+---------------+---------------+
| Buffer Number |
+---------------+---------------+---------------+---------------+
| High Consecutive Seq Num Rcvd | Packet Number |
+---------------+---------------+---------------+---------------+
| Data Area Checksum Value | Reserved (MBZ) |L
+---------------+---------------+---------------+---------------+
Buffer number: a 32 bit unique number assigned to every buffer
Numbers are monotonically increasing
High Consecutive Sequence Number Received: Highest
message sequence number below which all sequence numbers
are consecutive
Packet number: monotonically increasing DATA packet
Data Area Checksum Value: Checksum of the DATA packet's data
Algorithm used is the same as that used to compute checksums
other NETBLT packets
"L" is a flag set when the buffer that this DATA packet
to is the last buffer in the transfer
Clark, Lambert, & Zhang [Page 18]
RFC 998 March 1987
NULL-ACK (type 8)
1 2 3
1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2
+---------------+---------------+---------------+---------------+
| Checksum | Version | Type |
+---------------+---------------+---------------+---------------+
| Length | Local Port |
+---------------+---------------+---------------+---------------+
| Foreign Port | Longword Alignment Padding |
+---------------+---------------+---------------+---------------+
| High Consecutive Seq Num Rcvd | New Burst Size |
+---------------+---------------+---------------+---------------+
| New Burst Rate | Longword Alignment Padding |
+---------------+---------------+---------------+---------------+
High Consecutive Sequence Number Received: same as in DATA/
New Burst Size: Burst size as negotiated from value given
receiving NETBLT in OK
New burst rate: Burst rate as negotiated from value
by receiving NETBLT in OK message. Value is in milliseconds
CONTROL (type 9):
1 2 3
1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2
+---------------+---------------+---------------+---------------+
| Checksum | Version | Type |
+---------------+---------------+---------------+---------------+
| Length | Local Port |
+---------------+---------------+---------------+---------------+
| Foreign Port | Longword Alignment Padding |
+---------------+---------------+---------------+---------------+
Followed by any number of messages, each of which is
aligned, with the following formats
GO message (type 0):
1 2 3
1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2
+---------------+---------------+---------------+---------------+
| Type | Word Padding | Sequence Number |
+---------------+---------------+---------------+---------------+
| Buffer Number |
+---------------+---------------+---------------+---------------+
Type: message type (GO = 0, OK = 1, RESEND = 2)
Clark, Lambert, & Zhang [Page 19]
RFC 998 March 1987
Sequence number: A 16 bit unique message number.
numbers must be monotonically increasing, starting from 1.
Buffer number: as in DATA/LDATA
OK message (type 1):
1 2 3
1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2
+---------------+---------------+---------------+---------------+
| Type | Word Padding | Sequence Number |
+---------------+---------------+---------------+---------------+
| Buffer Number |
+---------------+---------------+---------------+---------------+
| New Offered Burst Size | New Offered Burst Rate |
+---------------+---------------+---------------+---------------+
| Current control timer value | Longword Alignment Padding |
+---------------+---------------+---------------+---------------+
New offered burst size: burst size for subsequent
transfers, possibly based on performance information for
buffer transfers
New offered burst rate: burst rate for subsequent
transfers, possibly based on performance information for
buffer transfers. Rate is in milliseconds
Current control timer value: Receiving NETBLT's control
value in milliseconds
RESEND Message (type 2):
1 2 3
1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2
+---------------+---------------+---------------+---------------+
| Type | Word Padding | Sequence Number |
+---------------+---------------+---------------+---------------+
| Buffer Number |
+---------------+---------------+---------------+---------------+
| Number of Missing Packets | Longword Alignment Padding |
+---------------+---------------+---------------+---------------+
| Packet Number (2 bytes) ...
+---------------+---------------+----------
| Padding (if necessary) |
-----------+---------------+---------------+
Packet number: the 16 bit data packet identifier found in
DATA packet
Clark, Lambert, & Zhang [Page 20]
RFC 998 March 1987
NOTES
<1> When the buffer size is large, the variances in the round
delays of many packets may cancel each other out; this means
variance value need not be very big. This expectation will
explored in further testing
Clark, Lambert, & Zhang [Page 21]
if you see any problems within the linking, don't worry be happy,
this is version 0.1 of the Relevance System and you gotta expect some crappy subroutines sometimes,
just be content we did not write this in Java, which would have made this "bigger and better" HAHAHHA.
RFC documents can be found at I.E.T.F.
Relevance System Copyright © 2002 Spectrum WorldResearch
other technical nosh by ServerMasters Corporation
collaboration of BobX
