As per Relevance of the word problems, we have this rfc below:
Network Working Group I.
Request for Comments: 3143 Equinix, Inc
Category: Informational J.
Akamai Technologies, Inc
June 2001
Known HTTP Proxy/Caching
Status of this
This memo provides information for the Internet community. It
not specify an Internet standard of any kind. Distribution of
memo is unlimited
Copyright
Copyright (C) The Internet Society (2001). All Rights Reserved
This document catalogs a number of known problems with World Wide
(WWW) (caching) proxies and cache servers. The goal of the
is to provide a discussion of the problems and proposed workarounds
and ultimately to improve conditions by illustrating problems.
construction of this document is a joint effort of the Web
community
Table of
1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 2
1.1 Problem Template . . . . . . . . . . . . . . . . . . . . . . 2
2. Known Problems . . . . . . . . . . . . . . . . . . . . . . . 4
2.1 Known Specification Problems . . . . . . . . . . . . . . . . 5
2.1.1 Vary header is underspecified and/or misleading . . . . . . 5
2.1.2 Client Chaining Loses Valuable Length Meta-Data . . . . . . 9
2.2 Known Architectural Problems . . . . . . . . . . . . . . . . 10
2.2.1 Interception proxies break client cache directives . . . . . 10
2.2.2 Interception proxies prevent introduction of new
methods . . . . . . . . . . . . . . . . . . . . . . . . 11
2.2.3 Interception proxies break IP address-based authentication . 12
2.2.4 Caching proxy peer selection in heterogeneous networks . . . 13
2.2.5 ICP Performance . . . . . . . . . . . . . . . . . . . . . . 15
2.2.6 Caching proxy meshes can break HTTP serialization of content 16
2.3 Known Implementation Problems . . . . . . . . . . . . . . . 17
2.3.1 User agent/proxy failover . . . . . . . . . . . . . . . . . 17
2.3.2 Some servers send bad Content-Length headers for files
contain CR . . . . . . . . . . . . . . . . . . . . . . . 18
Cooper & Dilley Informational [Page 1]
RFC 3143 Known HTTP Proxy/Caching Problems June 2001
3. Security Considerations . . . . . . . . . . . . . . . . . . 18
References . . . . . . . . . . . . . . . . . . . . . . . . . 19
Authors' Addresses . . . . . . . . . . . . . . . . . . . . . 20
A. Archived Known Problems . . . . . . . . . . . . . . . . . . 21
A.1 Architectural . . . . . . . . . . . . . . . . . . . . . . . 21
A.1.1 Cannot specify multiple URIs for replicated resources . . . 21
A.1.2 Replica distance is unknown . . . . . . . . . . . . . . . . 22
A.1.3 Proxy resource location . . . . . . . . . . . . . . . . . . 23
A.2 Implementation . . . . . . . . . . . . . . . . . . . . . . . 23
A.2.1 Use of Cache-Control headers . . . . . . . . . . . . . . . . 23
A.2.2 Lack of HTTP/1.1 compliance for caching proxies . . . . . . 24
A.2.3 ETag support . . . . . . . . . . . . . . . . . . . . . . . . 25
A.2.4 Servers and content should be optimized for caching . . . . 26
A.3 Administration . . . . . . . . . . . . . . . . . . . . . . . 27
A.3.1 Lack of fine-grained, standardized hierarchy controls . . . 27
A.3.2 Proxy/Server exhaustive log format standard for analysis . . 27
A.3.3 Trace log timestamps . . . . . . . . . . . . . . . . . . . . 28
A.3.4 Exchange format for log summaries . . . . . . . . . . . . . 29
Full Copyright Statement . . . . . . . . . . . . . . . . . . 32
1.
This memo discusses problems with proxies - which act
application-level intermediaries for Web requests - and
specifically with caching proxies, which retain copies of
requested resources in the hope of improving overall quality
service by serving the content locally. Commonly used terminology
this memo can be found in the "Internet Web Replication and
Taxonomy"[2].
No individual or organization has complete knowledge of the
problems in Web caching, and the editors are grateful to
contributors to this document
1.1 Problem
A common problem template is used within the following sections.
gratefully acknowledge RFC2525 [1] which helped define an
format for this known problems list. The template format
summarized in the following table and described in more detail below
Name: short, descriptive name of the problem (3-5 words
Classification: classifies the problem: performance, security,
Description: describes the problem
Significance: magnitude of problem, environments where it
Implications: the impact of the problem on systems and
See Also: a reference to a related known
Indications: states how to detect the presence of this
Cooper & Dilley Informational [Page 2]
RFC 3143 Known HTTP Proxy/Caching Problems June 2001
Solution(s): describe the solution(s) to this problem, if
Workaround: practical workaround for the
References: information about the problem or
Contact: contact name and email address for this
A short, descriptive, name (3-5 words) name associated with
problem
Problems are grouped into categories of similar problems for
of reading of this memo. Choose the category that best
the problem. The suggested categories include three
categories and several more specific categories
* Architecture: the fundamental design is incomplete,
* Specification: the spec is ambiguous, incomplete, or incorrect
* Implementation: the implementation of the spec is incorrect
* Performance: perceived page response at the client
excessive; network bandwidth consumption is excessive;
on origin or proxy servers exceed reasonable bounds
* Administration: care and feeding of caches is, or causes,
problem
* Security: privacy, integrity, or authentication concerns
A definition of the problem, succinct but including
background information
Significance (High, Medium, Low
May include a brief summary of the environments for which
problem is significant
Why the problem is viewed as a problem. What
behavior results from it? This section should substantiate
magnitude of any problem indicated with High significance
See
Optional. List of other known problems that are related to
one
Cooper & Dilley Informational [Page 3]
RFC 3143 Known HTTP Proxy/Caching Problems June 2001
How to detect the presence of the problem. This may
references to one or more substantiating documents
demonstrate the problem. This should include the
configuration that led to the problem such that it can
reproduced. Problems that are not reproducible will not appear
this memo
Solution(s
Solutions that permanently fix the problem, if such are known.
example, what version of the software does not exhibit
problem? Indicate if the solution is accepted by the community
one of several solutions pending agreement, or open possibly
experimental solutions
Practical workaround if no solution is available or usable.
workaround should have sufficient detail for someone
the problem to get around it
References to related information in technical publications or
the web. Where can someone interested in learning more go to
out more about this problem, its solution, or workarounds
Contact name and email address of the person who supplied
information for this section. The editors are listed as
for anonymous submissions
2. Known
The remaining sections of this document present the
documented known problems. The problems are ordered
classification and significance. Issues with protocol
or architecture are first, followed by implementation issues.
of high significance are first, followed by lower significance
Some of the problems initially identified in the previous versions
this document have been moved to Appendix A since they discuss
where resolution primarily involves education rather than
work
A full list of the problems is available in the table of contents
Cooper & Dilley Informational [Page 4]
RFC 3143 Known HTTP Proxy/Caching Problems June 2001
2.1 Known Specification
2.1.1 Vary header is underspecified and/or
The "Vary" header is underspecified and/or
The Vary header in HTTP/1.1 was designed to allow a caching
to safely cache responses even if the server's choice of
is not entirely understood. As RFC 2616 says
The Vary header field can be used to express the parameters
server uses to select a representation that is subject
server-driven negotiation
One might expect that this mechanism is useful in general
extensions that change the response message based on some
of the request. However, that is not true
During the design of the HTTP delta encoding specification[9]
was realized that an HTTP/1.1 proxy that does not understand
encoding might cache a delta-encoded response and then
deliver it to a non-delta-capable client, unless the
included some mechanism to prevent this. Initially, it
thought that Vary would suffice, but the following scenario
this wrong
NOTE: It is likely that other scenarios exhibiting the same
problem with "Vary" could be devised, without reference to
encoding. This is simply a concrete scenario used to explain
problem
A complete description of the IM and A-IM headers may be found
the "Delta encoding in HTTP" specification. For the purpose
this problem description, the relevant details are
1. The concept of an "instance manipulation" is introduced.
some ways, this is similar to a content-coding, but there
differences. One example of an instance manipulation name
"vcdiff".
2. A client signals its willingness to accept one or
instance-manipulations using the A-IM header
Cooper & Dilley Informational [Page 5]
RFC 3143 Known HTTP Proxy/Caching Problems June 2001
3. A server indicates which instance-manipulations are used
encode the body of a response using the IM header
4. Existing implementations will ignore the A-IM and IM headers
following the usual HTTP rules for handling unknown headers
5. Responses encoded with an instance-manipulation are sent
the (proposed) 226 status code, "IM Used".
6. In response to a conditional request that carries an IM header
if the request-URI has been modified then a server may
a compact encoding of the modifications using a delta-
instead of a status-200 response. The encoded response
be understood by an implementation that does not support
encodings
This summary omits many details
Suppose client A sends this request via proxy P
GET http://example.com/foo.html HTTP/1.1
Host: example.
If-None-Match: "abc
A-IM:
and the origin server returns, via P, this response
HTTP/1.1 226 IM
Etag: "def
Date: Wed, 19 Apr 2000 18:46:13
IM:
Cache-Control: max-age-60
Vary: A-IM, If-None-
the body of which is a delta-encoded response (it encodes
difference between the Etag "abc" instance of foo.html, and
"def" instance). Assume that P stores this response in its cache
and that P does not understand the vcdiff encoding
Later, client B, also ignorant of delta-encoding, sends
request via P
GET http://example.com/foo.html HTTP/1.1
Host: example.
What can P do now? According to the specification for the
header in RFC2616,
Cooper & Dilley Informational [Page 6]
RFC 3143 Known HTTP Proxy/Caching Problems June 2001
The Vary field value indicates the set of request-header
that fully determines, while the response is fresh, whether
cache is permitted to use the response to reply to a
request without revalidation
Implicitly, however, the cache would be allowed to use the
response in response to client B WITH "revalidation". This is
potential bug
An obvious implementation of the proxy would send this request
test whether its cache entry is fresh (i.e., to revalidate
entry):
GET /foo.html HTTP/1.1
Host: example.
If-None-Match: "def
That is, the proxy simply forwards the new request, after
the usual transformation on the URL and tacking on the "obvious
If-None-Match header
If the origin server's Etag for the current instance is
"def", it would naturally respond
HTTP/1.1 304 Not
Etag: "def
Date: Wed, 19 Apr 2000 18:46:14
thus telling the proxy P that it can use its stored response.
this cache response actually involves a delta-encoding that
not be sensible to client B, signaled by a header field that
be ignored by B, and so the client displays garbage
The problem here is that the original request (from client A
generated a response that is not sensible to client B, not
one that is not "the appropriate representation" (as the result
server-driven negotiation).
One might argue that the proxy P shouldn't be storing status-226
responses in the first place. True in theory, perhaps,
unfortunately RFC2616, section 13.4, says
A response received with any [status code other than 200, 203,
206, 300, 301 or 410] MUST NOT be returned in a reply to
subsequent request unless there are cache-control directives
another header(s) that explicitly allow it. For example,
Cooper & Dilley Informational [Page 7]
RFC 3143 Known HTTP Proxy/Caching Problems June 2001
include the following: an Expires header (section 14.21);
"max-age", "s-maxage", "must-revalidate", "proxy-revalidate",
"public" or "private" cache-control directive (section 14.9).
In other words, the specification allows caching of responses
yet-to-be-defined status codes if the response carries a
Cache-Control directive. So unless we ban servers
this kind of extension from using these Cache-Control
at all, the Vary header just won't work
Certain plausible extensions to the HTTP/1.1 protocol might
interoperate correctly with older HTTP/1.1 caches, if
extensions depend on an interpretation of Vary that is not
same as is used by the cache implementer
This would have the effect either of causing hard-to-debug
transparency failures, or of discouraging the deployment of
extensions, or of encouraging the implementers of such
to disable caching entirely
The problem is visible when hand-simulating plausible
exchanges, especially when using the proposed delta
extension. It probably has not been visible in practice yet
Solution(s
1. Section 13.4 of the HTTP/1.1 specification should probably
changed to prohibit caching of responses with status codes
the cache doesn't understand, whether or not they
Expires headers and the like. (It might require some care
define what "understands" means, leaving room for
extensions with new status codes.) The behavior in this
needs to be defined as equivalent to "Cache-Control: no-store
rather than "no-cache", since the latter allows revalidation
Possibly the specification of Vary should require that it
treated as "Cache-Control: no-store" whenever the status
is unknown - that should solve the problem in the
given here
Cooper & Dilley Informational [Page 8]
RFC 3143 Known HTTP Proxy/Caching Problems June 2001
2. Designers of HTTP/1.1 extensions should consider
mechanisms other than Vary to prevent false caching
It is not clear whether the Vary mechanism is
implemented in caches; if not, this favors solution #1.
A cache could treat the presence of a Vary header in a response
an implicit "Cache-control: no-store", except for "known"
codes, even though this is not required by RFC 2616. This
avoid any transparency failures. "Known status codes" for
HTTP/1.1 caches probably include: 200, 203, 206, 300, 301, 410
(although this list should be re-evaluated in light of the
discussed here).
See [9] for the specification of the delta encoding extension,
well as for an example of the use of a Cache-Control
instead of "Vary."
Jeff Mogul
2.1.2 Client Chaining Loses Valuable Length Meta-
Client Chaining Loses Valuable Length Meta-
HTTP/1.1[3] implementations are prohibited from sending Content
Length headers with any message whose body has been Transfer
Encoded. Because 1.0 clients cannot accept chunked Transfer
Encodings, receiving 1.1 implementations must forward the body
1.0 clients must do so without the benefit of information that
discarded earlier in the chain
Lacking either a chunked transfer encoding or Content-
indication creates negative performance implications for how
proxy must forward the message body
Cooper & Dilley Informational [Page 9]
RFC 3143 Known HTTP Proxy/Caching Problems June 2001
In the case of response bodies, the server may either forward
response while closing the connection to indicate the end of
response or must utilize store and forward semantics to buffer
entire response in order to calculate a Content-Length.
former option defeats the performance benefits of
connections in HTTP/1.1 (and their Keep-Alive cousin in HTTP/1.0)
as well as creating some ambiguously lengthed responses.
latter store and forward option may not even be feasible given
size of the resource and it will always introduce
latency
Request bodies must undertake the store and forward process as 1.0
request bodies must be delimited by Content-Length headers.
with response bodies this may place unacceptable
constraints on the proxy and the request may not be able to
satisfied
The lack of HTTP/1.0 style persistent connections between 1.0
clients and 1.1 proxies, only when accessing 1.1 servers, is
strong indication of this problem
Solution(s
An HTTP specification clarification that would allow origin
identity document Content-Lengths to be carried end to end
alleviate this issue
None
Patrick McManus
2.2 Known Architectural
2.2.1 Interception proxies break client cache
Interception proxies break client cache
HTTP[3] is designed for the user agent to be aware if it
connected to an origin server or to a proxy. User
believing they are transacting with an origin server but which
Cooper & Dilley Informational [Page 10]
RFC 3143 Known HTTP Proxy/Caching Problems June 2001
really in a connection with an interception proxy may fail to
critical cache-control information they would have
included in their request
Clients may receive data that is not synchronized with the
even when they request an end to end refresh, because of the
of inclusion of either a "Cache-control: no-cache" or "must
revalidate" header. These headers have no impact on origin
behavior so may not be included by the browser if it believes
is connected to that resource. Other related data
are possible as well. For instance, data security may
compromised by the lack of inclusion of "private" or "no-store
clauses of the Cache-control header under similar conditions
Easily detected by placing fresh (un-expired) content on a
proxy while changing the authoritative copy, then requesting
end-to-end reload of the data through a proxy in both
and explicit modes
Solution(s
Eliminate the need for interception proxies and IP spoofing,
will return correct context awareness to the client
Include relevant Cache-Control directives in every request at
cost of increased bandwidth and CPU requirements
Patrick McManus
2.2.2 Interception proxies prevent introduction of new HTTP
Interception proxies prevent introduction of new HTTP
A proxy that receives a request with a method unknown to it
required to generate an HTTP 501 Error as a response.
methods are designed to be extensible so there may be
deployed with initial support just for the user agent and
Cooper & Dilley Informational [Page 11]
RFC 3143 Known HTTP Proxy/Caching Problems June 2001
server. An interception proxy that hijacks requests which
new methods destined for servers that have implemented
methods creates a de-facto firewall where none may be intended
Medium within interception proxy environments
Renders new compliant applications useless unless
are made to proxy software. Because new methods are not
to be globally standardized it is impossible to keep up to date
the general case
Solution(s
Eliminate the need for interception proxies. A client receiving
501 in a traditional HTTP environment may either choose to
the request to the origin server directly, or perhaps
configured to use a different proxy
Level 5 switches (sometimes called Level 7 or application
switches) can be used to keep HTTP traffic with unknown
out of the proxy. However, these devices have heavy
responsibilities, still require TCP sequence number spoofing,
do not interact well with persistent connections
The HTTP/1.1 specification allows a proxy to switch over to
mode when it receives a request with a method or HTTP version
does not understand how to handle
Patrick McManus
Henrik Nordstrom (HTTP/1.1 clarification
2.2.3 Interception proxies break IP address-based
Interception proxies break IP address-based
Some web servers are not open for public access, but
themselves to accept only requests from certain IP address
for security reasons. Interception proxies alter the
(client) IP addresses to that of the proxy itself, without
Cooper & Dilley Informational [Page 12]
RFC 3143 Known HTTP Proxy/Caching Problems June 2001
knowledge of the client/user. This breaks such
mechanisms and prohibits otherwise allowed clients access to
servers
Creates end user confusion and frustration
Users may start to see refused connections to servers
interception proxies are deployed
Solution(s
Use user-based authentication instead of (IP) address-
authentication
Using IP filters at the intercepting device (L4 switch) and
all requests to such servers concerned
Keith K. Chau
2.2.4 Caching proxy peer selection in heterogeneous
Caching proxy peer selection in heterogeneous
ICP[4] based caching proxy peer selection in networks with
variance in latency and bandwidth between peers can lead to non
optimal peer selection. For example take Proxy C with
siblings, Sib1 and Sib2, and the following network
(summarized).
* Cache C's link to Sib1, 2 Mbit/sec with 300 msec
* Cache C's link to Sib2, 64 Kbit/sec with 10 msec latency
ICP[4] does not work well in this context. If a user submits
request to Proxy C for page P that results in a miss, C will
an ICP request to Sib1 and Sib2. Assume both siblings have
Cooper & Dilley Informational [Page 13]
RFC 3143 Known HTTP Proxy/Caching Problems June 2001
requested object P. The ICP_HIT reply will always come from Sib
before Sib1. However, it is clear that the retrieval of
objects will be faster from Sib1, rather than Sib2.
The problem is more complex because Sib1 and Sib2 can't have
100% hit ratio. With a hit rate of 10%, it is more efficient
use Sib1 with resources larger than 48K. The best choice
on at least the hit rate and link characteristics; maybe
parameters as well
By using the first peer to respond, peer selection algorithms
not optimizing retrieval latency to end users. Furthermore
are causing more work for the high-latency peer since it
respond to such requests but will never be chosen to serve
if the lower latency peer has a copy
Inherent in design of ICP v1, ICP v2, and any cache mesh
that selects peers based upon first response
This problem is not exhibited by cache digest or other
which (attempt to) maintain knowledge of peer contents and
hit peers that are believed to have a copy of the requested page
Solution(s
This problem is architectural with the peer selection protocols
Cache mesh design when using such a protocol should be done
such a way that there is not a high latency variance among peers
In the example presented in the above description the high
high bandwidth peer could be used as a parent, but should not
used as a sibling
Ivan Lovric
John Dilley
Cooper & Dilley Informational [Page 14]
RFC 3143 Known HTTP Proxy/Caching Problems June 2001
2.2.5 ICP
ICP
Architecture(ICP),
ICP[4] exhibits O(n^2) scaling properties, where n is the
of participating peer proxies. This can lead ICP traffic
dominate HTTP traffic within a network
If a proxy has many ICP peers the bandwidth demand of ICP can
excessive. System managers must carefully regulate ICP peering
ICP also leads proxies to become homogeneous in what they serve
if your proxy does not have a document it is unlikely your
will have it either. Therefore, ICP traffic requests are
unable to locate a local copy of an object (see [6]).
Inherent in design of ICP v1, ICP v2.
Solution(s
This problem is architectural - protocol redesign or
is required to solve it if ICP is to continue to be used
Implementation workarounds exist, for example to turn off use
ICP, to carefully regulate peering, or to use another mechanism
available, such as cache digests. A cache digest protocol
a summary of cache contents using a Bloom Filter technique.
allows a cache to estimate whether a peer has a document.
are updated regularly but are not always up-to-date so cannot
when a spike in popularity occurs. They also increase traffic
not as much as ICP
Proxy clustering protocols organize proxies into a mesh
another alternative solution. There is ongoing research on
topic
John Dilley
Cooper & Dilley Informational [Page 15]
RFC 3143 Known HTTP Proxy/Caching Problems June 2001
2.2.6 Caching proxy meshes can break HTTP serialization of
Caching proxy meshes can break HTTP serialization of
Architecture (HTTP protocol
A caching proxy mesh where a request may travel different paths
depending on the state of the mesh and associated caches,
break HTTP content serialization, possibly causing the end user
receive older content than seen on an earlier request, where
request traversed another path in the mesh
Can cause end user confusion. May in some situations (
cache hit, object has changed state from cacheable to uncacheable
be close to impossible to get the caches properly updated with
new content
Older content is unexpectedly returned from a caching proxy
after some time
Solutions(s
Work with caching proxy vendors and researchers to find a
protocol for maintaining proxy relations and object state in
mesh
When designing a hierarchy/mesh, make sure that for each end
user/URL combination there is only one single path in the
during normal operation
Henrik Nordstrom
Cooper & Dilley Informational [Page 16]
RFC 3143 Known HTTP Proxy/Caching Problems June 2001
2.3 Known Implementation
2.3.1 User agent/proxy
User agent/proxy
Failover between proxies at the user agent (using a proxy.pac[8]
file) is erratic and no standard behavior is defined
Additionally, behavior is hard-coded into the browser, so
proxy administrators cannot use failover at the user
effectively
Architects are forced to implement failover at the proxy itself
when it may be more appropriate and economical to do it within
user agent
If a browser detects that its primary proxy is down, it will
n minutes before trying the next one it is configured to use.
will then wait y minutes before asking the user if they'd like
try the original proxy again. This is very confusing for
users
Solution(s
Work with browser vendors to establish standard extensions
JavaScript proxy.pac libraries that will allow configuration
these timeouts
User education; redundancy at the proxy level
Mark Nottingham
Cooper & Dilley Informational [Page 17]
RFC 3143 Known HTTP Proxy/Caching Problems June 2001
2.3.2 Some servers send bad Content-Length headers for files
contain
Some servers send bad Content-Length headers for files
contain
Certain web servers send a Content-length value that is
than number of bytes in the HTTP message body. This happens
the server strips off CR characters from text files with
terminated with CRLF as the file is written to the client.
server probably uses the stat() system call to get the file
for the Content-Length header. Servers that exhibit this
include the GN Web server (version 2.14 at least).
Low. Surveys indicate only a small number of sites run
servers
In this case, an HTTP client (e.g., user agent or proxy)
believe it received a partial response. HTTP/1.1 [3] advises
caches MAY store partial responses
Count the number of bytes in the message body and compare to
Content-length value. If they differ the server exhibits
problem
Upgrade or replace the buggy server
Some browsers and proxies use one TCP connection per object
ignore the Content-Length. The document end of file is
by the close of the TCP socket
Duane Wessels measurement-factory.com
3. Security
This memo does not raise security considerations in itself. See
individual submissions for details of security concerns and issues
Cooper & Dilley Informational [Page 18]
RFC 3143 Known HTTP Proxy/Caching Problems June 2001
[1] Paxson, V., Allman, M., Dawson, S., Fenner, W., Griner, J.,
Heavens, I., Lahey, K., Semke, J. and B. Volz, "Known
Implementation Problems", RFC 2525, March 1999.
[2] Cooper, I., Melve, I. and G. Tomlinson, "Internet
Replication and Caching Taxonomy", RFC 3040, January 2001.
[3] Fielding, R., Gettys, J., Mogul, J., Frystyk, H., Masinter, L.,
Leach, P. and T. Berners-Lee, "Hypertext Transfer Protocol --
HTTP/1.1", RFC 2616, June 1999.
[4] Wessels, D. and K. Claffy, "Internet Cache Protocol (ICP),
Version 2", RFC 2186, September 1997.
[5] Davison, B., "Web Traffic Logs: An Imperfect Resource
Evaluation", in Proceedings of the Ninth Annual Conference
the Internet Society (INET'99), July 1999.
[6] Melve, I., "Relation Analysis, Cache Meshes", in Proceedings
the 3rd International WWW Caching Workshop, June 1998,
workshop/29/magicnumber.html>.
[7] Krishnamurthy, B. and M. Arlett, "PRO-COW: Protocol
on the Web", AT&T Labs Technical Report #990803-05-TM,
1999, research.att.com/~bala/papers/procow-1.ps.gz>.
[8] Netscape, Inc., "Navigator Proxy Auto-Config File Format",
1996,
http://home.netscape.com/eng/mozilla/2.0/relnotes/demo/proxy
live.
[9] Mogul, J., Krishnamurthy, B., Douglis, F., Feldmann, A., Goland
Y., van Hoff, A. and D. Hellerstein, "HTTP Delta in HTTP",
in Progress
Cooper & Dilley Informational [Page 19]
RFC 3143 Known HTTP Proxy/Caching Problems June 2001
Authors'
Ian
Equinix, Inc
2450 Bayshore
Mountain View, CA 94043
Phone: +1 650 316 6065
EMail: icooper@equinix.
John
Akamai Technologies, Inc
1400 Fashion Island
Suite 703
San Mateo, CA 94404
Phone: +1 650 627 5244
EMail: jad@akamai.
Cooper & Dilley Informational [Page 20]
RFC 3143 Known HTTP Proxy/Caching Problems June 2001
Appendix A. Archived Known
The following sub-sections are an archive of problems identified
the initial production of this memo. These are typically
requiring further work/research, or user education. They
included here for reference purposes only
A.1
A.1.1 Cannot specify multiple URIs for replicated
Cannot specify multiple URIs for replicated
There is no way to specify that multiple URIs may be used for
single resource, one for each replica of the resource. Similarly
there is no way to say that some set of proxies (each
by a URI) may be used to resolve a URI
Forces users to understand the replication model and mechanism
Makes it difficult to create a replication framework
protocol support for replication and naming
Inherent in HTTP/1.0, HTTP/1.1.
Solution(s
Architectural - protocol design is necessary
Replication mechanisms force users to locate a replica or
site for replicated content
Daniel LaLiberte
Cooper & Dilley Informational [Page 21]
RFC 3143 Known HTTP Proxy/Caching Problems June 2001
A.1.2 Replica distance is
Replica distance is
There is no recommended way to find out which of several
or proxies is closer either to the requesting client or to
machine, either geographically or in the network topology
Clients must guess which replica is closer to them when
a copy of a document that may be served from multiple locations
Users must know the set of servers that can serve a
object. This in general is hard to determine and maintain.
must understand network topology in order to choose the
copy. Note that the closest copy is not always the one that
result in quickest service. A nearby but heavily loaded
may be slower than a more distant but lightly loaded server
Inherent in HTTP/1.0, HTTP/1.1.
Solution(s
Architectural - protocol work is necessary. This is a
instance of a general problem in widely distributed systems.
general solution is unlikely, however a specific solution in
web context is possible
Servers can (many do) provide location hints in a
selection web page. Users choose one based upon their location
Users can learn which replica server gives them best performance
Note that the closest replica geographically is not
the closest in terms of network topology. Expecting users
understand network topology is unreasonable
Daniel LaLiberte
Cooper & Dilley Informational [Page 22]
RFC 3143 Known HTTP Proxy/Caching Problems June 2001
A.1.3 Proxy resource
Proxy resource
There is no way for a client or server (including another proxy
to inform a proxy of an alternate address (perhaps including
proxy to use to reach that address) to use to fetch a resource
If the client does not trust where the redirected resource
from, it may need to validate it or validate where it came from
Proxies have no systematic way to locate resources within
proxies or origin servers. This makes it more difficult to
information among proxies. Information sharing would
global efficiency
Inherent in HTTP/1.0, HTTP/1.1.
Solution(s
Architectural - protocol design is necessary
Certain proxies share location hints in the form of
digests of their contents (e.g., Squid). Certain proxy
enable a proxy query another for its contents (e.g., ICP). (
however "ICP Performance" issue (Section 2.2.5).)
Daniel LaLiberte
A.2
A.2.1 Use of Cache-Control
Use of Cache-Control
Cooper & Dilley Informational [Page 23]
RFC 3143 Known HTTP Proxy/Caching Problems June 2001
Many (if not most) implementations incorrectly interpret Cache
Control response headers
Cache-Control headers will be spurned by end users if there
conflicting or non-standard implementations
-
Solution(s
Work with vendors and others to assure proper
None
Mark Nottingham
A.2.2 Lack of HTTP/1.1 compliance for caching
Lack of HTTP/1.1 compliance for caching
Although performance benchmarking of caches is starting to
explored, protocol compliance is just as important
Caching proxy vendors implement their interpretation of
specification; because the specification is very large,
vague and ambiguous, this can lead to inconsistent
between caching proxies
Caching proxies need to comply to the specification (or
specification needs to change).
Cooper & Dilley Informational [Page 24]
RFC 3143 Known HTTP Proxy/Caching Problems June 2001
There is no currently known compliance test being used
There is work underway to quantify how closely servers comply
the current specification. A joint technical report between AT&
and HP Labs [7] describes the compliance testing. This
examines how well each of a set of top traffic-producing
support certain HTTP/1.1 features
The Measurement Factory (formerly IRCache) is working to
protocol compliance testing software. Running such a
test suite against caching proxy products would measure
and ultimately would help assure they comply to the specification
Solution(s
Testing should commence and be reported in an open industry forum
Proxy implementations should conform to the specification
There is no workaround for non-compliance
Mark Nottingham
Duane Wessels measurement-factory.com
A.2.3 ETag
ETag
Available caching proxies appear not to support ETag (strong
validation
Last-Modified/If-Modified-Since validation is inappropriate
many requirements, both because of its weakness and its use
dates. Lack of a usable, strong coherency protocol
developers and end users not to trust caches
-
Cooper & Dilley Informational [Page 25]
RFC 3143 Known HTTP Proxy/Caching Problems June 2001
Solution(s
Work with vendors to implement ETags; work for better
protocols
Use Last-Modified/If-Modified-Since validation
Mark Nottingham
A.2.4 Servers and content should be optimized for
Servers and content should be optimized for
Implementation (Performance
Many web servers and much web content could be implemented to
more conducive to caching, reducing bandwidth demand and page
delay
By making poor use of caches, origin servers encourage longer
times, greater load on caching proxies, and increased
demand
The problem is most apparent for pages that have low or
expires time, yet do not change
Solution(s
-
Servers could start using unique object identifiers for write-
content: if an object changes it gets a new name, otherwise it
considered to be immutable and therefore have an infinite
age. Certain hosting providers do this already
Peter
Cooper & Dilley Informational [Page 26]
RFC 3143 Known HTTP Proxy/Caching Problems June 2001
A.3
A.3.1 Lack of fine-grained, standardized hierarchy
Lack of fine-grained, standardized hierarchy
There is no standard for instructing a proxy as to how it
resolve the parent to fetch a given object from.
therefore vary greatly, and it can be difficult to make
interoperate correctly in a complex environment
Complications in deployment of caches in a complex
(especially corporate networks
Inability of some proxies to be configured to direct traffic
on domain name, reverse lookup IP address, raw IP address,
normal operation and in failover mode. Inability in some
to set a preferred parent / backup parent configuration
Solution(s
-
Work with vendors to establish an acceptable configuration
the limits of their product; standardize on one product
Mark Nottingham
A.3.2 Proxy/Server exhaustive log format standard for
Proxy/Server exhaustive log format standard for
Cooper & Dilley Informational [Page 27]
RFC 3143 Known HTTP Proxy/Caching Problems June 2001
Most proxy or origin server logs used for characterization
evaluation do not provide sufficient detail to
cacheability of responses
Low (for operationality; high significance for research efforts
Characterizations and simulations are based on non-
workloads
See
W3C Web Characterization Activity, since they are also
with collecting high quality logs and building
from them
-
Solution(s
To properly clean and to accurately determine cacheability
responses, a complete log is required (including all
headers as well as all response headers such as "User-agent" [
removal of spiders] and "Expires", "max-age", "Set-cookie", "no
cache", etc.)
-
See "Web Traffic Logs: An Imperfect Resource for Evaluation"[5]
for some discussion of this
Brian D. Davison
Terence Kelly
A.3.3 Trace log
Trace log
Cooper & Dilley Informational [Page 28]
RFC 3143 Known HTTP Proxy/Caching Problems June 2001
Some proxies/servers log requests without sufficient
detail. Millisecond resolution is often too small to
request ordering and either the servers should record
reception time in addition to completion time, or elapsed
plus either one
Low (for operationality; medium significance for research efforts
Characterization and simulation fidelity is improved with
timing and ordering information. Since logs are generally
in order of request completion, these logs cannot be re-
without knowing request generation times and
accordingly
See
-
Timestamps can be identical for multiple entries (when
millisecond resolution is used). Request orderings can be
when clients open additional connections for embedded
while still receiving the container object
Solution(s
Since request completion time is common (e.g., Squid),
continuing to use it (with microsecond resolution if possible
plus recording elapsed time since request reception
-
See "Web Traffic Logs: An Imperfect Resource for Evaluation"[5]
for some discussion of this
Brian D. Davison
A.3.4 Exchange format for log
Exchange format for log
Administration/Analysis
Cooper & Dilley Informational [Page 29]
RFC 3143 Known HTTP Proxy/Caching Problems June 2001
Although we have (more or less) a standard log file format
proxies (plain vanilla Common Logfile and Squid), there isn't
commonly accepted format for summaries of those log files
Summaries could be generated by the cache itself, or by post
processing existing log file formats such as Squid's
High, since it means that each log file summarizing/analysis
is essentially reinventing the wheel (un-necessary repetition
code), and the cost of processing a large number of large
files through a variety of analysis tools is (again for no
reason) excessive
In order to perform a meaningful analysis (e.g., to
performance in relation to loading/configuration over time)
access logs from multiple busy caches, it's often necessary to
first one tool then another, each against the entire log file (
a significantly large subset of the log). With log files
into hundreds of MB even after compression (for a cache
with millions of transactions per day) this is a non-trivial task
See
IP packet/header sniffing - it may be that individual
are at a level of granularity which simply isn't sensible to
attempting on extremely busy caches. There may also be
implications in some countries, e.g., if this analysis
individuals
Disks/memory full(!) Stats (using multiple programs) take too
to run. Stats crunching must be distributed out to
machines because of its high computational cost
Solution(s
Have the proxy produce a standardized summary of its
either automatically or via an external (e.g., third party) tool
in a commonly agreed format. The format could be something
XML or the Extended Common Logfile, but the format and
are subjects for discussion. Ideally this approach would
individual cache server products to supply subsets of the
summary info, since it may not be feasible for all servers
provide all of the information which people would like to see
Cooper & Dilley Informational [Page 30]
RFC 3143 Known HTTP Proxy/Caching Problems June 2001
Devise a private summary format for your own personal use -
this complicates or even precludes the exchange of summary
with other interested parties
See the web pages for the commonly used cache stats
programs, e.g., Calamaris, squidtimes, squidclients, etc
Martin Hamilton
Cooper & Dilley Informational [Page 31]
RFC 3143 Known HTTP Proxy/Caching Problems June 2001
Full Copyright
Copyright (C) The Internet Society (2001). All Rights Reserved
This document and translations of it may be copied and furnished
others, and derivative works that comment on or otherwise explain
or assist in its implementation may be prepared, copied,
and distributed, in whole or in part, without restriction of
kind, provided that the above copyright notice and this paragraph
included on all such copies and derivative works. However,
document itself may not be modified in any way, such as by
the copyright notice or references to the Internet Society or
Internet organizations, except as needed for the purpose
developing Internet standards in which case the procedures
copyrights defined in the Internet Standards process must
followed, or as required to translate it into languages other
English
The limited permissions granted above are perpetual and will not
revoked by the Internet Society or its successors or assigns
This document and the information contained herein is provided on
"AS IS" basis and THE INTERNET SOCIETY AND THE INTERNET
TASK FORCE DISCLAIMS ALL WARRANTIES, EXPRESS OR IMPLIED,
BUT NOT LIMITED TO ANY WARRANTY THAT THE USE OF THE
HEREIN WILL NOT INFRINGE ANY RIGHTS OR ANY IMPLIED WARRANTIES
MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE
Funding for the RFC Editor function is currently provided by
Internet Society
Cooper & Dilley Informational [Page 32]
if you see any problems within the linking, don't worry be happy,
this is version 0.1 of the Relevance System and you gotta expect some crappy subroutines sometimes,
just be content we did not write this in Java, which would have made this "bigger and better" HAHAHHA.
RFC documents can be found at I.E.T.F.
Relevance System Copyright © 2002 Spectrum WorldResearch
other technical nosh by ServerMasters Corporation
collaboration of BobX
