How important is it that this is specifically "web pages" i.e. html loaded in a web browser as opposed to non www? And http versus other protocols? I ask for two reasons. First in 1996 the www had not yet won the day so if you read the academic work from that time www is clearly important and seen as the likely winner but a lot of attention is also give to similar problems in usenet, gopher, and other protocols besides http and of course today just as then lots of mobile was not on the web.
This article is from 1995
http://ftp.isi.edu/touch/pubs/jsac95.pdf
Touch, J.D., "Defining high-speed protocols: five challenges and an
example that survives the challenges," Selected Areas in
Communications, IEEE Journal on , vol.13, no.5, pp.828,835, Jun 1995
doi: 10.1109/49.391751 Abstract: The First IEEE Gigabit Networking
(GBN) Workshop defined a set of characteristics of “Interesting”
high-speed applications. The GBN criteria ensure that the application
addresses a significant problem, and that it actually requires a
gigabit network. This paper presents five challenges that augment the
GBN criteria. These challenges ask whether gigabit applications
require new research into different protocols, or can be supported by
existing protocols that merely run faster. It shows a class of
applications, interactive distributed multimedia, namely interactive
real-time World Wide Web (WWW) access, that survive the challenges. It
also shows how source presenting is a way to use excess
bandwidth-delay product to reduce the browser response time, and how
this is one example of a truly gigabit protocol keywords: {computer
networks;delays;distributed processing;interactive systems;multimedia
communication;professional aspects;protocols;societies;First IEEE
Gigabit Networking Workshop;bandwidth-delay product;browser response
time;gigabit network;gigabit protocol;high-speed
applications;high-speed protocols;interactive distributed
multimedia;interactive real-time World Wide
Web;research;Bandwidth;Broadcasting;Computational modeling;Concurrent
computing;Conferences;Distributed computing;Government;Multimedia
communication;Protocols;Telemedicine}, URL:
http://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=391751&isnumber=8876
In the article Touch discusses the need to "keep the pipe full" in order to optimize speed. In particular he discusses the World Wide Web as a gigabyte application and on pages 829-830
2) Even with Large Windows, There Is Not Enough Stuff: Even if the
windowing mechanism allows large amounts of pipelining, there may not
be enough data with which to fill the pipe. A gigabit WAN has 30-100
Mb in the pipe4-I2 Mbytes. That is more RAM than many systems have,
and certainly larger than most messages an application has to
transmit.
One solution is to use multiplexing to share the channel among user
processes 181. This is a parallel of process- swapping in OS-when one
process runs out of data to send, another is activated. This works,
provided that the process activation is deterministic, i.e., that the
other side of the channel knows what the process activation order is
[l 11. This is also tantamount to not having a gigabit application
protocol+ach application does not use a gigabit channel, so bandwidth
needs to be aggregated over a set of multiplexed applications. The
case where the application order is not predictable is more
interesting, as will be discussed later
So in other words, to have the best performance no "space" should be left unused, this is almost always the case when loading a single page of html, so more data should be sent than just one page worth. However the issue of predictability is key for solutions.
On 830 he continues
B. Challenge #2--Multiplux (Determinisfic) Multiplexing has been
proposed as a solution to the “do not have enough stuff to send”
issue, as mentioned before [SI. This is equivalent to not having a
gigabit application-the gigabits are shared among a set of
applications on a workstation. Using multiple hosts, users, or
processes are all ways of providing aggregate gigabits only. In the
deterministic multiplexer case, this avoids the domain examined here
[ 1 11. In the nondeterministic case, the problem has just moved down
a level in the protocol stack. In the original case, there was not
enough data to send, primarily because data was sent to be sufficient
for the current state of the other end of the channel, and not any
possible subsequent states. If there were enough for subsequent
states, that data would have been sent too, and so on, increasing the
amount of data available to send ad infinitum. This is how sliding
windows works-by predicting subsequent states, in a linear manner.
If the subsequent state is not predictable, neither is the subsequent
data [12]. It does not matter whether it is the application state, or
the multiplexer state. Nondeterminis- tic multiplexing moves the
state prediction problem to the multiplexer-synchronization level,
i.e., lower in the protocol stack.
I'm not going to copy all of the rest of the related portions but among other things he discusses the difference between prefetching and preloading and the problem of unpredictability. Although I don't see statistical prediction the article does propose differentiating preloads of "local" hrefs (from the same domain) and "all" hrefs and has a figure where they appear to have collected data on this. I wonder if this may be enough differentiation.
He does mention the issue of predictability being of key importance and I think that using statistical analysis to proxy deterministic predictibility is part of what is described in this patent.
To me the next article is really to the point in that they really implemented a variety of solutions but across many different kinds of systems, with their discussion of what they did for Usenet particularly interesting.
H. Shrikumar, Rehmi Post, Thinternet: life at the end of a tether, Computer Networks and ISDN Systems, Volume 27, Issue 3, December 1994, Pages 375-385, ISSN 0169-7552,
http://dx.doi.org/10.1016/0169-7552(94)90111-2.
(http://www.sciencedirect.com/science/article/pii/0169755294901112)
Abstract: As the Internet continues its exponential growth, the user
profile is changing. Many of the newer Internet hosts are personal
workstations, often connected by dial-up or other slow links. We
examine some factors that motivate or mandate “thin” (low-bandwidth)
connections to the Internet. We notice that the motivation for
adopting thin links in the West can be different from those in
developing countries. Using a profile of such typical users, we show
how techniques exist that allow practical and adequately efficient use
of the Internet even “at the end of a tether”.
We are exploring the use of these methods in routine Internet use from
a site in India (a software development laboratory, multi-user LAN,
connected to an Internet service provider through an expensive dial-up
link) and from mobile computers (e.g., HP100LX and Gateway Handbook)
in the US. In each case the user's Internet access is through a thin
link, with a bandwidth somewhere between 2400 bps and 28.8 kbps.
Local caching and prefetching of resources naturally suggests itself
as a useful candidate. It appears that transparent replay of
application protocols is a practical way to retrofit resource caching
into existing (shrinkwrapped) software. One promising method which
works with most services of interest is Postel spoofing.
Given the “browsing” mode of network usage, progressive encoding
mechanisms are shown to effectively reduce the access time for
particularly large Internet objects, such as Web pages. An ideal
progressive encoding of a resource sends a gross quality rendering
followed by successive refinements. Since only a fraction of the
images retrieved in a session actually have long-term value, such
techniques can reduce on-line bandwidth demands by an order of
magnitude. Obviously, such encoding methods apply also to large
archive and distribution files (such as from FTP archives).
Filtering and relevance feedback have been recognised as effective
tools in overcoming information overload. Many sophisticated general
techniques are a subject of active research. However, we found that
exploiting certain behaviour patterns typical in Internet usage
permits particularly efficient filtering using surprisingly simple
methods.
We apply this to USENET communication, and extend this to other
services (FTP, HTTP, Gopher, mail, etc.), and outline a method of
filtering network hypermedia on the basis of relevance contours. Our
method recognises the amount of selected information that can be
digested by a user in a day, and maximises the value of the packet so
selected. It also differs from others by integrating all network
hypermedia and selecting and filtering items without regard to the
service they were accessed from (USENET, Web, Gopher, FTP, etc.). It
does not depend on a-priori categorisation such as news groups and the
consequent need for explicit subscription and unsubscription.
On page 222-09 they say
It also appear that cache hit ratios can be improved considerably by
pre-fetching objects based on the branching probabilities of threads
of browsing. These can either be statically determined or dynamically
discovered. Quantitative information concerning patterns of reference
and locality in the Web is hard to come by, but this appears to be an
av enue worth exploring
There is also a dissertation that mainly focuses on preloading in file systems but says that similar approaches would apply on the WWW and an NSF document that talks about machine learning based on user click patterns to develop a way of predicting likely browsing patterns and using that to optimize.
SOme of the work on Rover may also be relevant http://scholar.google.com/scholar?q=rover+prefetching&btnG=&hl=en&as_sdt=0%2C33&as_yhi=1996
For example page 14 of
http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.45.6672&rep=rep1&type=pdf describes the issue
There are a lot of other examples from around the same time.
Hope that gives some help or clues.
prior-art-request
