caching special kind of Web request

Hi everybody,

I've got a particular (?) case here and I'd like to
know if any program (Squid?) can be used in such
a situation. The answer may simply be "yes, of
course", but before making changes to the setup
here I'd like to be sure...

Here's the setup: a webserver, which I've got no
control at all on, dynamically serves webpages to
many machines in our LAN (the people here are
using a special 'webapp'). The webserver itself
is not on the LAN.

The machines on the LAN typically 'refresh' some
elements on their webpage every minute or so and
those machines all issue those request simultaneously
(or nearly simultaneously).

When a 'new' request is made, it is impossible that
the resource are cached: the webserver just
dynamically generated them.

Here's an example :

at t0 machine A asks for "dynamic18748.jpg"
(that the server just created)

(typically, in our case, 'dynamic18748.jpg' will
be quite big... and there'll be more than one
picture asked for)

at t0 + 100 ms machine B and C asks for that same "dynamic18748.jpg"

at t0 + 150 ms machine D asks for that same "dynamic18748.jpg"

etc.

Does a caching proxy realize, when B, C, D (and all the
others) ask for "dynamic18748.jpg" that the file is "being
transferred" or does the proxy start a new (unnecessary)
request to the webserver?

I'm mostly (if not 'only') interested in caching HTTP GET
request for pictures... It doesn't have to be "transparent",
the clients here on the LAN can be configured to access
a proxy.

I'd like to cache all these pictures for a few minutes, then
discard them after a few minutes (I mean, I would care
about keeping them for a few hours, but they'd serve no
purpose). Due to how the setup/webapp works in our case,
each of these pictures would be accessed at least one by
every machine in our LAN (more or less), which is why I'd
like them to be cached even though all the request will be
'simultaneous'.

My question may boil down to...

When does a 'web proxy cache' (like Squid) start caching?

As soon as the resource as been fully transferred to the proxy
or as soon as the resource is starting to be transferred to the proxy
(which in my case makes a lot of difference)?

I'm sorry if it's not very clear: english is not my first language.

Any explanations are very welcome,

Tristan

In comp.os.linux.networking (E-Mail Removed):
[ http proxy? ]

> I'm mostly (if not 'only') interested in caching HTTP GET
> request for pictures... It doesn't have to be "transparent",
> the clients here on the LAN can be configured to access a
> proxy.

You can run squid in transparent mode, forcing web traffic at
your firewall/gateway to be redirected to the squid proxy
server. No client configuration needed.

[..]

> When does a 'web proxy cache' (like Squid) start caching?

> As soon as the resource as been fully transferred to the proxy
> or as soon as the resource is starting to be transferred to the proxy
> (which in my case makes a lot of difference)?

Dunno the exact answer, though it should be trivial to find out,
running a proxy with a few clients and checking squid access logs
when it starts handing out cached content. Perhaps you could even
slow down clients a bit, to get squid the file cached once.
Though it might depend on the client request and what the html
page says concerning caching.

I'd recommend to do some testing or/and ask (search the archives
before!) http://www.squid-cache.org/ some squid mailing list
about the matter. Probably you get better advice there or/and
some idea what to do?

Good luck

--
Michael Heiming (X-PGP-Sig > GPG-Key ID: EDD27B94)
mail: echo (E-Mail Removed) | perl -pe 'y/a-z/n-za-m/'
#bofh excuse 285: Telecommunications is upgrading.

Michael Heiming wrote:
....
> Dunno the exact answer, though it should be trivial to find out,
....
> I'd recommend to do some testing or/and ask (search the archives
> before!) http://www.squid-cache.org/ some squid mailing list
> about the matter. Probably you get better advice there or/and
> some idea what to do?

You're right, I'd probably better simply try it out. I've got some
machines at home, I'll install Squid and see what's going on.

I may try the mailing lists first though

Thanks for your answer,

Tristan