[svn.haxx.se] · SVN Dev · SVN Users · SVN Org · TSVN Dev · TSVN Users · Subclipse Dev · Subclipse Users · this month's index

(forw) Re: [PATCH] Compressed streams (take 2)

From: Eric Dorland <eric.dorland_at_mail.mcgill.ca>
Date: 2003-04-01 03:00:59 CEST

Sorry, meant to send this to the list.

-- 
Eric Dorland <eric.dorland@mail.mcgill.ca>
ICQ: #61138586, Jabber: hooty@jabber.com
1024D/16D970C6 097C 4861 9934 27A0 8E1C  2B0A 61E9 8ECF 16D9 70C6
-----BEGIN GEEK CODE BLOCK-----
Version: 3.12
GCS d- s++: a-- C+++ UL+++ P++ L++ E++ W++ N+ o K- w+ 
O? M++ V-- PS+ PE Y+ PGP++ t++ 5++ X+ R tv++ b+++ DI+ D+ 
G e h! r- y+ 
------END GEEK CODE BLOCK------

attached mail follows:


* Philip Martin (philip@codematters.co.uk) wrote:
> Eric Dorland <eric.dorland@mail.mcgill.ca> writes:
>
> > It was basically arbitrary. I picked something that wasn't too small
> > and that would likely be the size of a page, or some constant multiple
> > of the size of a page.
>
> That's the sort of information that should go in a comment beside the
> constant.

Good idea.
 
> > I'm sure different sizes would yield different
> > performance characteristics in different situations (eg a bigger
> > buffer would be better for larger reads, but penalize smaller ones). I
> > don't think it really matters too much, since I think the IO system
> > under Linux and other Unices would cache things enough to make it
> > pretty irrelevant what I choose (as long as it's not too insane).
>
> APR will also provides buffering of files if requested.

Right, but I need to allocate my own buffer to pass to the zlib
functions, and I don't think I can get a hold of APR's buffer.
 
> > I could even make the buffer dynamic in size, with some heuristic
> > based on the amount of data the user wants to read it.
>
> I'm not sure a heuristic is a good idea, mainly because I don't know
> how to evaluate it. What about a buffer size argument to
> svn_stream_compressed? Is that a good idea? Perhaps with a #define
> (your 4096) for a default value.

Well generally the buffer would probably in the worst case be slightly
bigger than the amount of data being read in (can't be too scientific,
probably someone knew a lot about compression could be). So maybe the
size being read + 20%. I think passing a parameter is total
overkill. I never had to pass a buffer size to a buffered IO function
before... I think this exposes the implementation too much.

> > It's probably
> > not worth it though. But hey, if it gets my patch in, I'll do it :)
>
> I'm not trying to be awkward, I'm just trying to understand the patch
> without going to the trouble of reading the zlib documentation.

Well if you take a peek at /usr/include/zlib.h, there's not too much
to it. Feel free to ask any questions about zlib, but I'm by no means
an expert.

-- 
Eric Dorland <eric.dorland@mail.mcgill.ca>
ICQ: #61138586, Jabber: hooty@jabber.com
1024D/16D970C6 097C 4861 9934 27A0 8E1C  2B0A 61E9 8ECF 16D9 70C6
-----BEGIN GEEK CODE BLOCK-----
Version: 3.12
GCS d- s++: a-- C+++ UL+++ P++ L++ E++ W++ N+ o K- w+ 
O? M++ V-- PS+ PE Y+ PGP++ t++ 5++ X+ R tv++ b+++ DI+ D+ 
G e h! r- y+ 
------END GEEK CODE BLOCK------

  • application/pgp-signature attachment: stored
  • application/pgp-signature attachment: stored
Received on Tue Apr 1 03:01:40 2003

This is an archived mail posted to the Subversion Dev mailing list.