[svn.haxx.se] · SVN Dev · SVN Users · SVN Org · TSVN Dev · TSVN Users · Subclipse Dev · Subclipse Users · this month's index

Re: Total failure in dav layer when >1024 files already open

From: Greg Hudson <ghudson_at_MIT.EDU>
Date: 2006-06-03 18:21:11 CEST

On Sat, 2006-06-03 at 08:12 -0400, Michael Sweet wrote:
> > fd_set cannot be a dynamically allocated structure because you have to
> > be able to copy it with assignment
>
> Why? No code I have ever seen does this...

Because select() modifies its input arguments, it's natural to keep a
master copy of the fds you want to track, and copy it before invoking
select(). Since there is no copy method, structure assignment has to
suffice. A lot of code sets up the fd_set from scratch every time, but
there certainly exists code which does not.

At any rate, there's no deallocation function. If FD_SET() and friends
were to dynamically allocate memory pointed to by the fd_set structure,
that memory would be leaked when the lifetime of the fd_set structure
ends.

(There's also no failure return for FD_SET to handle out of memory
conditions.)

> Every OS other than Linux allows select() to work with an arbitrary
> number of file descriptors; even the Linux kernel allows it, just not
> glibc.

I believe you're simply mistaken. Solaris has a default limit of 1024
(or 65536 for 64-bit code). NetBSD has a default limit of 256. It's
extremely unlikely that you can find even one OS which tries to do a
dynamically-sized fd_set.

The Linux kernel can allow an arbitrary number of file descriptors
because it isn't responsible for the particular semantics of FD_SET and
friends.

> Try managing and array of thousands of poll entries sometime, and
> then compare the efficiency of a bit test vs. scanning an array
> after the fact...

Both select() and poll() require you to scan the fd set structure after
the call to find out what I/O events actually happened. No difference
there. This is not normally an issue, as performing a few thousand
memory accesses is generally cheap compared to a single I/O operation,
but there have been some stabs at solving the theoretical scaling
problem, such as Linux's epoll(), Solaris's /dev/poll, and the like.

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Received on Sat Jun 3 18:22:26 2006

This is an archived mail posted to the Subversion Dev mailing list.