[JDEV] Stress testing and connection limits
xkahn at cybersites.com
Wed Jun 14 15:09:05 CDT 2000
There are two things at issue here. On my Linux box, the
FD_SETSIZE macro is set to 1024 -- so that's all the FDs that select will
take. You can check your machine by looking at:
But also, each process is limited to 1024 file descriptors. This
is a list by OS: (old)
For more information on the select trouble, you can look at:
An interesting note from that page is:
Macro: int FD_SETSIZE
The value of this macro is the maximum number of file descriptors
that a fd_set object can hold information about. On systems with a
fixed maximum number, FD_SETSIZE is at least that number. On some
systems, including GNU, there is no absolute limit on the number
of descriptors open, but this macro still has a constant value
which controls the number of bits in an fd_set; if you get a file
descriptor with a value as high as FD_SETSIZE, you cannot put that
descriptor into an fd_set.
In other words, yes, it is limited, but no... It doens't look
like it needs to be.
As a side note, I seem to recall a conversation on the kernel list
about a limit of 64k sockets in the select list!
On Wed, 14 Jun 2000, Eric Bowersox wrote:
> Date: Wed, 14 Jun 2000 13:38:24 -0600
> From: Eric Bowersox <ebowersox at jabber.com>
> Reply-To: jdev at jabber.org
> To: jdev at jabber.org
> Subject: RE: [JDEV] Stress testing and connection limits
> > ulimit on my Linux server that jserver is running on reports
> > 1024 open file
> > handles. I've tried changing that number with ulimit but I'm
> > not sure if the
> > number of handles really changes. I don't think so.
> It's not the number of open file handles that is set with ulimit that
> matters, it's the maximum number of file handles in a select() call. This
> is limited by factors which are set at compile time and difficult to change
> (see below).
> > The
> > implication from
> > what you've said is that the max number of jabber users
> > connected at one
> > point is limited. In my case, 1024. Is that true? Isn't that as fairly
> > serious restriction? I was expecting to have a powerful
> > server with 10's
> > maybe 100's of thousands of connected users.
> I agree, it's serious. That's why the server developers (Jer and temas,
> namely) are working their asses off to develop the "mux" solution.
> > Can the kernel
> > be rebuilt with
> > a larger size? If I change the limit to 100,000 or 200,000,
> > will Linux still
> > run?
> Again, the number of file handles the kernel has is *not relevant* here,
> because the kernel has way more handles available already than a single
> select() can use. The problem is with the size of the FD_SET structure that
> select() uses. This structure's size is fixed at compile time, and almost
> certainly cannot be changed without severe breakage in many places (such as
> the C library).
> I have a feeling this is why poll() was introduced, to get around the
> restrictions of select(). However, that doesn't help us, as pth uses
> select() as part of the very core of its event-handling system, and pth is
> some heavy hoodoo-voodoo at the center of the Jabber server, so attempting
> to modify it would be "Not Recommended."
> jdev mailing list
> jdev at jabber.org
------------------------------------ |\ _,,,--,,_ ,) ----------
Benjamin Kahn /,`.-'`' -, ;-;;'
(212) 924 - 2220 ext 201 |,4- ) )-,_ ) /\
ben at cybersites.com --------------- '---''(_/--' (_/-' ---------------
I just want to make the world a better place for me to live in.
-Lucy in a Peanuts comic
More information about the JDev