[Libevent-users] Possible configure changes for epoll

[Scott Lamb]

Rolf Vandevaart wrote:
> Hello:
> 
> We embed libevent in the Open MPI project; long ago, we folded 
> libevent's configure m4 code into our own configure code.  Over time, we 
> tweaked the libevent m4 code for our own purposes as we ran into 
> compatibility issues, etc.  We're submitting these changes to you in the 
> hopes that they will be useful.  Feel free to use them or disregard them.
> 
> We've changed the form of the original m4 so that it's not directly 
> [re-]importable to libevent, but perhaps the code can be copied and 
> morphed back into your configure.in.  Here's a link to our current 
> revision of the libevent-specific m4 in our configure system:
> 
> https://svn.open-mpi.org/trac/ompi/browser/trunk/config/ompi_setup_libevent.m4 
> 
> 
> Here's a list of the changes that we have made that you might care about 
> (there are other changes, too, but I doubt you'll care/want them):
> 
> 1. Checking for epoll_ctl with AC_CHECK_FUNCS is unfortunately not 
> sufficient on some older Linux kernels (e.g., whatever was in Fedora 9) 
> because although the function exists and is linkable, it's hardwired to 
> return ENOSYS.  So if AC_CHECK_FUNCS with epoll reports that the 
> function exists, we also run a short program to ensure that epoll 
> actually works.  Doesn't work with cross-compiling, of course.

I suspect what's happening is that the epoll functions exist in libc but 
not the kernel, so the system calls get issued but always fails with 
ENOSYS. glibc and the Linux kernel are versioned independently, so this 
isn't too surprising.

Why do this detection at compile-time rather than run-time? The run-time 
approach seems superior because it does the right thing if run with a 
newer or older kernel than compiled with (which I wouldn't describe as 
cross-compiling per se).

libevent's current run-time ENOSYS support looks okay, in fact. Looks 
like it (incorrectly) returns epoll in event_get_supported_methods(), 
but it will accept an ENOSYS in epoll_create() without complaint and 
move on to the next method. What problem did you encounter?

> 
> 2. The Sun Studio 12 compilers on Linux currently don't properly support 
> the "packed" attribute.  As such, (struct epoll_event) generated by Sun 
> Studio 12 on 64 bit architectures will not match the same memory layout 
> as in the Linux kernel.  Badness ensues.  In conjunction with #1, our 
> test  that checks whether epoll works will also fail if the packed-ness 
> of (struct epoll_event) doesn't match between the user application and 
> the kernel.  Specifically: the test passes with Sun Studio 12 32-bit 
> builds, but fails with Sun Studio 12 64-bit builds (exactly as it should).
> 
> We extended the #1 and #2 tests into the syscall test for epoll as well.
> 
> --> Including tests #2 and #3 would be most helpful to Sun, because it   
> makes libevent compilable by the Sun Studio 12 compilers on Linux.

Yuck. As I'm sure you know, there are many occurrences of packed in 
system headers used by more than just libevent. I'm not the one you'd 
need to convince (I'm not a libevent maintainer), but IMHO, it's not 
feasible to put workarounds in the upstream version of every affected 
package. libevent's relatively easy because it could just not use epoll, 
but other packages would have to do unpleasant things to their structures.

Instead, you (by which I mean Sun, not necessarily you personally) can 
fix your own compiler and, until then, you can (attempt to) maintain 
patches in your distribution to work around its flaws.

> 4. All versions of OS X kqueue up to and including 10.5.3 are broken 
> when used with pty's.  We therefore disable kqueue on OS X because Open 
> MPI uses libevent with pty's.  I don't think you want this in the 
> general case, but perhaps this a useful datapoint for you.

With libevent trunk, you might be able to use the 
event_get_supported_methods() and event_config_*() support I alluded to 
above and test methods for this bug and exclude them if necessary at 
run-time. Then if run on a hypothetical newer OS X with a fixed kqueue, 
your program would use it without code changes or recompiling.

If using a pty with libevent is common, maybe this test could be put 
into libevent for use via event_config_require_features(cfg, 
EV_FEATURE_PTY). Then all callers could make their own decision without 
having to do the hard work of testing it themselves.

> 
> That's it.  Thanks for all your hard work on libevent.
> 
> Rolf
> 
> 
> 

Best regards,
Scott

-- 
Scott Lamb <http://www.slamb.org/>