From liusifan at gmail.com Sun Jun 1 01:51:53 2008 From: liusifan at gmail.com (liusifan) Date: Sun Jun 1 01:52:39 2008 Subject: [Libevent-users] Integrate Windows I/O Completion Port into Libevent Message-ID: <200806011351494530332@gmail.com> Integrate Windows I/O Completion Port into Libevent IOCP is true async i/o model, and libevent is event-driven i/o model. 1. How to emulate event-driven recv/send ? It is possible to use IOCP to emulate event-driven mode. When you make read or write calls, pass in a zero-byte buffer (ie, you can manage your i/o buffers using event driven semantics). So if you issue a read passing in a zero-byte length buffer, your will get notified when there is data to read. 2. How to emulate event-driven accept ? The WSAEventSelect API provides an event-driven accept mechanism. 3. How to integrate these two mechanism ? When we issue a WSARecv or WSASend, we need to pass a OVERLAPPED structure. This structure has a hEvent member. http://msdn.microsoft.com/en-us/library/ms742203(VS.85).aspx If the lpCompletionRoutine parameter is NULL, the hEvent parameter of lpOverlapped is signaled when the overlapped operation completes if it contains a valid event object handle. An application can use WSAWaitForMultipleEvents or WSAGetOverlappedResult to wait or poll on the event object. We could use one event for all WSARecv/WSASend. http://msdn.microsoft.com/en-us/library/ms686211(VS.85).aspx Setting an event that is already set has no effect. So it is possible to use WSAWaitForMultipleEvents to integrate IOCP and WSAEventSelect. 4. Solution /* objects[0] for iocp operations, object[1..63] for accept */ HANDLE objects[64]; struct event * accepts[64]; struct win32iocp_event event1; event1.overlapped.hEvent = objects[0]; WSARecv( ..., &event1.overlapped, ... ); .... struct win32iocp_event event2; event2.overlapped.hEvent = objects[0]; WSASend( ..., &event2.overlapped, ... ); ... WSAEventSelect( ev1->ev_fd, objects[1], FD_ACCEPT ); accepts[1] = ev1; ... WSAEventSelect( ev2->ev_fd, objects[2], FD_ACCEPT ); accepts[2] = ev2; ... int index = WSAWaitForMultipleEvents( 64, objects, FALSE, timeout, FALSE ); index = index - WSA_WAIT_EVENT_0; if( index > 0 ) { struct event * event = win32iocp_op->accepts[index]; event_active (event, EV_READ | EV_ACCEPT, 1); } if( index == 0 ) { for( ; ; ) { GetQueuedCompletionPort( ...... ); } } 5. Limitation It can only support 63 accept fds. 6. Source code http://spserver.googlecode.com/files/libevent-1.4.4-iocp.zip diff file : libevent-1.4.4-iocp\libevent-iocp\diff.txt add file : libevent-1.4.4-iocp\WIN32-Code\win32iocp.cpp liusifan 2008-06-01 -------------- next part -------------- An HTML attachment was scrubbed... URL: http://monkeymail.org/archives/libevent-users/attachments/20080601/b4b86372/attachment.htm From liusifan at gmail.com Sun Jun 1 09:03:08 2008 From: liusifan at gmail.com (liusifan) Date: Sun Jun 1 09:03:31 2008 Subject: [Libevent-users] Re: Integrate Windows I/O Completion Port into Libevent References: <200806011351494530332@gmail.com> Message-ID: <200806012102549211693@gmail.com> Found a bug in the source code. I have forget to get rid of some debug code. After fix the problem, the echo_iocp program can accept 1000 online connections propriety. Test result : Clients : 1000, Messages Per Client : 1000, Byte Per Message : 52 ExecTimes: 34.229569 seconds client Send Recv total : 1000000 1000000 average : 29215/s 29215/s http://spserver.googlecode.com/files/libevent-1.4.4-iocp.zip >>> >Integrate Windows I/O Completion Port into Libevent > >IOCP is true async i/o model, and libevent is event-driven i/o model. > >1. How to emulate event-driven recv/send ? >It is possible to use IOCP to emulate event-driven mode. When you make read or write calls, pass in a zero-byte buffer (ie, you can manage your i/o buffers using event driven semantics). So if you issue a read passing in a zero-byte length buffer, your will get notified when there is data to read. > >2. How to emulate event-driven accept ? >The WSAEventSelect API provides an event-driven accept mechanism. > >3. How to integrate these two mechanism ? >When we issue a WSARecv or WSASend, we need to pass a OVERLAPPED structure. This structure has a hEvent member. > >http://msdn.microsoft.com/en-us/library/ms742203(VS.85).aspx >If the lpCompletionRoutine parameter is NULL, the hEvent parameter of lpOverlapped is signaled when the overlapped operation completes if it contains a valid event object handle. An application can use WSAWaitForMultipleEvents or WSAGetOverlappedResult to wait or poll on the event object. > >We could use one event for all WSARecv/WSASend. > >http://msdn.microsoft.com/en-us/library/ms686211(VS.85).aspx >Setting an event that is already set has no effect. > >So it is possible to use WSAWaitForMultipleEvents to integrate IOCP and WSAEventSelect. > >4. Solution > >/* objects[0] for iocp operations, object[1..63] for accept */ >HANDLE objects[64]; >struct event * accepts[64]; > >struct win32iocp_event event1; >event1.overlapped.hEvent = objects[0]; >WSARecv( ..., &event1.overlapped, ... ); >.... > >struct win32iocp_event event2; >event2.overlapped.hEvent = objects[0]; >WSASend( ..., &event2.overlapped, ... ); >... > >WSAEventSelect( ev1->ev_fd, objects[1], FD_ACCEPT ); >accepts[1] = ev1; >... > >WSAEventSelect( ev2->ev_fd, objects[2], FD_ACCEPT ); >accepts[2] = ev2; >... > >int index = WSAWaitForMultipleEvents( 64, objects, FALSE, timeout, FALSE ); >index = index - WSA_WAIT_EVENT_0; > >if( index > 0 ) { > struct event * event = win32iocp_op->accepts[index]; > event_active (event, EV_READ | EV_ACCEPT, 1); >} > >if( index == 0 ) { > for( ; ; ) { > GetQueuedCompletionPort( ...... ); > } >} > >5. Limitation >It can only support 63 accept fds. > >6. Source code >http://spserver.googlecode.com/files/libevent-1.4.4-iocp.zip >diff file : libevent-1.4.4-iocp\libevent-iocp\diff.txt >add file : libevent-1.4.4-iocp\WIN32-Code\win32iocp.cpp > > > > > > >liusifan >2008-06-01 > From brofield2 at jellycan.com Sun Jun 1 19:57:20 2008 From: brofield2 at jellycan.com (Brodie Thiesfield) Date: Sun Jun 1 19:57:41 2008 Subject: [Libevent-users] Status win32 build in libevent Message-ID: Hi all, I've been trying to build libevent on win32 but am finding a few problems. 1) The VC6 workspace/projects in the 1.4.4 tar file are "corrupted" because they don't use CRLF line endings (VC6 and VC2003 refuse to open them, but easy fixed when you know the problem). 2) Once opened, libevent doesn't build in VC6 because of missing sys/time.h. This appears to be because although config.h is okay, the file event-config.h defines a heap of stuff that shouldn't be defined. 3) Win32-Code/misc.c is missing So the 1.4.4 stable release appears quite broken as released. Getting the latest from subversion I find: - VC6 has the same problems as 2 & 3 above. - opening the VC6 workspace/project in VC 2003 gives missing event2/util.h and event2/event.h errors - opening the VC2005 project builds libevent but with a number of serious stylistic warnings (e.g. void functions returning values. deleting const pointers. missing function definitions. mismatched signed and unsigned). Apologies if my problems are because I am using it incorrectly. I couldn't find a readme.win32 or similar file to let me know how to build, or what versions of compilers are required. The VC6 workspaces/projects made me assume that VC6 was still supported. I see from the list that some people at least are still using it. Would someone please clarify for me the current state of win32 support in libevent? Is it still beta and not designed to be used? Regards, Brodie From nickm at freehaven.net Mon Jun 2 11:35:58 2008 From: nickm at freehaven.net (Nick Mathewson) Date: Mon Jun 2 11:36:39 2008 Subject: [Libevent-users] Integrate Windows I/O Completion Port into Libevent In-Reply-To: <200806011351494530332@gmail.com> References: <200806011351494530332@gmail.com> Message-ID: <20080602153558.GZ30848@moria.seul.org> On Sun, Jun 01, 2008 at 01:51:53PM +0800, liusifan wrote: [...] > Integrate Windows I/O Completion Port into Libevent > > IOCP is true async i/o model, and libevent is event-driven i/o model. > > 1. How to emulate event-driven recv/send ? > It is possible to use IOCP to emulate event-driven mode. When you > make read or write calls, pass in a zero-byte buffer (ie, you can > manage your i/o buffers using event driven semantics). So if you issue > a read passing in a zero-byte length buffer, your will get notified > when there is data to read. This seems pretty promising, and could wind up being a better short-term idea than our older plan to get IOCP support by making it a new backend for bufferevents. How well tested is this? Can you get the unit tests to pass, at least, to the extent that the unit tests in 1.4.4 currently pass on win32? I'd like people who use libevent on windows to try this code out; if it works well, we should integrate it. A useful but not-immediately-necessary followup benchmark would be to see how well this does against using IOCP in the "normal" (not "event-driven") way. If there's no big difference, we can defer bufferevents IOCP work for a while; but if there's still a significant amount of overhead here, we should try to apply this patch *and* get IOCP working with bufferevents. [...] > 6. Source code > http://spserver.googlecode.com/files/libevent-1.4.4-iocp.zip > diff file : libevent-1.4.4-iocp\libevent-iocp\diff.txt (For future reference, please use unified diffs (generated with diff -u) rather than old-style diffs. They're shorter and (usually) easier to read. This diff is small enough that there's not much worry about this one, though.) > add file : libevent-1.4.4-iocp\WIN32-Code\win32iocp.cpp ^^^^ You mean ".c"? Hm. This might be better applied against trunk (which will eventually become libevent 2.0) than against 1.4.4; the 1.4 series is supposed to be stable, and Big New Features are usually out there... ...but now that I look at the code, it looks like the only big change is to add a new backend file, which (as far as I can tell) doesn't require any big changes elsewhere in the code. I'd be curious to see what Niels thinks here, but I wouldn't be opposed to checking it into the stable branch. BTW: I haven't read the code fully yet, but it looks relatively clean and straightforward. I'd appreciate it if people who know Windows networking better than I do would check it out and comment. Finally: Thanks, Stephen! This looks like great work to me so far. peace, -- Nick From Rolf.Vandevaart at Sun.COM Mon Jun 2 15:04:21 2008 From: Rolf.Vandevaart at Sun.COM (Rolf Vandevaart) Date: Mon Jun 2 15:04:53 2008 Subject: [Libevent-users] Possible configure changes for epoll Message-ID: <48444435.8030100@Sun.COM> Hello: We embed libevent in the Open MPI project; long ago, we folded libevent's configure m4 code into our own configure code. Over time, we tweaked the libevent m4 code for our own purposes as we ran into compatibility issues, etc. We're submitting these changes to you in the hopes that they will be useful. Feel free to use them or disregard them. We've changed the form of the original m4 so that it's not directly [re-]importable to libevent, but perhaps the code can be copied and morphed back into your configure.in. Here's a link to our current revision of the libevent-specific m4 in our configure system: https://svn.open-mpi.org/trac/ompi/browser/trunk/config/ompi_setup_libevent.m4 Here's a list of the changes that we have made that you might care about (there are other changes, too, but I doubt you'll care/want them): 1. Checking for epoll_ctl with AC_CHECK_FUNCS is unfortunately not sufficient on some older Linux kernels (e.g., whatever was in Fedora 9) because although the function exists and is linkable, it's hardwired to return ENOSYS. So if AC_CHECK_FUNCS with epoll reports that the function exists, we also run a short program to ensure that epoll actually works. Doesn't work with cross-compiling, of course. 2. The Sun Studio 12 compilers on Linux currently don't properly support the "packed" attribute. As such, (struct epoll_event) generated by Sun Studio 12 on 64 bit architectures will not match the same memory layout as in the Linux kernel. Badness ensues. In conjunction with #1, our test that checks whether epoll works will also fail if the packed-ness of (struct epoll_event) doesn't match between the user application and the kernel. Specifically: the test passes with Sun Studio 12 32-bit builds, but fails with Sun Studio 12 64-bit builds (exactly as it should). We extended the #1 and #2 tests into the syscall test for epoll as well. --> Including tests #2 and #3 would be most helpful to Sun, because it makes libevent compilable by the Sun Studio 12 compilers on Linux. 4. All versions of OS X kqueue up to and including 10.5.3 are broken when used with pty's. We therefore disable kqueue on OS X because Open MPI uses libevent with pty's. I don't think you want this in the general case, but perhaps this a useful datapoint for you. That's it. Thanks for all your hard work on libevent. Rolf -- ========================= rolf.vandevaart@sun.com 781-442-3043 ========================= From william at 25thandClement.com Mon Jun 2 15:30:29 2008 From: william at 25thandClement.com (William Ahern) Date: Mon Jun 2 15:31:09 2008 Subject: [Libevent-users] Integrate Windows I/O Completion Port into Libevent In-Reply-To: <20080602153558.GZ30848@moria.seul.org> References: <200806011351494530332@gmail.com> <20080602153558.GZ30848@moria.seul.org> Message-ID: <20080602193029.GB31656@wilbur.25thandClement.com> On Mon, Jun 02, 2008 at 11:35:58AM -0400, Nick Mathewson wrote: > On Sun, Jun 01, 2008 at 01:51:53PM +0800, liusifan wrote: > [...] > > Integrate Windows I/O Completion Port into Libevent > > > > IOCP is true async i/o model, and libevent is event-driven i/o model. > I'm curious, how close are IOCP semantics to POSIX AIO? I'm read through most of the documentation, but not having actually used IOCP before.... I'm writing a kqueue(2) compat library, and AIO support is next on my list. From al-libevent at none.at Mon Jun 2 15:46:18 2008 From: al-libevent at none.at (Aleksandar Lazic) Date: Mon Jun 2 15:51:41 2008 Subject: [Libevent-users] Integrate Windows I/O Completion Port into Libevent In-Reply-To: <20080602193029.GB31656@wilbur.25thandClement.com> References: <200806011351494530332@gmail.com> <20080602153558.GZ30848@moria.seul.org> <20080602193029.GB31656@wilbur.25thandClement.com> Message-ID: <20080602194618.GA23868@none.at> Hi William, On Mon 02.06.2008 12:30, William Ahern wrote: > >I'm writing a kqueue(2) compat library, and AIO support is next on my >list. http://software.schmorp.de/pkg/libeio.html what do you think about this lib, looks quite nice for me? Cheers Aleks From liusifan at gmail.com Tue Jun 3 00:40:22 2008 From: liusifan at gmail.com (lau stephen) Date: Tue Jun 3 00:40:28 2008 Subject: [Libevent-users] Integrate Windows I/O Completion Port into Libevent In-Reply-To: <20080602153558.GZ30848@moria.seul.org> References: <200806011351494530332@gmail.com> <20080602153558.GZ30848@moria.seul.org> Message-ID: 2008/6/2, Nick Mathewson : > > On Sun, Jun 01, 2008 at 01:51:53PM +0800, liusifan wrote: > [...] > > This seems pretty promising, and could wind up being a better > short-term idea than our older plan to get IOCP support by making it a > new backend for bufferevents. > > How well tested is this? Can you get the unit tests to pass, at > least, to the extent that the unit tests in 1.4.4 currently pass on > win32? I'd like people who use libevent on windows to try this code > out; if it works well, we should integrate it. Do you mean the uni tests in libevent/test directory ? I will try to get these unit tests to pass. > (For future reference, please use unified diffs (generated with diff -u) > rather than old-style diffs. They're shorter and (usually) easier > to read. This diff is small enough that there's not much worry > about this one, though.) OK, will use -u option. > add file : libevent-1.4.4-iocp\WIN32-Code\win32iocp.cpp > ^^^^ You mean ".c"? Yes. Just a type mistake. It's win32iocp.c . Hm. This might be better applied against trunk (which will eventually > become libevent 2.0) than against 1.4.4; the 1.4 series is supposed to > be stable, and Big New Features are usually out there... > > ...but now that I look at the code, it looks like the only big change > is to add a new backend file, which (as far as I can tell) doesn't > require any big changes elsewhere in the code. I'd be curious to see > what Niels thinks here, but I wouldn't be opposed to checking it into > the stable branch. These some thing i need to explain here. This patch requires to add a EV_ACCEPT macro to indicate the accept event. This is incompatible with the old code. The old code use EV_READ to indicate the accept event. Best Regards, Stephen Liu -------------- next part -------------- An HTML attachment was scrubbed... URL: http://monkeymail.org/archives/libevent-users/attachments/20080603/eb047535/attachment.htm From liusifan at gmail.com Tue Jun 3 11:40:22 2008 From: liusifan at gmail.com (liusifan) Date: Tue Jun 3 11:50:29 2008 Subject: [Libevent-users] Integrate Windows I/O Completion Port into Libevent References: <200806011351494530332@gmail.com>, <20080602153558.GZ30848@moria.seul.org>, Message-ID: <200806032340168905940@gmail.com> Mostly test cases of regress.c are passed. These are some changes since the libevent-1.4.4-iocp.zip Please check the diff file as attachment. The following test cases are failure or cann't be compiled. I will pay more time to get these test cases to pass. test_bufferevent_watermarks(); fail, block on WSAWaitForMultipleEvents test_multiple(); fail, block on WSAWaitForMultipleEvents test_persistent(); fail, block on WSAWaitForMultipleEvents http_suite(); fail on http.c::name_from_addr , vc6 has't getnameinfo function rpc_suite(); it seem so complex to complile, give up rpc_test(); same as above dns_suite(); cannot compile evdns.c, because vc6's winsock2.h is too old Test result of regress.c evutil_stroll: OK Testing Priorities 1: OK Testing Priorities 2: OK Testing Priorities 3: OK Testing Evbuffer: OK Testing evbuffer_find 1: OK Testing evbuffer_find 2: OK Testing evbuffer_find 3: OK Bufferevent: OK Free active base: OK Event base new: OK Simple read: OK Simple write: OK Combined read/write: OK Simple timeout: OK Loop exit: OK Loop break: OK Loop Multiple exit: OK Multiple events for same fd: OK Want read only once: OK Testing Tagging: encoded 0x00000af0 with 2 bytes encoded 0x00001000 with 3 bytes encoded 0x00000001 with 1 bytes encoded 0xdeadbeef with 5 bytes encoded 0x00000000 with 1 bytes encoded 0x00bef000 with 4 bytes ??: OK ??: OK encoded 0x00000af0 with 2 bytes encoded 0x00001000 with 2 bytes encoded 0x00000001 with 1 bytes encoded 0xdeadbeef with 5 bytes encoded 0x00000000 with 1 bytes encoded 0x00bef000 with 4 bytes ??: OK OK -------------- next part -------------- A non-text attachment was scrubbed... Name: diff.txt Type: application/octet-stream Size: 9661 bytes Desc: not available Url : http://monkeymail.org/archives/libevent-users/attachments/20080603/f75ad4e1/diff.obj From fan at hlrs.de Tue Jun 3 13:08:57 2008 From: fan at hlrs.de (Shiqing Fan) Date: Tue Jun 3 13:10:23 2008 Subject: [Libevent-users] libevent on windows: select() fails In-Reply-To: <47D91B68.7070001@bserved.nl> References: <47D91B68.7070001@bserved.nl> Message-ID: <48457AA9.5090904@hlrs.de> Hi all, I'm testing Libevent 1.3 with VS2005 on Windows. The same problem below there happens on my machine, select() always returns -1. Does anyone know the reason? Sorry for polling this old mail up, I just found someone has asked the same question that I got, but no answer. Thanks a lot. Regards, Shiqing > But when I run 'event-test', it seems to die at select(): > > V:\dev\libevent-svn\WIN32-Prj\event_test\Debug>event_test.exe > [msg] libevent using: win32 > [debug] event_add: event: 002DFF1C, EV_READ call 00433CE0 > [debug] win32_insert: adding event for 916 > [debug] win32_dispatch: select returned -1 > > Any idea why it would do that, and what I can do to fix it? I would > really like to start using libevent on windows. > > Cheers, > Bas Verhoeven > > _______________________________________________ > Libevent-users mailing list > Libevent-users@monkey.org > http://monkeymail.org/mailman/listinfo/libevent-users > -- -------------------------------------------------------------- Shiqing Fan http://www.hlrs.de/people/fan High Performance Computing Tel.: +49 711 685 87234 Center Stuttgart (HLRS) Fax.: +49 711 685 65832 POSTAL:Nobelstrasse 19 email: fan@hlrs.de ACTUAL:Allmandring 30 70569 Stuttgart From nickm at freehaven.net Tue Jun 3 13:18:47 2008 From: nickm at freehaven.net (Nick Mathewson) Date: Tue Jun 3 13:19:11 2008 Subject: [Libevent-users] libevent on windows: select() fails In-Reply-To: <48457AA9.5090904@hlrs.de> References: <47D91B68.7070001@bserved.nl> <48457AA9.5090904@hlrs.de> Message-ID: <20080603171847.GJ30848@moria.seul.org> On Tue, Jun 03, 2008 at 07:08:57PM +0200, Shiqing Fan wrote: > Hi all, > > I'm testing Libevent 1.3 with VS2005 on Windows. > > The same problem below there happens on my machine, select() always > returns -1. Does anyone know the reason? Sorry for polling this old > mail up, I just found someone has asked the same question that I got, > but no answer. Thanks a lot. Sometimes this means that you haven't called WSAStartup ? Winsock wants to be initialized. From libevent at bserved.nl Tue Jun 3 15:54:50 2008 From: libevent at bserved.nl (Bas Verhoeven) Date: Tue Jun 3 15:55:13 2008 Subject: [Libevent-users] libevent on windows: select() fails In-Reply-To: <48457AA9.5090904@hlrs.de> References: <47D91B68.7070001@bserved.nl> <48457AA9.5090904@hlrs.de> Message-ID: <4845A18A.9070303@bserved.nl> Shiqing Fan wrote: > Hi all, Hi, > > I'm testing Libevent 1.3 with VS2005 on Windows. > > The same problem below there happens on my machine, select() always > returns -1. Does anyone know the reason? Sorry for polling this old > mail up, I just found someone has asked the same question that I got, > but no answer. Thanks a lot. I believe the problem was that select() does not work on file descriptors on Windows, as it does on *NIX systems. Cheers, Bas Verhoeven > > > Regards, > Shiqing > >> But when I run 'event-test', it seems to die at select(): >> >> V:\dev\libevent-svn\WIN32-Prj\event_test\Debug>event_test.exe >> [msg] libevent using: win32 >> [debug] event_add: event: 002DFF1C, EV_READ call 00433CE0 >> [debug] win32_insert: adding event for 916 >> [debug] win32_dispatch: select returned -1 >> >> Any idea why it would do that, and what I can do to fix it? I would >> really like to start using libevent on windows. >> >> Cheers, >> Bas Verhoeven >> >> _______________________________________________ >> Libevent-users mailing list >> Libevent-users@monkey.org >> http://monkeymail.org/mailman/listinfo/libevent-users >> > > From awantik.das at rancoretech.com Wed Jun 4 03:17:14 2008 From: awantik.das at rancoretech.com (awantik.das) Date: Wed Jun 4 03:18:41 2008 Subject: [Libevent-users] Why this error mesg ???? "port_getn: Invalid argument" Message-ID: Hi all, We are trying to implement our messaging library using libevent . After few successful attempts we are getting this error. "port_getn: Invalid argument". Thanks & Regards Awantik Das +91 9987796932 -------------- next part -------------- An HTML attachment was scrubbed... URL: http://monkeymail.org/archives/libevent-users/attachments/20080604/eafeb412/attachment.htm From fan at hlrs.de Wed Jun 4 03:39:41 2008 From: fan at hlrs.de (Shiqing Fan) Date: Wed Jun 4 03:40:30 2008 Subject: [Libevent-users] libevent on windows: select() fails In-Reply-To: <4845A18A.9070303@bserved.nl> References: <47D91B68.7070001@bserved.nl> <48457AA9.5090904@hlrs.de> <4845A18A.9070303@bserved.nl> Message-ID: <484646BD.2080105@hlrs.de> Hi Bas Verhoeven , Thanks for your reply. > I believe the problem was that select() does not work on file > descriptors on Windows, as it does on *NIX systems. Yes, that's right. But is there any way to fix it? Thanks, Shiqing From fan at hlrs.de Wed Jun 4 04:01:24 2008 From: fan at hlrs.de (Shiqing Fan) Date: Wed Jun 4 04:03:23 2008 Subject: [Libevent-users] libevent on windows: select() fails In-Reply-To: <20080603171847.GJ30848@moria.seul.org> References: <47D91B68.7070001@bserved.nl> <48457AA9.5090904@hlrs.de> <20080603171847.GJ30848@moria.seul.org> Message-ID: <48464BD4.60506@hlrs.de> Hi Nick, > Sometimes this means that you haven't called WSAStartup ? Winsock wants > to be initialized. > > I was testing with the event-test, which is coming with Libevent source. Although I added WSAStartup, nothing was changed. Thanks, Shiqing From valery+libevent at grid.net.ru Wed Jun 4 04:48:33 2008 From: valery+libevent at grid.net.ru (Valery Kholodkov) Date: Wed Jun 4 04:50:15 2008 Subject: [Libevent-users] Why this error mesg ???? "port_getn: Invalid argument" In-Reply-To: References: Message-ID: <58422.62.218.60.210.1212569313.squirrel@mail.grid.net.ru> Could you please specify which operating system of which version are you using and the version of libevent? > We are trying to implement our messaging library using libevent . After > few successful attempts we are getting this error. "port_getn: Invalid > argument". -- Best regards, Valery Kholodkov From awantik.das at rancoretech.com Wed Jun 4 04:53:56 2008 From: awantik.das at rancoretech.com (awantik.das) Date: Wed Jun 4 04:54:09 2008 Subject: [Libevent-users] Why this error mesg ???? "port_getn: Invalid argument" In-Reply-To: <58422.62.218.60.210.1212569313.squirrel@mail.grid.net.ru> References: <58422.62.218.60.210.1212569313.squirrel@mail.grid.net.ru> Message-ID: OS solaris SunOS D001RANCOR5 5.10 Generic_125100-10 sun4v sparc SUNW,Sun-Fire-T200 Libevent version libevent-1.4.4-stable.tar -----Original Message----- From: Valery Kholodkov [mailto:valery+libevent@grid.net.ru] Sent: Wednesday, June 04, 2008 2:19 PM To: awantik.das Cc: libevent-users@monkey.org Subject: Re: [Libevent-users] Why this error mesg ???? "port_getn: Invalid argument" Could you please specify which operating system of which version are you using and the version of libevent? > We are trying to implement our messaging library using libevent . After > few successful attempts we are getting this error. "port_getn: Invalid > argument". -- Best regards, Valery Kholodkov -------------- next part -------------- An HTML attachment was scrubbed... URL: http://monkeymail.org/archives/libevent-users/attachments/20080604/761d3d94/attachment.htm From awantik.das at rancoretech.com Wed Jun 4 07:00:59 2008 From: awantik.das at rancoretech.com (awantik.das) Date: Wed Jun 4 07:01:24 2008 Subject: [Libevent-users] Why this error mesg ???? "port_getn: Invalid argument" In-Reply-To: <58422.62.218.60.210.1212569313.squirrel@mail.grid.net.ru> References: <58422.62.218.60.210.1212569313.squirrel@mail.grid.net.ru> Message-ID: Hi All, What does the function port_getn() do ??? if ((res = port_getn(epdp->ed_port, pevtlist, EVENTS_PER_GETN, (unsigned int *) &nevents, ts_p)) == -1) { if (errno == EINTR || errno == EAGAIN) { evsignal_process(base); return (0); } else if (errno == ETIME) { if (nevents == 0) return (0); } else { event_warn("port_getn"); return (-1); } } else if (base->sig.evsignal_caught) { evsignal_process(base); My program control is running to the highlighted part & prints port_getn: Invalid argument Thanks & Regards Awantik Das -----Original Message----- From: Valery Kholodkov [mailto:valery+libevent@grid.net.ru] Sent: Wednesday, June 04, 2008 2:19 PM To: awantik.das Cc: libevent-users@monkey.org Subject: Re: [Libevent-users] Why this error mesg ???? "port_getn: Invalid argument" Could you please specify which operating system of which version are you using and the version of libevent? > We are trying to implement our messaging library using libevent . After > few successful attempts we are getting this error. "port_getn: Invalid > argument". -- Best regards, Valery Kholodkov -------------- next part -------------- An HTML attachment was scrubbed... URL: http://monkeymail.org/archives/libevent-users/attachments/20080604/9581e22b/attachment-0001.htm From wouter at nlnetlabs.nl Wed Jun 4 07:06:47 2008 From: wouter at nlnetlabs.nl (W.C.A. Wijngaards) Date: Wed Jun 4 07:06:57 2008 Subject: [Libevent-users] Why this error mesg ???? "port_getn: Invalid argument" In-Reply-To: References: <58422.62.218.60.210.1212569313.squirrel@mail.grid.net.ru> Message-ID: <48467747.5010802@nlnetlabs.nl> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Hi Awantik, port_getn is like select(), and solaris specific. check that a) all filedescriptors that you pass are OK. If you pass a bad file descriptor (closed file, random number), then this message would appear. b) the timeouts you ask for are reasonable (not negative or infinite). If your program is OK, you can try to disable the dev/port or solaris ports mechanism using the shell defines, and see if your program works with good old select. If it doesn't you missed a bug in your program, if it does, the port code may be buggy. Best regards, ~ Wouter awantik.das wrote: ~ > My program control is running to the highlighted part & prints | *port_getn: Invalid argument* -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.9 (GNU/Linux) Comment: Using GnuPG with Fedora - http://enigmail.mozdev.org iEYEARECAAYFAkhGd0cACgkQkDLqNwOhpPhgXQCdG7yk57iH5miuVIQXZnyKV8k6 hegAn3+KHB1mFQRxP6N8WBtcVP0wj/Df =5aDa -----END PGP SIGNATURE----- From liusifan at gmail.com Wed Jun 4 09:30:55 2008 From: liusifan at gmail.com (liusifan) Date: Wed Jun 4 09:31:27 2008 Subject: [Libevent-users] Integrate Windows I/O Completion Port into Libevent References: <200806011351494530332@gmail.com>, <20080602153558.GZ30848@moria.seul.org>, Message-ID: <200806042130477658425@gmail.com> Hi, >>3. How to integrate these two mechanism ? >>When we issue a WSARecv or WSASend, we need to pass a OVERLAPPED structure. This structure has a hEvent member. >>We could use one event for all WSARecv/WSASend. http://msdn.microsoft.com/en-us/library/ms686358(VS.85).aspx Synchronization and Overlapped Input and Output If an event is used, the hEvent member of the OVERLAPPED structure specifies a handle to the allocated event object. The system sets the state of the event object to nonsignaled when a call to the I/O function returns before the operation has been completed. The integrate strategy for IOCP and WSAEventSelect is not safe. Because WSARecv/WSASend will sets the state of the event object to nonsignaled. This can lead to a race condition: 1.the first WSARecv/WSASend operations have completed, hEvent is signaled 2.the second WSARecv/WSASend has been called, hEvent has been reseted 3.WSAWaitForMultipleEvents will block until the second operations have completed It need to try other way to integrate IOCP and WSAEventSelect. Best Regards, Stephen Liu From liusifan at gmail.com Wed Jun 4 11:53:48 2008 From: liusifan at gmail.com (liusifan) Date: Wed Jun 4 11:56:30 2008 Subject: [Libevent-users] Integrate Windows I/O Completion Port into Libevent References: <200806011351494530332@gmail.com>, <20080602153558.GZ30848@moria.seul.org>, , <200806042130477658425@gmail.com> Message-ID: <200806042353442039211@gmail.com> Hi, >The integrate strategy for IOCP and WSAEventSelect is not safe. >Because WSARecv/WSASend will sets the state of the event object to nonsignaled. >This can lead to a race condition: >1.the first WSARecv/WSASend operations have completed, hEvent is signaled >2.the second WSARecv/WSASend has been called, hEvent has been reseted >3.WSAWaitForMultipleEvents will block until the second operations have completed The following sequences can deal with the race condition: 1.the first WSARecv/WSASend operations have completed, hEvent is signaled 2.the second WSARecv/WSASend has been called, hEvent has been reseted 3.use GetQueuedCompletionStatus with zero timeout, try to process I/O completion event first if it has some I/O completion event, then don't need to call WSAWaitForMultipleEvents 4.if it has not I/O completion event, then calling WSAWaitForMultipleEvents It still has race condition: 1.the second WSARecv/WSASend operation has completed before calling GQCS 2.this completion event has been process by GQCS, but left the hEvent as signaled 3.the next time to call WSAWaitForMultipleEvents, it will success, but GQCS cannot get any completion event. But it don't lead to a logic error, it lead to do a unwanted GQCS call in the worst situation. The last source code: http://spserver.googlecode.com/files/libevent-1.4.4-iocp-1.zip A a echo_iocp.exe test program is also included. This program seen to work fine, it can easy to deal with 1000 simultaneous connections propriety. I hope someone can try this code, any and all comments are appreciated. ===================================================================================== With this strategy, the following test cases are passed: test_bufferevent_watermarks(); test_multiple(); test_persistent(); The following test cases still fail, but it's enviroment problem, not buggy. http_suite(); fail on http.c::name_from_addr , vc6 has't getnameinfo function rpc_suite(); it seem so complex to complile, give up rpc_test(); same as above dns_suite(); cannot compile evdns.c, because vc6's winsock2.h is too old Best Regards, Stephen Liu From liusifan at gmail.com Thu Jun 5 11:31:59 2008 From: liusifan at gmail.com (liusifan) Date: Thu Jun 5 11:33:07 2008 Subject: [Libevent-users] libevent + iocp + memcached References: <200806011351494530332@gmail.com>, <20080602153558.GZ30848@moria.seul.org>, Message-ID: <200806052331525780656@gmail.com> Hi, all libevent + iocp + memcached memcached for Win32 : http://jehiah.cz/projects/memcached-win32/ It need to add EV_ACCEPT flag for listen socket, the adapted source code can download from: http://spserver.googlecode.com/files/memcached-1.2.1-iocp.zip VC6 project files: memcached-1.2.1-iocp\win32-iocp\ The last source code for libevent + iocp http://spserver.googlecode.com/files/libevent-1.4.4-iocp-2.zip VC6 project files: libevent-1.4.4-iocp-2\libevent-iocp Changes of libevent + iocp since libevent-1.4.4-iocp-1.zip: * More robustious error handling This combination looks work very well. Best Regards, Stephen Liu From liusifan at gmail.com Fri Jun 6 13:05:24 2008 From: liusifan at gmail.com (liusifan) Date: Fri Jun 6 13:05:53 2008 Subject: [Libevent-users] Stress test for libevent + iocp References: <200806011351494530332@gmail.com>, <20080602153558.GZ30848@moria.seul.org> Message-ID: <200806070105099536952@gmail.com> Hi, http://spserver.googlecode.com/files/libevent-1.4.4-iocp-3.zip Add iocpserver.cpp and testiocpstress.cpp. iocpserver.cpp comes from : ftp://ftp.microsoft.com/bussys/winsock/winsock2/iocp.zip Make some changes: when client sends quit, server forces to close the connection. To let the iocpserver.exe has the same behavior as echo_iocp.exe. Using testiocpstress.exe to stress echo_iocp.exe and iocpserver.exe . It is used libevent+iocp to implement echo_iocp.exe . I guesss iocpserver.cpp is not very well implemented, it seems echo_iocp.exe faster than iocpserver.exe. I hope people can try this code, any and all comments are appreciated. result of echo_iocp.exe ======================================================================= testiocpstress.exe -p 5555 -c 10000 -m 100 Test result : Clients : 10000, Messages Per Client : 100 Failure : send 0, WSASend 0, recv 0, WSARecv 0, GQCS 0 ExecTimes : 50.999765 seconds client Send Recv total : 1000000 1000000 average : 19608/s 19608/s result of echo_iocp.exe ======================================================================= testiocpstress.exe -p 5001 -c 10000 -m 100 Test result : Clients : 10000, Messages Per Client : 100 Failure : send 0, WSASend 0, recv 0, WSARecv 0, GQCS 0 ExecTimes : 69.999781 seconds client Send Recv total : 1000000 1000000 average : 14286/s 14286/s Best Regards, Stephen Liu From dengminwen at gmail.com Sun Jun 8 10:03:00 2008 From: dengminwen at gmail.com (=?GB2312?B?tcvD8c7E?=) Date: Sun Jun 8 10:03:14 2008 Subject: [Libevent-users] why event_del() failed so much Message-ID: <19583a390806080703h1aa25b74na27f7b692b3efa29@mail.gmail.com> recently, i try to use libcurl-7.18.2 + libevent-1.4.4, and i found that the event_del() call failed so much, but event_add() call are all return successfully i use the curl's example code hiperfifo.c, and dump some runtime info to stderr.log below is the beginning part of stderr.log: ============================below====================================== after epoll_init, epollop->(epfd, nevents, nfds) = 3,31999,31999 socket callback: s=8 e=0x805c390 what=IN Adding data: IN socket callback: s=9 e=0x8072820 what=IN Adding data: IN socket callback: s=10 e=0x8088ba0 what=IN Adding data: IN socket callback: s=11 e=0x809ef20 what=IN Adding data: IN socket callback: s=12 e=0x80b52c8 what=IN Adding data: IN socket callback: s=13 e=0x80cb658 what=IN Adding data: IN socket callback: s=14 e=0x80e19e8 what=IN Adding data: IN socket callback: s=15 e=0x80f7d98 what=IN Adding data: IN socket callback: s=16 e=0x810e168 what=IN Adding data: IN socket callback: s=17 e=0x8124548 what=IN Adding data: IN socket callback: s=18 e=0x813a900 what=IN Adding data: IN socket callback: s=19 e=0x8150c90 what=IN Adding data: IN socket callback: s=8 e=0x805c390 what=OUT Changing action from IN to OUT Error: epoll_del: epoll_ctl return false, fd = 8, op = EPOLL_CTL_MOD, epoll_ctl() returns ENOENT epollop->(epfd, nevents, nfds) = 3,31999,31999 Error: event_del in setsock() return false socket callback: s=9 e=0x8072820 what=OUT Changing action from IN to OUT Error: epoll_del: epoll_ctl return false, fd = 9, op = EPOLL_CTL_MOD, epoll_ctl() returns ENOENT epollop->(epfd, nevents, nfds) = 3,31999,31999 Error: event_del in setsock() return false ...... ============================above====================================== my input is 3000 different urls >> grep ENOENT stderr.log | wc -l 2002 >> grep EBADF stderr.log | wc -l 1510 is it a epoll_ctl system call 's bug? or libevent misuse the epoll ? hoping for your help sincerely -- dengminwen@gmail.com -------------- next part -------------- An HTML attachment was scrubbed... URL: http://monkeymail.org/archives/libevent-users/attachments/20080608/6315b3b6/attachment.htm From dengminwen at gmail.com Mon Jun 9 01:41:13 2008 From: dengminwen at gmail.com (=?GB2312?B?tcvD8c7E?=) Date: Mon Jun 9 01:41:41 2008 Subject: [Libevent-users] Re: why event_del() failed so much In-Reply-To: <19583a390806080703h1aa25b74na27f7b692b3efa29@mail.gmail.com> References: <19583a390806080703h1aa25b74na27f7b692b3efa29@mail.gmail.com> Message-ID: <19583a390806082241y4f823eafy438d9a9f1a8385ec@mail.gmail.com> i am sorry, the previous problem i posted seems to be that libcurl behave different with libevent, but not the fault of libevent. i think the problem can be explain like this: 1. libcurl detect a sockfd to be BAD or GONE 2. libcurl tell the user's callback func to do a REMOVE to this sockfd 3. user's callback func call epoll_del() to this sockfd 4. in function epoll_del(), epoll_ctl() returns ENOENT or EBADF 2008/6/8 : > recently, i try to use libcurl-7.18.2 + libevent-1.4.4, and i found that > the event_del() call failed so much, but event_add() call are all return > successfully > > i use the curl's example code hiperfifo.c, and dump some runtime info to > stderr.log > below is the beginning part of stderr.log: > ============================below====================================== > after epoll_init, epollop->(epfd, nevents, nfds) = 3,31999,31999 > socket callback: s=8 e=0x805c390 what=IN Adding data: IN > socket callback: s=9 e=0x8072820 what=IN Adding data: IN > socket callback: s=10 e=0x8088ba0 what=IN Adding data: IN > socket callback: s=11 e=0x809ef20 what=IN Adding data: IN > socket callback: s=12 e=0x80b52c8 what=IN Adding data: IN > socket callback: s=13 e=0x80cb658 what=IN Adding data: IN > socket callback: s=14 e=0x80e19e8 what=IN Adding data: IN > socket callback: s=15 e=0x80f7d98 what=IN Adding data: IN > socket callback: s=16 e=0x810e168 what=IN Adding data: IN > socket callback: s=17 e=0x8124548 what=IN Adding data: IN > socket callback: s=18 e=0x813a900 what=IN Adding data: IN > socket callback: s=19 e=0x8150c90 what=IN Adding data: IN > socket callback: s=8 e=0x805c390 what=OUT Changing action from IN to OUT > Error: epoll_del: epoll_ctl return false, fd = 8, op = EPOLL_CTL_MOD, > epoll_ctl() returns ENOENT > epollop->(epfd, nevents, nfds) = 3,31999,31999 > Error: event_del in setsock() return false > socket callback: s=9 e=0x8072820 what=OUT Changing action from IN to OUT > Error: epoll_del: epoll_ctl return false, fd = 9, op = EPOLL_CTL_MOD, > epoll_ctl() returns ENOENT > epollop->(epfd, nevents, nfds) = 3,31999,31999 > Error: event_del in setsock() return false > ...... > ============================above====================================== > > my input is 3000 different urls > >> grep ENOENT stderr.log | wc -l > 2002 > >> grep EBADF stderr.log | wc -l > 1510 > > is it a epoll_ctl system call 's bug? > or libevent misuse the epoll ? > > hoping for your help sincerely > > -- > dengminwen@gmail.com -- dengminwen@gmail.com -------------- next part -------------- An HTML attachment was scrubbed... URL: http://monkeymail.org/archives/libevent-users/attachments/20080609/31f59749/attachment-0001.htm From slamb at slamb.org Wed Jun 11 01:46:42 2008 From: slamb at slamb.org (Scott Lamb) Date: Wed Jun 11 01:46:48 2008 Subject: [Libevent-users] Possible configure changes for epoll In-Reply-To: <48444435.8030100@Sun.COM> References: <48444435.8030100@Sun.COM> Message-ID: <484F66C2.6090104@slamb.org> Rolf Vandevaart wrote: > Hello: > > We embed libevent in the Open MPI project; long ago, we folded > libevent's configure m4 code into our own configure code. Over time, we > tweaked the libevent m4 code for our own purposes as we ran into > compatibility issues, etc. We're submitting these changes to you in the > hopes that they will be useful. Feel free to use them or disregard them. > > We've changed the form of the original m4 so that it's not directly > [re-]importable to libevent, but perhaps the code can be copied and > morphed back into your configure.in. Here's a link to our current > revision of the libevent-specific m4 in our configure system: > > https://svn.open-mpi.org/trac/ompi/browser/trunk/config/ompi_setup_libevent.m4 > > > Here's a list of the changes that we have made that you might care about > (there are other changes, too, but I doubt you'll care/want them): > > 1. Checking for epoll_ctl with AC_CHECK_FUNCS is unfortunately not > sufficient on some older Linux kernels (e.g., whatever was in Fedora 9) > because although the function exists and is linkable, it's hardwired to > return ENOSYS. So if AC_CHECK_FUNCS with epoll reports that the > function exists, we also run a short program to ensure that epoll > actually works. Doesn't work with cross-compiling, of course. I suspect what's happening is that the epoll functions exist in libc but not the kernel, so the system calls get issued but always fails with ENOSYS. glibc and the Linux kernel are versioned independently, so this isn't too surprising. Why do this detection at compile-time rather than run-time? The run-time approach seems superior because it does the right thing if run with a newer or older kernel than compiled with (which I wouldn't describe as cross-compiling per se). libevent's current run-time ENOSYS support looks okay, in fact. Looks like it (incorrectly) returns epoll in event_get_supported_methods(), but it will accept an ENOSYS in epoll_create() without complaint and move on to the next method. What problem did you encounter? > > 2. The Sun Studio 12 compilers on Linux currently don't properly support > the "packed" attribute. As such, (struct epoll_event) generated by Sun > Studio 12 on 64 bit architectures will not match the same memory layout > as in the Linux kernel. Badness ensues. In conjunction with #1, our > test that checks whether epoll works will also fail if the packed-ness > of (struct epoll_event) doesn't match between the user application and > the kernel. Specifically: the test passes with Sun Studio 12 32-bit > builds, but fails with Sun Studio 12 64-bit builds (exactly as it should). > > We extended the #1 and #2 tests into the syscall test for epoll as well. > > --> Including tests #2 and #3 would be most helpful to Sun, because it > makes libevent compilable by the Sun Studio 12 compilers on Linux. Yuck. As I'm sure you know, there are many occurrences of packed in system headers used by more than just libevent. I'm not the one you'd need to convince (I'm not a libevent maintainer), but IMHO, it's not feasible to put workarounds in the upstream version of every affected package. libevent's relatively easy because it could just not use epoll, but other packages would have to do unpleasant things to their structures. Instead, you (by which I mean Sun, not necessarily you personally) can fix your own compiler and, until then, you can (attempt to) maintain patches in your distribution to work around its flaws. > 4. All versions of OS X kqueue up to and including 10.5.3 are broken > when used with pty's. We therefore disable kqueue on OS X because Open > MPI uses libevent with pty's. I don't think you want this in the > general case, but perhaps this a useful datapoint for you. With libevent trunk, you might be able to use the event_get_supported_methods() and event_config_*() support I alluded to above and test methods for this bug and exclude them if necessary at run-time. Then if run on a hypothetical newer OS X with a fixed kqueue, your program would use it without code changes or recompiling. If using a pty with libevent is common, maybe this test could be put into libevent for use via event_config_require_features(cfg, EV_FEATURE_PTY). Then all callers could make their own decision without having to do the hard work of testing it themselves. > > That's it. Thanks for all your hard work on libevent. > > Rolf > > > Best regards, Scott -- Scott Lamb From liusifan at gmail.com Thu Jun 12 09:16:51 2008 From: liusifan at gmail.com (liusifan) Date: Thu Jun 12 09:17:31 2008 Subject: [Libevent-users] event->min_heap_idx may be not -1 after min_heap_pop Message-ID: <200806122116054218659@gmail.com> Hi, all After reading the min_heap.h, I found the min_heap_pop may be has a bug struct event* min_heap_pop(min_heap_t* s) { if(s->n) { struct event* e = *s->p; e->min_heap_idx = -1; min_heap_shift_down_(s, 0u, s->p[--s->n]); return e; } return 0; } ( e->min_heap_idx = -1 ) is before ( min_heap_shift_down_ ), but e->min_heap_idx may be change in min_heap_shift_down_ . The following source code will cause a core dump. Calling min_heap_erase after min_heap_pop with the same event. #include "min_heap.h" int main( int argc, char * argv[] ) { min_heap_t heap; struct event ev, * evp; min_heap_ctor( &heap ); min_heap_push( &heap, &ev ); evp = min_heap_pop( &heap ); min_heap_erase( &heap, evp ); return 0; } The following min_heap_pop could fix this problem. Reverse the ( e->min_heap_idx = -1 ) and ( min_heap_shift_down_ ). Because min_heap_erase will detect the event->min_heap_idx. struct event* min_heap_pop(min_heap_t* s) { if(s->n) { struct event* e = *s->p; min_heap_shift_down_(s, 0u, s->p[--s->n]); e->min_heap_idx = -1; return e; } return 0; } liusifan 2008-06-12 From provos at citi.umich.edu Thu Jun 12 10:46:17 2008 From: provos at citi.umich.edu (Niels Provos) Date: Thu Jun 12 10:46:20 2008 Subject: [Libevent-users] event->min_heap_idx may be not -1 after min_heap_pop In-Reply-To: <200806122116054218659@gmail.com> References: <200806122116054218659@gmail.com> Message-ID: <850f7cbe0806120746g3710e79ft4998fe2f21545df0@mail.gmail.com> On Thu, Jun 12, 2008 at 6:16 AM, liusifan wrote: > After reading the min_heap.h, I found the min_heap_pop may be has a bug [...] > evp = min_heap_pop( &heap ); > min_heap_erase( &heap, evp ); Hi Liusifan, thanks for the report; I changed min_heap_pop to set the index to -1 after the shift down. Niels. From teunis at wintersgift.com Thu Jun 12 10:59:45 2008 From: teunis at wintersgift.com (Teunis Peters) Date: Thu Jun 12 10:59:55 2008 Subject: [Libevent-users] files and libevent Message-ID: <485139E1.2050206@wintersgift.com> I've not found any code to work with - but is there any reason that libevent would not work with standard files? I keep getting permission denied more or less: fd = open(filename, O_RDONLY) event_set(&ev, EV_READ | EV_PERSIST, rd_callback, rd_data); event_add(&ev, NULL) -> EPERM Or does libevent only work with network connections? aside: the other thing that I've been looking into is the possibility of getting the http package to pass data through a callback rather than store to memory. I've been testing it with podcasts and 50 mb files are proving too heavyweight. (the lack of ability to display progress is proving annoying as well) Thank you! - Teunis Peters From liusifan at gmail.com Thu Jun 12 11:00:28 2008 From: liusifan at gmail.com (liusifan) Date: Thu Jun 12 11:00:41 2008 Subject: [Libevent-users] patch for building libevent with vs2008 Message-ID: <200806122300174688931@gmail.com> Hi, all Libevent svn main trunk cannot build with vs2008. The patch is in attachment. Changes: Rename INPUT to EVRPC_INPUT and OUTPUT to EVRPC_INPUT The following definition is conflict with system header. ========================================================== libevent/evrpc.h enum EVRPC_HOOK_TYPE { INPUT, /**< apply the function to an input hook */ OUTPUT /**< apply the function to an output hook */ }; ========================================================== windows\v6.0a\include\winuser.h typedef struct tagINPUT { DWORD type; union { MOUSEINPUT mi; KEYBDINPUT ki; HARDWAREINPUT hi; }; } INPUT, *PINPUT, FAR* LPINPUT; Stephen Liu 2008-06-12 -------------- next part -------------- A non-text attachment was scrubbed... Name: vs2008.diff.txt Type: application/octet-stream Size: 2862 bytes Desc: not available Url : http://monkeymail.org/archives/libevent-users/attachments/20080612/057bcf90/vs2008.diff.obj From brofield2 at jellycan.com Fri Jun 13 02:42:35 2008 From: brofield2 at jellycan.com (Brodie Thiesfield) Date: Fri Jun 13 02:42:49 2008 Subject: [Libevent-users] Re: Status win32 build in libevent In-Reply-To: References: Message-ID: Hi, I created my own win32 build for libevent 1.4.4 since there wasn't one available that works. No guarantees, no support. http://code.jellycan.com/memcached/ Cheers, Brodie On Mon, Jun 2, 2008 at 8:57 AM, Brodie Thiesfield wrote: > Hi all, > > I've been trying to build libevent on win32 but am finding a few problems. > > [etc etc] From ron.arts at neonova.nl Fri Jun 13 04:01:33 2008 From: ron.arts at neonova.nl (Ron Arts) Date: Fri Jun 13 04:01:54 2008 Subject: [Libevent-users] [err] event_queue_insert: 0xb6a4517c(fd 17) already on queue 2 Message-ID: <4852295D.7040307@neonova.nl> Hello, I get this error on libevent 1.3e on CentOS5. Upgrading to a higher version is not easy to push through, to say the least, so that's why I am asking the list first. This error happens after my program has been running for a day or so. There are only 10 clients that stay connected all the time. It's a multithreaded program, only one thread ever calls libevent. What could I be doing wrong to cause this error? Thanks, Ron -- NeoNova BV, The Netherlands Professional internet and VoIP solutions http://www.neonova.nl Kruislaan 419 1098 VA Amsterdam info: 020-5628292 servicedesk: 020-5628292 fax: 020-5628291 KvK Amsterdam 34151241 The following disclaimer applies to this email: http://www.neonova.nl/maildisclaimer From chris.brody at gmail.com Fri Jun 13 05:44:12 2008 From: chris.brody at gmail.com (Chris Brody) Date: Fri Jun 13 05:44:27 2008 Subject: [Libevent-users] Re: Status win32 build in libevent In-Reply-To: References: Message-ID: <7365f28e0806130244n5fd33ec8ybc623ad0a798d4bd@mail.gmail.com> On Fri, Jun 13, 2008 at 8:42 AM, Brodie Thiesfield wrote: > Hi, > > I created my own win32 build for libevent 1.4.4 since there wasn't one > available that works. No guarantees, no support. > > http://code.jellycan.com/memcached/ Good work to support such an environment! I liked some of your other products as well. >From what I have seen, it is extremely difficult if not impossible to support both Visual C++ 6.0 and VS 2003+ in the same project. I had to use VC++ 6.0 to build a commercial Windows application and want to switch to something like Dev-C++ as soon as I possibly can. For example, the CR-LF issue is a real problem for VS and is only an annoyance for everyone else. Perhaps we should just store all files in Unix format and let the VS users use something like ASCII FTP to convert to DOS format. From liusifan at gmail.com Fri Jun 13 11:27:04 2008 From: liusifan at gmail.com (liusifan) Date: Fri Jun 13 11:27:34 2008 Subject: [Libevent-users] win32 iocp patch for libevent main trunk Message-ID: <200806132327030782261@gmail.com> Hi, I want to submit win32 iocp patch for main trunk ( with VS2008 ). This one integerates the iocp backend into libevent's main trunk. This one is passed mostly testcases in regress.c, except the following http_suite, rpc_suite, test_edgetriggered, rpc_test. The patch is in attachment. Include a diff file and win32iocp.c . Compiles and works under VS2008, and it should compile and word under VC6. The changes are: 1.Add a win32iocp.c, this file is implemented the iocp backend. It use a macro (_EVENT_NOIOCP) to disable iocp backend on the platform which is not support iocp. iocp is only supported on NT4 and later. 2.Add EV_ACCEPT flag for accept event ( in include\event.h ) #define EV_ACCEPT 0x40 3.Add win32iocpops into the eventops array ( in event.c ), also use _EVENT_NOIOCP to guard. 4.Rename INPUT to EVRPC_INPUT and OUTPUT to EVRPC_INPUT ( in evrpc.h/evrpc.c ) Because these two symbols are conflict with Windows SDK's header. The full package: http://spserver.googlecode.com/files/libevent-main-iocp.zip Include: 1.libevent : library 2.regress : regress.c testcases 3.echo_test : an echo server, default listen on 5555 4.echo_stress : stress test tools for echo server Stress Test Tools Usage: echo_stress.exe [-h ] [-p ] [-c ] [-m ] -h default is 127.0.0.1 -p default is 5555 -c how many clients, default is 10 -m messages per client, default is 10 c:\>echo_stress -c 10000 -m 1000 Test result : Clients : 10000, Messages Per Client : 1000 ExecTimes: 927.078000 seconds client Send Recv total : 10000000 9999694 average : 10787/s 10786/s Stephen Liu 2008-06-13 -------------- next part -------------- A non-text attachment was scrubbed... Name: WIN32-Code-IOCP.zip Type: application/octet-stream Size: 6027 bytes Desc: not available Url : http://monkeymail.org/archives/libevent-users/attachments/20080613/312e7be4/WIN32-Code-IOCP.obj From juanito at staff.epita.fr Sat Jun 14 13:23:19 2008 From: juanito at staff.epita.fr (juanito) Date: Sat Jun 14 13:32:19 2008 Subject: [Libevent-users] [WINDOWS] Libevent Message-ID: <4853FE87.5080507@staff.epita.fr> Hi, Anyone could give me some tips how to compile libevent with windows ? When openning WIN32-Prj\libevent.dsw, it says the project can't be openned because the file is truncated. Any tips ? Thanks. -- Juanito IONIS System & Network Administrator juanito@iit.ionis-group.com 14-16 rue Voltaire 94270 Le Kremlin-Bicetre From nickm at freehaven.net Sat Jun 14 13:50:47 2008 From: nickm at freehaven.net (Nick Mathewson) Date: Sat Jun 14 13:51:25 2008 Subject: [Libevent-users] patch for building libevent with vs2008 In-Reply-To: <200806122300174688931@gmail.com> References: <200806122300174688931@gmail.com> Message-ID: <20080614175047.GC28348@moria.seul.org> On Thu, Jun 12, 2008 at 11:00:28PM +0800, liusifan wrote: > Hi, all > > Libevent svn main trunk cannot build with vs2008. > > The patch is in attachment. > Changes: Rename INPUT to EVRPC_INPUT and OUTPUT to EVRPC_INPUT Hi! This patch has the same problem as the last three patches that did this: it doesn't keep backward compatibility with existing code that uses the old names. We try very hard not to change source APIs in ways that will break correct code that worked before. Instead, I'd suggest changing enum EVRPC_HOOK_TYPE { INPUT, OUTPUT }; to enum EVRPC_HOOK_TYPE { EVRPC_INPUT, EVRPC_OUTPUT }; #ifndef WIN32 #define INPUT EVRPC_INPUT #define OUTPUT EVRPC_INPUT #endif I've checked in a fix that does this; please let me know if it has problems. yrs, -- Nick From nickm at freehaven.net Sat Jun 14 14:03:20 2008 From: nickm at freehaven.net (Nick Mathewson) Date: Sat Jun 14 14:03:38 2008 Subject: [Libevent-users] files and libevent In-Reply-To: <485139E1.2050206@wintersgift.com> References: <485139E1.2050206@wintersgift.com> Message-ID: <20080614180320.GD28348@moria.seul.org> On Thu, Jun 12, 2008 at 07:59:45AM -0700, Teunis Peters wrote: > I've not found any code to work with - but is there any reason that > libevent would not work with standard files? > > I keep getting permission denied > > more or less: > > fd = open(filename, O_RDONLY) > event_set(&ev, EV_READ | EV_PERSIST, rd_callback, rd_data); > event_add(&ev, NULL) > -> EPERM > > Or does libevent only work with network connections? Libevent uses underlying nonblocking IO mechanisms the platform gives it. Some of these work well with non-socket file descriptors; some don't. By default, libevent uses the fastest (best-scaling) backend that it knows about for your platform, even if that backend doesn't support all fds. In the current svn trunk (which will eventually become libevent 2.0), there's a feature to let you specify that you want a backend that works with file descriptors, even if it doesn't scale well. yrs, -- Nick From nickm at freehaven.net Sat Jun 14 14:13:28 2008 From: nickm at freehaven.net (Nick Mathewson) Date: Sat Jun 14 14:14:01 2008 Subject: [Libevent-users] [err] event_queue_insert: 0xb6a4517c(fd 17) already on queue 2 In-Reply-To: <4852295D.7040307@neonova.nl> References: <4852295D.7040307@neonova.nl> Message-ID: <20080614181328.GE28348@moria.seul.org> On Fri, Jun 13, 2008 at 10:01:33AM +0200, Ron Arts wrote: > Hello, > > I get this error on libevent 1.3e on CentOS5. Upgrading to > a higher version is not easy to push through, to say the least, > so that's why I am asking the list first. The error message looks like an event that's already been added to a queue is getting added again, and that's not supposed to happen. "queue 2" is EVLIST_INSERTED, the list of all events that have been event_add()ed for IO events and not event_del()d. The double insert is pretty weird, though, since event_add() is pretty good about not adding an event that's already been added. The code where it puts an event into the EVLIST_INSERTED queue is (in 1.3-svn): if ((ev->ev_events & (EV_READ|EV_WRITE)) && !(ev->ev_flags & (EVLIST_INSERTED|EVLIST_ACTIVE))) { event_queue_insert(base, ev, EVLIST_INSERTED); return (evsel->add(evbase, ev)); } so unless I can't read C this morning, it should only add the event to the queue if it's already not-inserted and not-active. In other words, something fishy is afoot here. Having a stack trace and the contents of the offending struct event might help track it down. yrs, -- Nick From provos at citi.umich.edu Sat Jun 14 14:18:37 2008 From: provos at citi.umich.edu (Niels Provos) Date: Sat Jun 14 14:18:41 2008 Subject: [Libevent-users] [err] event_queue_insert: 0xb6a4517c(fd 17) already on queue 2 In-Reply-To: <20080614181328.GE28348@moria.seul.org> References: <4852295D.7040307@neonova.nl> <20080614181328.GE28348@moria.seul.org> Message-ID: <850f7cbe0806141118q58b46e36r8270c7e7a6c11f0f@mail.gmail.com> On Sat, Jun 14, 2008 at 11:13 AM, Nick Mathewson wrote: > In other words, something fishy is afoot here. Having a stack trace > and the contents of the offending struct event might help track it down. To me this seems like some kind of memory corruption might be happening; such as accessing an event that was freed or creating an event on the stack that becomes invalid when the function returns. Niels. From liusifan at gmail.com Sun Jun 15 12:57:00 2008 From: liusifan at gmail.com (liusifan) Date: Sun Jun 15 12:57:11 2008 Subject: [Libevent-users] [WINDOWS] Libevent References: <4853FE87.5080507@staff.epita.fr> Message-ID: <200806160056483437332@gmail.com> SGksIGp1YW5pdG8NCg0KV2hpY2ggdmVyc2lvbiBhcmUgeW91IHVzaW5nPw0KDQpZb3UgY2FuIHRy eSB0aGlzIG9uZToNCg0KaHR0cDovL3d3dy5tb25rZXkub3JnL35wcm92b3MvbGliZXZlbnQtMS40 LjQtc3RhYmxlLnRhci5neg0KDQoNCkJlc3QgcmVnYXJkc6OsDQoNClN0ZXBoZW4gTGl1DQoyMDA4 LTA2LTE2DQoNCj4+Pg0KPkhpLA0KPg0KPkFueW9uZSBjb3VsZCBnaXZlIG1lIHNvbWUgdGlwcyBo b3cgdG8gY29tcGlsZSBsaWJldmVudCB3aXRoIHdpbmRvd3MgPw0KPg0KPldoZW4gb3Blbm5pbmcg IFdJTjMyLVByalxsaWJldmVudC5kc3csIGl0IHNheXMgdGhlIHByb2plY3QgY2FuJ3QgYmUgDQo+ b3Blbm5lZCBiZWNhdXNlIHRoZQ0KPmZpbGUgaXMgdHJ1bmNhdGVkLg0KPg0KPkFueSB0aXBzID8N Cj4NCj5UaGFua3MuDQo+DQo+LS0gDQo+SnVhbml0bwkgICAgICAgICAgICAgICAgICAgIElPTklT IFN5c3RlbSAmIE5ldHdvcmsgQWRtaW5pc3RyYXRvcg0KPmp1YW5pdG9AaWl0LmlvbmlzLWdyb3Vw LmNvbSAxNC0xNiBydWUgVm9sdGFpcmUgOTQyNzAgTGUgS3JlbWxpbi1CaWNldHJlDQo+DQo+DQo+ X19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX18NCj5MaWJldmVu dC11c2VycyBtYWlsaW5nIGxpc3QNCj5MaWJldmVudC11c2Vyc0Btb25rZXkub3JnDQo+aHR0cDov L21vbmtleW1haWwub3JnL21haWxtYW4vbGlzdGluZm8vbGliZXZlbnQtdXNlcnMNCgkJCQ0KDQo= From Joerg.Richter at pdv-FS.de Sun Jun 15 13:12:05 2008 From: Joerg.Richter at pdv-FS.de (=?iso-8859-1?Q?Richter=2C_J=F6rg?=) Date: Sun Jun 15 13:12:18 2008 Subject: AW: [Libevent-users] win32 iocp patch for libevent main trunk References: <200806132327030782261@gmail.com> Message-ID: Hi Stephen Liu, > The changes are: > 2.Add EV_ACCEPT flag for accept event ( in include\event.h ) > #define EV_ACCEPT 0x40 I like your work that you have put into this. Keep on the good work! But what I don't like is the new EV_ACCEPT flag. Have you thought about detecting somehow if a socket is a listen socket? I just browsed msdn to find a way. Looks like getsockopt(SO_ACCEPTCONN) can be used for this task. Sorry, but can't test it right now. If your are able to use iocp without a change in the interface everyone will benefit without changing their code. This would be a huge win. J?rg -------------- next part -------------- An HTML attachment was scrubbed... URL: http://monkeymail.org/archives/libevent-users/attachments/20080615/63b3bf9d/attachment.htm From ron.arts at neonova.nl Sun Jun 15 15:14:47 2008 From: ron.arts at neonova.nl (Ron Arts) Date: Sun Jun 15 23:36:54 2008 Subject: [Libevent-users] [err] event_queue_insert: 0xb6a4517c(fd 17) already on queue 2 Message-ID: <48556A27.3060005@neonova.nl> (reposted because it didn't go through according to the archives) Hello, I get this error on libevent 1.3e on CentOS5. Upgrading to a higher version is not easy to push through, to say the least, so that's why I am asking the list first. This error happens after my program has been running for a day or so. There are only 10 clients that stay connected all the time. It's a multithreaded program, only one thread ever calls libevent. What could I be doing wrong to cause this error? Thanks, Ron -- NeoNova BV, The Netherlands Professional internet and VoIP solutions http://www.neonova.nl Kruislaan 419 1098 VA Amsterdam info: 020-5628292 servicedesk: 020-5628292 fax: 020-5628291 KvK Amsterdam 34151241 The following disclaimer applies to this email: http://www.neonova.nl/maildisclaimer -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/x-pkcs7-signature Size: 3295 bytes Desc: S/MIME Cryptographic Signature Url : http://monkeymail.org/archives/libevent-users/attachments/20080615/ba48fda3/smime.bin From ron.arts at neonova.nl Mon Jun 16 05:26:10 2008 From: ron.arts at neonova.nl (Ron Arts) Date: Mon Jun 16 05:26:35 2008 Subject: [Libevent-users] [err] event_queue_insert: 0xb6a4517c(fd 17) already on queue 2 In-Reply-To: <48556A27.3060005@neonova.nl> References: <48556A27.3060005@neonova.nl> Message-ID: <485631B2.4000507@neonova.nl> Ron Arts schreef: > (reposted because it didn't go through according to the archives) > Sorry, my mistake. The post *did* come through, I just missed it for some reason. Ron > Hello, > > I get this error on libevent 1.3e on CentOS5. Upgrading to > a higher version is not easy to push through, to say the least, > so that's why I am asking the list first. > > This error happens after my program has been running for a day > or so. There are only 10 clients that stay connected all the time. > It's a multithreaded program, only one thread ever calls libevent. > > What could I be doing wrong to cause this error? > > Thanks, > Ron > > > ------------------------------------------------------------------------ > > _______________________________________________ > Libevent-users mailing list > Libevent-users@monkey.org > http://monkeymail.org/mailman/listinfo/libevent-users -- NeoNova BV, The Netherlands Professional internet and VoIP solutions http://www.neonova.nl Kruislaan 419 1098 VA Amsterdam info: 020-5628292 servicedesk: 020-5628292 fax: 020-5628291 KvK Amsterdam 34151241 The following disclaimer applies to this email: http://www.neonova.nl/maildisclaimer From hannah at schlund.de Mon Jun 16 09:11:33 2008 From: hannah at schlund.de (Hannah Schroeter) Date: Mon Jun 16 09:12:45 2008 Subject: [Libevent-users] files and libevent In-Reply-To: <20080614180320.GD28348@moria.seul.org> References: <485139E1.2050206@wintersgift.com> <20080614180320.GD28348@moria.seul.org> Message-ID: <20080616131133.GD927@schlund.de> Hi! On Sat, Jun 14, 2008 at 02:03:20PM -0400, Nick Mathewson wrote: >On Thu, Jun 12, 2008 at 07:59:45AM -0700, Teunis Peters wrote: >> I've not found any code to work with - but is there any reason that >> libevent would not work with standard files? >> I keep getting permission denied >> more or less: >> fd = open(filename, O_RDONLY) >> event_set(&ev, EV_READ | EV_PERSIST, rd_callback, rd_data); >> event_add(&ev, NULL) >> -> EPERM >> Or does libevent only work with network connections? >Libevent uses underlying nonblocking IO mechanisms the platform gives >it. Some of these work well with non-socket file descriptors; some >don't. By default, libevent uses the fastest (best-scaling) backend >that it knows about for your platform, even if that backend doesn't >support all fds. >In the current svn trunk (which will eventually become libevent 2.0), >there's a feature to let you specify that you want a backend that >works with file descriptors, even if it doesn't scale well. On most OSes, however, checking for "readable" on plain files always immediately returns/yields a readable event anyway, so it's of no use (e.g. select/poll on Unix, I guess the same for epoll/kqueue, ok, kqueue differs from select/poll in that it returns readable if you're not at EOF, unless you set an additional flag, and then it returns in all cases). You have to use different mechanisms (like aio) if you want to hide disk latencies on plain files without using separate threads for their manipulation. So I think, using libevent events on plain files usually just doesn't make much sense in my eyes. Kind regards, Hannah. From chris.brody at gmail.com Mon Jun 16 09:23:58 2008 From: chris.brody at gmail.com (Chris Brody) Date: Mon Jun 16 09:24:02 2008 Subject: [Libevent-users] files and libevent In-Reply-To: <20080616131133.GD927@schlund.de> References: <485139E1.2050206@wintersgift.com> <20080614180320.GD28348@moria.seul.org> <20080616131133.GD927@schlund.de> Message-ID: <7365f28e0806160623r443ca87dn9b96e0e3906a5042@mail.gmail.com> On Mon, Jun 16, 2008 at 3:11 PM, Hannah Schroeter wrote: [...] > On most OSes, however, checking for "readable" on plain files always > immediately returns/yields a readable event anyway, so it's of no use > (e.g. select/poll on Unix, I guess the same for epoll/kqueue, ok, > kqueue differs from select/poll in that it returns readable if you're > not at EOF, unless you set an additional flag, and then it returns in > all cases). You have to use different mechanisms (like aio) if you want > to hide disk latencies on plain files without using separate threads > for their manipulation. So I think, using libevent events on plain files > usually just doesn't make much sense in my eyes. For good AIO, I suggest you take a look at: http://software.schmorp.de/pkg/libeio.html CB From lyesjob at gmail.com Mon Jun 16 10:16:47 2008 From: lyesjob at gmail.com (Lyes Amazouz) Date: Mon Jun 16 10:17:08 2008 Subject: [Libevent-users] libevent under cygwin Message-ID: <60d886530806160716s7e1a821dt848fb4574eb6f34c@mail.gmail.com> Hi list! more then one month ago, I asked about installation errors I had while I was compiling the libevent under cygwin. Is there some thing new about this ?? thank you -- =========== | Lyes Amazouz | USTHB, Algiers =========== -------------- next part -------------- An HTML attachment was scrubbed... URL: http://monkeymail.org/archives/libevent-users/attachments/20080616/e981d965/attachment-0001.htm From liusifan at gmail.com Mon Jun 16 11:20:00 2008 From: liusifan at gmail.com (liusifan) Date: Mon Jun 16 11:20:09 2008 Subject: AW: [Libevent-users] win32 iocp patch for libevent main trunk References: <200806132327030782261@gmail.com>, Message-ID: <200806162319530317438@gmail.com> SGksIFJpY2h0ZXINCg0KVGhhbmtzIGZvciB5b3VyIHRpcHMgISBUaGUgd2F5IHlvdSBtZW50aW9u IHdvcmtzIHdlbGwuDQoNClRoZSBmdWxsIHBhY2thZ2U6DQpodHRwOi8vc3BzZXJ2ZXIuZ29vZ2xl Y29kZS5jb20vZmlsZXMvbGliZXZlbnQtbWFpbi1pb2NwLTEuemlwCQ0KDQpDaGFuZ2VzOg0KMS5S ZW1vdmUgRVZfQUNDRVBUIGZsYWcsIHVzZSBnZXRzb2Nrb3B0KFNPX0FDQ0VQVENPTk4pIHRvIGRp c3Rpbmd1aXNoIGFjY2VwdC9yZWFkLA0KICBzbyBpdCdzIGNvbXBhdGlibGUgd2l0aCB0aGUgb2xk IGNvZGUgbm93Lg0KMi5Vc2luZyByZWQtYmxhY2sgdHJlZSB0byBtYXAgc29ja2V0cyB0byBldmVu dHMsIGl0IHVzZSBhIGFycmF5IHByZXZpb3VzbHkuDQozLlJlcGxhY2UgV1NBUmVjdiBhbmQgV1NB U2VuZCBieSBSZWFkRmlsZSBhbmQgV3JpdGVGaWxlLCBzbyBpdCBjYW4gcHJvY2VzcyBGaWxlSGFu ZGxlLg0KICBUaGUgZXZlbnRfdGVzdC5jIGlzIHdvcmtpbmcgbm93ICggYnV0IG5lZWQgc29tZSBj aGFuZ2VzICkuDQo0LlN1cHBvcnQgbm9ucGVyc2lzdCBhY2NlcHQgZXZlbnQsIGl0IGFsd2F5cyB0 cmVhdCBhY2NlcHQgZXZlbnQgaXMgcGVyc2lzdCBwcmV2aW91c2x5Lg0KDQpBbnkgYW5kIGFsbCBj b21tZW50cyBhcmUgYXBwcmVjaWF0ZWQuIA0KDQoNCkJlc3QgcmVnYXJkc6OsDQoNClN0ZXBoZW4g TGl1DQoyMDA4LTA2LTE2DQoNCj4+Pg0KPkhpIFN0ZXBoZW4gTGl1LA0KPg0KPj4gVGhlIGNoYW5n ZXMgYXJlOg0KPj4gMi5BZGQgRVZfQUNDRVBUIGZsYWcgZm9yIGFjY2VwdCBldmVudCAoIGluIGlu Y2x1ZGVcZXZlbnQuaCApDQo+PiAgI2RlZmluZSBFVl9BQ0NFUFQgICAweDQwDQo+DQo+SSBsaWtl IHlvdXIgd29yayB0aGF0IHlvdSBoYXZlIHB1dCBpbnRvIHRoaXMuIEtlZXAgb24gdGhlIGdvb2Qg d29yayENCj5CdXQgd2hhdCBJIGRvbid0IGxpa2UgaXMgdGhlIG5ldyBFVl9BQ0NFUFQgZmxhZy4N Cj4NCj5IYXZlIHlvdSB0aG91Z2h0IGFib3V0IGRldGVjdGluZyBzb21laG93IGlmIGEgc29ja2V0 IGlzIGEgbGlzdGVuIHNvY2tldD8gSSBqdXN0IGJyb3dzZWQgbXNkbiB0byBmaW5kIGEgd2F5LiBM b29rcyBsaWtlIGdldHNvY2tvcHQoU09fQUNDRVBUQ09OTikgY2FuIGJlIHVzZWQgZm9yIHRoaXMg dGFzay4gU29ycnksIGJ1dCBjYW4ndCB0ZXN0IGl0IHJpZ2h0IG5vdy4gDQo+DQo+SWYgeW91ciBh cmUgYWJsZSB0byB1c2UgaW9jcCB3aXRob3V0IGEgY2hhbmdlIGluIHRoZSBpbnRlcmZhY2UgZXZl cnlvbmUgd2lsbCBiZW5lZml0IHdpdGhvdXQgY2hhbmdpbmcgdGhlaXIgY29kZS4gVGhpcyB3b3Vs ZCBiZSBhIGh1Z2Ugd2luLg0KPg0KPg0KPiAgIEr2cmcNCgkJCQ0KDQo= From brofield2 at jellycan.com Mon Jun 16 19:28:10 2008 From: brofield2 at jellycan.com (Brodie Thiesfield) Date: Wed Jun 18 01:07:34 2008 Subject: [Libevent-users] [WINDOWS] Libevent In-Reply-To: <200806160056483437332@gmail.com> References: <4853FE87.5080507@staff.epita.fr> <200806160056483437332@gmail.com> Message-ID: <4856F70A.7050105@jellycan.com> liusifan wrote: >juanito wrote: >> Anyone could give me some tips how to compile libevent with windows ? >> >> When openning WIN32-Prj\libevent.dsw, it says the project can't be >> openned because the >> file is truncated. >> >> Any tips ? > > Which version are you using? > > You can try this one: > > http://www.monkey.org/~provos/libevent-1.4.4-stable.tar.gz That version still doesn't work for me either. I had no reply to my message a while ago pointing this out. If you are still searching, try http://code.jellycan.com/memcached/ for a version that works. Perhaps the next stable release will work better since there appears to be action on the list regarding Windows. Regards, Brodie From support at unixservice.com Thu Jun 19 10:09:15 2008 From: support at unixservice.com (Unixservice Support) Date: Thu Jun 19 10:31:26 2008 Subject: [Libevent-users] Access to POST data in evhttp Message-ID: <485A688B.8080007@unixservice.com> Hello, I have just started using libevent and find it very useful. Quick question. After reading through headers and docs, it seems that I only have access to get URI data from forms method=get. I need to post large amounts of a form data and would prefer to use POST. Any ideas? getenv()? Cheers! Gary Wallis From mail2arthur at gmail.com Fri Jun 20 10:27:13 2008 From: mail2arthur at gmail.com (arthur) Date: Fri Jun 20 10:27:57 2008 Subject: [Libevent-users] connected event missing on Solaris 10 Message-ID: <002c01c8d2e1$c418bfd0$9b080f0a@usr.ingenico.loc> My porxy is built with libevent v1.4.4-stable. After accepting client and connecting to server, on Linux (epoll) I can get event upon server connected, but on Solaris 10 (which is using event ports) this event is missing. On both cases the connect() call returns none zero and errno==EINPROGRESS. How can I get this event? Or is there any way to use poll on Solaris? Thanks. Arthur -------------- next part -------------- An HTML attachment was scrubbed... URL: http://monkeymail.org/archives/libevent-users/attachments/20080620/482ce177/attachment.htm From liusifan at gmail.com Fri Jun 20 12:51:37 2008 From: liusifan at gmail.com (liusifan) Date: Fri Jun 20 12:51:58 2008 Subject: [Libevent-users] connected event missing on Solaris 10 References: <002c01c8d2e1$c418bfd0$9b080f0a@usr.ingenico.loc> Message-ID: <200806210051334847592@gmail.com> SGksIGFydGh1cg0KDQpUcnkgdGhpcyA6DQoNCgkvKg0KCSAqIERpc2FibGUgZXZlbnQgcG9ydHMg d2hlbiB0aGlzIGVudmlyb25tZW50IHZhcmlhYmxlIGlzIHNldCANCgkgKi8NCglpZiAoZ2V0ZW52 KCJFVkVOVF9OT0VWUE9SVCIpKQ0KCQlyZXR1cm4gKE5VTEwpOwkNCg0KDQpCZXN0IHJlZ2FyZHOj rA0KDQpsaXVzaWZhbg0KMjAwOC0wNi0yMQ0KDQo+Pj4NCj5NeSBwb3J4eSBpcyBidWlsdCB3aXRo IGxpYmV2ZW50IHYxLjQuNC1zdGFibGUuIEFmdGVyIGFjY2VwdGluZyBjbGllbnQgYW5kIGNvbm5l Y3RpbmcgdG8gc2VydmVyLCBvbiBMaW51eCAoZXBvbGwpIEkgY2FuIGdldCBldmVudCB1cG9uIHNl cnZlciBjb25uZWN0ZWQsIGJ1dCBvbiBTb2xhcmlzIDEwICh3aGljaCBpcyB1c2luZyBldmVudCBw b3J0cykgdGhpcyBldmVudCBpcyBtaXNzaW5nLiBPbiBib3RoIGNhc2VzIHRoZSBjb25uZWN0KCkg Y2FsbCByZXR1cm5zIG5vbmUgemVybyBhbmQgZXJybm89PUVJTlBST0dSRVNTLiBIb3cgY2FuIEkg Z2V0IHRoaXMgZXZlbnQ/IE9yIGlzIHRoZXJlIGFueSB3YXkgdG8gdXNlIHBvbGwgb24gU29sYXJp cz8gVGhhbmtzLg0KPg0KPkFydGh1cg0KPl9fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19f X19fX19fX19fX19fX19fDQo+TGliZXZlbnQtdXNlcnMgbWFpbGluZyBsaXN0DQo+TGliZXZlbnQt dXNlcnNAbW9ua2V5Lm9yZw0KPmh0dHA6Ly9tb25rZXltYWlsLm9yZy9tYWlsbWFuL2xpc3RpbmZv L2xpYmV2ZW50LXVzZXJzDQo+DQoJCQkNCg0K From matti.savolainen at dynamoid.com Mon Jun 23 02:53:07 2008 From: matti.savolainen at dynamoid.com (Matti Savolainen) Date: Mon Jun 23 03:17:39 2008 Subject: [Libevent-users] Access to POST data in evhttp In-Reply-To: <485A688B.8080007@unixservice.com> References: <485A688B.8080007@unixservice.com> Message-ID: <485F4853.8030706@dynamoid.com> Hello, The POST data is avaible in the body, thus you can find it in request->input_buffer. There is no builtin mechanism to parse multipart/formdata so you will need to handle that by your self. -- Matti Savolainen Unixservice Support kirjoitti: > Hello, > > I have just started using libevent and find it very useful. > > Quick question. After reading through headers and docs, it seems that > I only have access to get URI data from forms method=get. > > I need to post large amounts of a form data and would prefer to use POST. > > Any ideas? getenv()? > > Cheers! > Gary Wallis > _______________________________________________ > Libevent-users mailing list > Libevent-users@monkey.org > http://monkeymail.org/mailman/listinfo/libevent-users From mail2arthur at gmail.com Tue Jun 24 12:25:43 2008 From: mail2arthur at gmail.com (arthur) Date: Tue Jun 24 12:26:18 2008 Subject: [Libevent-users] connected event missing on Solaris 10 References: <002c01c8d2e1$c418bfd0$9b080f0a@usr.ingenico.loc> <200806210051334847592@gmail.com> Message-ID: <001301c8d616$f5eaa7c0$9b080f0a@usr.ingenico.loc> Thanks liusifan. That works. Arthur > Hi, arthur > > Try this : > > /* > * Disable event ports when this environment variable is set > */ > if (getenv("EVENT_NOEVPORT")) > return (NULL); > > > Best regards£¬ > > liusifan > 2008-06-21 > > >>> > >My porxy is built with libevent v1.4.4-stable. After accepting client and connecting to server, on Linux (epoll) I can get event upon server connected, but on Solaris 10 (which is using event ports) this event is missing. On both cases the connect() call returns none zero and errno==EINPROGRESS. How can I get this event? Or is there any way to use poll on Solaris? Thanks. > > > >Arthur > >_______________________________________________ > >Libevent-users mailing list > >Libevent-users@monkey.org > >http://monkeymail.org/mailman/listinfo/libevent-users > > > > > From support at unixservice.com Tue Jun 24 20:19:20 2008 From: support at unixservice.com (Unixservice Support) Date: Tue Jun 24 20:19:43 2008 Subject: [Libevent-users] Access to POST data in evhttp In-Reply-To: <485D9276.90106@slamb.org> References: <485A688B.8080007@unixservice.com> <485D9276.90106@slamb.org> Message-ID: <48618F08.7060105@unixservice.com> Thanks, the binary req->input_buffer has the posted name/value pairs in it. The remaining problem is the offset where the name/value pairs start and where they end: Any clues, code etc, much appreciated: --- More details if you need them: I'm used to a different C cgi API and if anyone has any C name/value parser (name0=val0&nam1=val1&...&nameN=valN posted data) to use with libevent's quick and cool evhttp interface let me know. There is one in the library but I can not get it to work. (Have the latest stable version.) Can't seem to find out how to get the input_buffer length. Or how to find the beginning of the name value pair without using a binary string search function that depends on the name of the first name/value pair. The CGI specs for post data refer to the client informing the server about CONTENT_LENGTH bytes. But it seems that the input_buffer has other libevent data in it, maybe the CONTENT_LENGTH and CONTENT_TYPE of the data sent but in the libevent headers. I can't find more on this issue. Probably obvious to the initiated. I realize that maybe another callback function works when all of the "chunks" arrive. --- Thanks! From a libevent newbie. From william at 25thandClement.com Tue Jun 24 21:10:02 2008 From: william at 25thandClement.com (William Ahern) Date: Tue Jun 24 21:10:10 2008 Subject: [Libevent-users] Access to POST data in evhttp In-Reply-To: <48618F08.7060105@unixservice.com> References: <485A688B.8080007@unixservice.com> <485D9276.90106@slamb.org> <48618F08.7060105@unixservice.com> Message-ID: <20080625011001.GA3745@wilbur.25thandClement.com> On Tue, Jun 24, 2008 at 09:19:20PM -0300, Unixservice Support wrote: > Thanks, the binary req->input_buffer has the posted name/value pairs in it. > > The remaining problem is the offset where the name/value pairs start and > where they end: Any clues, code etc, much appreciated: > > --- > More details if you need them: > > I'm used to a different C cgi API and if anyone has any C name/value > parser (name0=val0&nam1=val1&...&nameN=valN posted data) to use with > libevent's quick and cool evhttp interface let me know. There is one in > the library but I can not get it to work. (Have the latest stable version.) Appended below (uri.h first, then uri.c) is a zero-allocation URI parser. uri_parse() isn't stateful, so you have to pass the whole URI upfront. (I figured that was a tad too sophisticated in this case, and in any event termination condition is different depending on context anyhow.) But it keeps pointers into the buffer you pass it, so if the buffer becomes invalid, the contents of the URI structure does as well. To see how it works, from the commandline do: $ make uri CPPFLAGS="-DURI_DEBUG" $ ./uri 'http://google.com/?foo=bar&bar=#frag' $ ./uri 'http://google.com/?foo=bar&bar=#frag' 01 # mask 2nd param For the scheme, auth, path, params, and fragment you'll see whether it was defined, undefined, or an empty string. For the query params, you have to loop using uri_next_query()--this one is stateful. See main() for usage. You can recompose the URI and mask out params by position. I wrote this for a zero-copy transparent HTTP/RTSP proxy. I re-write the traffic w/o additional allocations (or rather O(1) allocation, independent of content), thus the peculiar API. There's a small bug, I think, w/ setting default parts, like the scheme. But it's just a bug w/ the semantics, nothing critical. If you fix anything, please send a patch. This bit of code is too small to make a project out of. Oh, also you need to use a C99 compiler (or GCC in [default] gnu mode). This won't compile in Visual Studio w/o re-writing the array initialization and perhaps some other bits. /* ========================================================================== * uri.h - URI parser. * -------------------------------------------------------------------------- * Copyright (c) 2007 William Ahern * * Permission is hereby granted, free of charge, to any person obtaining a * copy of this software and associated documentation files (the "Software"), * to deal in the Software without restriction, including without limitation * the rights to use, copy, modify, merge, publish, distribute, sublicense, * and/or sell copies of the Software, and to permit persons to whom the * Software is furnished to do so, subject to the following conditions: * * The above copyright notice and this permission notice shall be included in * all copies or substantial portions of the Software. * * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS * OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF * MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN * NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, * DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR * OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE * USE OR OTHER DEALINGS IN THE SOFTWARE. * ========================================================================== */ #ifndef URI_H #define URI_H #include #include /* struct iovec */ #define URI_PART_ISDECLARED(u, m) \ ((u)->m.iov_base != (u)->initializer->m.iov_base) #define URI_PART_ISDEFINED(u, m) \ (URI_PART_ISDECLARED((u), m) && (u)->m.iov_len > 0) #define URI_PART_ISEMPTY(u, m) \ ((u)->m.iov_len == 0) enum uri_errno { URI_ESUCCESS = 0, URI_ENOMORE, URI_ESYNTAX, }; /* enum uri_errno */ struct uri { const struct uri *initializer; struct iovec scheme; struct iovec authority; struct iovec path; struct iovec query; struct iovec fragment; }; /* struct uri */ extern const struct uri uri_initializer; extern const struct uri uri_http_initializer; struct uri_authority { const struct uri_authority *initializer; struct iovec userinfo; struct iovec host; struct iovec port; }; /* struct uri_authority */ extern const struct uri_authority uri_authority_initializer; struct uri_query { const struct uri_query *initializer; struct iovec key; struct iovec value; }; /* struct uri_query */ extern const struct uri_query uri_query_initializer; enum uri_errno uri_parse(struct uri *, const void *, size_t, int flags); enum uri_errno uri_parse_authority(struct uri_authority *, const void *, size_t, int flags); #define URI_SKIP_NO_KEY 1 #define URI_SKIP_NO_VALUE 2 #define URI_SKIP_EMPTY (URI_SKIP_NO_KEY | URI_SKIP_NO_VALUE) enum uri_errno uri_next_query(struct uri_query *, const void *, size_t, unsigned long *, unsigned); typedef unsigned long uri_decode_t; size_t uri_decode(void *, size_t, const void *, size_t, char **, const void *, size_t, uri_decode_t *); size_t uri_print_query(void *, size_t, struct uri *, const unsigned char *, size_t, unsigned); size_t uri_print(void *, size_t, struct uri *, const unsigned char *, size_t, unsigned); #endif /* URI_H */ /* ========================================================================== * uri.c - URI parser. * -------------------------------------------------------------------------- * Copyright (c) 2007 William Ahern * * Permission is hereby granted, free of charge, to any person obtaining a * copy of this software and associated documentation files (the "Software"), * to deal in the Software without restriction, including without limitation * the rights to use, copy, modify, merge, publish, distribute, sublicense, * and/or sell copies of the Software, and to permit persons to whom the * Software is furnished to do so, subject to the following conditions: * * The above copyright notice and this permission notice shall be included in * all copies or substantial portions of the Software. * * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS * OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF * MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN * NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, * DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR * OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE * USE OR OTHER DEALINGS IN THE SOFTWARE. * ========================================================================== */ #if URI_DEBUG #include #endif #include /* memchr(3) */ #include /* CHAR_BIT */ #include /* isalnum(3) */ #include /* assert(3) */ #include /* isset howmany MIN */ #include #include /* struct iovec */ #include "uri.h" const struct uri uri_initializer = { &uri_initializer, { 0, 0 }, }; /* uri_initializer */ const struct uri uri_http_initializer = { &uri_http_initializer, { "http", sizeof "http" - 1 }, { 0, 0 }, { "/", sizeof "/" - 1 }, { 0, 0 }, { 0, 0 }, }; /* uri_http_initializer */ const struct uri_authority uri_authority_initializer = { &uri_authority_initializer, { 0, 0 }, }; /* uri_authority_initializer */ const struct uri_query uri_query_initializer = { &uri_query_initializer, { 0, 0 }, }; /* uri_query_initializer */ static const unsigned char *uri_scheme_next_invalid(const unsigned char *pos, const unsigned char *end) { /* RFC 3986 Sec 3.1. */ while (pos < end && (isalnum(*pos) || *pos == '+' || *pos == '-' || *pos == '.')) pos++; return pos; } /* uri_scheme_next_invalid() */ static const unsigned char *uri_authority_next_invalid(const unsigned char *pos, const unsigned char *end) { /* RFC 3986 Sec 3.2. */ while (pos < end && *pos != '/' && *pos != '?' && *pos != '#') pos++; return pos; } /* uri_authority_next_invalid() */ static const unsigned char *uri_path_next_invalid(const unsigned char *pos, const unsigned char *end) { /* RFC 3986 Sec 3.3. */ while (pos < end && *pos != '?' && *pos != '#') pos++; return pos; } /* uri_path_next_invalid() */ static const unsigned char *uri_query_next_invalid(const unsigned char *pos, const unsigned char *end) { /* RFC 3986 Sec 3.4. */ while (pos < end && *pos != '#') pos++; return pos; } /* uri_query_next_invalid() */ static const unsigned char *uri_fragment_next_invalid(const unsigned char *pos, const unsigned char *end) { /* RFC 3986 Sec 3.5. */ while (pos < end) pos++; return pos; } /* uri_fragment_next_invalid() */ enum uri_errno uri_parse(struct uri *uri, const void *str, size_t slen, int flags) { struct { const unsigned char *pos, *nxt, *end; } src; src.pos = str; src.end = src.pos + slen; /* Find scheme. */ src.nxt = uri_scheme_next_invalid(src.pos, src.end); if (src.nxt < src.end && *src.nxt == ':' && src.nxt != src.pos) { uri->scheme.iov_base = (void *)src.pos; uri->scheme.iov_len = src.nxt - src.pos; src.pos = src.nxt + 1; /* Skip ':' */ } if (src.pos >= src.end) return 0; /* Find authority. */ if (&src.pos[1] < src.end && src.pos[0] == '/' && src.pos[1] == '/') { src.pos += 2; src.nxt = uri_authority_next_invalid(src.pos, src.end); uri->authority.iov_base = (void *)src.pos; uri->authority.iov_len = src.nxt - src.pos; src.pos = src.nxt; } /* Find path. */ src.nxt = uri_path_next_invalid(src.pos, src.end); if (src.nxt != src.pos) { uri->path.iov_base = (void *)src.pos; uri->path.iov_len = src.nxt - src.pos; src.pos = src.nxt; } /* Find query. */ if (src.pos < src.end && *src.pos == '?') { src.pos++; src.nxt = uri_query_next_invalid(src.pos, src.end); uri->query.iov_base = (void *)src.pos; uri->query.iov_len = src.nxt - src.pos; src.pos = src.nxt; } /* Find fragment. */ if (src.pos < src.end && *src.pos == '#') { src.pos++; src.nxt = uri_fragment_next_invalid(src.pos, src.end); uri->fragment.iov_base = (void *)src.pos; uri->fragment.iov_len = src.nxt - src.pos; src.pos = src.nxt; } return 0; } /* uri_parse() */ enum uri_errno uri_parse_authority(struct uri_authority *auth, const void *str, size_t slen, int flags) { struct { const unsigned char *pos, *nxt, *end; } src; src.pos = str; src.end = src.pos + slen; /* Find the end of either the userinfo or host:port. */ for (src.nxt = src.pos; src.nxt < src.end && *src.nxt != '@'; src.nxt++) ;; if (src.nxt < src.end) { auth->userinfo.iov_base = (void *)src.pos; auth->userinfo.iov_len = src.nxt - src.pos; src.pos = src.nxt + 1; /* Skip '@' */ if (src.pos >= src.end) return 0; /* Nothing left. */ } /* Find any port delimiter. */ for (src.nxt = src.end - 1; src.nxt >= src.pos && *src.nxt != ':'; src.nxt--) ;; if (src.nxt >= src.pos) { auth->port.iov_base = (void *)src.nxt + 1; auth->port.iov_len = src.end - src.nxt; } else src.nxt = src.end; auth->host.iov_base = (void *)src.pos; auth->host.iov_len = src.nxt - src.pos; return 0; } /* uri_parse_authority() */ enum uri_errno uri_next_query(struct uri_query *q, const void *str, size_t slen, unsigned long *state, unsigned flags) { struct { const unsigned char *pos, *nxt, *end; } src; struct iovec key = { 0, 0 }; struct iovec value = { 0, 0 }; src.pos = (unsigned char *)str + *state; src.end = (unsigned char *)str + slen; next: if (src.pos >= src.end) return URI_ENOMORE; /* Find '=' or '&'. */ for (src.nxt = src.pos; src.nxt < src.end && *src.nxt != '=' && *src.nxt != '&'; src.nxt++) ;; key.iov_base = (void *)src.pos; key.iov_len = src.nxt - src.pos; src.pos = src.nxt + 1; /* Skip '=' or '&' */ if (src.nxt >= src.end || *src.nxt == '&') goto done; /* Find '&'. */ for (src.nxt = src.pos; src.nxt < src.end && *src.nxt != '&'; src.nxt++) ;; value.iov_base = (void *)src.pos; value.iov_len = src.nxt - src.pos; src.pos = src.nxt + 1; done: *state = src.pos - (unsigned char *)str; if ((key.iov_len == 0 && (flags & URI_SKIP_NO_KEY)) || (value.iov_len == 0 && (flags & URI_SKIP_NO_VALUE))) goto next; if (key.iov_base != 0) q->key = key; if (value.iov_base != 0) q->value = value; return 0; } /* uri_next_query() */ /* * Invalid values have all bits set, the only value guaranteed not to occur * in an "encoding" (if all bits are available it would be raw, not * encoded). */ #define BASE16_VALUE_INVALID ((1U << CHAR_BIT) - 1) static const unsigned char base16_value[1U << CHAR_BIT] = { [0 ... (1U << CHAR_BIT) - 1] = BASE16_VALUE_INVALID, /* * We're assuming our compiler's locale's alnum range is mapped like * ASCII (or is ASCII). */ ['0'] = 0, ['1'] = 1, ['2'] = 2, ['3'] = 3, ['4'] = 4, ['5'] = 5, ['6'] = 6, ['7'] = 7, ['8'] = 8, ['9'] = 9, ['A'] = 10, ['B'] = 11, ['C'] = 12, ['D'] = 13, ['E'] = 14, ['F'] = 15, ['a'] = 10, ['b'] = 11, ['c'] = 12, ['d'] = 13, ['e'] = 14, ['f'] = 15, }; /* base16_value */ #define URI_DECODE_ESCAPED (1UL << ((sizeof (uri_decode_t) * CHAR_BIT) - 1)) #define URI_DECODE_PUTC_NEEDMORE -1 #define URI_DECODE_PUTC_INVALID -2 static inline int uri_decode_putc(unsigned long *chars, int *nchars, unsigned char ch, uri_decode_t *state) { unsigned char hi, lo; if (!(*state & URI_DECODE_ESCAPED) && ch != '%') return ch; if ((*state & URI_DECODE_ESCAPED) && BASE16_VALUE_INVALID == base16_value[ch]) return URI_DECODE_PUTC_INVALID; *chars = (*chars << CHAR_BIT) | (0xff & ch); ++(*nchars); switch (*nchars) { case 3: hi = 0xff & (*chars >> (1 * CHAR_BIT)); lo = 0xff & (*chars >> (0 * CHAR_BIT)); *chars = 0; *nchars = 0; *state &= ~URI_DECODE_ESCAPED; return (base16_value[hi] << 4) | base16_value[lo]; case 2: assert(*state & URI_DECODE_ESCAPED); return URI_DECODE_PUTC_NEEDMORE; case 1: *state |= URI_DECODE_ESCAPED; return URI_DECODE_PUTC_NEEDMORE; } assert(0); return URI_DECODE_PUTC_INVALID; } /* uri_decode_putc() */ size_t uri_decode(void *buf, size_t blen, const void *str, size_t slen, char **end, const void *stop, size_t nstop, uri_decode_t *state) { struct { unsigned char *pos, *end; } src, dst; unsigned long chars = *state & ((1 << (3 * CHAR_BIT)) - 1); int nchars = (*state >> (3 * CHAR_BIT)) & 0x03; int ch; src.pos = (unsigned char *)str; src.end = (slen == -1)? (unsigned char *)-1 : src.pos + slen; dst.pos = buf; dst.end = buf + blen; for (; src.pos < src.end && dst.pos < dst.end; src.pos++) { if (nstop && memchr(stop, *src.pos, nstop)) break; switch (ch = uri_decode_putc(&chars, &nchars, *src.pos, state)) { case URI_DECODE_PUTC_INVALID: goto finish; case URI_DECODE_PUTC_NEEDMORE: break; default: *(dst.pos++) = ch; } } finish: *state &= ~(0x03 << (3 * CHAR_BIT)); *state |= (nchars & 0x03) << (3 * CHAR_BIT); *state |= chars & ((3 * CHAR_BIT) - 1); *end = (char *)src.pos; return dst.pos - (unsigned char *)buf; } /* uri_decode() */ size_t uri_print_query(void *dbuf, size_t dlen, struct uri *u, const unsigned char *mask, size_t msiz, unsigned flags) { struct { unsigned char *pos, *end; } dst = { dbuf, (unsigned char *)dbuf + dlen }; struct uri_query q = uri_query_initializer; unsigned long s = 0; unsigned i = 0; unsigned n = 0; for (; 0 == uri_next_query(&q, u->query.iov_base, u->query.iov_len, &s, flags); i++, q = uri_query_initializer) { if (msiz >= howmany(i + 1, CHAR_BIT) && isset(mask, i)) continue; if (n++ > 0) { if (dst.pos < dst.end) *dst.pos = '&'; dst.pos++; } if (dst.pos < dst.end) (void)memcpy(dst.pos, q.key.iov_base, MIN((size_t)(dst.end - dst.pos), q.key.iov_len)); dst.pos += q.key.iov_len; if (URI_PART_ISDECLARED(&q, value)) { if (dst.pos < dst.end) *dst.pos = '='; dst.pos++; if (dst.pos < dst.end) (void)memcpy(dst.pos, q.value.iov_base, MIN((size_t)(dst.end - dst.pos), q.value.iov_len)); dst.pos += q.value.iov_len; } } return dst.pos - (unsigned char *)dbuf; } /* uri_print_query() */ size_t uri_print(void *dbuf, size_t dlen, struct uri *u, const unsigned char *mask, size_t msiz, unsigned flags) { struct { unsigned char *pos, *end; } dst = { dbuf, (unsigned char *)dbuf + dlen }; if (!URI_PART_ISEMPTY(u, scheme)) { if (dst.pos < dst.end) (void)memcpy(dst.pos, u->scheme.iov_base, MIN((size_t)(dst.end - dst.pos), u->scheme.iov_len)); dst.pos += u->scheme.iov_len; if (dst.pos < dst.end) *dst.pos = ':'; dst.pos++; if (dst.pos < dst.end) *dst.pos = '/'; dst.pos++; if (dst.pos < dst.end) *dst.pos = '/'; dst.pos++; } if (!URI_PART_ISEMPTY(u, authority)) { if (dst.pos < dst.end) (void)memcpy(dst.pos, u->authority.iov_base, MIN((size_t)(dst.end - dst.pos), u->authority.iov_len)); dst.pos += u->authority.iov_len; } if (!URI_PART_ISEMPTY(u, path)) { if (dst.pos < dst.end) (void)memcpy(dst.pos, u->path.iov_base, MIN((size_t)(dst.end - dst.pos), u->path.iov_len)); dst.pos += u->path.iov_len; } if (URI_PART_ISDECLARED(u, query) || !URI_PART_ISEMPTY(u, query)) { if (dst.pos < dst.end) *dst.pos = '?'; dst.pos++; dst.pos += uri_print_query(dst.pos, (dst.pos < dst.end)? dst.end - dst.pos : 0, u, mask, msiz, flags); } if (URI_PART_ISDECLARED(u, fragment) || !URI_PART_ISEMPTY(u, fragment)) { if (dst.pos < dst.end) *dst.pos = '#'; dst.pos++; if (dst.pos < dst.end) (void)memcpy(dst.pos, u->fragment.iov_base, MIN((size_t)(dst.end - dst.pos), u->fragment.iov_len)); dst.pos += u->fragment.iov_len; } return dst.pos - (unsigned char *)dbuf; } /* uri_print() */ #if URI_DEBUG #include /* printf(3) puts(3) */ #include /* strlen(3) */ int main(int argc, char *argv[]) { struct uri uri; struct uri_query q; int i, j, n; unsigned long s; char buf[256]; char dbuf[256]; unsigned char mask[16] = { 0 }; size_t dlen; if (argc < 2) return 0; for (n = 0, i = 2; i < argc; i++) { for (j = 0; argv[i][j] != '\0' && n < sizeof mask * CHAR_BIT; j++, n++) { if ('1' == argv[i][j]) setbit(mask, n); } } uri = uri_http_initializer; uri_parse(&uri, argv[1], strlen(argv[1]), 0); dlen = uri_print(dbuf, sizeof dbuf, &uri, mask, sizeof mask, 0); printf("recomposed: %.*s\n", (int)MIN(dlen, sizeof dbuf), dbuf); if (argc > 2) return 0; printf("%.*s[%s]\n", (i > 1), "\n", argv[i]); printf("scheme: [%.*s]\n", (int)uri.scheme.iov_len, (char *)uri.scheme.iov_base); printf("authority: [%.*s]\n", (int)uri.authority.iov_len, (char *)uri.authority.iov_base); printf("path: [%.*s]\n", (int)uri.path.iov_len, (char *)uri.path.iov_base); printf("query: [%.*s]", (int)uri.query.iov_len, (char *)uri.query.iov_base); for (s = 0, q = uri_query_initializer; 0 == uri_next_query(&q, uri.query.iov_base, uri.query.iov_len, &s, URI_SKIP_NO_KEY); q = uri_query_initializer) { uri_decode_t st = 0; char *beg, *pos, *end; size_t n, len = 0; pos = beg = q.value.iov_base; end = pos + q.value.iov_len; while (0 < (n = uri_decode(&buf[len], 1, pos, end - pos, &pos, 0, 0, &st))) { len += n; } printf(" [%.*s=(%d)%.*s]", (int)q.key.iov_len, (char *)q.key.iov_base, (int)len, (int)len, buf); } puts(""); printf("fragment: [%.*s]\n", (int)uri.fragment.iov_len, (char *)uri.fragment.iov_base); return 0; } /* main() */ #endif /* URI_DEBUG */ From provos at citi.umich.edu Wed Jun 25 22:31:32 2008 From: provos at citi.umich.edu (Niels Provos) Date: Wed Jun 25 22:32:11 2008 Subject: [Libevent-users] ANN: Libevent 1.4.5-stable released Message-ID: <850f7cbe0806251931n1ea83e99l1a32b8d7a817eeff@mail.gmail.com> Hi, I am happy to announce the release of libevent 1.4.5-stable. You can download the source here: http://monkey.org/~provos/libevent-1.4.5-stable.tar.gz There have been a few bug fixes since 1.4.4-stable. Here's a brief summary: - Several HTTP fixes - Fixed the Windows port - event_rpcgen.py correctly generates fixed length entries now See the changelog for full details: http://levent.svn.sourceforge.net/viewvc/levent/branches/patches-1.4/libevent/ChangeLog?revision=885&view=markup We would like to thank the people who have reported bugs including Forest Wilkinson, liusifan and others. To report a bug, make a feature request, or submit code, you can use our sourceforge interface here: https://sourceforge.net/projects/levent Thanks again, Niels. From brofield2 at jellycan.com Wed Jun 25 22:51:38 2008 From: brofield2 at jellycan.com (Brodie Thiesfield) Date: Wed Jun 25 22:51:54 2008 Subject: [Libevent-users] ANN: Libevent 1.4.5-stable released In-Reply-To: <850f7cbe0806251931n1ea83e99l1a32b8d7a817eeff@mail.gmail.com> References: <850f7cbe0806251931n1ea83e99l1a32b8d7a817eeff@mail.gmail.com> Message-ID: <4863043A.9020501@jellycan.com> Niels Provos wrote: > I am happy to announce the release of libevent 1.4.5-stable. You can > download the source here: > > http://monkey.org/~provos/libevent-1.4.5-stable.tar.gz This still doesn't build for me on Windows. If I am missing some requirements please let me know. Perhaps there should be a readme for Windows building as well? Steps to reproduce: * extract 1.4.5 archive * open .dsw in Visual Studio 6 (or 2003, or 2005) * batch build, rebuild all Results: * lots of errors like the following --------------------Configuration: libevent - Win32 Debug-------------------- Compiling... log.c d:\downloads\libevent-1.4.5-stable\event.h(167) : fatal error C1083: Cannot open include file: 'sys/time.h': No such file or directory event.c d:\downloads\libevent-1.4.5-stable\event.h(167) : fatal error C1083: Cannot open include file: 'sys/time.h': No such file or directory misc.c fatal error C1083: Cannot open source file: 'D:\Downloads\libevent-1.4.5-stable\WIN32-Code\misc.c': No such file or directory win32.c d:\downloads\libevent-1.4.5-stable\event.h(167) : fatal error C1083: Cannot open include file: 'sys/time.h': No such file or directory Error executing cl.exe. event_test.exe - 4 error(s), 0 warning(s) --------------------Configuration: signal_test - Win32 Debug-------------------- Compiling... signal-test.c d:\downloads\libevent-1.4.5-stable\event.h(167) : fatal error C1083: Cannot open include file: 'sys/time.h': No such file or directory Error executing cl.exe. signal_test.exe - 1 error(s), 0 warning(s) From provos at citi.umich.edu Wed Jun 25 23:16:55 2008 From: provos at citi.umich.edu (Niels Provos) Date: Wed Jun 25 23:17:01 2008 Subject: [Libevent-users] ANN: Libevent 1.4.5-stable released In-Reply-To: <4863043A.9020501@jellycan.com> References: <850f7cbe0806251931n1ea83e99l1a32b8d7a817eeff@mail.gmail.com> <4863043A.9020501@jellycan.com> Message-ID: <850f7cbe0806252016o603c38x4b6162fd67cb548@mail.gmail.com> On Wed, Jun 25, 2008 at 7:51 PM, Brodie Thiesfield wrote: > This still doesn't build for me on Windows. If I am missing some > requirements please let me know. Perhaps there should be a readme for > Windows building as well? > > Steps to reproduce: > * extract 1.4.5 archive > * open .dsw in Visual Studio 6 (or 2003, or 2005) > * batch build, rebuild all We really need some better way of building in Windows like Makefiles. Try opening WIN32-Proj/libevent.sln with Visual Studio and let us know if that works any better for you. Niels. From brofield2 at jellycan.com Thu Jun 26 00:18:35 2008 From: brofield2 at jellycan.com (Brodie Thiesfield) Date: Thu Jun 26 00:18:44 2008 Subject: [Libevent-users] ANN: Libevent 1.4.5-stable released In-Reply-To: <850f7cbe0806252016o603c38x4b6162fd67cb548@mail.gmail.com> References: <850f7cbe0806251931n1ea83e99l1a32b8d7a817eeff@mail.gmail.com> <4863043A.9020501@jellycan.com> <850f7cbe0806252016o603c38x4b6162fd67cb548@mail.gmail.com> Message-ID: <4863189B.5020200@jellycan.com> Niels Provos wrote: > On Wed, Jun 25, 2008 at 7:51 PM, Brodie Thiesfield > wrote: > >> This still doesn't build for me on Windows. If I am missing some >> requirements please let me know. Perhaps there should be a readme for >> Windows building as well? >> >> Steps to reproduce: >> * extract 1.4.5 archive >> * open .dsw in Visual Studio 6 (or 2003, or 2005) >> * batch build, rebuild all >> > > We really need some better way of building in Windows like Makefiles. > Try opening > > WIN32-Proj/libevent.sln > > with Visual Studio and let us know if that works any better for you. > This only opens in Visual Studio 2005 because it is the latest version. When opening it is missing all of the required .vcproj files (the actual project definition files) except for "regress". Attempting to build regress results in the same errors as the .dsw project (see below). I think that workspace / project files are fine. The problem appears to me that no-one is actually building libevent on Windows, or that they are using different project files than are being released. I would suggest that if the .dsw / .dsp (Visual Studio 6) files are updated (if necessar), and the source files are updated as necessary then this build will work. For the changes I found necessary to build 1.4.4 on Windows with Visual Studio 2003 with no warnings, see http://code.jellycan.com/files/libevent.diff In summary, this is: * changing include of to "event.h" * changing include of to "evutil.h" * changing include of to "event-config.h" * fixing incorrect definitions in "win32/event-config.h" * changing a couple of variable from signed to unsigned * explicitly using the A versions of Win32 functions (to be compatible with Unicode builds) * removing the .dsw / .dsp files and adding .sln / .vcproj files * casting away signed / unsigned warnings At some stage I will get around to fixing this patch for 1.4.5 too. Regards, Brodie Results from building "regress" project: 1>------ Build started: Project: regress, Configuration: Debug Win32 ------ 1>Compiling... 1>regress.c 1>d:\downloads\libevent-1.4.5-stable\event.h(167) : fatal error C1083: Cannot open include file: 'sys/time.h': No such file or directory 1>regress_dns.c 1>d:\downloads\libevent-1.4.5-stable\event.h(167) : fatal error C1083: Cannot open include file: 'sys/time.h': No such file or directory 1>regress_http.c 1>d:\downloads\libevent-1.4.5-stable\event.h(167) : fatal error C1083: Cannot open include file: 'sys/time.h': No such file or directory 1>Generating Code... 1>Build log was saved at "file://d:\Downloads\libevent-1.4.5-stable\WIN32-Prj\regress\Debug\BuildLog.htm" 1>regress - 3 error(s), 0 warning(s) ========== Build: 0 succeeded, 1 failed, 0 up-to-date, 0 skipped ========== From provos at citi.umich.edu Fri Jun 27 21:20:55 2008 From: provos at citi.umich.edu (Niels Provos) Date: Fri Jun 27 21:21:16 2008 Subject: [Libevent-users] ANN: Libevent 1.4.5-stable released In-Reply-To: <4863189B.5020200@jellycan.com> References: <850f7cbe0806251931n1ea83e99l1a32b8d7a817eeff@mail.gmail.com> <4863043A.9020501@jellycan.com> <850f7cbe0806252016o603c38x4b6162fd67cb548@mail.gmail.com> <4863189B.5020200@jellycan.com> Message-ID: <850f7cbe0806271820i2add6007pea9455a6b889169d@mail.gmail.com> On Wed, Jun 25, 2008 at 9:18 PM, Brodie Thiesfield wrote: > I think that workspace / project files are fine. The problem appears to me > that no-one is actually building libevent on Windows, or that they are using > different project files than are being released. I would suggest that if the > .dsw / .dsp (Visual Studio 6) files are updated (if necessar), and the > source files are updated as necessary then this build will work. I built and tested 1.4.5-stable on Windows before releasing it. Although, it looks that some vc project files are missing from the tar ball. They are in svn though. I don't know how to keep the .dsw/.dsp files up to date, VS2005 refuses to use the dsw/dsp files. I guess it would be better to have Makefiles instead. > At some stage I will get around to fixing this patch for 1.4.5 too. Perhaps, you might have time to look at creating Makefiles? :-) Thank you, Niels.