[Libevent-users] Add support for WINDOWS Overlapped IO

[Toby Douglass]

Kevin Sanders wrote:
> On 12/10/06, Toby Douglass <toby.douglass at summerblue.net> wrote:
>> I've not added support for writes, because I
>> think people generally issue blocking writes, since non-blocking means
>> that if you return from a function which issues a send you have to
>> ensure the lifetime of the buffer you've sent is non-local.
> 
> If you're doing even a small amount of writing, you're going to have
> dismal performance (at best) using blocking writes.  If you're talking
> to a peer socket (real world) that is no longer responding, this write
> may take more than a minute to error out.

True.  I non-block and select and give it a couple of seconds, but of 
course even a couple of seconds is still hugely long for some uses.

> One of my coworkers recently observed, that handles associated with a
> IOCP seem to have CPU affinity, at least sometimes.  In a read
> completion callback, he posted another read (which is fine and
> encouraged) and then went off and did a lot of processing which
> preventing it from calling GQCS for about 20 seconds (very bad).  Even
> though there were 3 other threads waiting on GQCS, they couldn't pop
> the completion status for the read from the IOCP even though the read
> had completed.  Finally, as soon as the original thread came back
> around and called GQCS, it popped the completion instead of the other
> threads which had been waiting the whole time.

Oh man...!

> The lesson here is you don't want the IOCP threads doing anything
> except issuing async IO, popping completions and a quick state machine
> change (see below) and issue another async IO if needed.  If more
> processing is needed, put a work item in a queue for another thread to
> process.  That work thread can call your state machine callback when
> it is finished, and that may in turn cause further async IO.

True.  But you realise in the code I've written the IOCP threads are 
internal to the library itself - the user doesn't see them or touch 
them.  All they do is call GQCS.  If the user did blocking writes, they 
would be outside the IOCP mechanism and they wouldn't be using the IOCP 
threads.

>> Single-thread event-driven seems to me to basically mean state machine.
>> State machines are wonderful things for achieving simple, bug-free
>> code, but they have a cost; they are implicitly single-threaded.  This
>> can mean you cannot use them in some situations, because you will
>> inherently block whenever you perform work.
> 
> I'm not sure I follow this.  Are you saying you can't use state
> machines in a multi-threaded application because they cause threads to
> block?

No - I'm saying state machines inherently serialise, and so if you need 
parallelism, you have a problem.