[Libevent-users] Add support for WINDOWS Overlapped IO

[Kevin Sanders]

On 12/10/06, Toby Douglass <toby.douglass at summerblue.net> wrote:
> I've not added support for writes, because I
> think people generally issue blocking writes, since non-blocking means
> that if you return from a function which issues a send you have to
> ensure the lifetime of the buffer you've sent is non-local.

If you're doing even a small amount of writing, you're going to have
dismal performance (at best) using blocking writes.  If you're talking
to a peer socket (real world) that is no longer responding, this write
may take more than a minute to error out.

Make the writes async, use a callback when the write completes,
similar to the read completion.  Basically your buffer management
should be careful to never reuse or free a buffer that has a pending
IO.  You're already doing this for the reads, AcceptEx and ConnectEx.

A very tempting and huge mistake would be to call a blocking write
from your read completion callback (using its IOCP thread).  That
would then block that IOCP thread, which can't process completions
until the write completes or errors out.  This is very bad.  However,
issuing an async write from the read callback is a good idea if you
have the data available.

One of my coworkers recently observed, that handles associated with a
IOCP seem to have CPU affinity, at least sometimes.  In a read
completion callback, he posted another read (which is fine and
encouraged) and then went off and did a lot of processing which
preventing it from calling GQCS for about 20 seconds (very bad).  Even
though there were 3 other threads waiting on GQCS, they couldn't pop
the completion status for the read from the IOCP even though the read
had completed.  Finally, as soon as the original thread came back
around and called GQCS, it popped the completion instead of the other
threads which had been waiting the whole time.

This makes sense, because a running thread that is reading & writing
would suffer a CPU cache flush if it changed CPUs.  This was on a true
dual CPU box, not a dual core, or hyperthreaded.  I've never read
anything that confirms this, but we did see it in this case.

The lesson here is you don't want the IOCP threads doing anything
except issuing async IO, popping completions and a quick state machine
change (see below) and issue another async IO if needed.  If more
processing is needed, put a work item in a queue for another thread to
process.  That work thread can call your state machine callback when
it is finished, and that may in turn cause further async IO.

> Single-thread event-driven seems to me to basically mean state machine.
>   State machines are wonderful things for achieving simple, bug-free
> code, but they have a cost; they are implicitly single-threaded.  This
> can mean you cannot use them in some situations, because you will
> inherently block whenever you perform work.

I'm not sure I follow this.  Are you saying you can't use state
machines in a multi-threaded application because they cause threads to
block?

State machines hold the status of some hypothetical object.  If that
state changes, and the new state dictates to send or receive data, the
IO should be issued asynchronously, and state changed to reflect it is
pending.  In other words, if you have some "state" that requires
calculations or IO, that can't be determined or completed quickly then
the task of doing the calculation or IO should, in itself, be a
different specific case of the state machine.

> Also note that with GetQueuedCompletionStatus(), you're supposed, for
> optimal performance, to have multiple threads (2xCPU) blocking on the
> IOCP object.
>
> Given the advent and growing popularity of multi-core CPUs, single
> threaded blocking seems inappropriate.

Bingo, use async writes!

> Note though that with GetQueuedCompletionStatus(), the user has no
> synchronization work to do.  The API handles it all behind the scenes.

Kevin