<Skufca@discussions.microsoft.com>> wrote:

> [...]
> My initial thought was to spawn a thread to handle the parsing/inserting
> of
> the incoming messages, but with messages streaming in continusously I
> don't
> think this would be the best way.

> My other idea which i don't have much experience with is to create a
> queue
> and stick each incoming message in the queue and have another thread
> handle
> retrieving the messages from the queue and doing the SQL insert/update.
>
> What would be the pros and cons of using the queue?

"Peter Duniho" wrote:

> On Wed, 26 Dec 2007 18:38:00 -0800, Ron Skufca <Ron
> <Skufca@discussions.microsoft.com>> wrote:
>
> > [...]
> > My initial thought was to spawn a thread to handle the parsing/inserting
> > of
> > the incoming messages, but with messages streaming in continusously I
> > don't
> > think this would be the best way.
>
> Do you mean start a new thread for each message received? If so, yes...I
> agree this wouldn't be the best way, or even a very good way.
>
> > My other idea which i don't have much experience with is to create a
> > queue
> > and stick each incoming message in the queue and have another thread
> > handle
> > retrieving the messages from the queue and doing the SQL insert/update.
> >
> > What would be the pros and cons of using the queue?
>
> I can't think of any real con per se. It's potentially more complicated,
> but not much and it's a fairly common and natural solution to the problem
> you're describing. See "producer/consumer". Producer would be the code
> recieving datagrams, consumer would be the code handling the thread doing
> the SQL operations.
>
> Assuming that a single SQL operation can be faster than a single UDP
> datagram can be received, this should work fine. If not, then you have a
> problem, as the consumer thread won't be able to keep up with the
> producer. You'll either need to know you have moments when you can catch
> up, or you need to be able to discard datagrams, or you need to speed the
> SQL operations up.
>
> If the latter is necessary, it seems likely to me that you could improve
> performance by consolidating your SQL operations. Presumably most of the
> overhead comes from moving data over the SQL server connection and any
> latency in the server's response. If you have a way to do a single SQL
> operation that handles data from multiple datagrams, that could improve
> performance enough that the consumer thread can easily outpace the
> producer.
>
> Pete
>