dotnet clr: Hi, I have a question. I have a DLL created in C that does some file
conversion. I have gone through the code to ensure that it is thread-safe,
i.e. removing globals from the library, etc. I have a wrapper class in C#.

I have tested the DLL/wrapper by creating a couple threads on different
files in a test application, and it works. I can observe the simultaneous
conversions, and the files are correct when done.

For the real application, I create a loop over a number of files (N), and
launch N threads that call the above DLL (each thread runs on a method in an
object created in an object array).

Initially, I was creating the threads as fast as the loop would go, but then
the application started crashing on the "Init" function of the above DLL. I
set some breakpoints in the debugger on Init, and stepped through the code,
but nothing failed when I did this. Not sure, I put a Thread.Sleep(5)
statement at the end of my loop over the files. Now it crashed, but a little
less frequently, so I changed the statement to Thread.Sleep(100), and I
haven't had it crash since, and have converted four files simultaneously
without errors.

So the code that works is, effectively:

for each file in list:
setup the thread
start the thread
sleep 100 ms

Why would this be? Am I possibly overwhelming the threading system when I
try to launch threads in a loop against a single DLL, even if it is
thread-safe? That seems a little dubious.

Thanks,

It is hard to say without some code... but for what it is worth,
starting N threads seems overkill anyways... unless you are expecting
high latency (e.g. remote comms), why not have 1 thread per CPU (or
some factor there-of)? With disk-IO-bound operations, even with this
you are likely to swamp the HDD. Each thread would perhaps be picking
items of a (synchroised) queue until it is empty.

It isn't clear how you are initialising your threads; I'd guess that
you missed something when synchronising; can you post any code of how
the threads are started, and how they get their "what to do"?

Marc

[quoted text, click to view]

The setting for how many files to convert simultaneously is
adjustable, as is the amount of time the threads sleep. I want the
application to work so that you can convert files in batch and push as
many files as can be handled, as determined by the user. On my
machine, which has two processor cores, if I convert four files
simultaneously, making each thread sleep 1 ms, the system is
responsive, and the files convert nearly as fast as if they were
converted sequentially. If I want to walk away from my computer and
let it work, I set it to sleep no time at all, and the system becomes
unresponsive until the task is done; however, by my estimates, the
time taken for the four files to convert is noticeably faster.


[quoted text, click to view]

Here's a snippet. The method that gets put on the thread is from an
object of the class ConvertFile. I have an array of ConvertFile
objects. I iterate over the number of threads available for use, and
start them with the object in the array at the current index. Each
object is passed a string to a file to convert, and that's all the
thread needs. The task is "embarrassingly parallel" in this regard.
The thread accesses the DLL and works on converting the file.

for ( int i=0; ... /* loop over number of threads user specifies */ )
{
cvf[i] = new ConvertFile();
cvf[i].InFile = (string)listBox1.Items[i]; // Pass in input
file path
tsCvf[i] = new ThreadStart( cvf[i].FromFormatX );
tCvf[i] = new Thread( tsCvf[i] );
Thread.Sleep(100); // This keeps it from crashing... ?
}

And what does it do at completion? I'm wondering if there is some
collection / list manipulation perhaps?

But either way, the code as-is does some major forking... I'd be far
more inclined to have a fixed set of threads pumping a shared queue.
IIRC, Jon Skeet has a suitable custom pool in his MiscUtil library
(http://www.yoda.arachsys.com/csharp/miscutil/); alternatively, I
believe some of the new async MS stuff has a lot of custom pools...
but I can't remember where I was reading about the last...

Marc

[quoted text, click to view]

After the threads are started, another thread is kicked off that
monitors the state of the running threads, but the code crashes before
reaching that point. The method FromFormatX() does nothing outside of
manipulating the passed file.

[quoted text, click to view]

I do have a fixed limit set for the amount of threads that can be
kicked off, currently this is about eight. But as it is, I've only run
four threads concurrently, and am getting the crash. I can't imagine
that I've overwhelmed anything with only four threads.

[quoted text, click to view]

Ditto; it /sounds/ like it should just work. I don't know what is
failing... ;-(

If the crash does not happen when the access to Init() is serial, it
indicates that something in the Init() and it's codepath is not very
thread-safe. Can you put the Init() under some mutex/lock so that only one
thread at a time executes it? It should not be very bad for the performance
since the real work, as I understand it, is done on other functions.
If that works even with 100 threads...

"JA" <JA@discussions.microsoft.com> ha scritto nel messaggio
news:7FBDC1BA-6A35-4805-8839-E6856CB37768@microsoft.com...
[quoted text, click to view]