dotnet clr: Hello,

I am currently using the Enterprise Library Caching Block on a server
that receives incoming calls and stores/retrieves random information.
I noticed while load testing, the system is slow to respond (a call
can take several seconds) during scavenging of old items when the
threshold is met. It is reclaiming approx 4gb of data during each
scavenge on approximately 14gb of cache. I realize this is a lot of
information but am looking for strategies to implement a performant
cache in .Net. I'm not particularly married to entlib so anything is
an option.

One thought I had was to perhaps rewrite the cache (or a new one from
scratch) in C++/CLI to handle all the memory storage/reclamation in
native code but am concerned with marshalling data back and forth
between native/managed code. That said, the GC multisecond swings are
worse then knowing that a call to the cache will take, for example,
5ms, instead of 1ms.

Barry, thanks for the response, see inline.

[quoted text, click to view]

As mentioned, this was stress testing so I wanted to push the cache to
the size limit and see how it would respond while scavenging.

[quoted text, click to view]

Can you explain this further? Do you mean you alloc a huge block of
memory to insure it sits in the LOH and suballocate from there (ala
your own memory management on the managed heap?) If so, how does the
suballocations look? Does C# have a placement new like C++?


[quoted text, click to view]

How does one serialize/deserialize managed objects onto the native
heap if these managed objects can be derived from a common base (e.g.
CacheItem<-MyCacheItem, CacheItem<-MyOtherCacheItem)? I'm (also)
wondering if memory fragmentation will start occurring in the native
heap?

[quoted text, click to view]

It sounds like you need a more aggressive cache expiration policy. As
Rico Mariani [Google] likes to say, a poorly implemented expiration
policy is just a memory leak by another name. What you say makes me
think that the cache is getting a too-low hit rate, so it needs throw
away most of it too often. Perhaps it shouldn't be caching much of this
stuff at all in the first place?

When dealing with large amounts of data, I find GC is often not the best
way to go. I end up writing manual allocators for byte arrays. This
makes most sense when the data you're caching is in fact a byte array,
or other value-typed array, of course. However, I still allocate those
byte arrays from the .NET GC. It's just that they're all over the large
block limit (80K or so), and I carefully recycle them through a pool.

[quoted text, click to view]

BTW, you can still do manual memory allocation in C#, using the Marshal
class or by writing the PInvoke signatures yourself. However, I'd take a
closer look at your expiration policy before I'd go off and try and
reimplement a cache.

-- Barry

--

[quoted text, click to view]

Sure. All I was saying is that if stress testing shows problems, then
perhaps it indicates a lax expiration policy.

[quoted text, click to view]

No, though I have done something like that too, and used a kind of
ArraySlice<T> handle to work with buffers. I worked strictly with byte
arrays, though the code was generic and would have made sense with any
value type. The primary use for the byte arrays in my case was as
buffers for I/O. However, if you implement your cache store such that it
can be processed in a streaming fashion, then you can rely on the LOH
and manual management for memory and GC generation 0 for efficient
cleanup of temporary structures.

[quoted text, click to view]

No. The CLR doesn't support directly mixing managed and unmanaged
memory. Managed->unmanaged needs to be an opaque pointer, usually
IntPtr, while unmanaged->managed needs to be a GC root, e.g. GCHandle,
or gcroot<> in C++.

[quoted text, click to view]

Some ideas: using Stream wrapped around one of the aforementioned LOH
buffers, use ordinary .NET serialization to iterate through structures,
a bit like IDataReader or XmlReader etc. It seems to me that the better
performance you want to get, the more specific your solution becomes to
your problem set.

[quoted text, click to view]

If you allocate memory manually or semi-manually (e.g. GC LOH recycled
buffers), you can implement it such that you don't need to worry about
fragmentation, given some constraints. For example, emulate generations
by having a list of stack-oriented allocators which trivially deallocate
by changing the top location. Alternatively, use circular buffers, etc.

-- Barry

--