There are a few options. It normally is a good idea to use the Cache
whenever possible to hold onto data. In this case you want to ensure that
the cache does not remove it for a long time.

What you can do is set up a cache policy which you use when you add the item
to the cache. You will use this value for the CacheItemPriority.

System.Web.Caching.CacheItemPriority.NotRemovable

You can attache the CacheItemRemovedCallback.

Here is a VB.NET snippet.

Dim onRemove As CacheItemRemovedCallback = New
CacheItemRemovedCallback(AddressOf RemovedCallback)

Cache.Add(key, obj, Nothing, DateTime.Now.AddSeconds(seconds), _
TimeSpan.Zero, CacheItemPriority.NotRemovable, onRemove)

What you could do is have the item expire once every 60 minutes and add it
back into the cache if it is between midnight and 1am. At that point you
get a fresh copy and add it to the cache.

However, if this DataTable does take a long time to load, you may still
experience problems. At any time during the day the Application Pool could
reset itself and cause the web application to restart and reload this
DataTable. It may be best to tune the loading of this data. I will assume
this data is being loaded from a query joining across many tables with
multiple subselects and inner and outer joins. With ASP.NET 2.0 you should
be able to quickly load a lot of data as long as the query is simple. So
what you can do is schedule a DTS package to run that complex query to
populate a simple table with the results, called WarehouseTable1. Then you
just run this query to populate your DataTable.

select * from WarehouseTable1

That should load very quickly. You may be surprised how fast it can be.

If this is not an option because the query is still slow due to a slow
network connection, another method would be to run a scheduled job to run a
..NET utility to load this query into a DataTable and serialize it out the an
XML file. Then place that XML file onto the web server and have it use that
file as the datasource. You can then update the XML file and use the Cache
dependency to reload the XML file whenever the file changes.

http://www.eggheadcafe.com/articles/20030716.asp

You have a lot of great ways to tune this performance.

Brennan Stehling
http://brennan.offwhite.net/blog/


[quoted text, click to view]