Often times we have to use large amounts of memory with the Event Store (up to 70 gb in our current running environments). I have gotten the question many times of how do we do this and not get killed by the garbage collector.
When the garbage collector runs it will stop all of your threads, iterate through your heaps and figure out what to kill and what not to based upon what is rooted via other references or current stacks http://msdn.microsoft.com/en-us/magazine/bb985010.aspx is a pretty good description of how it works. There have also recently been changes to the collector to make it much better http://blogs.msdn.com/b/dotnet/archive/2012/07/20/the-net-framework-4-5-includes-new-garbage-collector-enhancements-for-client-and-server-apps.aspx discusses.
There are however two ways that you can help lower the impact of the garbage collector on you. The first is to use the large object heap http://msdn.microsoft.com/en-us/magazine/cc534993.aspx. The Large Object Heap is for large objects and does not get compacted. This can dramatically lower your use of memory. Allocate big chunks and reuse them (or just allocating big chunks for things that are actually big like say an index of a file). Always try to use value types here and don’t hold references if you can (will take a lot of work off the gc, 5m longs is much better to deal with than 5m objects). There is a drawback here however in that very often you will end up with a fragmented heap if you aren’t careful.
The other mechanism to avoid the managed memory problems is to stop using managed memory and use unmanaged memory. The main thing that needs to get loaded into memory for something like the event store is the data on disk. The OS can cache this as well for you but you can force it to happen by putting it into unmanaged memory. Unmanaged memory is not used enough from managed code. Think about all the places where you need MB of memory. The ES uses unmanaged memory. Very often if you see it using 16gb of memory it has a managed heap size of 200mb + unmanaged for all the rest. This helps to keep GC rates down.
We have not seen a huge perf difference between the LOH and putting things into unmanaged memory for things that live a long time. For short lived things beware the LOH (not compacted) though I have not tested yet with new GC to see if its changed.