AppFabric of Our Lives

A couple weeks ago, I was assigned to a project creating the cross-cutting concerns in our new software architecture. Specifically, I was tasked with the caching aspect. As I searched for distributed caching solutions, I came across "Velocity", a Microsoft solution. As it turns out, Velocity is now a production release, part of the Windows Server AppFabric. I've spent the last few days setting up a virtual machine with AppFabric as my "cache cluster" and writing some tests against it. Here's some thoughts:

• The server-side installation is very streamlined. Much of the configuration is done after the fact through Powershell cmdlets, which I actually really like.
• When using Powershell to configure AppFabric, it must be run with elevated privileges, aka Run as Administrator. My account is a domain admin, but I kept receiving InvalidOperationException when I tried to use the Start-CacheCluster command.
• I looked through the MSDN documentation on how to set the size of the cache, and came across this: http://msdn.microsoft.com/en-us/library/ee790935.aspx. The Powershell commands listed for Cache size, Low watermark and High watermark under the Host Settings header are actually incorrect in the article. The correct commands are Get-CacheHostConfig and Set-CacheHostConfig.
• I played with adjusting the cache size and was actually able to adjust the size to more than the physical memory available on the machine. My guess is that it will just consume virtual memory after the physical memory is exhausted. The architects on my project were concerned with cost of physical hardware, so this may be an interesting compromise. I'll have to evaluate with some load testing.
  This is actually an incorrect assumption. Per Microsoft, you are allowed to configure the size past the physical memory in anticipation of growth, but AppFabric will throttle it back to the physical memory available.
• I noticed quite a bit of overhead on an object's size in the cache. I was pulling back a 350kb object (according to Fiddler) and seeing the cache size increase 560kb when it was added. I plan to follow up on this to see if I can determine what makes up the difference.
  Per Microsoft, there should be about 500 bytes of overhead per object stored in the cache. I believe my results were due to the object being returned from our WCF service being significantly larger when hydrated as opposed to the serialized data coming across the wire.
• Implementing caching in code is straightforward and intuitive with the AppFabric API.
• Performance is pretty great. I was able to pull 150 of those 350kb objects from the cache in the time it took to pull the 151st object from the service (although, maybe that says more about our ESB).

First impression: Windows Server AppFabric is pretty killer. If you're looking for a distributed caching solution, it's definitely worth a look. Check it out!