Technical requirements for CQS/Event Sourcing/Eager Read Derivation

Hi all,

I'm just wondering what thoughts people have on the minimum tech specs  
to achieve some of the features of the Event Sourcing model,  
especially the storing of explicit state transitions (events).

We're currently deploying a version of our app on Azure, which is  
still in early days and doesn't have, as yet, as many features as we'd  
like from a cloud platform, but that's where we're at. It does have  
BigTable-like storage (Azure Table Storage) and a basic message queue  
(not guaranteed to be FIFO) and blob storage, but there's no  
distributed caching for example.

So here are some issues we're thinking about:

* How can we store our event stream in a manner that's performant?

   - When we process commands, we need to build up our aggregates, so  
we'll need to read all events up until the last snapshot - how can we  
construct optimised queries for this given the limitations of the  
storage and query language? And is there a way to avoid doing this for  
every new command we need to process?

   - We could try caching aggregates in memory on the worker nodes,  
but given that nodes are homogeneous, there's no (straightforward) way  
to ensure the workers will retrieve command messages off the queue for  
the aggregates they're caching. As there's no distributed in-memory  
caching capabilities, the only other option would be to try to use the  
blob storage as a cache, which would still be writing/reading from disk.

* How can we "ensure that our audit is correct"?

   - This is one of the touted features of state transition storing  
and sourcing, but if the events are just stored in an open table  
format, then what's to stop them being modified (as long as they're  
modified within your validation constraints)? Do we need to store a  
checksum along with a server-side secret, or something like that, for  
each event?

* How can we construct our read layer to apply event changes?

   - Again, given homogeneous worker roles, how do we coordinate the  
reading of the event storage so that events are only applied once - do  
we need to keep a whole lot of versioning information in the read  
layer as well? And generally how does each worker node know where to  
start reading from in the event storage?

Now, I'm not actually expecting answers to these questions themselves,  
but more broadly just interested in what people's thoughts are on the  
necessary features that you need in a system to support this design.

Can CQS/Event Sourcing only be achieved in certain environments, and  
if so, what sort of features do you need?

Cheers,

Michael

------------------------------------