Projections 6: A use case of indexing

As we went through in Projections 5 : Indexing. The linkTo() function is capable of emitting pointers to another stream. This can allow you to break apart streams in order to change their partitioning and to allow fast indexing. In this post we will look at a use case of how you can use this functionality.

I have seen no less than 10 custom auditing systems for nservicebus. Everyone seems to want to build out their own custom auditing system. Most involve writing out the messages to a database table then querying off the database table to show results for say a given correlationid or a certain user. Projections and the Event Store can handle this workload quite easily with two small indexing projections.

Let’s assume we have hooked the audit queue in nservicebus and we are writing into the Event Store all the messages. When writing we write metadata containing the correlationId and the username.

correlationId : “guid”,
username : “greg”

We would then write our two indexing projections (correlationid and username).

fromAll().when({$any : function(s,e) { linkTo(e.metadata.correlationId, e); }}})

fromAll().when({$any : function(s,e) { linkTo(e.metadata.username, e); }}})

This will create a stream for every correlationId and a stream for every username. Once those are run you would then just go to http://node:port/streams/greg and you would see all the messages I am doing in the system (the ui even updates on its own as I am watching it). This is a very common and simple usage. It should also be noted that while here I am basically promoting a piece of metadata, the code is in javascript and you could do pretty much anything there (maybe the modulus of the combined string of four properties of the to 42?)


  1. Posted February 27, 2013 at 5:03 pm | Permalink | Reply

    Is there a good way to create an collection-like index? For example, in the InventoryItem domain, what would be a way to create an index that allowed look up of say 10 most recently changed inventory items? One way is to create an projection off of a stream of flattened read models and store the set of recent items in projection state, however I’m not sure this is the best way? Or is this scenario best delegated to a document store?

    • Posted March 1, 2013 at 4:05 pm | Permalink | Reply

      For the 10 most recently updated items its very simple in fact. Just do a projection like the one done here and set the maxLength of the stream to 10. As they update they will write the entry and the old ones will fall off.

      • Posted March 1, 2013 at 6:26 pm | Permalink

        But in that case, suppose that a single item was changed 10 times in a row. Won’t the stream, capped at 10, only contain events for that single item?

      • Posted March 1, 2013 at 6:33 pm | Permalink

        you can also hold state of last ten to make sure you dont double up.

  2. Posted March 1, 2013 at 7:36 pm | Permalink | Reply

    Do you have any guidelines for state storage capacity?

    • Posted March 1, 2013 at 7:41 pm | Permalink | Reply

      less is better. 10 is no problem. 100 a bit more. 10000 and its a bit much.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: