Ouro’s Birthday

We are now over 3 years working on Event Store! Ouro’s second birthday will be happening in London Sept 17. In general Ouro’s birthday is always a lot of fun!

This year we will have some short talks by varying contributors on the OSS side (what they have done, what they are working on etc) and I will do a talk on major functionality completed/on the way including showing off some new goodies.

There will be plenty of people from the community there and I am sure lots of good discussions. After there will be a bit of a celebration with free food/beer and of course some ouro swag. Come on out and hang out for the evening! RSVP required: http://geteventstore.com/two-years-on/#tickets

Sublime is Sublime Closing

Well its an early morning. I can blame the travel from London for that. I managed to struggle through to the end of the second period watching the canadiens game last night. I was a bit worried entering the third but was quite happy to see they won when I woke up :)

In this post I just want to sum up the other posts from the sublime series as well as add a few tidbits. In the post series we have learned how to setup sublime for .net development. We have covered how to setup project/solution support. How to get intellisense and some basic refactoring. Even how to get automated builds and tests running (all in linux).

We have also looked at a lot of other things that are built on top of sublime that are fairly useful if you are doing other types of development such as javascript or html5. Many of these tools far outclass the Visual Studio equivalents and are usable with many other environments (such as using a ruby backend).

I have personally given up on using Visual Studio as a whole. I will however keep a vm with it on it for some very specific tasks that it does well (such as line by line debugging). These are not things I use in my daily worksflow but are nice to have when you absolutely need them.

Some other changes have come about in the use of sublime as my primary editor. A big one is that when I am writing one off code (which I do alot) I do not bother creating project or solution files any more. I instead just create C# files then either run the command line directly to the compiler or create a small make file. It sounds odd but its actually much simpler than creating project/solution files overall.

There will also be much going on in this space coming up. As of now the sublime plugin supports maybe 20% of what omnisharp is capable of. There will be quite a bit of further support coming in. As an example I was looking the other day at supporting run tests in context from inside of sublime (in test->run test, on fixture->run fixture). There is also much coming in for refactoring support and my guess is that you will see even more coming in on this due to nrefactory moving to roslyn. I think within a year you will find most of this tooling built in.

Another thing that I added to sublime though there isn’t really an official plugin for it yet is sublime repl + script cs. I find it quite common to grab a function and work it out in the repl first and then move it back into the code. A perfect example of this happened to me while in London. I was trying to combine to Uris and was getting some odd behaviour. Three minutes in the repl showed exactly what the issue was.

Moving to sublime will change the way that you work a bit but is definitely worth trying. Remember that a primary benefit of working in this way is that everything that you are doing is a composition of pieces that will also apply to any other code you happen to be working on (whether its C/Ruby/Erlang/even F#).

Banking Example Again

I was reading through this yesterday on my way out of London. Go on take a minute and read it.

http://hackingdistributed.com/2014/04/06/another-one-bites-the-dust-flexcoin/

I do find it funny that the bitcoin exchanges were taken down by such things but the article is pretty ridiculous in how it presents its problem/solution. Banks don’t actually work as described in this post. There is not a column “balance” in an account table as presented unless the developers just had no clue what they were doing.

mybalance = database.read("account-number") newbalance = mybalance - amount database.write("account-number", newbalance) dispense_cash(amount) // or send bitcoins to customer - See more at: http://hackingdistributed.com/2014/04/06/another-one-bites-the-dust-flexcoin/#sthash.esawajq9.dpuf

This is absurd your balance while perhaps being denormalized on your account is really the result of an equation (summation of the value of your transactions). All of these problems discussed would just go away if the system had been designed to record a journal properly (and as the journal is append only most other issues would go away).

I have always hated that the typical example of distributed transactions is transfering between two accounts. Banks don’t work this way!

Sublime is Sublime 12

OK so we are at the last blog post in the sublime series. I have I hope saved the best one for last. One of the largest questions I have received during this series is how do I get intelisense and that R# is awesome because it supports things like goto definition and rename…

In this post we will add all of these features to sublime. There is a great project out there called OmniSharp that supports most of them (and in the future can support many many more!). Let’s get going then and add our sublime support.

https://github.com/moonrabbit/OmniSharpSublime is the project

So to install:

goto your packages directory (linux here so it may be different in windows or mac, just look in packages in sublime to find the folder)

cd ~/.config/sublime-text-3/Packages/
git clone git@github.com:moonrabbit/OmniSharpSublime.git
cs OmniSharpSublime
git submodule update --init 

Now you have gotten all the needed files. The next thing we will need to do is build OmniSharp

./build.sh

Now edit your project file for sublime and add at the root level

 "solution_file": "./EventStore.sln"

Remember the path is relative from your project file! restart sublime if its running. Try typing out variable names and you will see you have intellisense. If you hit a . you will notice that it does not come up :( by default the auto complete keystroke is alt + / you can remap this to ctrl+space if you want by editing your keymap in sublime.

Want to go to definition of a method? Try f12 by default (again can be remapped its up to you!)

In the next post we will recap everything that we have done so far!

Sublime is Sublime 11

So we are now into the 11th post. There are only two more to go after this one. One with some more functionality and one as a summary. In the last post we installed OpenIDE and showed the very basics of its functionality, adding a file to an existing project.

OpenIDE can do much more than this. It has most support for sln/prj that you will need. Let’s start by making a new project.

greg@goblin:~/src/foo$ oi create console src/HelloConsole
Created src/HelloConsole
greg@goblin:~/src/foo$ ls
src
greg@goblin:~/src/foo/src$ ls
HelloConsole.csproj  Program.cs  Properties

This will create a proejct from a template. The following are the available templates (listed from help).

	create : Uses the create template to create what ever project related specified by the template
		console : Creates a new C# console application
			ITEM_NAME : The name of the Project/Item to create
		library : Creates a new C# library project
			ITEM_NAME : The name of the Project/Item to create
		service : Creates a new C# windows service
			ITEM_NAME : The name of the Project/Item to create

You could remove Program.cs with oi deletefile foo/Program.cs if you wanted and it would also be removed from the project as well.

You can create your own templates as well they are just scripts. This applies to both new items and project templates. If for example you wanted to make a custom item for a new item (say a custom xunit testfixture).

Go to your OpenIDE release. cd .OpenIDE/languages/C#-files/

You will see here there is create and new. These hold the templates for the create and new commands they are implemented as python but can be scripted in any language

As an example here is the template for a new interface

#!/usr/bin/env python
import sys

if __name__ == "__main__":
	if sys.argv[1] == 'get_file_extension':
		print(".cs")
	elif sys.argv[1] == 'get_position':
		print("6|3")
	elif sys.argv[1] == 'get_definition':
		print("Creates an new C# interface")
	else:
		classname = sys.argv[1]
		namespace = sys.argv[2]
		parameterfile = sys.argv[3]
		print("using System;")
		print("")
		print("namespace " + namespace)
		print("{")
		print("	interface " + classname)
		print("	{")
		print("	}")
		print("}")

and here is the template for a new Console Application.

#!/usr/bin/env python
import sys
from files.copydir import copy as copydir

if __name__ == "__main__":
	if sys.argv[1] == 'get_file':
		print("Program.cs")
	elif sys.argv[1] == 'get_position':
		print("8|3")
	elif sys.argv[1] == 'get_definition':
		print("Creates a new C# console application")
	else:
		copydir("console", sys.argv[1])

There is still much that can be added to OpenIDE (and it does a ton of other things we have not covered). But in general it can get you around the issues of dealing with project and solution files including references.

Sublime is Sublime 10

Ok we have been moving right along through the sublime features and getting setup for .net development. I have been half saving the next few posts deliberately until the end as they will be covering the largest arguments I hear against the use of other editors than VS when dealing with .NET code.

But my team uses Visual Studio, I can’t just give up on using project/solution files and use some hipster editor.

This has for a long time been the single largest hurdle in using non-VS editors. If you want to get an idea of how bad it is and you have been following along the posts, try adding a file to a project or reference another project in Sublime. Ouch manually editing project files. How do you know that your manual edit will work when opened in Visual Studio?

To be fair even if it took you 15 seconds per file/reference that you added in Sublime the overall time on the project would be minimal but it is a serious pain in the ass. Nothing makes you feel slower than having to manually edit xml that was automatically done for you previously.

To get some of this functionality we will install a new tool though a whole new tool is not really needed for this. It could be done with some basic shell scripts. The tool is OpenIDE by @ackenpacken. OpenIDE does a whole lot more than what we need it to. I have been chatting with him recently about maybe making it more modular hell even Mighty Moose is contained within it as of now.

OpenIDE supports some of the generic things you would want when working with .NET code. The ability to edit project/solution files. The ability to handle templating for new additions. Reference management. There are also some other tools out there as well such as omnisharp https://github.com/nosami/Omnisharp but I fear all of them are too complex and not modular enough as there hasn’t been much of a push for that kind of tooling. Part of this post series is to help mold demand for these kinds of tools.

Now for OpenIDE install. You can grab the sources for OpenIDE here https://github.com/continuoustests/OpenIDE. Svein has recently added a binary repository here https://github.com/continuoustests/OpenIDE.binaries.git. Pull the binaries repository or build from sources. Put the output into your $PATH. OpenIDE also comes with bash completion if you want to install it which can help greatly! Now you are good to start.

Let’s make sure OpenIDE works: oi

You should get help.

oi package install C-Sharp

In the root of your project type oi init C#

Now oi is setup and ready to go. From the command line let’s try

greg@goblin:~/src/EventStore/src/EventStore$ oi new class esquery/bar
Created class esquery.bar
Full path /home/greg/src/EventStore/src/EventStore/esquery/bar.cs

Note that I did not put bar.cs just esquery/bar and yes you get tab completion on this.

If I now look at what changed.

greg@goblin:~/src/EventStore/src/EventStore$ git status
# On branch dev
# Changes not staged for commit:
#   (use "git add <file>..." to update what will be committed)
#   (use "git checkout -- <file>..." to discard changes in working directory)
#
#	modified:   esquery/esquery.csproj
#
# Untracked files:
#   (use "git add <file>..." to include in what will be committed)
#
#	esquery/bar.cs
greg@goblin:~/src/EventStore/src/EventStore$ git diff esquery/esquery.csproj
diff --git a/src/EventStore/esquery/esquery.csproj b/src/EventStore/esquery/esqu
index dec282f..9f3c95f 100644
--- a/src/EventStore/esquery/esquery.csproj
+++ b/src/EventStore/esquery/esquery.csproj
@@ -84,6 +84,7 @@
     <Compile Include="CommandProcessor.cs" />
     <Compile Include="Program.cs" />
     <Compile Include="Properties\AssemblyInfo.cs" />
+    <Compile Include="bar.cs" />
   </ItemGroup>
   <ItemGroup>
     <None Include="app.config" />

You can also run this command directly inside of sublime. Just use ctrl+shift+c and type in your command. This is just the beginning though. OpenIDE and such tools can support most of your integration with things like project/solution files.

Sublime is Sublime 9

Yesterday I took a survey of .net developers as I was curious how most are working. In particular I was wondering what % of time people actually work on .NET code vs working on code such as html or javascript and in particular whether they were using VS for html and javascript development.

I would love to put all of the raw data here but apparently surveymonkey wants 25/euros per month to actually export the results of a survey which sucks. Need to find a new survey tool.

What I found was a few groups of .NET developers one that was quite interesting for me answered the questions in this way:

Q1: What % of your time are you actively coding at work?
80
Q2: What % of your coding time is in .NET code (C#/F#/VB.NET/etc)
30
Q3: What % of your coding time is in HTML?
30
Q4: What % of your coding time is in JavaScript?
40
Q5: Do you use Visual Studio for javascript/html?

There were a huge number of developers who spent 40-70% of their coding time in Visual Studio not working on C# code but instead working on javascript and html code. For these developers in particular I point you to the html and javascript support in sublime without comment just watch a few videos, getting setup is just installing a few packages through your package manager (discussed previously in this series)

 

cookie-monster

WebInspector – http://sokolovstas.github.io/SublimeWebInspector/ seriously watch the video
Emmet – http://emmet.io/ formerly known as zencoding.
Code Intel – http://sublimecodeintel.github.io/SublimeCodeIntel/ go to definition etc support

Tomorrow we will continue the series getting into .net specific stuff and fixing some areas we are weak in now. How do you add a reference?

Anyone used any of these plugins? Please share your experiences in comments.

Sublime is Sublime 8

In this post we will look at how to become a bit more efficient in our sublime setup without adding lots of new tools or complexity. In particular I don’t like running my builds and my tests manually (and we will look at some varying options for this later). In this post however we will look at a dead simple method for doing it.

Let’s take a look at the full script I briefly wrote that does all of this.

#!/bin/sh
notify-send "Auto Started" "" -i $PWD/images/circleWIN.png -t 1000
while true
do
      if [ ! -z $(inotifywait --recursive -qre modify --format "%w%f" ./ | grep -v -f includes) ]
          then
	  sleep .25
          /opt/mono/bin/xbuild src/EventStore/EventStore.sln /p:Configuration=Release /verbosity:0 /nologo
          if [ $? -eq 0 ]
            then
               ./run_tests.sh -m /opt/mono -x LongRunning 
                if [ $? -eq 0 ]
                then
                      notify-send "Passed" "Tests Passed" -i $PWD/images/circleWIN.png -t 1000
                else
                      notify-send "Failed" "Tests Failed" -i $PWD/images/circleFAIL.png -t 1000
                fi
            else
                notify-send "Failed" "Build Failed" -i $PWD/images/circleFAIL.png -t 1000
          fi 
      fi
done

So what this does is wait for file changes in the directory structure (notify wait). It then checks to see if the file returned matches an include filter eg “*.cs” if it does it will then build the code followed by running the tests. When you get a failure/success you will receive a notification including a red/green circle in xwindows (regardless of if you are still in sublime). If you want to see run_tests.sh you can view it here https://github.com/EventStore/EventStore/blob/dev/run_tests.sh its a shell script in EventStore to ease in the running of tests on linux.

This is a great showing of the “unix philosophy” of small tools being composed together to get something larger. It was not hard to make this but it provides 80% of the value of autotest.net and is vastly more configurable if you understand shell script (want an exclude vs an include just edit the script).

There is no fancy installer. There is no visual studio integration to do (I will never do that ever again). On EventStore source base which is by all means not a small source base (roughly 2000 tests which are run locally) the entire cycle is 15-20 seconds on my machine. Although MightyMoose could have gotten this to around 5 which would be even better for simplicity this wins out.

One problem that I ran into had to deal with nunit-console when you have ignored tests there is no way to get it to not print out all the ignored tests (which it does of course last! so you can’t see your failures). To work around that I wrote a small xslt to format its results

<?xml version="1.0" encoding="UTF-8" ?>
<xsl:stylesheet version="1.0"	
                exclude-result-prefixes="msxsl"
                xmlns:xsl="http://www.w3.org/1999/XSL/Transform"
		xmlns:msxsl="urn:schemas-microsoft-com:xslt">
<xsl:output method="html" indent="yes" />
<xsl:template match="test-results">
			<xsl:variable name="failedtests.list" select="//test-case[@success='False']"/>
			<xsl:variable name="total" select="@total + @not-run" />
   			<xsl:variable name="success" select="@total - @failures - @errors" />
          	Total  : <xsl:value-of select="$total" /> Success: <xsl:value-of select="$success" /> Failed : <xsl:value-of select="@failures" /> Not Run: <xsl:value-of select="@not-run" />
			<xsl:apply-templates select="$failedtests.list"/>
</xsl:template>
<xsl:template match="test-case[@success='False']">
*************************************************************************************************	
	<xsl:value-of select="@name"/>
	<xsl:value-of select="failure/message"/>
	<xsl:value-of select="failure/stack-trace" />
</xsl:template>
</xsl:stylesheet>

This will give you a printout that looks like this:

          	Total  : 1748 Success: 1699 Failed : 1 Not Run: 48
*************************************************************************************************
	EventStore.Core.Tests.Services.UserManagementService.password_change_notification_reader.when_notification_has_been_written.publishes_reset_password_cache  Expected: 1
  But was:  0
at EventStore.Core.Tests.Services.UserManagementService.password_change_notification_reader.when_notification_has_been_written.publishes_reset_password_cache () [0x00000] in &lt;filename unknown&gt;:0

Again we have not done anything complicated here we are just composing our exists tools. I should add that you can easily make those links clickable back into sublime to go to a line based on stack trace but you either need to switch your terminal from gnometerminal or you need to add one line of code and custom build gnome terminal which may be a bit much for some people :)

As a quick exercise could you write a script like above for javascript?

Sublime is Sublime 7

So as we have come along the path our sublime instance is starting to feel quite a bit more customized than it was before. We can now build, have git/github integration, and of course have awesome command line integration but what about running tests?

We will start with the simplest way of running tests. This may not work for all of your scenarios and does somewhat require you to have thought about things in advance.

If you for instance started off from the beginning with resharper as a test runner you are probably have a test suite that will be difficult to use. Likely it is all in one assembly regardless of whether its a unit test or an integration test. Quite likely the tests have large numbers of dependencies. And to be frank your unit tests probably don’t run properly in linux anyways. Yeah this kind of sucks but can be fixed.

To start with there is no integrated right click->debug test in sublime. I find however that this is not something I missed. Maybe its having worked for a few years building embedded systems where there was no debugger (except dumping as hex pieces of memory).

Let’s talk about the simplest way to run tests from sublime and the general workflow. Normally when working with sublime I have multiple test suites either separated by category or by assembly. I have unit tests which are quick to run and then all other tests. I rarely if ever run the other tests locally. Instead I let a CI server run those. My unit tests should be run in and this would give you a new buffer with the output of your tests running. For a huge portion of code you will be working on this is *good enough*. If you have a 10 second cycle on tests running it is completely reasonable. eg

ctrl+shift+c run >

where I have a script called run that runs nunit in the way I want.

There are however some other things you can do if you want to be more fancy. The first is you could assign the running of the script to a key in sublime. This is actually quite easy any command can be assigned to any key by editing your keymap http://sublimetext.info/docs/en/reference/key_bindings.html

Another cool option that is quite easy to enable to get a notification from linux on the status of your build/test run. This can be done trivial using notify-send in your small script above based on return values from processes.

Try:
notify-send ‘Hello world!’ ‘This is an example notification.’ –icon=dialog-information

You can even change the icons to say red/green circles for the notification.

Now if your tests are more than 10 seconds. The first strategy would be to start putting them into categories that make sense to you. This way you can only run tests within a given category. This step alone will normally be good enough to get your test runs back under 10 seconds (I prefer 1-2 seconds). If this is not then perhaps your code base has become too dependent on the tool (15 categories 10 seconds each…). One thing you will notice is that your tests in general will run far faster than through tools like VS/R# when being run from the command line.

OK so now we have basic unit test integration. There are still a lot of problems with .NET code. In the next post we will look at what some of these are and how to deal with them.

Sublime is Sublime 6

So far we have done quite a bit getting sublime up and going for use with .NET work but we still at this point have to compile our code from the command prompt which well sucks. In this post we will look at how to get compiles working from within sublime by adding a custom build system for xbuild.

Go to Tools->Build System->new build. A file will open. Drop in:

{
“cmd”: ["xbuild"],
“path”: “/opt/mono/bin/”,
“working_dir”: “${project_path:${folder}}”,
“file_regex”: “^ (.*)\\(([0-9]*),([0-9]*)”
}

Then save this file named xbuild. You can now choose xbuild as a build platform.

Is is important to note that you have to point at the correct xbuild (I run off trunk so I compile with a /opt prefix if you get from say apt-get it will be some place else) and you must have the correct mcs in your path when doing this. This code also assumes that you have only a single solution in the root folder of your code.

Once this is done you can choose xbuild as your current build provider. Then you can use either f7 or ctrl+b to build your code on demand.

Now we are really starting to get there. In the next post we will look at how to run unit tests from within sublime manually.

Follow

Get every new post delivered to your Inbox.

Join 8,241 other followers