Banking Example Again

I was reading through this yesterday on my way out of London. Go on take a minute and read it.

http://hackingdistributed.com/2014/04/06/another-one-bites-the-dust-flexcoin/

I do find it funny that the bitcoin exchanges were taken down by such things but the article is pretty ridiculous in how it presents its problem/solution. Banks don’t actually work as described in this post. There is not a column “balance” in an account table as presented unless the developers just had no clue what they were doing.

mybalance = database.read("account-number") newbalance = mybalance - amount database.write("account-number", newbalance) dispense_cash(amount) // or send bitcoins to customer - See more at: http://hackingdistributed.com/2014/04/06/another-one-bites-the-dust-flexcoin/#sthash.esawajq9.dpuf

This is absurd your balance while perhaps being denormalized on your account is really the result of an equation (summation of the value of your transactions). All of these problems discussed would just go away if the system had been designed to record a journal properly (and as the journal is append only most other issues would go away).

I have always hated that the typical example of distributed transactions is transfering between two accounts. Banks don’t work this way!

Sublime is Sublime 12

OK so we are at the last blog post in the sublime series. I have I hope saved the best one for last. One of the largest questions I have received during this series is how do I get intelisense and that R# is awesome because it supports things like goto definition and rename…

In this post we will add all of these features to sublime. There is a great project out there called OmniSharp that supports most of them (and in the future can support many many more!). Let’s get going then and add our sublime support.

https://github.com/moonrabbit/OmniSharpSublime is the project

So to install:

goto your packages directory (linux here so it may be different in windows or mac, just look in packages in sublime to find the folder)

cd ~/.config/sublime-text-3/Packages/
git clone git@github.com:moonrabbit/OmniSharpSublime.git
cs OmniSharpSublime
git submodule update --init 

Now you have gotten all the needed files. The next thing we will need to do is build OmniSharp

./build.sh

Now edit your project file for sublime and add at the root level

 "solution_file": "./EventStore.sln"

Remember the path is relative from your project file! restart sublime if its running. Try typing out variable names and you will see you have intellisense. If you hit a . you will notice that it does not come up :( by default the auto complete keystroke is alt + / you can remap this to ctrl+space if you want by editing your keymap in sublime.

Want to go to definition of a method? Try f12 by default (again can be remapped its up to you!)

In the next post we will recap everything that we have done so far!

Sublime is Sublime 11

So we are now into the 11th post. There are only two more to go after this one. One with some more functionality and one as a summary. In the last post we installed OpenIDE and showed the very basics of its functionality, adding a file to an existing project.

OpenIDE can do much more than this. It has most support for sln/prj that you will need. Let’s start by making a new project.

greg@goblin:~/src/foo$ oi create console src/HelloConsole
Created src/HelloConsole
greg@goblin:~/src/foo$ ls
src
greg@goblin:~/src/foo/src$ ls
HelloConsole.csproj  Program.cs  Properties

This will create a proejct from a template. The following are the available templates (listed from help).

	create : Uses the create template to create what ever project related specified by the template
		console : Creates a new C# console application
			ITEM_NAME : The name of the Project/Item to create
		library : Creates a new C# library project
			ITEM_NAME : The name of the Project/Item to create
		service : Creates a new C# windows service
			ITEM_NAME : The name of the Project/Item to create

You could remove Program.cs with oi deletefile foo/Program.cs if you wanted and it would also be removed from the project as well.

You can create your own templates as well they are just scripts. This applies to both new items and project templates. If for example you wanted to make a custom item for a new item (say a custom xunit testfixture).

Go to your OpenIDE release. cd .OpenIDE/languages/C#-files/

You will see here there is create and new. These hold the templates for the create and new commands they are implemented as python but can be scripted in any language

As an example here is the template for a new interface

#!/usr/bin/env python
import sys

if __name__ == "__main__":
	if sys.argv[1] == 'get_file_extension':
		print(".cs")
	elif sys.argv[1] == 'get_position':
		print("6|3")
	elif sys.argv[1] == 'get_definition':
		print("Creates an new C# interface")
	else:
		classname = sys.argv[1]
		namespace = sys.argv[2]
		parameterfile = sys.argv[3]
		print("using System;")
		print("")
		print("namespace " + namespace)
		print("{")
		print("	interface " + classname)
		print("	{")
		print("	}")
		print("}")

and here is the template for a new Console Application.

#!/usr/bin/env python
import sys
from files.copydir import copy as copydir

if __name__ == "__main__":
	if sys.argv[1] == 'get_file':
		print("Program.cs")
	elif sys.argv[1] == 'get_position':
		print("8|3")
	elif sys.argv[1] == 'get_definition':
		print("Creates a new C# console application")
	else:
		copydir("console", sys.argv[1])

There is still much that can be added to OpenIDE (and it does a ton of other things we have not covered). But in general it can get you around the issues of dealing with project and solution files including references.

Sublime is Sublime 10

Ok we have been moving right along through the sublime features and getting setup for .net development. I have been half saving the next few posts deliberately until the end as they will be covering the largest arguments I hear against the use of other editors than VS when dealing with .NET code.

But my team uses Visual Studio, I can’t just give up on using project/solution files and use some hipster editor.

This has for a long time been the single largest hurdle in using non-VS editors. If you want to get an idea of how bad it is and you have been following along the posts, try adding a file to a project or reference another project in Sublime. Ouch manually editing project files. How do you know that your manual edit will work when opened in Visual Studio?

To be fair even if it took you 15 seconds per file/reference that you added in Sublime the overall time on the project would be minimal but it is a serious pain in the ass. Nothing makes you feel slower than having to manually edit xml that was automatically done for you previously.

To get some of this functionality we will install a new tool though a whole new tool is not really needed for this. It could be done with some basic shell scripts. The tool is OpenIDE by @ackenpacken. OpenIDE does a whole lot more than what we need it to. I have been chatting with him recently about maybe making it more modular hell even Mighty Moose is contained within it as of now.

OpenIDE supports some of the generic things you would want when working with .NET code. The ability to edit project/solution files. The ability to handle templating for new additions. Reference management. There are also some other tools out there as well such as omnisharp https://github.com/nosami/Omnisharp but I fear all of them are too complex and not modular enough as there hasn’t been much of a push for that kind of tooling. Part of this post series is to help mold demand for these kinds of tools.

Now for OpenIDE install. You can grab the sources for OpenIDE here https://github.com/continuoustests/OpenIDE. Svein has recently added a binary repository here https://github.com/continuoustests/OpenIDE.binaries.git. Pull the binaries repository or build from sources. Put the output into your $PATH. OpenIDE also comes with bash completion if you want to install it which can help greatly! Now you are good to start.

Let’s make sure OpenIDE works: oi

You should get help.

oi package install C-Sharp

In the root of your project type oi init C#

Now oi is setup and ready to go. From the command line let’s try

greg@goblin:~/src/EventStore/src/EventStore$ oi new class esquery/bar
Created class esquery.bar
Full path /home/greg/src/EventStore/src/EventStore/esquery/bar.cs

Note that I did not put bar.cs just esquery/bar and yes you get tab completion on this.

If I now look at what changed.

greg@goblin:~/src/EventStore/src/EventStore$ git status
# On branch dev
# Changes not staged for commit:
#   (use "git add <file>..." to update what will be committed)
#   (use "git checkout -- <file>..." to discard changes in working directory)
#
#	modified:   esquery/esquery.csproj
#
# Untracked files:
#   (use "git add <file>..." to include in what will be committed)
#
#	esquery/bar.cs
greg@goblin:~/src/EventStore/src/EventStore$ git diff esquery/esquery.csproj
diff --git a/src/EventStore/esquery/esquery.csproj b/src/EventStore/esquery/esqu
index dec282f..9f3c95f 100644
--- a/src/EventStore/esquery/esquery.csproj
+++ b/src/EventStore/esquery/esquery.csproj
@@ -84,6 +84,7 @@
     <Compile Include="CommandProcessor.cs" />
     <Compile Include="Program.cs" />
     <Compile Include="Properties\AssemblyInfo.cs" />
+    <Compile Include="bar.cs" />
   </ItemGroup>
   <ItemGroup>
     <None Include="app.config" />

You can also run this command directly inside of sublime. Just use ctrl+shift+c and type in your command. This is just the beginning though. OpenIDE and such tools can support most of your integration with things like project/solution files.

Sublime is Sublime 9

Yesterday I took a survey of .net developers as I was curious how most are working. In particular I was wondering what % of time people actually work on .NET code vs working on code such as html or javascript and in particular whether they were using VS for html and javascript development.

I would love to put all of the raw data here but apparently surveymonkey wants 25/euros per month to actually export the results of a survey which sucks. Need to find a new survey tool.

What I found was a few groups of .NET developers one that was quite interesting for me answered the questions in this way:

Q1: What % of your time are you actively coding at work?
80
Q2: What % of your coding time is in .NET code (C#/F#/VB.NET/etc)
30
Q3: What % of your coding time is in HTML?
30
Q4: What % of your coding time is in JavaScript?
40
Q5: Do you use Visual Studio for javascript/html?

There were a huge number of developers who spent 40-70% of their coding time in Visual Studio not working on C# code but instead working on javascript and html code. For these developers in particular I point you to the html and javascript support in sublime without comment just watch a few videos, getting setup is just installing a few packages through your package manager (discussed previously in this series)

 

cookie-monster

WebInspector – http://sokolovstas.github.io/SublimeWebInspector/ seriously watch the video
Emmet – http://emmet.io/ formerly known as zencoding.
Code Intel – http://sublimecodeintel.github.io/SublimeCodeIntel/ go to definition etc support

Tomorrow we will continue the series getting into .net specific stuff and fixing some areas we are weak in now. How do you add a reference?

Anyone used any of these plugins? Please share your experiences in comments.

Sublime is Sublime 8

In this post we will look at how to become a bit more efficient in our sublime setup without adding lots of new tools or complexity. In particular I don’t like running my builds and my tests manually (and we will look at some varying options for this later). In this post however we will look at a dead simple method for doing it.

Let’s take a look at the full script I briefly wrote that does all of this.

#!/bin/sh
notify-send "Auto Started" "" -i $PWD/images/circleWIN.png -t 1000
while true
do
      if [ ! -z $(inotifywait --recursive -qre modify --format "%w%f" ./ | grep -v -f includes) ]
          then
	  sleep .25
          /opt/mono/bin/xbuild src/EventStore/EventStore.sln /p:Configuration=Release /verbosity:0 /nologo
          if [ $? -eq 0 ]
            then
               ./run_tests.sh -m /opt/mono -x LongRunning 
                if [ $? -eq 0 ]
                then
                      notify-send "Passed" "Tests Passed" -i $PWD/images/circleWIN.png -t 1000
                else
                      notify-send "Failed" "Tests Failed" -i $PWD/images/circleFAIL.png -t 1000
                fi
            else
                notify-send "Failed" "Build Failed" -i $PWD/images/circleFAIL.png -t 1000
          fi 
      fi
done

So what this does is wait for file changes in the directory structure (notify wait). It then checks to see if the file returned matches an include filter eg “*.cs” if it does it will then build the code followed by running the tests. When you get a failure/success you will receive a notification including a red/green circle in xwindows (regardless of if you are still in sublime). If you want to see run_tests.sh you can view it here https://github.com/EventStore/EventStore/blob/dev/run_tests.sh its a shell script in EventStore to ease in the running of tests on linux.

This is a great showing of the “unix philosophy” of small tools being composed together to get something larger. It was not hard to make this but it provides 80% of the value of autotest.net and is vastly more configurable if you understand shell script (want an exclude vs an include just edit the script).

There is no fancy installer. There is no visual studio integration to do (I will never do that ever again). On EventStore source base which is by all means not a small source base (roughly 2000 tests which are run locally) the entire cycle is 15-20 seconds on my machine. Although MightyMoose could have gotten this to around 5 which would be even better for simplicity this wins out.

One problem that I ran into had to deal with nunit-console when you have ignored tests there is no way to get it to not print out all the ignored tests (which it does of course last! so you can’t see your failures). To work around that I wrote a small xslt to format its results

<?xml version="1.0" encoding="UTF-8" ?>
<xsl:stylesheet version="1.0"	
                exclude-result-prefixes="msxsl"
                xmlns:xsl="http://www.w3.org/1999/XSL/Transform"
		xmlns:msxsl="urn:schemas-microsoft-com:xslt">
<xsl:output method="html" indent="yes" />
<xsl:template match="test-results">
			<xsl:variable name="failedtests.list" select="//test-case[@success='False']"/>
			<xsl:variable name="total" select="@total + @not-run" />
   			<xsl:variable name="success" select="@total - @failures - @errors" />
          	Total  : <xsl:value-of select="$total" /> Success: <xsl:value-of select="$success" /> Failed : <xsl:value-of select="@failures" /> Not Run: <xsl:value-of select="@not-run" />
			<xsl:apply-templates select="$failedtests.list"/>
</xsl:template>
<xsl:template match="test-case[@success='False']">
*************************************************************************************************	
	<xsl:value-of select="@name"/>
	<xsl:value-of select="failure/message"/>
	<xsl:value-of select="failure/stack-trace" />
</xsl:template>
</xsl:stylesheet>

This will give you a printout that looks like this:

          	Total  : 1748 Success: 1699 Failed : 1 Not Run: 48
*************************************************************************************************
	EventStore.Core.Tests.Services.UserManagementService.password_change_notification_reader.when_notification_has_been_written.publishes_reset_password_cache  Expected: 1
  But was:  0
at EventStore.Core.Tests.Services.UserManagementService.password_change_notification_reader.when_notification_has_been_written.publishes_reset_password_cache () [0x00000] in &lt;filename unknown&gt;:0

Again we have not done anything complicated here we are just composing our exists tools. I should add that you can easily make those links clickable back into sublime to go to a line based on stack trace but you either need to switch your terminal from gnometerminal or you need to add one line of code and custom build gnome terminal which may be a bit much for some people :)

As a quick exercise could you write a script like above for javascript?

Sublime is Sublime 7

So as we have come along the path our sublime instance is starting to feel quite a bit more customized than it was before. We can now build, have git/github integration, and of course have awesome command line integration but what about running tests?

We will start with the simplest way of running tests. This may not work for all of your scenarios and does somewhat require you to have thought about things in advance.

If you for instance started off from the beginning with resharper as a test runner you are probably have a test suite that will be difficult to use. Likely it is all in one assembly regardless of whether its a unit test or an integration test. Quite likely the tests have large numbers of dependencies. And to be frank your unit tests probably don’t run properly in linux anyways. Yeah this kind of sucks but can be fixed.

To start with there is no integrated right click->debug test in sublime. I find however that this is not something I missed. Maybe its having worked for a few years building embedded systems where there was no debugger (except dumping as hex pieces of memory).

Let’s talk about the simplest way to run tests from sublime and the general workflow. Normally when working with sublime I have multiple test suites either separated by category or by assembly. I have unit tests which are quick to run and then all other tests. I rarely if ever run the other tests locally. Instead I let a CI server run those. My unit tests should be run in and this would give you a new buffer with the output of your tests running. For a huge portion of code you will be working on this is *good enough*. If you have a 10 second cycle on tests running it is completely reasonable. eg

ctrl+shift+c run >

where I have a script called run that runs nunit in the way I want.

There are however some other things you can do if you want to be more fancy. The first is you could assign the running of the script to a key in sublime. This is actually quite easy any command can be assigned to any key by editing your keymap http://sublimetext.info/docs/en/reference/key_bindings.html

Another cool option that is quite easy to enable to get a notification from linux on the status of your build/test run. This can be done trivial using notify-send in your small script above based on return values from processes.

Try:
notify-send ‘Hello world!’ ‘This is an example notification.’ –icon=dialog-information

You can even change the icons to say red/green circles for the notification.

Now if your tests are more than 10 seconds. The first strategy would be to start putting them into categories that make sense to you. This way you can only run tests within a given category. This step alone will normally be good enough to get your test runs back under 10 seconds (I prefer 1-2 seconds). If this is not then perhaps your code base has become too dependent on the tool (15 categories 10 seconds each…). One thing you will notice is that your tests in general will run far faster than through tools like VS/R# when being run from the command line.

OK so now we have basic unit test integration. There are still a lot of problems with .NET code. In the next post we will look at what some of these are and how to deal with them.

Sublime is Sublime 6

So far we have done quite a bit getting sublime up and going for use with .NET work but we still at this point have to compile our code from the command prompt which well sucks. In this post we will look at how to get compiles working from within sublime by adding a custom build system for xbuild.

Go to Tools->Build System->new build. A file will open. Drop in:

{
“cmd”: ["xbuild"],
“path”: “/opt/mono/bin/”,
“working_dir”: “${project_path:${folder}}”,
“file_regex”: “^ (.*)\\(([0-9]*),([0-9]*)”
}

Then save this file named xbuild. You can now choose xbuild as a build platform.

Is is important to note that you have to point at the correct xbuild (I run off trunk so I compile with a /opt prefix if you get from say apt-get it will be some place else) and you must have the correct mcs in your path when doing this. This code also assumes that you have only a single solution in the root folder of your code.

Once this is done you can choose xbuild as your current build provider. Then you can use either f7 or ctrl+b to build your code on demand.

Now we are really starting to get there. In the next post we will look at how to run unit tests from within sublime manually.

Sublime is Sublime 5

So far we have learned how to install packages and have setup some basic packages for color schemes and for git. In this post we will get better support for the command line as there is lots of useful things that can be done on the command line in a linux system and as a developer you tend to be there often.

To do this bring up the command pallette (ctrl+shift+p) and type install which should bring you to install package. Hit enter and you will get the same popup as when installing the git support in the last post.

In the box type shell turtlestein to install this package https://github.com/misfo/Shell-Turtlestein. This is a very simple package but its also very powerful.

Once it is installed type ctrl+shift+c and a textbox will come up at the bottom of your screen. Any text you type here will be executed. As an example you can try ls -al and you will see the results of your command. This is how you can run arbitrary commands from directly in sublime.

There are however some other features of this command prompt such as being able to take text from in sublime and move them to the command prompt or to take text from the shell and move it to a buffer.

To move text from your command back to a buffer you use the > key trailing your command. As example try ctrl+shift+c then type ps -ax >. This will create a new buffer in sublime with the results of your command. We could even build this way xbuild yoursolution.sln > would bring up a new buffer with the build results but it is not the best way of doing this. This is very useful when writing documentation of say a restful service and interacting via curl

You can also take a document and pipe it to the command (it gets passed as stdin). An example of this can be seen with | wc or | wc > to get the result into a new buffer. The | will either take your selected text in the buffer or the whole text if nothing else is selected.

You can even combine things and do a transformation on your buffer through the command line. An example of this might be | sort | or | transform | which transforms the text then replaces the text within your document.

As you can imagine there are a huge number of possibilities that are now available to you from just interacting with the command line and we will use some of these in the next few posts to do other things that you will need to do on a daily basis.

Sublime is Sublime 4

Now we are heading into our fourth post about sublime. So far we have looked at why we might want to use sublime. We have installed sublime and learned some basic key shortcuts and we have installed a package manager with our first package. In this post we will install git integration into sublime and look at how its used.

This post assumes that the code you have opened in sublime is in a git repository!

So let’s start by installing the git package.

Hit ctrl+shift+p (you should be good at this)
Type install wait for 2-3 seconds
Type git and hit enter to install https://github.com/kemayo/sublime-text-git/wiki

After a few moments the git plugin will be installed. You may or may not need to restart sublime at this point I have heard mixed results.

Now let’s try using our git plugin. Let’s start by being in a file under source control and hitting

ctrl+shift+p git log

You will see the full log of your code. If you select one you can view the commit in a text window.

Let’s try another common one

ctrl+shift+p git pull

This will do a git pull. This is just the very beginning of the integration though. One of my favorite features. Add a new file foo.cs. Then in that file do a

ctrl+shift+p git add (current file)

if you then do a ctrl+shift+p git status you can see that the file has been added. We can also add such fun functionality as ctrl+shift+p blame :)

When you want to commit:

ctrl+shift+p git commit

this will bring up a buffer of your commit. Just type it your comment and hit ctrl+w to close the window and commit.

This plugin has very deep git integration including diff viewing and annotating code while working in it! You can find more on its feature set here https://github.com/kemayo/sublime-text-git/wiki

It takes a bit of exploring but you can do quite a bit with git integration in sublime and its language agnostic which is really nice.

While we are on the topic let’s install the github plugin since many of you are probably using github. The package is sublime-github https://github.com/bgreenlee/sublime-github. You can read more about the configuration of the plugin on their website but it allows you to do things like

view on github (opens code in browser)
manage gists (eg file or selection to public gist)
browse gists
gist->clipboard

Overall its quite well done as well.

So now you have gotten full git and github integration into sublime very soon we will start getting into the C# specific stuff but before that we need some better linux specific handling.

Follow

Get every new post delivered to your Inbox.

Join 7,557 other followers