Monthly Archives: July 2013

Why I cancelled my teched talks

This was written the morning of my session at TechEd and has been sitting for a while. I wanted to give some time to see if Microsoft would have some kind of follow up.

A bit jetlagged now. Why? I flew from Bali into New Orleans to speak at TechEd, yeah that’s about two days of travel and 15 hours of jetlag. It was awesome waking up at midnight my first night here (at least in New Orleans you know you can get a good breakfast at 0200).

This morning I managed to get out of bed to go do my talk at TechEd. Upon arrival at the venue we went to registration for speakers. I got my badge. I asked how we should handle my wife coming with me for the hour duration of my talk (she wanted to take pictures). I was told they can’t handle it and we need to go to the speakers’ room at 238.

As such we walked to room 238. Security however told my wife she can’t go to room 238 and escorted her from the building (even though we explained registration told us to go there). Thinking its a small misunderstanding (security is almost always contracted through the venue not the conference) I went to room 238 to try to figure things out. After explaining the situation going on to three staff members I was given the “opportunity” to buy a ticket (full conference on the last day) for her. They then handed me a shirt and said “you need to head over to your talk”. Are you kidding me? I immediately gave them the choice. Either they fix the problem or my talks (including european teched will be cancelled). They replied there is nothing they can do and I need to get moving to my talk, as such I cancelled my talks. What man would just leave his wife in a foreign country stranded outside a conference hall for almost two hours without even letting her know what was going on.

I apologize to anyone that really looked forward to attending my talk. Drop me a line offline and I can at the least send you over my slides.

This is not how you treat people. There were a huge number of ways this situation could have been diffused. How hard would it be to even just say “hey, you have to get to your talk, I will walk outside to find your wife, explain to her what’s going on and bring her to starbucks”. Or perhaps “your wife can’t go to the talk but we will grab her and let her hang out in the speakers lounge”. There are a lot of things that could have been done here to alleviate the problem.

But this is only the last straw, let’s start at the beginning of the “experience”.

For travel. They pay for your travel but only from your home city and only through their travel agency. As I was not at home and travelling with my wife I suggested that they could get two tickets one way (my wife and I) from Los Angeles instead of me round trip from Vilnius (no brainer the two one ways are $400 vs >$2000 round trip from Vilnius). After 2 weeks of emails and escalation we managed to make this work (all the while I am receiving emails about my travel not being booked and it needs my immediate attention). What’s missed in this? Yes we paid our own way from Bali to Los Angeles and then from New Orleans to London.

Along with this they tried booking us on the “best flight” 0600 take off. From some quick looking it was to save $20. How many people do you know enjoy waking up at 0300 to be able to meet their flight?

Now the fun starts. I get an email asking if I can do some Birds of a Feather. I say sure no worries. I love talking with interesting people and gave a list of five I would be happy to participate in. I get back an email scheduling me for two Birds of a Feather on days before I arrive! I inform them that I will not be able to do them as I won’t actually be there (travel booked by Microsoft). I offered to adjust our travel but requested to be reimbursed the $400 that it would cost us to rebook our tickets from Los Angeles. They thought better to cancel sessions. Apparently sessions are not worth $200 each (of course we are adjusting our travel by two days).

On Monday I received an email from the same contractor asking why I did not attend my Bird of a Feather session and asking if she should cancel the other one. What was hilarious was it was a reply in the same email chain where we discussed my not being able to attend them a week before and her saying she will cancel them.

I was also asked to do an interview for Channel 9. It was scheduled to be right after my talk (eg finish talk and walk straight to interview). I mentioned to them that normally after a talk there are people with questions that want to discuss them. I was told “tell them to find you later in the conference”. Really? How many thousands of people are there?

TechEd is on the whole the worst organized conference I have ever seen. It contains some lessons however.

1) You need a decision maker on site. No competent decision maker would decide that throwing my wife out is a good idea <30 minutes before my talk. Every single person we talked to was a v- contractor who were selected by being the lowest bidder.

2) People need to be empowered. The travel agent had a no brainer choice before her but was unable to act even if in their own best interest

3) Bureaucracy is often needed in scaled up situations but does not work well in things that require personal relations. Good support managers realized this decades ago. Give a point person that handles everything instead of forcing the client to handle your bureaucracy a conference the size of teched could easily have 15-20 temp employees to make sure the insanity of their internal process ends at a point person and is not publicly visible.

4) Conferences are not about filling in numbers. They are about personal interactions. Speakers are not commodities, they are people who are giving much to help the conference.

fail

Update: Received an email from Microsoft. While you would expect it to be an apology (over 200 tweets the first hour about it) it was not. It was letting me know about breach of contract and to pay for my hotel. This was 30 minutes after I cancelled. I guess its good they are efficient at some things.

Update 2: It is now about six weeks after the conference and I still have not heard a word from Microsoft over the incident. They are aware what happened and apparently its not important. I waited this amount of time to see if there would some reasonable response from them.

Pair Programming

Read about a week ago an interesting article challenging Pair Programming

I have to admit the first time I read it it seemed really devastating to pair programming. I however re-read it yesterday when James Nugent sent it by the second time. This time reading through I picked up on something I didn’t the first time.

Table 1 is extracted from a pair-programming calculator developed by the author.  It allows researchers to input a number of variables including staff compensation, application size in lines of code, and coding speeds for both pairs and individual programmers.  Table 1 shows a typical pattern for average pairs and average individual programmers for 1000 code statements

All of the numbers in the paper aren’t actual research! They are assumptions of the author that were then plugged into a “pair programming calculator”. Which as Mark Nijhof pointed out should “really have more than one author”.

In other words even though its on a “research” site and appears professional its just an opinion piece with made up statistics 😦

I personally think sometimes pairing is good and sometimes its not. I think the bigger question is getting into a more formal discussion of when it is good or not.

Reactivate Manifesto

This is worth a read for people http://www.reactivemanifesto.org/ there is a lot of wisdom in the document (though it would be great if they had more links)

Self Connects

About six weeks ago we had a bizarre randomly failing integration test over at http://geteventstore.com.

The test would bring up a node. And it was doing some writing/reading/verifying. It would fail maybe 3% of the time. In trying to debug the problem we came across some quite interesting results when it would actually fail!

During a failure we found that the client was receiving back a message that normally it sends to the server. This was quite frightening for us because we do a lot of things with re-using of buffers as opposed to making new buffers every time in both our client and our server (we reuse socketargs/buffers/etc). Our initial thought was crap we have a usually not hit threading bug somewhere in our heavily multi-threaded code this should be fun!

After some looking around though we found that it was not in fact anything wrong with our buffers. We were getting hit with a TCP self connect! Most people have never heard of a self connect or would even imagine that its possible but as we were running the server and the client on ephemeral ports the client was actually connecting to itself! Anything you sent through the client would end up in the client’s receive.

You can actually see this behaviour in the tcp state diagram (syn/syn+ack).

Image

This ended up costing us a ton of time. When we found it after debugging through (and philosophically debugging thousands of lines of multi-threaded buffer caching code). This is something to keep in mind when debugging tcp code, we knew tcp fairly well but had never run into this particular problem.

Want to try it yourself?

while true

do

telnet 127.0.0.1 50000

done

 

Want to prevent getting bit by it? Our mistake was the server was running on an ephemeral port. If we didn’t run on an ephemeral port in these tests we never would have been hit by the bug.

 

 

HttpListener in .NET

I have heard at least 20 different presentations inform me that one of the super awesome benefits of using a HttpListener in .NET is that it goes through http.sys which will allow you to have kernel mode caching inside of http.sys. They then explain why this is so much better because it doesn’t require a transition to user mode and blah blah blah.

Guess what they lied. In actually measuring this possible performance gain I found it was non-existent. After some poking through reference sources I came across:

 
line 525
 
UnsafeNclNativeMethods.HttpApi.HttpSendHttpResponse(
HttpListenerContext.RequestQueueHandle,
HttpListenerRequest.RequestId,
(uint)flags,
pResponse,
null,
null,
SafeLocalFree.Zero,
0,
asyncResult==null ? null : asyncResult.m_pOverlapped,
null );
 
 
Argument 5 is the CACHE_POLICY. It is set to null which means no caching.
 
The bigger question is why on earth would microsoft not allow you to use the http caching you pay a big complexity penalty to have access to?