Saturday, November 15, 2014

Balancing resources in SQL Server

Level: 3 where 1 is noob and 5 is totally awesome
System: SQL Server

As software developer I believe in decoupling, and I'm certain that decoupling is just as important in database design as well. I really like having things in minimum chunks, and keeping them independent as possible. Independence and decoupling equals scalability. This post is based on a professional experience, which supports my belief.

At my work we are developing a billing platform, and this platform has several clients. Each client use our platform to bill a certain amount of customers. So this set the scene.

Some time ago, we had one database for each client in one instance, which is a easy pattern. But in SQL Server it can easily be an expensive anti-pattern, and it is certainly a pattern which is my last choice. The problem with this pattern, it is only effective, when the databases is equal regarding size and usage.

The buffer pool


The explanation is, there is only one buffer pool pr. instance. SQL Server is an in-memory database, but in most cases, there is not enough memory for a whole database in the memory. And if there is more databases in an instance, then it is almost certain there no room in the memory for all the bases. To solve this issue, SQL server has the buffer pool, which is a cache. When querying, SQL Server will get the result from the buffer pool. If it not available in the buffer pool, it reads from the storage to the buffer pool. If there if no room in the buffer pool, it will flush some of the buffer pool, to make room for the requested data. Everything has to go through the buffer pool.

The process of getting data from storage to the bufferpool, generates PAGEIOLATCH waits. A high count of PAGEIOLATCH waits can be a sign of much data is being loaded from disk to the buffer pool.

Let us say, we have 3 databases in one instance. They respectively supports 25.000, 150.000 and 300.000 customers. The sizes of the databases reflects the number of customer, because more customers generates more data. What will happen, is the database with the most usage, will push out the data from the other bases, out of the buffer pool. This will give the lesser bases more read from the disk, producing more PAGEIOLATCH wait, adding more latency to the lesser bases. Actually, the smallest databases will suffer the most.

The poor solution  


I'm tempted to call this, the poormans solution, but it can be very expensive. The solution is to extend the buffer pool, with more memory. But there is a physical and economical limit to this. SQL Server 2014 has a new feature called buffer pool extensions, where it is possible to extends the buffer with storage(preferable SSD drives). I guess extending the buffer pool, will only cache more of the database with most uses. Besides, we have tried this feature, and have yet to see some great results. I would really like to hear, if some has used this feature with success.

The better solution


A easy way to balance databases resourcing over an instance, is by using the resource governor feature in sql server. It works excellent, but it is really a doubled edged sword. Used wrong it can really downgrade performance. It would also require some nursing, which I prefer to keep to a minimum.

The best solution


I find the best solution is to have more instances. If I have 3 clients, I will assign an instance to each, make it thier domain. Each instance, should have hard assigned a certain amount of memory, and their databases would never be able to inflict each others performance. 

This was the solution we did end up with, and it gave us better performance overall. 

  


Wednesday, October 22, 2014

Smart and Useful are not implicit the same

This special topic is something, I do philosophy on often. Why do we see, the smartest solution looses to the less smart solution? Like why did Betamax loose to VHS? Amiga to PC? HD-DVD to Blu-ray? And so on. My conclusion is; people in generel did find the looser less useful, no matter how smart they where compared to the winner.

Our job as developers(I assume you are a developer), no matter sort of developer, is to make the life easier for somebody or some, usually in an organization. At least, making life easier a.k.a. creating value, should be rule number one, for every developer.

I believe most developers understand the business domain, for the job they are hired to do, but I don't believe it is enough. By knowing the business domain, a smart solution can be developed, but is it useful? And what are the difference of smart and useful? I can state it, by making an example of financial data which need to be presented for a CEO.

Let's say the financial data is produced with the finest practices. The code behind is clean code, full maintainable and maybe even very innovative so the data can be produced in half the time as normal. The numbers are spot on error free. The data is presented in XML, so everybody with an understanding of XSLT, can transform it any way the like. It is smart!!! At least from a developer's point of view. But is it useful from a CEO's point of view. I don't think so. For being useful to, the data could have been sent, as an Excel file to the CEO's inbox. Everyone in such position know some degree of Excel, and done completely useful, the CEO should be able to present the report for his peers without further changes. XML and transformation should not be considered CEO knowledge, but Excel could.

My point is, to develop something smart and useful, you should not only know the business domain, you should also know the recipient(s) of your solution.

A good way to know a recipient is to talk with the recipient, and recognise the person's workflow. See if the solution you create, can be easily integrated into the recipient's workflow with little impact as possible, while still solve the recipient's problem and make the life easier for this person. That would be useful. Thing you could consider in advance, could be:


  • If your recipient, are using a coperate computer, then avoid things which means something has to be installed. Sometimes installation of components, is not even possible. Often components often involves regular involvement from support. Try to target you solution to something already on the your recipient(s) computer. E.g. Microsoft office. It will give you less fuzz and frustration.

  • If possible and depending of the type of the recipient, try to push the result of your solution to the recipient. As an example, again consider a report to a CEO. He shouldn't use time on digging out a report from a system you build, it should just be mailed to him. It would fit his busy workflow better. What would be even worse, would be if the system didn't use his OS credentials for logging in as single sign on, but if he had to use a 2nd credentials for this. Then the digging would be much harder. It would have even more negative impact on his workflow.

  • Optimise your result if possible. A report example again, instead of pages full of numbers, try to optimise with graphs in the start of the report. This would point out the important faster. Numbers could be added as appendix if details are needed. Optimise the report, so it is ready for be shown at meetings. CEO it save time, his life is more easy. 

  • Maybe most importantly. As a developer you are often a minority, compared to the number of recipients of your solution. They very seldom have the knowledge and technical insight on your solution as you. They will see your solution from another perspective, and you should respect this and learn from it. It is the key to make your solution more useful.

  • The purpose of education of recipients is to have a win win situation, you have less to do while your recipients would get more value. But educating people is hard, and choose you battles wisely here. E.g. Learning business people reading XML, compared to learning them a new functionality in Excel is tedious. You will often find the urge to learn people something, when they don't have time for it. E.g. When a business man who really need some data, and you have it in XML. The issues here are, his mind is on getting data, not on learning XML, He can't speak XML with his peers, He is not interested in XML at all and so on. On the other, if you extend a tool he already knows or show a feature, so you can provide him silent with XML. Then he would properly be more keen to learn about the tool, and then the education would both help you and him.  
It was few example, but is easy to come up with more.

My examples has primarily been with reporting, but this can be transferred to what ever. The golden rule is basically to know you audience and their habits. 

Thursday, October 2, 2014

SQL Server, SSIS or SSAS: Fastest and easiest way to create a time dimension

Level: 2 where 1 is noob and 5 is totally awesome
System: SQL Server 2005 and above

One of the most vital components of a dataware house, is the time dimension. It also seems to be the part which people to have most difficulties to generate. I seen all from black magic in MS Excel to big complex SQL scripts and even fancy SSIS solutions, just to generate time dimensions.

I do believe, I have found a better solution.

My solution is this script, I have written(inspired from a script I did found on Stack Overflow). You set a start and end date, plus the granularity. My recommendation is to generate times for at least a decade. A decade with the granularity of one hour, takes less than 5 Mb of data storage and 87601 records with this script as default. A decade is generated in a few seconds.


 DECLARE @StartDate DATETIME = '2010-01-01';   
 DECLARE @EndDate DATETIME = '2019-12-31';   
   
 WITH DateTimeGenerator AS  
 (  
  SELECT @StartDate AS DateAndTime  
  UNION ALL  
  SELECT DATEADD(HOUR, 1, DateAndTime) -- Change datepart to ajust granularity, see http://msdn.microsoft.com/en-us/library/ms186819.aspx for more  
  FROM  DateTimeGenerator    
  WHERE  DateAndTime + 1 < @EndDate  
 )  
 SELECT ROW_NUMBER() OVER (ORDER BY DateAndTime) As Id,   
 DateAndTime,   
 DATEPART(HOUR, DateAndTime) AS [Hour],  
 DATEPART(DAY, DateAndTime) AS [Day],  
 DATEPART(WEEK, DateAndTime) AS [Week],  
 DATEPART(MONTH, DateAndTime) AS [Month],  
 DATEPART(QUARTER, DateAndTime) AS [Quarter],  
 DATEPART(Year, DateAndTime) AS [Year]  
 INTO DimTime  
 FROM  DateTimeGenerator  
 OPTION (MAXRECURSION 0)  

Enjoy

Tuesday, June 24, 2014

SQL 2014: Getting started with OLTP, OLAP, SSIS, SSAS and SSRS

As passionate developer, I come in touch with many technologies, but only a few of them has been so tough to crack as SSAS and SSRS. While learning SSIS, I felt i was learning a lot about datawarehousing, and I couldn't really imagine there could be more to it. At the same time I knew there must be more to it, as SSAS and SSRS was a part of the SQL Server suite.

Now I know that SSIS, SSAS and SSRS can be translated to data gathering, data modelling and data visualisation.

And I have written this post to give a swift explenation of SSIS, SSAS and SSRS so it might be easier for you to get started.

My dream database setup 


Starting with a greenfield project, is often difficult. There no experiences and no knowledge, thereby there is paved no road, to go by when developing the system. In my opinion, the 1st mistake which the most make at this point, is to put it all data in same database. This leads to a monolith system, which is hard to maintain and expand. The 2nd mistake is not to divide the database setup in OLTP and OLAP. This lead to analysis with heavy querying in production and passive historic data also stored in production.

OLTP is an acronym for Online Transaction Processing. In other words, it is the database, which is in the line of business. You can also call it, the database with live data. In my dream setup for at system, i would have a bunch micro services, all supported by their own database. Not necessarily relational databases, I would have the database which fitted the job best. The data in these databases, would be on a journey, which ended in a OLAP system. This journey could span from instant to the lifetime of the system.

OLAP is an acronym for Online analytical Processing. OLAP is almost always, analysed for financial reasons, and therefor it should be the last stop for data. It must not return to OLTP, because it would mean, that history could be re-written which is a problem when it comes to finance.

The process of getting data from OLTP to OLAP, would be data gathering, which is the perfect intro for next topic: SSIS.

SQL Server Integration Services (SSIS)


SSIS has become one of my favourite SQL Server tools. With few drag and drops, it is possible to setup data copy or transformation. Compared to scripting(as a developer I usually prefers scripting), it is so much faster to get something up and running, and also easier to structure. Enough praising.

Referring to OLTP about having several different databases, and SSIS being able to read from different source. It is the perfect tool to support this solution.

DQS (Data Quality Service) and MDS (Master Data Service) are often mention together with SSIS. These tool works only with SQL Server, but if you are all-in with SQL Server these tools are quite good. DQS is for correcting data, as they are gathered. MDS is sort of an authority which distribute a data model through out a system to the connected SQL Server database. It makes sure data integrity is kept.

There is other suppliers of ETL tool for SQL Server, but I only use SSIS.

SQL Server Analysis Services (SSAS)


Even thou data is arranged by stars, and it browsable through excel, it can be better. While a star is easy to imagine, it can be more difficult to imagine an cube, and in the beginning even to realize the difference. A cube is like a dimensional system, as we now it from math. Most of us can imagine 1st, 2nd, 3rd dimensional system. Some of us can even imagine a 4th dimensional system, by collapsing 3 dimensions and add a 4th dimension, so it look 2 dimensional again.

Each dim table in a star, would be a dimension in a cube, but they are still not the same. In principle you could take a star, print out all tables, put the papers on a table and then draw lines for each relation from the dimension tables to the fact table. Where a cube is arranged (as mentioned before) an a virtual dimensional system, it is really not printable, but fast for analysing/browsing.

SSAS Helps you building cubes, so that would be data modelling. It will also assist you to do some data mining. I'll will refrain to speak of data mining with SSAS, because I haven't touch that topic yet.

Working with cubes, is a way in SSAS which called dimensional mode. SSAS has two more modes: PowerPivot and Tabular. It is other ways to model data, and which one is best? It depends. 

But with there is no point in having data modelled, if you can't deliver their information in a readable way.

SQL Server Reporting Services (SSRS)


As you might have guessed SSRS is the visualisation part of the SQL Server suite. There not much to say. It easy to design reports with the editor, you can render reports to various formats and make them available from a reporting server or a share point site.


Take advantages of having SSIS, SSAS and SSRS available


SSIS, SSAS and SSRS are available from SQL Server standard, BI and Enterprise. Not using these tools, when having SQL Server licenses which includes them is as failure. Often have I seen, and have properly in past done myself, custom made reports and based on some analysis directly from the production environment. 

Nobody finds it funny to create custom reports and nobody finds it funny to maintain them. So using SSIS, SSAS and SSRS is a win-win, because it would mean full value from licenses, and developers would use less time on reports and more time on funny stuff.


Getting started 


Getting started with SSAS and SSRS, was a bit hard for me, but fortunately I did stumble in the book: Microsoft SQL Server 2014 Business Intelligence Development: Beginner’s Guide by Raza Red. It is real page turner, which is nice, if you reads a lot of books per year.

It is very comprehensive and filled examples, with good explanations, which I like a lot.

Saturday, April 5, 2014

SQL Server 2014: SQL Server Data Tools(SSDT) and Visual Studio 2013 Challenges in Training Kit 70-463 and in general

Level: 2 where 1 is noob and 5 is totally awesome
System: SQL Server 2014

Notice: I'm using the SQL Server 2014 Developer edition, so if you are using a different edition, things might be different.

Some History


I'm currently studying for the exam 70-463 Implementing a Data Warehouse with Microsoft SQL Server 2012, and I'm using the 70-463 training kit for the exam. I have decided to use SQL Server 2014 for the practical training(regarding to SSDT, I now know it is the best to use SQL Server 2014). It has just been released and regarding to exam 70-463 there is no change. People who took the exam for SQL Server 2012 is also certified for SQL Server 2014. For more see FAQ here at http://www.microsoft.com/learning/en-us/sql-certification.aspx.

When using the training kit, you sooner or later realize, it has become a bit outdated. Which is quite natural, because it was published 25. Dec. 2012, and lot have happened since. There has been new releases and updates of Visual Studio, and we even have a new version of SQL Server. Which, all in all is nice.

The first sign of you might be challenged when using the training kit with SQL Server 2014, is in the beginning. The exam kit provides a list of features you have to install, and last on this list is SQL Server Data Tools, also known as SSDT. As you might discover, this feature is not shipped with SQL Server 2014, and it is the most important component for SSIS and the exam 70-463.

SSDT is a Visual Studio plugin, for developing SQL Server Integration Service(SSIS) packages. SSDT is a replacement from Business Intelligence Development Studio, also known as BIDS.Among things, SSDT delivers SSIS packages templates to Visual Studio. SSIS 2012 depends on Visual Studio 2010, Like SSIS 2014 does, but SQL Server 2012 installs the Visual Studio 2010 IDE if none is present. SQL Server 2014 has failed to do so, everytime I have tried to install it. If there is no SSDT in the SQL Server 2014 installation, it might make sense. Also it is okay, because I prefers to use Visual Studio 2013, and if you are a developer like me, I guess you prefer the same.

And Now the Tricky Part


I discovered Microsoft had developed SSDT (SSDT-BI which it is called now) for Visual Studio 2012, it the most easy package to find, and I thought it was the latest. I installed it, and was hit hard in chapter 3 of the training kit 70-463. Because I has installed SSDT-BI 2012 for Visual Studio 2012.

The problem was I have created a SSIS package with the SQL Server Import and Export tool, the version which comes with SQL Server 2014. It stores it SSIS Packages in the SSIS Package format version 8, while the SSDT 2012 stores in the SSIS Package format version 6. The SSIS Package format is apparently not forward compatible. 

The pain I experienced, in chapter 3 of the training kit, was I had to create a package with the SQL Server Import and Export tool (version 2014 in my case) and add it to a SSIS Project created with SSDT 2012. The because the version numbers didn't match, result was this:



Fortunately I succeeded to find SSDT-BI 2014 well hidden at 

It saves SSIS Packages in SSIS Package format version 8, like the rest of the SQL Server 2014 Suite, plus it works with Visual Studio 2013. I'll expect it will work it with the rest of the training kit 70-463, or else I'll provide a solution on this blog. As bonus info, you have to select "Perform a new installation of SQL Server 2014", or you will get some kind of architecture error. Despite the option name, it only installs SSTD-BI 2014.


Enjoy




 

Tuesday, March 18, 2014

Migrating from HaveBox 1.6.1 to 2.0.0

It's here, the lastest version of HaveBox. HaveBox 2.0.0. A bump in the major vision number means there are some breaking changes, when migrating from a prior version of HaveBox.

The breaking changes was needed, to get HaveBox in the direction I wanted. My goal is still to keep HaveBox simple, it doesn't mean less features in the future, but on the other hand I will not bloat HaveBox with features. HaveBox 2.0.0 now consist of a core with the basic features build in. Auxiliary features like scanners, config injection or xml configuration, will be added via sub-configs. Sub-config is the key word to extend HaveBox from now on. It should now be possible for everybody, to write a sub-config to customize HaveBox.

The Breaking Changes


Scanners

I have removed the scanner concept from config. I deprecated scanners in version 1.6.0, and from now on, scanner has to be defined sub-configs. HaveBox still comes with a scanner called SimpleScanner, but now it is a sub-config which have to be merged in to the config. See more in the documentation: HaveBox Scanners

Config injection

Config injection has to be enabled by merging in the Config Injection sub-config into the config.  See more in the documentation: Config Injection

Spawning Child Containers

The Spawn methods has been removed. Now you have to configure the life-time, of the singletons in the config section and call CreateChildContainer to create a child container. In this way, it is possible to combine the functionality of the spawn methods. See more in the documentation: Custom life-times

The New Features


TryGetInstance

It now possible to try-getting an instance. It is the same try-get functionality known from the rest of the .NET frameworks. If it succeeds the method returns true, and there will be an instance in the out parameter, else it return false, and there will be NULL in the out parameter.

How About Performance?


HaveBox 2.0.0 has also been optimized for performance. The ilasm has been cleaned and reduced, plus I have written a hashtable to replace Dictionary. The new hashtable is almost twice as fast compared to Dictionary(which also is a hashtable, but less optimal written, check the .NET reference sources.).

The concept app, is showing some nice improvement in performance, but I urge you download HaveBox and do your own testing.

That's all for now


HaveBox 2.0.0 can be downloaded from Nuget: http://www.nuget.org/packages/HaveBox/
The project page: https://bitbucket.org/Have/havebox/downloads

And don't forget to checkout HaveBoxJSON and HaveBoxStates.

Enjoy

Wednesday, February 19, 2014

C#: Speeding up Dictionary

Level: 2 where 1 is noob and 5 is totally awesome
System: C# on .NET

I like micro optimizing, I find it very funny, relaxing and I learn a lot while doing it. When working on HaveBox, I sometimes discover odd things, which can add performance to HaveBox. Most of the work on HaveBox is micro optimization, which is often expensive and not worth the effort in larger system. But some days ago, when I worked on HaveBox, I did stumble onto something regarding the Dictionary class. It is still micro optimization, but it is so simple and cheap, it might be interesting for other than me.

My Discovery was


When trying to get a value from a key in a Dictionary, the Dictionary class is using a comparer class, to compare the given key with the keys in the dictionary. When instantiating a Dictionary class with the default constructor, The Dictionary class is using the EqualityComparer class as default comparer class. The EqualityComparer class is designed to handle general situations, and is used in other contexts than Dictionary, so it have null checks. These null checks can slow down the Dictionary.

The Trick


As you might have guess, the trick is to write you own comparer, which is amazingly simple. It is inheriting the interface IEqualityComparer, implement the 2 method and then inject it into the Dictionary contructor. Like:

1:  using System;  
2:  using System.Collections.Generic;  
3:  using System.Diagnostics;  
4:    
5:  namespace DictionaryPerformance  
6:  {  
7:    public class Program  
8:    {  
9:      public class DIYCompare : IEqualityComparer<string>  
10:      {  
11:        public bool Equals(string x, string y)  
12:        {  
13:          return x == y;  
14:        }  
15:    
16:        public int GetHashCode(string obj)  
17:        {  
18:          return obj.GetHashCode();  
19:        }  
20:      }  
21:    
22:      static void Main(string[] args)  
23:      {  
24:        IDictionary<string, string> strings = new Dictionary<string, string>();  
25:        IDictionary<string, string> stringsWithDIYCompare = new Dictionary<string, string>(new DIYCompare());  
26:    
27:        for(var i = 0; i < 1000; i++)  
28:        {  
29:          strings.Add("Ipsum " + i, "Not important");  
30:          stringsWithDIYCompare.Add("Ipsum " + i, "Not important");  
31:        }  
32:    
33:        var stopwatch = new Stopwatch();  
34:    
35:        Console.WriteLine("@Intel I5-3337");  
36:    
37:        string tempStr = "";  
38:    
39:        stopwatch.Restart();  
40:        for (var i = 0; i < 1000000; i++)  
41:        {  
42:          strings.TryGetValue("Ipsum 999", out tempStr);  
43:        }  
44:        stopwatch.Stop();  
45:    
46:        Console.WriteLine("Fetched 1 of 1000 elements 1.000.000 times with default comparer and TryGet : {0} ms", stopwatch.ElapsedMilliseconds);  
47:    
48:        stopwatch.Restart();  
49:        for (var i = 0; i < 1000000; i++)  
50:        {  
51:          stringsWithDIYCompare.TryGetValue("Ipsum 999", out tempStr);  
52:        }  
53:        stopwatch.Stop();  
54:    
55:        Console.WriteLine("Fetched 1 of 1000 elements 1.000.000 times with DIY comparer and TryGet : {0} ms", stopwatch.ElapsedMilliseconds);  
56:      }  
57:    }  
58:  }  
59:    

The Numbers


 @Intel I5-3337  
 Fetched 1 of 1000 elements 1.000.000 times with default comparer and TryGet : 56  
  ms  
 Fetched 1 of 1000 elements 1.000.000 times with DIY comparer and TryGet : 49 ms  
 Press any key to continue . . .  

That's all.

Friday, February 14, 2014

Introducing HaveBoxStates. The .NET framework for better flows in your code

Level: 3 where 1 is noob and 5 is totally awesome
System: C# on .NET

Does this looks familiar?
You sketch a program, e.g. a games, like this:


And the implementation ended up, looking like this:


Often it is quite easy to draw a flow of a program or a part of it, but expressing it in code turns out to be hard. This is here where HaveBoxStates comes in. The purpose with HaveBoxStates, is to make it easy to express flows in code, just as we draw them,

HaveBoxStates 

HaveBoxStates is a statemachine/flow engine, to express flows in code. It is very flex and lightweight, and very easy to use. People who has worked with state machines before, might find HaveBoxStates a bit unusual, because the state machine/flow is not defined/configured before use. Even thou it doesn't have strictness predefined in a model, it still as strict. Trying to enter an invalid state would blow up the machine, just as if it was predefined in a model.

Best pratices 

The most important thing I can come up with is, you should always constructor dependency inject your logic into the states. HaveBoxStates comes with HaveBox out of the box, but through a ContainerAdaptor, you can use any container.

How to use it

The following examples are taken from the concept app. So if something needs to be elaborated, you can get the full picture there. Questions in comments is also okay :-). Notice the dependency injection in the PlayGame state.

Here is how to use it:

You draw a flow. It could be this flow:


For each state, you creates a class like this(in separated files, of course):


 using GameStateMachineConcept.Dtos;  
 using GameStateMachineConcept.Dtos;  
 using GameStateMachineConcept.GameEngine;  
 using HaveBoxStates;  
   
 public class Init : IState  
 {  
      public void ExecuteState(IStateMachine statemachine)  
      {  
           statemachine.SetNextStateTo<PlayGame>(new PlayerContext{ Points = 0, Lives = 3, });  
      }  
 }  
   
   
 public class PlayGame : IState  
 {  
      private IGameEngine _gameEngine;  
   
      public PlayGame(IGameEngine gameEngine)  
      {  
           _gameEngine = gameEngine;  
      }  
   
      public void ExecuteState(IStateMachine statemachine, PlayerContext playerContext)  
      {  
           var gameContext = new GameContext { PlayerContext = playerContext, GainedPoints = 0, };  
   
           _gameEngine.Play(gameContext);  
   
           statemachine.SetNextStateTo<Scores>(gameContext);  
      }  
 }  
   
   
 public class Scores : IState  
 {  
      public void ExecuteState(IStateMachine statemachine, GameContext GameContext)  
      {  
           GameContext.PlayerContext.Points += GameContext.GainedPoints;  
   
           if (GameContext.PlayerContext.Lives == 0)  
           {  
                statemachine.SetNextStateTo<Stop>();  
           }  
           else  
           {  
                statemachine.SetNextStateTo<PlayGame>(GameContext.PlayerContext);  
           }  
      }  
 }  
   

Each state must implement an ExecuteState of one of the following signatures:

public void ExecuteState(IStateMachine statemachine)

or

public void ExecuteState(IStateMachine statemachine, <any type> <any name>)

<any type> can be any reference type and <any name> can be any name. The second parameter, is for  DTOs and transferring data between the states. HaveBoxStates automatically cast data to right type, before calling the ExecuteState method.  You just have to give the type of the DTO, forget about casting and just use it.

Because the DTO type can be of any type, and we are not using generics. IState do not enforce implementing of the ExecuteStates, so you have to remember it or else you will get a runtime exception.

The Stop state

HaveBoxStates have one builtin state. The Stop state. If the state machine needs stop, go to the Stop state.

Wrapping it up


Setting up and starting a state machine is also quite easy. Register all states with their dependencies in a container and then instantiate a state machine with the container. Notice, do not register states to IState.

 using GameStateMachineConcept.GameEngine;  
 using GameStateMachineConcept.States;  
 using HaveBox;  
 using HaveBoxStates;  
 using HaveBoxStates.ContainerAdaptors;  
 using System;  
 using System.Collections.Generic;  
   
 namespace GameStateMachineConcept  
 {  
   public class Program  
   {  
     static void Main(string[] args)  
     {  
       var container = new Container();  
       container.Configure(config =>  
       {  
         config.For<IGameEngine>().Use<GameEngine.GameEngine>();  
         config.For<Init>().Use<Init>();  
         config.For<PlayGame>().Use<PlayGame>();  
         config.For<Scores>().Use<Scores>();  
       });  
   
       var stateMachine = new StateMachine(new HaveBoxAdaptor(container));  
       stateMachine.StartStateMachinesAtState<Init>();  
     }  
   }  
 }  
   

That is all :-)

Monday, February 10, 2014

C#: Count() or Length?


Level: 4 where 1 is noob and 5 is totally awesome
System: C# on .NET

Intro


When working with some data structures such as Arrays, strings or alike in C#, we often have 2 choices if we want to know the count of elements in the structure. For most I guess the choice might by easy, people are used to use Count() from LINQ or other structures, so Count() it often is. Length might feel like as, it is just something which pops up, when you scroll the intellisense. You see it, you know what it does, but you prefer Count(), because it makes more sense prosa wise in you code.

Are there any arguments for using Length? Yes.

Performance wise, Length is superior to Count(). Under Count() is some evaluation logic, to count the elements of the structure. Under Length is a pure IL mnemonic, meaning there is directly CLR support for Length.

An Example


I have a reduced IL example, to show what happens IL wise when Length/Count() in C# is compiled:

 .method private hidebysig static void Main(string[] args) cil managed  
 {  
      ...  
      ...  
      ...  
   
      // We initialize 3 variables  
  [0] string[] strings,     // The string array  
  [2] int32 length,         // The length variable  
  [3] int32 count,          // The count variable  
             
      ...  
      ...  
      ...  
   
      // Getting array element count by Length  
  IL_0020: ldloc.0          // puts the string array on the stack  
  IL_0021: ldlen            // using the IL mnemonic for to get length for the array. An unsigned int is returned  
  IL_0022: conv.i4          // The Length variable is a signed int so we convert the output from ldlen  
  IL_0023: stloc.2          // Stores the value, in the variable Length  
   
      ...  
      ...  
      ...  
   
      // Getting array element count by Count()  
  IL_005f: ldloc.0          // puts the string array on the stack  
                            // Next, calling Count() and gets a signed int.  
  IL_0060: call    int32 [System.Core]System.Linq.Enumerable::Count<string>(class [mscorlib]System.Collections.Generic.IEnumerable`1<!!0>)  
  IL_0065: stloc.3          // Stores the value, in the variable count  
      ...  
      ...  
      ...  
    
 } // end of method Program::Main  
   


Performance

To show the difference in performance, I will use the following example:


 using System;  
 using System.Diagnostics;  
 using System.Linq;  
   
 namespace ArrayCount  
 {  
   public class Program  
   {  
     static void Main(string[] args)  
     {  
       var strings = new string[1000];  
       var stopwatch = new Stopwatch();  
       var length = 0;  
       var count = 0;  
   
       Console.WriteLine("@Intel I5-3337");  
   
       stopwatch.Start();  
       for (var i = 0; i < 1000000; i++)  
       {  
         length = strings.Length;  
       }  
       stopwatch.Stop();  
   
       Console.WriteLine("Counted {0} elements by using Length : {1} ms", length, stopwatch.ElapsedMilliseconds);  
   
       stopwatch.Restart();  
       for (var i = 0; i < 1000000; i++)  
       {  
         count = strings.Count();  
       }  
       stopwatch.Stop();  
   
       Console.WriteLine("Counted {0} elements by using Count() : {1} ms", count, stopwatch.ElapsedMilliseconds);  
     }  
   }  
 }  
   

When compiling for release and running it without debugger (shift+f5) the output is:

 @Intel I5-3337  
 Counted 1000 elements by using Length : 0 ms <--- Less than millisecond, not no time:-) 
 Counted 1000 elements by using Count() : 66 ms  
 Press any key to continue . . .  

Conclusion

Now you know the difference between Length and Count(). When performance is preferred Length should always be used, but in general it depends on the context. 


Thursday, February 6, 2014

Specification by Small Talk

Level: 4 where 1 is noob and 5 is totally awesome
Disclaimer: If you use anything from any of my blog entries, it is on your own responsibility.

Intro


This a term/method I have coined, based on some good experiences I have had. Just to rule out any misunderstandings, this has nothing to do with the programming language SmallTalk. It is hardly new, and I guess most of you do it to some degree. It is not supposed to replace any specification methods. It is a suggestion for a preamble for whatever specification method is used. It is quite simple, because it is just informel communication.

Motivation


We have come far with different specifications techniques, but how the techniques are executed, sometimes troubles me. Most a companies, often have a pipeline with tasks/projects. The tasks or projects is prioritised after importance, decided on some meeting. When a task or project comes to an end, it usually releases some human resources, which has to be allocated for a new task or projects. Usually a meeting or a workshop is arranged for specifying the task or project.

This is the point which troubles me, and here is why. Without any preamble, we are ironically using a waterfall alike model for specifying. It is like, meet up with you thoughts, specify within this period, see you again at next specify sprint. I know, this is very roughly, but it is for emphasising my point. The thing is, at the first meetings, people mindset and understanding, are not aligned. Often the best responses to input, comes when the input has circled in the mind for couple of days, but by then the specification meeting or workshop is done. Another issues is also, do we have the right people at the meeting? And most importantly, is this task or project in reality the most important, or should another project have the priority? We simply don't have the best start.

So what am I suggesting?


As mention before it is quite simple, it is just talking. The concept is about to find values, share thoughts maybe discuss something technical in a relaxed way, and in the end have an aligned mindset for a specification workshop. The process can run from an idea is born, to when there are resources to bring it to live. Regardless of the work pipeline. The point is, it has to be informal. The idea can be discussed on small informal meetings, maybe by mails or at lunch. Nothing must be settled or promised. The most important part is, involved people should have the time to think about it.

The process starts by someone wants something. As an example, lets say a customers gets an idea for a new feature in the system. He takes contacts his contact from the system supplier company, and tells about the idea. The contact person talk with a coworker about it. Maybe they invite the customer to come by to hear more about it. The co-worker mention the idea at lunch, and then finds out that another coworker, knows a lot about the domain of the idea. Thereby the best person for the job is chosen, instead if the available person at the time. The point is to let idea spread like rings in water through out the company/department. 

Another gain. Maybe the idea is has more value than first anticipated. Maybe it has so much value, that the pipeline has to be reorganised. Projects without values, could even be avoided, by this early evaluation. Value is always best discovered sooner than later. 

Maybe involved developers can test or try things, to improve the idea or give it the best solution. When running a time boxed project, developers do not have much time to do so.

It all sounds good, are there any pitfalls?


Oh Yes. The biggest pitfall is to settle on a solution, so the specification workshop is all about that solution. Some specification techniques, like specification by example, is about focusing on problems, instead of the solutions. In my book, it is the way to specify.

Summary

Specification by small talk, is preparing a project for specification workshop. Less introduction and more focus. It is also for an early evaluation, and preparing for developing. It is a interesting concept I'll explore some more.

Friday, January 24, 2014

The SQL NULL paradox

Level: 1 where 1 is noob and 5 is totally awesome
System: MS SQL Server or PostgreSQL

The Paradox 


One of the best things about computing, it is so logical. We can always trust in 1 + 1 = 2, 1 < 2 or 1 = 1. So now I'm going to throw 4 queries and their results, as an example:

SELECT CASE WHEN 42 > 42 THEN 'True' ELSE 'False' END; -- Result: False
SELECT CASE WHEN 42 < 42 THEN 'True' ELSE 'False' END; -- Result: False
SELECT CASE WHEN 42 != 42 THEN 'True' ELSE 'False' END;-- Result: False
SELECT CASE WHEN 42 = 42 THEN 'True' ELSE 'False' END; -- Result: True

Hopefully no surprise there. Now I'm going to throw 4 queries more, and their result:

SELECT CASE WHEN NULL > NULL THEN 'True' ELSE 'False' END; -- Result: False
SELECT CASE WHEN NULL < NULL THEN 'True' ELSE 'False' END; -- Result: False
SELECT CASE WHEN NULL != NULL THEN 'True' ELSE 'False' END;-- Result: False
SELECT CASE WHEN NULL = NULL THEN 'True' ELSE 'False' END; -- Result: False

The three first lines, seems logical, but wait a minute:-). How is this possible? If NULL != NULL then NULL = NULL  must be true, else it is illogical. Or is it?

The Explanation


Actually it is not paradox. The reason, some might expect NULL = NULL to be true, is because they misunderstand NULL. They mistake NULL for being a value. If NULL was a value, then NULL = NULL would be true. So if it is not a value, what is it then?

The ANSI SQL-92 Specification says about NULL:
"null value (null): A special value, or mark, that is used to indicate the absence of any data value."   
I can understand the confusion, because have just written, it is not a value. But you have to notice it says "A special value". Absence of data value, is the right way to see NULL,  unknown or missing.

To be able to handle NULL values defined in the SQL-92 specs. Databases servers such as SQL Server and PostgreSQL uses trinary logic and not binary logic. It means, logic operations can have 3 out comes: True, False or NULL.

Because NULL is considered to be unknown, any comparison to NULL, even NULL vs. NULL will have False as result, because it is unknown what we compares. While using any arithmetic operator with NULL, will have NULL as result. The latter makes good sense. Lets say you add something to unknown, then result is unknown.

So in the end, it's all logical. For more read Handling Null Values for SQL Server, or 9.2. Comparison Operators for PostgreSQL. Even better read them both:-)

Wednesday, January 22, 2014

Data Modelling with Immutability

Level: 3 where 1 is noob and 5 is totally awesome
Disclaimer: If you use anything from any of my blog entries, it is on your own responsibility.

Intro


Data modelling with immutability, is a thing I have been thinking about for a while now, and if I had green field project I would think it in, from start. The concept is simple, instead of change states on objects, we create new ones(Well, sometimes it makes more sense to use mutable entities, but I'll get back on that). It is a concept, which makes it possible to create better data models, which resembles the things, we want to model more precisely.

Through out this blog post, I'll use the following Person-Address model, when clarifying the concept. The model is simple. It resembles a person and an address for this person.

The Concept


The conception is as mention before; instead of change the state of entities in the data model, we create a new ones. As an example, let us look, at our Person-Address model. Lets say we have a Person, who decides to move, which is the same as change address. A way to model this change of address, could be updating the address entity with the new address. But, not only from a modelling perspective is this wrong, but it can also give some technical challenges later on. Such querying the person former addressed, can be difficult.

The right thing to do, is to somehow mark the old address entity obsolete(but do not delete), create a new address entity, with the new address, and connect it to the person as the current address. Why is this right? It is right, because when a person is moving, they a moving to a new location/address. The new address has another house, while the old house is still standing at the old address. So by modelling address a immutable, we have the old address and the new address, just like in the real world. Actually, if the address entity was updated with a new address, it would be the same as saying, we are  dumping a new address and a house, on the old address. Not really possible in reality, eh?

The tricky part of modelling with immutability, is identifying what should be immutable and what should be mutable. Lets take a look at the Person-Address model again. Person. No matter the name of a person, the person would ideally always be the same. Thereby the person is mutable. So if a person changes name, the person entity should be updated. The entity should not be replaced, like in the in address example. If the entity was replaced, it would be same as saying it is a new person. So thing which remains the same, no matter the state, should be mutable.

Versioning and event sourcing

First a quick definition, of versioning and event sourcing. Event sourcing, is storing the event which caused an entity to change. A version is the result of an event.

Some might think, by using immutable entities in a model, we would get event sourcing, as an positive side-effect. Well, if we have to be absolute correctly, it wouldn't be event sourcing. Because we are not storing an event, we ares storing a result of an event a.k.a. a version. We are also only getting versioning for the immutable entities, because we are creating a new entity for every change. Mutable entities state changes should, like immutable entities events, be event sourced.

By modelling this way, you should always have the most true data model. As positive side effect, you should always be able, to query versions and events for every entity in your model. Happy modelling.

Friday, January 17, 2014

T-SQL: Comma Separating Values into Fields

Level: 2 where 1 is noob and 5 is totally awesome
System: SQL Server from 2005 and up


Visualising data


When querying data from databases, I'm always trying to get resultsets which are most presentable as possible. It save me a lot of time, because I then don't need to transform data further. The upsides are, I can either save a resultset as csv or just copy paste it to some spreadsheet or to a mail, right after the querying. The down side is, I have to type a bit more SQL.

In this blog post, I'll present one of my favorites methods, to easily visualize data in a smart way. I'm going to explain the method, using these 2 tables:

Items:
Id          Name                                               ColorId
----------- -------------------------------------------------- -----------
1           Cranberries                                        1
2           Plants                                             2
3           Shrimps                                            1
4           Ocean                                              3
5           Sky                                                3
6           Roses                                              1  

And

Colors:
Id          Color
----------- --------------------------------------------------
1           Red
2           Green
3           Blue

The most used way to show, the color for each item would be to join the tables like:

 SELECT Items.Id, Items.Name, Color FROM Items  
 INNER JOIN Colors ON Items.ColorId = Colors.Id  

Which gives:

Name                                               Color
-------------------------------------------------- -----------------
Cranberries                                        Red
Plants                                             Green
Shrimps                                            Red
Ocean                                              Blue
Sky                                                Blue
Roses                                              Red

It is readable, and if we sort by color it might be more readable. It might be most readable, if we could group the items by color. It requires a bit more SQL; but the result is better:

 SELECT   
  STUFF(  
      (SELECT ', ' + Name  
      FROM Items  
      where Items.ColorId = Colors.Id  
      FOR XML PATH (''))  
      , 1, 1, '') AS Items,  
 Color  
 FROM Colors;  

Gives:

Items                                Color
------------------------------------ ------------------------------------------
Cranberries, Shrimps, Roses          Red
Plants                               Green
Ocean, Sky                           Blue

Explanation


The magic happens in the inner SQL. FOR XML PATH is used as a smart way for concatenating string columns to one string column. By adding the ',' to Name, we are getting a comma separated list, where the first char is ','. For more about FOR XML PATH see: http://technet.microsoft.com/en-us/library/ms189885.aspx

STUFF are used to remove the first char(in this case ','), from the comma separated lists. Stuff is a sql command for inserting strings into strings, but is can also remove chars. In this case we are overwriting first char with a empty string. For more about STUFF see: http://technet.microsoft.com/en-us/library/ms188043.aspx

The rest should be self explanatory :-)


Thursday, January 9, 2014

Starting and Running an Open Source Project

Level: 1 - 5 where 1 is noob and 5 is totally awesome


Introduction


This post is about my experiences and thoughts about starting and running an open source project. Let me start by introducing my own open source project called HaveBox. HaveBox is one the fastest IoC containers written for .NET. For more about HaveBox see http://www.havebox.net. Through developing and running the HaveBox project, I got some experiences which might help others with their projects. Some of the stuff in this post, might be common knowledge for some, and gold nugets for others.

Getting started


You might have a lot of reasons to start a project. Maybe you are unsatisfied with existing solutions, maybe you got a smart idea or maybe you just want to learn something. The most important factor is, for what ever the reason you choose to start a project, it has to be fun. The more fun, the more a project is likely to get released.

Make the project for your own sake only. Listen to others opinions, will be healthy for your project. Listening to much can throw your project of the course you did set for it. In general, only do what you feel is best for you project, because it is your project.

Most of all, don't be afraid to reinvent something, because your solution could end up being smarter or faster, or in general better. Also if you think about it. Maybe some of the established solutions is to matured. Maybe their concept has evolved over the years, and to keep up with evolution, they have been extended to a point where is impossible to add more. You maybe at a point in time, where all the basic features is thought of, so you can incorporate them from start, and leaving room for your own smart features.

Developing your project


When I develop open source for free, I would like my tool to be free to, or as cheap as possible. I also look at how much of my work, I can unload to the web. Ex. Version control and hosting. Specially if you are on your own, then reduce your devops tasks, and use your time on developing.

People sometimes ask me, why HaveBox is hosted at BitBucket and not at the more popular GitHub. The reason is BitBucket suits my workflow better. I really likes to work on things in private, and then release it in public when I feel my projects is ready. This kind of workflow cost money at GitHub, even thou it is an open source project when released. At BitBucket it is free, to have release and dev branches.

Selecting a language and a platform can be straight forward in some cases, but in many cases challenging decision. Some platforms and languages are more suited than others, for your project. Actually this might be the hardest part. Personally, I would like to do my private developing, on platforms and languages, which are not the same, as the ones I use in my professional work. This gives me the chance, to get experience with some more exotic an new stuff, plus it gives a broad knowledge.

Using hyped/new languages and platforms can be lot of fun, but I would only choose those for smaller project. The reason is, my experience says no matter how promising a hyped language seems for being consolidating, it only seems to last a couple of years. E.g. Ruby was the new black a couple of years, and even now I have heard talks about Node is yesterdays news. I would hate to have a big project, which had to be converted, because I choose a language which failed to settle.

Releasing the project


Develop and releasing your project, is actually a part of running the project. But usually there is no running of the project until first official release. 

Keep your release small, first release should contains only most basically functionality. Then else only a few new features per release. If you try to make all fit into one release, the project might be to boring and then it will die. Also smaller and often releases, might give some feedback which could point the project in a better direction.

Every time you release your project, remember to versioning it. I recommend using versioning like http://semver.org. Also the source code for each version should be available. Sometimes, people are using older versions of your project, and need the source to debug, then it will be a great help if they can use how your code works in that particular version.

Running the project


If you have done your  project and documented it properly, it should run it self more or less. Sometimes people wants to ask you questions, and you should think of how people should be able to ask you question.

Earlier I encouraged people to ask questions on Stack Overflow, so more people could gain from the answers.  My experience is, when you answer questions, and referring your documentation on the projects homepage, Stack Overflow thinks your are promoting your project and they don't like it. The consequence could be, they delete all your answers and write you a mail suggestion you can pay for promoting your project via Stack Overflow. Now I just encourage people, to write me directly or use the Issue Tracker on BitBucket. I refrain from crawling the web for HaveBox questions, it takes to much time, which could be used for development, so I only focus on direct contact now.

Most of the feedback I have received, has been good and constructive. Sometimes I also have received, harsh and unconstructive feedback. I have found the way to handle this, is to advise the feedbacker to use another product. If the feedbacker has misunderstood something, I will of course try to straight it out.

Most of all put yourself and your private life before your project. Don't feel bad about not developing on your project. Nothing good comes out of using force, so when an error is reported, sit back and analyse how big the problem is. Don't rush release, because of errors, but keep your users informed.  

How to promote you project

Promoting you project, is like selling a product. You have to identify what channels you can use for promoting. Personally I use Facebook, LinkedIn, 2 IoC container's benchmark pages and local .NET user groups. It all seems to work. 

If you get the chance, to give a speak about you project, take it. It is a very valuable way to promote your project. Even if it might sounds scary.

Do not use your project at work


Avoid using your project at work, because suddenly it can be hard to determine when it is your private project or a property of you company.

Also don't force your friends to use your project. Let them use the components for their project, they feel safe with. If you makes sure you project always works, and let it mature, then they might feel safe with it and start using it.

Good luck with your project

That is more or less, what I have for now. I might update this post when I some more. If you have something you would like to add, feel free to write me.




Wednesday, January 1, 2014

Key and Mouse events with NSViewController

Technical Level: 1 to 3 out of 5, where level 1 is noob and level 5 is totally awesome.
XCode version: 5

The Problem

Sometimes it is convenient for some apps, to be able to handle key and/or mouse events extraordinary.

While handling key and mouse events in many frameworks are straight forward. It is for what ever reason unnecessary complex in osx cocoa, when using the NSViewController class. Osx cocoa developers are encourage to use the NSViewController for their apps, and out of the box, it is suitable for most apps without extraordinary key handling.

NSViewController inherits from NSResponder, so theoretically, it should be possible to handle key/mouse events just by overriding the methods keyDown, keyUp, mouseDown or mouseUp. In practice, it doesn't work, yeah. Googling the solution to the problem, can just as frustrating as the problem, because information is very sparse. Plus the answers, can be hard to interpret for new cocoa developers.

The problem is, even thou NSViewController inherits from NSResponder, it is not added to the responder chain when initialising. Only responders on the responder chain, is able to handle events. We have to add NSViewControllers to the responder chain, for being able to handle key and mouse events. The only problem is, NSViewController is not capable to do this, by it self.
For more about the responder chain see: https://developer.apple.com/library/mac/documentation/cocoa/conceptual/eventoverview/EventArchitecture/EventArchitecture.html#//apple_ref/doc/uid/10000060i-CH3-SW2

The Solution
Actually the best solution can be found here: http://www.cocoawithlove.com/2008/07/better-integration-for-nsviewcontroller.html

So what I'm going to do, is to emphasise the solution cocoawithlove.com and making it clear, by making a walkthrough of how to implement it.

In short, we are going to create a subclass of the class NSView. We are going to use methods on the NSView class, to add the NSViewController class to the responder chain. The tricky part is to connect the NSViewController to the NSView, but we will go through that.

The next steps, may for some experienced cocoa developers, be extremely elementary. So if you know how to setup project with NSViewController, you might want to skip to the section: Implement keyDown and mouseDown
 

Setting up project with NSViewController


Create new project, and select Cocoa Application.


Fill out the properties for the new project.


Select a nice place for the project.


If everything went okay, you should have a screen more or less similar, to the one above.


Test the app, by pressing the play button. A window should pop up. When you see the window, you should stop the app, by pressing the stop button.


We are going to create a NSViewController. Select AppDelegate.m(to make sure the files we are going to create, ends up the in right folder) and press ⌘n. Should your newly created files, end in the supporting files folder. You can just drag them up to the KeyAndMouseEventExample folder.


Select Objective-C class


We are prefixing with App, just to prefix with something, because ViewController is an invalid name. Make sure With XIB for user interface is checked. Press next, and then Create on next page.

Click on AppDelegate.m and correct the code to:

 //  
 // AppDelegate.m  
 // KeyAndMouseEventExample  
 //  
 // Created by Christian Henrik Reich on 01/01/14.  
 //  
 #import "AppDelegate.h"  
 #include "AppViewController.h"  
 @interface AppDelegate()  
 @property (nonatomic,strong) IBOutlet AppViewController *ViewController;  
 @end  
 @implementation AppDelegate  
 - (void)applicationDidFinishLaunching:(NSNotification *)aNotification  
 {  
   self.ViewController = [[AppViewController alloc] initWithNibName:@"AppViewController"bundle:nil];  
   [self.window.contentView addSubview:self.ViewController.view];  
   self.ViewController.view.frame = ((NSView*)self.window.contentView).bounds;  
 }  
 @end  

Press the Play button again, to see everything works. Close app again with stop button.

I guess many of you have this NSViewController structure in your apps already. Nevertheless, we have something to build on now.

Implement keyDown and mouseDown

First we are going to create a subclass of NSView. Select AppDelegate.m(to make sure the files we are going to create, ends up the in right folder) and press ⌘n.



Select Objective-C class 


We are prefixing with App, just to prefix with something, because ViewController is an invalid name. NSView is properly not selectable in the combo box, but it can be typed in.

Select AppView.h and correct the code to:

 //  
 // AppView.h  
 // KeyAndMouseEventExample  
 //  
 // Created by Christian Henrik Reich on 01/01/14.  
 //  
 #import <Cocoa/Cocoa.h>  
 @interface AppView : NSView  
 {  
   IBOutlet NSViewController *viewController;  
 }  
 @end  

Select AppView.m and correct the code to:

 //  
 // AppView.m  
 // KeyAndMouseEventExample  
 //  
 // Created by Christian Henrik Reich on 01/01/14.  
 // Copyright (c) 2014 Christian Henrik Reich. All rights reserved.  
 //  
 #import "AppView.h"  
 @implementation AppView  
 - (void)setViewController:(NSViewController *)newController  
 {  
   if (viewController)  
   {  
     NSResponder *controllerNextResponder = [viewController nextResponder];  
     [super setNextResponder:controllerNextResponder];  
     [viewController setNextResponder:nil];  
   }  
   viewController = newController;  
   if (newController)  
   {  
     NSResponder *ownNextResponder = [self nextResponder];  
     [super setNextResponder: viewController];  
     [viewController setNextResponder:ownNextResponder];  
   }  
 }  
 - (void)setNextResponder:(NSResponder *)newNextResponder  
 {  
   if (viewController)  
   {  
     [viewController setNextResponder:newNextResponder];  
     return;  
   }  
   [super setNextResponder:newNextResponder];  
 }  
 - (BOOL)acceptsFirstResponder {  
   return YES;  
 }  
 @end  

Select AppViewController and correct the code to:

 //  
 // AppViewController.m  
 // KeyAndMouseEventExample  
 //  
 // Created by Christian Henrik Reich on 01/01/14.  
 //  
 #import "AppViewController.h"  
 @implementation AppViewController  
 - (id)initWithNibName:(NSString *)nibNameOrNil bundle:(NSBundle *)nibBundleOrNil  
 {  
   self = [super initWithNibName:nibNameOrNil bundle:nibBundleOrNil];  
   if (self) {  
     // Initialization code here.  
   }  
   return self;  
 }  
 -(void)keyDown:(NSEvent*)theEvent  
 {  
   NSLog(@"Caught key event");  
 }  
 -(void)mouseDown:(NSEvent*)theEvent  
 {  
   NSLog(@"Caught mouse event");  
 }  
 @end  

If you start the application, you will notice it is not working yet. We have to connect AppViewController to AppView's member viewController(See AppView.m), before it is working. Here comes the tricky and magic part :-)

Select AppViewController.xib


Select Custom View, and change Custom Class to AppView.
Select AppView formerly known as Custom View, it should be placed at the green circle to the left. Hold down control, and drag it to file owner.
On the outlet popup, select viewController

Press Play, to test the app. Click on the app's window, it should write "Caught mouse event" to output. Try to press some keys and see the "Caught key event" messages in output. As you might have noticed, handling of key events is not possible before, you have clicked the window once. Often it is best, if key events are available, when the app starts. To fix this issue, we have to set the first responder of the window holding AppView, to AppView.

It is done by adding 

[self.window makeFirstResponder:self.ViewController.view];

At the end of the applicationDidFinishLaunching method in AppDelegate.m

So the AppDelegate.m looks like:
 //  
 // AppDelegate.m  
 // KeyAndMouseEventExample  
 //  
 // Created by Christian Henrik Reich on 01/01/14.  
 //  
 #import "AppDelegate.h"  
 #include "AppViewController.h"  
 @interface AppDelegate()  
 @property (nonatomic,strong) IBOutlet AppViewController *ViewController;  
 @end  
 @implementation AppDelegate  
 - (void)applicationDidFinishLaunching:(NSNotification *)aNotification  
 {  
   self.ViewController = [[AppViewController alloc] initWithNibName:@"AppViewController"bundle:nil];  
   [self.window.contentView addSubview:self.ViewController.view];  
   self.ViewController.view.frame = ((NSView*)self.window.contentView).bounds;  
   [self.window makeFirstResponder:self.ViewController.view];  
 }  
 @end  

That's it. For more about the NSResponderClass see https://developer.apple.com/library/mac/documentation/cocoa/reference/applicationkit/classes/NSResponder_Class/Reference/Reference.html#//apple_ref/occ/instm/NSResponder/keyDown:

Enjoy and happy developing.

Cheers,
Christian Henrik Reich