Thursday, March 19, 2015

A generic JsonConverter for Dictionaries and JSON.NET

JSON.NET is a great tool and it already handles Dictionaries well, unless your keys are classes (as opposed to structs). Say you have the following class you want to serialize:
public class DataPoints
{
    public IDictionary<DataPointKey, int> Values { get; set; }
}
And the DataPointKey class:
public class DataPointKey
{
    public int Length { get; set; }
    public int Height { get; set; }
}
I'm just making stuff up here, don't mind the actual contents of the class. The important thing is that the key of our Dictionary is a class.

When you let JSON.NET serialize this, you will get the following:
{"Values":{"JsonDictionary.DataPointKey":200}}
Notice how the key is just the class name. This will not deserialize nicely.

In the past, I would create a custom JsonConverter, but this meant repeating code over and over when new keys were introduced. Inheriting from a base converter was my first solution, but I've finally found a converter that can handle any Type.

The result is a little more verbose, but serializes and deserializes nicely:
{"Values":[{"Key":{"Length":10,"Height":20},"Value":200}]}}
The entire code is a little too long to post here, but I've made a Gist you can check out.

Sunday, March 8, 2015

International Women's Day: women in tech

Contrary to what you might think today, the worldwide community of technology professionals (data entry, programming, etc etc) used to be predominantly female. In fact, our industry was built upon the hard work of women (there are more and longer articles on this subject, just google around).

And yet now, I haven't had a lot of female colleagues. When I'm at a tech show or conference, there are hardly any women. There is a slowly growing feeling that this has to change, along with all the sexist practices (booth babes, etc) and incidents that have happened in the past and are still happening today.

I'm confident any industry only has to gain with a more diverse composition. And because it's International Women's Day*, I thought I'd list some great female techies I follow and look up to.

Hopefully, this can
  • convince older female tech-professionals they're not alone
  • convince younger female tech-enthusiasts that a career in the tech industry is possible and worthwhile
  • convince the macho male techies out there to change their attitude
Here goes:
  • Leslie Cottenj√©: owner of Pureplexity, a startup from Bruges, my home town; I've seen her do a great session on Metro apps and the work her agency delivers is fabulous.
  • Iris Classon: possibly the wackiest and most energetic programmer out there; she went from no-programmer to great-programmer in a time I envy.
  • Kelly Sommers: more often than not, I have no idea what she's tweeting about because her work is so technically challenging (think big data and crazy algorithmic stuff).
  • Sheree Atcheson: founder of WomenWhoCode UK, actively encouraging more women to become coders.
  • Mary Jo Foley: not a coder, but a highly regarded tech journalist, mainly covering Microsoft. She's also on the (recommended) Windows Weekly podcast and a beer-enthusiast, which, as a Belgian and beer-lover myself, I can only encourage.
  • Jenn Lukas: a great front-end developer and co-host of the Ladies In Tech podcast.
  • Val Head: the other co-host of the Ladies In Tech podcast, a CSS lover and organizer of Web Design Day.
  • Kathleen Dollard: MVP, Pluralsight and Wintellect author, featured on DotNetRocks.
  • Suz Hinton: programmer, IOT enthusiast, coined the fabulous term 'Nerdiverse' (as far as I know)
This is just a short list of the great women in our small tech-world. Follow the links and you will definitely find even more women who are active and successful in the tech industry.

Also, if you have a daughter and want to get her started, check out GoldieBlox :)

And for the book-lover, O'Reilly recently published Women in Data

Finally, to my female colleagues and friends: happy International Women's Day!

Update: I was pointed to a great article by Uncle Bob on the subject. A must-read.

*If you think (like I've heard some people say) that International Women's Day is no longer necessary, you're not following the news. If you think it's no longer necessary in our Western world, you don't know the tech industry or haven't been paying attention (just two links I found while googling "sexism in tech industry".

Friday, March 6, 2015

Comparing a local file with a workspace file with C# and TFS

The situation is simple:
  • we need to send messages to another system
  • the developers of the other system need documentation on the layout of these messages
  • this is a company-specific protocol (no JSON or anything standard, because we're talking to old Fortran machines here)
  • naturally, the documentation is made in Word files
Taking an example from a session I saw at ngEurope, I decided we might be better off auto-generating this documentation. Keeping Word files in sync with the evolving code is a tedious, manual process, not to mention that the Word files are never in sync 10 years down the road.

And as the code is the single point of truth, I started on automating the process. I hate repeating manual things. It's boring, allows for manual error (I remember updating a production SQL database instead of the beta database!), and needs to be taught to newcomers time and again.

So:


After having written code to generate the documentation (too specific to put here), I needed a way of having the build automatically add the new file to the pending changes, or if nothing had changed, undo the checkout (sadly, I'm on TFS; git would not have this problem).

So this is how you can compare a local file with the server version of the file:
private bool HasFileChanged(string file, TfsTeamProjectCollection server, Workspace workspace)
{
    string diff;

    var versionControlServer = server.GetService<VersionControlServer>();

    var serverPath = workspace.GetServerItemForLocalItem(file);
    var serverVersion = new DiffItemVersionedFile(versionControlServer, serverPath, VersionSpec.Latest);
    var localVersion = new DiffItemLocalFile(file, Encoding.UTF8.CodePage, DateTime.Now, false);

    using (var stream = new MemoryStream())
    using (var writer = new StreamWriter(stream))
    {
        var diffOptions = new DiffOptions
        {
            Flags = DiffOptionFlags.EnablePreambleHandling,
            OutputType = DiffOutputType.Unified,
            TargetEncoding = Encoding.UTF8,
            SourceEncoding = Encoding.UTF8,
            StreamWriter = writer
        };

        Difference.DiffFiles(versionControlServer, serverVersion, localVersion, diffOptions, serverPath, true);
        writer.Flush();

        diff = Encoding.UTF8.GetString(stream.ToArray());
    }

    return diff != "";
}
The server and workspace variables can be made like this:
var workspaceInfo = Workstation.Current.GetLocalWorkspaceInfo(file);
var server = new TfsTeamProjectCollection(workspaceInfo.ServerUri);
var workspace = workspaceInfo.GetWorkspace(server);

Friday, February 6, 2015

Avoiding the information madness

A while ago, a colleague asked how I tracked all the information that's coming in: Twitter, RSS, Facebook, blogs, podcasts,... It's easy to drown in all the information if you want. I won't pretend I have the ultimate strategy, but here's what I do.

Facebook

I'm not on Facebook for a variety of reasons (mainly this), but I don't miss it. So that's one information-source less to deal with.

Twitter

This has been my usage of Twitter:
  • Ignore the hype
  • Check out the hype, follow a handful of people
  • Love it and check it everyday picking up where I left of the day before
  • Start following more people; create lists
  • Whoa, too much! Just check it now and then to pick up anything interesting that was posted that last hour or so. Plus, use it when I write a blog post (self-promotion yay!) 
So I gave up any hope of keeping up with Twitter. Twitter is like entering a newspaper-shop. There's all sorts of magazines, newspapers, books, comics, etc. And you can't go there everyday and read it all. Just go there when you feel like it and read some stuff, check out any favorites, then move on.

RSS

Since Google Reader was discontinued, I briefly used The Old Reader, but after a while, I switched to Feedly. I can't really remember why I made the switch. Possibly because the Old Reader was experiencing load problems at the time (because thousands of developers were moving from Google Reader to the Old Reader). I hear it no longer has these problems, but in the mean time, I'm happy at Feedly.

I use NextGen Reader on my devices to connect to my Feedly. Great app.

Podcasts 

After following blogs for a while you collect a large amount of blogs to follow and read. Lately, I notice I don't take as much time to read them as I used to. Also, reading blogs is subject to ups and downs based on how busy work is, how busy home is, how I feel, etc.

Podcasts are my new information source. While you can't follow as many podcasts as you do blogs, the good ones provide a wide range of information. My commute is about 45 minutes which is perfect to follow several podcasts throughout the week.

Here's my selection, in no particular order:
There are definitely episodes that I skip, but not all too often. Here, again, you have to be able to skip content without feeling bad or getting the feeling you'll be missing life-or-death-crucial stuff.

Must read but have no time 

A final tip is to have a way of setting things aside for later. I use Pocket, but there's also Instapaper, and possibly others. I find it's easier to add stuff than it is to actually read any of it later, but now and then I do take the time to catch up.

Conclusion

It is hard/impossible to keep up with everything in our fastly-evolving world. And that is exactly why you shouldn't try. Just have a steady stream of information via the channels you are most comfortable with. You might miss out on the most minute details, but anything remotely important will be repeated by so many sources, you can't miss out.

Thursday, January 29, 2015

Wrapping an old-school callback-javascript method in a promise

When you have an older javascript library that supports asynchronous calls via callbacks, you might want to wrap them in a promise instead. Particularly if you're using Angular. For my own reference, this is how to do it. I'm using Angular's $q, but I suspect it should be more or less the same with Q.

Let's assume this is the method with the callbacks:

function goAsync(input, success, error) {
    try {
        // do something here
        success(result);
    } catch {
        error();
    }
}

Wrapping this in a promise is quite easy:

promiseObject.go = function(input) {
    var deferred = $q.defer();
    oldSchoolObject.goAsync(
        input,
        function(result) {
            deferred.resolve(result); 
            $rootScope.$apply();
        },
        function(e) {
            deferred.reject(e); 
            $rootScope.$apply();
        }

    return deferred.promise;
}

Notice how I call $rootScope.$apply after resolving or rejecting the deferred. This is so Angular starts its digest-cycle and the UI is updated.

Now you can use the promise:

promiseObject.go(input)
    .then(function() {})
    .catch(function() {});

You can see a live example of this in one of my GitHub repositories.

Friday, January 16, 2015

Synchronizing multithreaded incoming messages and unit testing

I'm currently developing a Windows service application that receives data from multiple sources. It isn't a highly concurrent application, but the incoming messages are come fairly fast after one another.

Incoming messages enter the system via WCF and that part is multithreaded. But these messages must be handled sequentially. What's more, certain pieces of our application must run at regular intervals.

We can experience problems if component A is running and changing objects in memory, and component B is triggered and starts using these same objects.

That's why we introduced a simple command queue:

public interface ICommandQueue : IEnumerable<Action>
{
    void Add(Action action);
    void CompleteAdding();
}


The implementation uses a BlockingCollection:

public class ThreadSafeCommandQueue : ICommandQueue
{
    private readonly BlockingCollection<Action> _actions = new BlockingCollection<Action>();

    public void Add(Action action)
    {
        _actions.Add(action);
    }

    public IEnumerator<Action> GetEnumerator()
    {
        return _actions.GetConsumingEnumerable().GetEnumerator();
    }

    IEnumerator IEnumerable.GetEnumerator()
    {
        return GetEnumerator();
    }

    public void CompleteAdding()
    {
        _actions.CompleteAdding();
    }
}

This means another class can loop over it and execute every action that is added. If the queue is empty, it will wait until a new item is added. You can read more on BlockingCollections for the ins and outs.

This is our implementation:

Task.Factory.StartNew(() =>
{
    foreach (var action in _queue)
    {
        action();
    }
}, TaskCreationOptions.LongRunning);

So, what happens is we add stuff that needs to be executed safely to this command queue:

_commandQueue.Add(() =>
{
    _something.Do(_input);
});

As you can see, you have access to private fields, as you're just passing in the entire Action to be performed.

Now the cool thing is this is entirely unit-testable. Because we're using a DI-container (Autofac in our case), we can inject a different implementation when we're testing. Because we don't want our Assert methods to start asserting if the Action hasn't been executed yet, we can inject the following when we're testing:

public class ImmediateCommandQueue : ICommandQueue
{
    public IEnumerator<action> GetEnumerator()
    {
        throw new NotImplementedException();
    }

    IEnumerator IEnumerable.GetEnumerator()
    {
        return GetEnumerator();
    }

    public void Add(Action action)
    {
        action();
    }

    public void CompleteAdding()
    {
        throw new NotImplementedException();
    }
}

When an Action is added, it is immediately executed. So when we've done the 'Act'-part of our tests (see AAA), we can be sure there aren't any Actions still waiting to be executed.

But what this also allows us to do is control when the Actions are executed. Take this method:

public void Process()
{
    _a = new List<int> { 1, 2, 3 };
    _commandQueue.Add(() => { _innerProcessor.Process(_list); });
    _list.Clear();
}

This is of course a silly method, but bear with me. The point is that, at runtime, the innerProcessor might receive en empty list because we're clearing it immediately after we added the Action. But in our tests, this isn't very clear because we're immediately executing the Action.

So we can introduce yet another implementation of ICommandQueue, purely for testing purposes:

public class ManuallyTriggeredCommandQueue : ICommandQueue
{
    private readonly IList<Action> _actions = new List<Action>();

    public IEnumerator<Action> GetEnumerator()
    {
        throw new NotImplementedException();
    }

    IEnumerator IEnumerable.GetEnumerator()
    {
        return GetEnumerator();
    }

    public void Add(Action action)
    {
        _actions.Add(action);
    }

    public void CompleteAdding()
    {
    }

    public void ExecuteNextAction()
    {
        var action = _actions[0];
        _actions.RemoveAt(0);
        action();
    }
}

Now, in our tests, we can call the Process method first, and only then execute the Action by calling ExecuteNextAction. This way we can get a failing test (because _innerProcessor.Process was called with an empty list instead of a list with 3 integers). And after a failing test, we can turn it into a green test:

public void Process()
{
    _a = new List<int> { 1, 2, 3 };
    _commandQueue.Add(() => 
    { 
        _innerProcessor.Process(_list);
        _list.Clear();
    });
}

I'm very happy with this solution. It's geared towards our domain, as we're not going to flood our queue because we won't be getting thousands of messages every millisecond. But it allows us to avoid threads getting in each others way, and our code remains testable.

Wednesday, December 31, 2014

2014 is so 2014

Another year has past, and I suspect many bloggers will be making lists of what happened in 2014 and what will happen in 2015. And so will I, but I'll keep it short.

These lists are actually quite handy, because they allow you to keep track of what your goals are and whether or not you met them. Depending on your situation, you might decide you slacked too much, or maybe you had too many goals. Or unrealistic ones. Or maybe you met all your goals and in that case, good for you!

Revisiting my post from the previous year, I can list what I achieved and what I didn't.

Didn't achieve:
  • Acquire 2 Microsoft certifications: this is a subject for a future post, but those who know me, know I'm in the non-believer-camp regarding certifications.
  • Finetune pwa.js, FlitsLogo: these are still on the mental to-do-list, but aren't very high in priority.
  • Migrate my blog: still very much want to do this.
  • Finish my course on beer: I've actually read and studied the material, but I still need to plan the exam.
  • Kitesurfing: oh well :)

Achieved:
  • Find and follow a good training: I attended ngEurope this year where I really learnt a lot. Also Techorama, but ngEurope was more interesting for me because it's outside of the traditional Microsoft-world where I'm in most of the time. (Though I will most likely be attending Techorama again this year)
  • Write a Drupal module: I did write a Drupal module for my frisbee team. Actually, I wrote several custom modules. There's still a lot of work to be done there, but I'm happy with what I have already. Also, it's already being used on a site with 140+ users.
  • Write a SPA: I wrote a web application to have a small contest with some friends during the World Cup. The code isn't pretty, but it worked and wasn't meant to be a long-term project anyway. But I did learn Angular that way.
Didn't plan, but achieved:
  • Start my own company: I am now a freelance developer, so that's exciting. More on that later.
  • Learnt Cordova: but I still regard myself as a beginner.
 And finally, plans for 2015:
  • Finish the Cordova mobile app I'm writing for my frisbee team
  • Push out some changes I've made to my 8cam Windows Phone app.
  • Maybe: rewrite FlitsLogo in HTML5 and javascript (using Cordova) so I can put it on multiple platforms
This list is definitely shorter than my 2014-list. But with starting my own company I suspect there might be a little less time, plus there are also personal/private goals I have set. And I tend to avoid putting too many private stuff on the internet.

So there is my list for 2015. Let's hope I can achieve these things.

And with that, I wish you a great 2015 and good luck with your goals!