Other types of testing

Some interesting other types of testing paradigms and approaches I came across, since the last year or so up to recently.

Mutation testing

Mutation testing helps you in guarding your code coverage. The goal here is for your tests to kill all mutants. A mutant, is a mutation to your code. If you have strictly specified the behavior of your code, any significant change should trigger a failing test. A failing test, kills a mutant.

This is a bit abstract, so here’s a pseudo code example:

// Some business logic
public bool Foo(string input)
  if(input == "some-condition")
    return true;

  return false;

// Test that specifies expected behavior of business logic
public void WhenFooGivenInputIsSomeConditionThenResultShouldBeTrue()
  var input = "some-condition";
  var result = Foo(input);
// Mutant in action
public bool Foo(string input)
  if(input == "some-condition")
    return false; // If statement inverted by mutant

  return true; // If statement inverted by mutant

// The test method above now fails, mutant succesfully killed

The changing of your code and checking that a test fails, is something that is gets handled by a framework. Unfortunately, I’m only aware of 1 active mutation testing framework for .NET, I admit this particular framework doesn’t convince me to start using it.

A user friendly mutation testing framework for .NET, would convince me to integrate it into a CI pipeline. Maybe only execute it in a nightly build, for critical components.


I first heard about Scientist.NET on Pluralsight, then later learned on .NET Rocks that Phil Haacked is the author that ported it to .NET.

The idea here is to test code refactorings in a live system with real data. So you would make a change, but still keep the original code in place. Then you call both the original and the new piece of code via Scientist (result of original is used, result of new is recorded for comparison and discarded). This goes into production.

Now Scientist collects the results of both pieces of code and after a period of time, you can validate that the refactored code returned the same results under real use and did not introduce any unexpected behavioral changes.

I was almost completely sold, until I understood that of course, you have to ensure there are no unwanted side effects. Since you executing the same action twice, albeit with different versions, it’s up to you not to duplicate the impact.

So this would work great for purely CPU bound calculations, but other than that you need to pay attention. Thing is, this is typically an approach I would want to use in legacy code, which are traditionally an entangled mess where database access is spread over long methods.

Property based testing

I heard this before but it didn’t stick. After a presentation on the subject it did. Here the approach is to describe the expected behavior as properties, instead of writing them out in tests.

The advantage is, this allows the computer to write the tests. The computer can then generate hundreds or thousands of test cases, that all validate that your business logic acts as you specified in the properties. Kind of reminds me of Pex, which became IntelliTest I guess.

But this post is getting way longer than I intended, so I suggest to check out the presentation for a proper introduction. From the three different approaches to testing here, this is the one I’m most interesting in pursuing. FsCheck seems like the framework to get started with in .NET, so gonna give that a try soon!

Theme music for this blog post

House Shoes w/guest mix by Kutmah – Magic
(scroll down on the page for the player or download if you prefer, no embed available this time)

Other types of testing

Late Pass

Some new old things I discovered last week, this is news to me!

Configure proxy in .NET app

It has been a while since I developed applications that had to deal with proxies, so only recently I learned you can just add to your .config:

 <defaultProxy useDefaultCredentials="true" />

Which will then use the system proxy settings.

“If the defaultProxy element is empty, the proxy settings from Internet Explorer will be used. This behavior is different from version 1.1 of the .NET Framework.”


Linq Aggregate

I forgot in which presentation I came across this, so I cannot give credit, it was something about functional programming in C#.

Anyway, check out this example.

How cool is that, it allows you to traverse a sequence while accumulating logic on it.
Linq’s Count, Min, Max and Avg are implementations of Aggregate.

Encrypt .config values

I learned this from K2 ‘s use of it, which uses encrypted connection strings in their .config files.

So apparently you can encrypt config sections with aspnet_regiis, which are automatically decrypted by ASP.NET at runtime when reading the values. Nooice.

Theme music for this blog post


Late Pass

Abstract Factory

Revisiting the GoF patterns with a good friend and learned more about the Abstract Factory pattern.

For some reason at jobs in the past, developers always talked about the “factory pattern” and I kind of thought I knew what they meant. But looking closer at the GoF patterns, there is the Abstract Factory and the Factory Method pattern.
And the things I come across with “factory” tacked at the end, are neither.

There is so much literature on this, so go ahead and refresh your memory, if you need to (search Abstract Factory and Factory Method)

In the wild I will usually come across a FooFactory with a static method CreateIFoo(string argument), encapsulating some basic decision logic on how to decide which subclass to return.

But the value of an Abstract Factory, being able to replace a family of related objects by other concrete classes, is never leveraged. Because OO design is hard and thus there hardly ever are any rich object models, where the opportunity and need for this would arise.

And an advantage of the Factory Method, letting you extend the supported types in a clean modular way, also isn’t leveraged. Because FooFactory is implemented as a monolithic base structure that contains all the logic to create any subclass.

Theme music for this blog post

Abstract Factory

QnA Maker

More bots, this is time they are answering your questions. I heard about this on .NET Rocks #1410.

With Qna Maker you can point a bot to a FAQ. The bot will then be able to understand questions from your FAQ in natural language, allowing it to response with a useful answer.

Try it out

WordPress doesn’t allow iframes, so I can’t embed the web version here, instead clik the link below:

Try out my QnA bot for Velo Antwerpen here

Look at the FAQ source to get a sense of which topics you can ask questions about.

I expected more from the natural language capabilities. But, maybe it’s also the poor English translations from the FAQ. I mean, “How to proceed to use Velo”. C’mon, nobody talks like that. Looks like they got their translations from a bot to begin with.

Still, lot of things happening around bots, at this pace we might have replicants among us before the new Blade Runner is out!

Under the hood

This is powered by Azure Bot Service, the QnA Maker is apparently just a template of this service. Since this all integrates with Bot Framework, the options to connect with other channels (Skype, SMS, Slack, …) are impressive.

Theme music for this blog post

QnA Maker

The bots are running the Internet

Two inspiring applications of natural language processing I came across recently, the authors blogged and open sourced it, so jump to their posts for the full report.

Making Your Music Choose Your Desktop Wallpaper, his approach summarized:

  1. Read song title and artist
    1. That’s straightforward
  2. Get lyrics for song
    1. Ofcourse, I’m not surprised you can find the lyrics of songs on the Internet
  3. Analyze sentiment of lyrics
    1. Smart, I wasn’t aware of IBM Watson’s Alchemy Language
  4. Search Flickr images with keywords from sentiment analysis and set desktop wallpaper
    1. Nice, this ties it all together!

And This Machine Turns President Tweets into Planned Parenthood Donations, in short:

  1. Follow President‘s tweets
  2. When a publicly traded company is mentioned, analyze sentiment of the tweet
    1. This uses Google’s Natural Language API
  3. Depending on sentiment, trade stock

Automation in stock trading is not new, but I thought it was a smart demo, to do it based on the statements of a person that influences a lot of people’s opinions and actions.

Theme music for this blog post

The bots are running the Internet

Travel from your desk like this

I’m not a digital nomad, but spend my days stuck at a desk. So as the second best thing, I pretend that I’m out there traveling and developing remotely, by setting the desktop wallpaper of my second screen to some exotic place.


Seriously though, this also helps great in reducing stress on the job, by giving you a philosophical perspective on things that stops you from worrying about unimportant things, try it!

Anyway, I like reddit’s EarthPorn which I use regularly for downloading wallpapers. Looking to automate this task, I was surprised I had a hard time finding something for Windows.

There is a workaround to have Windows use an RSS feed to update your wallpaper, but it expects it in a different format than reddit provides.

So I quickly threw something together and a colleague improved it with a UI.
So here it is as simple as it can be, it sits in your system tray and regularly updates your wallpaper:


Enjoy and pretend to be a digital nomad for a day 😉

Theme music for this blog post


Travel from your desk like this