Testing is Developing

Developers do not like Testers. Testers do not like Developers.
I am a developer and I love testers or more precisely I love testing. I believe testing require two distinct and important skills: The first one is the ability to know what to test. The second is writing the tests. Writing tests is writing code and writing code is a developer skill. That is why I believe a tester should actually be a developer with additional skills (Testers are a superset of Developers). To make testing efficient and useful, it is important to know what not to test and how to write the tests well.

What (not) to test

When a test is executed quickly and automatically, temptation to multiply the number of configurations is great. The configuration matrix does grow exponentially and it is important to know when to stop. Suppose you have an application with the following configuration matrix:

  • Locales: English, French, Russian, Italian
  • Platforms: Windows 7, iPhone, Android
  • Resolutions: 320x480, 640x480, 800x600, 1280x1024, 1680x1050
  • Modes: offline, online, real-time

You could run the tests 4 times 3 times 5 times 3 times (180 times) but you will hit diminishing return quickly, probably after 3 or 4 configurations. If you were to display the code coverage (or whatever metrics) after each run, this is how it would look like:

Software I use

Here's some self promotion. Nothing is better than eating your own dog food so below is 100 software I use personally, at work or for www.romku.com and I have generated this image with romku of course.

What are your favourite software? If you register at romku (it's free) and mark the software you like, use, or want to follow, you'll get a monthly newsletter containing releases update and other news (more information).

Following news on a given website (hackernews, techcrunch,...) is great, but it's easy to get overwhelmed by the volume of news and more importantly it's easy to miss the news about the software you care.

Instead of adding RSS feeds for all the software you use (when there is one), romku send you a monthly email with all the news related to your favourite software. And if facebook, jquery, django, or mongodb are not in your list, then you will not be inundated with news about them (they are great software but can take a good proportion of your daily intake of news).

By the way, how many of the 100 software above can you recognise? All of them? Are you sure?

Don't stop here, come and join romku now.

Do we need and did we ask for so much choice?

There are only 5 main browsers: Internet Explorer, Firefox, Chrome, Safari, and Opera. Between them, they take 99.5% of the market and you don't even have to choose as you can install multiple browsers on the same hardware.

When it comes to most consumer goods, virtual or not, I feel there is way too much choice. For instance, there are more than 2000 distinct models of digital camera available on sales. Choosing one becomes a lot harder and random filters are usually applied. For instance, I usually focus only on Canon because of good experience from previous owned camera, I don't even look at Sony because of the memory stick format that would force me to put all the SD card I have in the bin (which no longer applies anyway as most of their products support both Memory Stick and Secure Digital format). I could be missing out on a great Samsung camera but I'll never know.

The App Store is another example with over half a million of applications. Even spending a split one second on one screenshot per app would requires 5 full days and again I'd be missing out on the not so good looking app that could increase my productivity or entertain me for a while.

Contest: Win a £100 Amazon Gift Card

To enter the prize draw, simply follow us on Twitter and re-tweet this contest, register on Romku and select your top 10 software in your favourites, or take a small quiz.

My weekend project: What can you really do in a weekend?

Several headlines on HackerNews (HN) have caught my attention recently regarding weekend project. Here're some examples:

The last one is what made me write this post. Can you really build a website in 15 minutes? The purpose of the projects mentioned is not of importance and I am not trying to criticise the work accomplished. What I am curious about is did it really take 15 minutes or even a weekend to create these projects?

It takes months or years to learn programming language, databases, networking... Frameworks, apis, and modern programming language makes it easy to develop but you still need to learn about them. It may have taken just the weekend to build the project but it doesn't include all the initial learning. That is my problem. It makes it sound so easy to develop an app or a website when actually it requires a lot of effort.

Faceted Search with Apache Solr

Apache solr is an open source search platform from the Apache Software foundation project. I installedit on my ubuntu and followed the introductory tutorial. I often spend an hour to read/install/play with new software even when their publishers claim you only need 5 minutes... That was not the case with solr, the tutorial is easy to follow and in minutes I was able to see the power of the platform.

The next step was to integrate it with drupal (that's the Content Management Framework we use for romku). Fortunately, a drupal module (apachesolr) exists and provides the heavy lifting instantly. I did not need to do much to set it up. Next was to provide faceted search: the ability to filter by OS/Platform (Windows, Linux, Mac, Mobile, Online...)

Popularity vs Rating: How to rank content accurately?

It is nearly impossible to organise a vast amount of data without some kind of sorting, scoring, ranking or rating. Websites commonly use relevance, date, popularity, download, sponsorship,... All the methods and algorithms are biased in one way or another. For instance the more popular a video becomes on youtube, the more it is viewed and the more it is viewed, the more it continues to be popular. It does not necessary means the clip was good at the first place. The same pattern emerges with download websites and e-commerce website where most downloaded software and best sellers products respectively are promoted on the front page and at the top of searches.

Is there an ultimate algorithm that can organize the data in an unbiased fashion? Of course not. Depending on the objective of the classification - for instance, promote best sellers to increase the volume of sales, or promote products with bigger margin - a multitude of techniques can be used, which I briefly describe here:

Popularity, Number of Downloads or Viewing
Popularity is usually inferred from other metrics such as the number of downloads or viewing. The more a page is viewed, the more the page is prominent and therefore the more it is viewed. Viral Phenomena are mostly due to this type of ranking. To me, this is the worst type of ranking.