Description of test environment
Custom TDD for Javascript version of D/a/y

Project background

The D/a/y project aims to provide a specification for manipulating dates as we use them in the real world. For example May 2012 or 13 August which are perfectly valid dates you can't use with any standard date routines. If you're populating a database of authors and you have date fields for Birth, Death and winning the Pulitzer prize, then you need Unknown (Aesop), Not yet (J.K.Rowling),Never(Charlotte Bronte), and other hacks. Chaucer was born about 1343 (no month or day) and died 25 Oct 1400. Chaucer wasn't born on 1st January 1343.

As well as defining a specification for representation, interaction with legacy systems, internationalisation we need a definite specification for dealing with these strange objects. For example is March 14 'before' March?, What is January 31 plus one month?

So we have a lengthy and detailed specification which of course we need to implement. So we're set to write a load of tests with expected results which was exactly how I started. Back in 2008 I was using Delphi/Pascal which was a slightly different workflow to the Javascript version but the test scripts were still valid.


Actually it was perfectly possible to test against the raw code so long as this was
(a) browser-only
(b) development only,
by using lots of script includes in the html.
As you've probably guessed if there is to be a single deliverable package there needs to be some package-building automation. (Remember good old make?) As soon as the decision to automate was made it made sense to start testing against the deliverable rather than the raw source files. It also meant that the testing aspects could be switched-in or out as required.
So now the stage was set for 'some automation'. (This is spring 2015).

Modular testing architecture


The basic format for scripts is Specific test (arguments) = expected // comment
Typically there might be a hundred lines of plain text in a file.

Now write lots of lines for lots of tests and throw at some test runner. The runner is a web page that lists available scripts and gives some control over which lines to run. For example an issue might need a lot of instrumentation or debugging for a single line in isolation. Here is a simple method of automatically listing the available stript files.

Test methods

Originally I hardcoded the various special test methods into the main test module. This was fine when it was only me but if I'm to get others involved and confident, they must be able to do the simpler task of modifying some existing test method and then make it available. For example DAYTODATE might convert a DAY object into a javascript Date object. There's a whole lot of stuff going on in the background of the test runners but the code itself is d = new DAYo(TestData[1]); r = d.ToDate().ToISOString(); which is ideal for letting someone hack a simple bit of code.

The next evolutionary stage was to remove the hard-coded functions from the test runner code and create a number of js files which used a 'load me into a methods libary' function around the test method.

The problem of how to incorporate these files into the complete development package. This is what gulp is good for. As soon as a method file was changed gulp would notice and rebuild. This worked fine, but for any other tester they'd need to rebuild using the fiddly build infrastructure.

The example on the right shows that we've embedded some basic documentation which tells us how to make use of the method. (In case you think there's a lot of complexity involved, the test runner needed 10 lines to implement adding methods and producing a html table for reference. It's actually simpler than a large switch statement.)

The current system uses a similar file discovery process to that used for scripts. When initialising the test runner we look for js files in a designated methods directory and load them. eval() is ideal and not something to worry about in this context.


If anyone wants to alter or add scripts or methods the only thing they need is an editor. A typical use case is where a different internationalisation appendix is used which will need careful testing for user/string inputs.
Live script file discovery
// this gets the index page of the scripts directory $.get('tests/scripts',{},MakeScriptButtons,'html'); // and passes the html to the following function to give a list of // buttons with their file name on. function MakeScriptButtons(D,S,X){ var sfnames = D.match(/href="\w+\.testScript"/g); var buts =,I,A){ var sn = V.replace('href="','').replace('.testScript"',''); return '>span class="M1 button" id="' + sn + '" onClick="RunTest(this);"<' + sn + '>/span<'; }); $('#scriptButtons').html(buts.join('')); };

Example method file

DAYx.AddLibraryTest('DAYTODATE', "Does the DAY get converted to Javascript Date properly.", "DAY input str", "JS Date .toISOString()", function(TestData){ var jsd,r,d,d2,ok; try{ d = new DAYo(TestData[1]); jsd = d.ToDate(); r = jsd.ToISOString(); ok= StartsWith(TestData[2],r); }catch(e){ r = 'EXCEPTION:
' + DAYx._ErrorPlace(e); ok = false; } return [ok,TestData,r,null]; } );

Build automation

Extract from gulp file

It's actual javascript with replace, uglify, and concat being plug-ins loaded with require().

gulp.task('buildMinJs', function() { // register and label this function gulp.src(mainSources) // list of files/patterns to work with .pipe(replace('{{BUILDDATE}}', NowDate())) // pipe through a string replacer .pipe(uglify()) // minify .pipe(concat(minJs)) // stick together and name minJs .pipe(gulp.dest(packagePath)); // shove in the output path });
One of the key functions is setting it to watch for file changes then re-build what's required so there's no need to manually start a build after making edits.
The key to this is gulp.js which reacts to changes in source files then kicks off various processes to modify, merge and deliver. The basic model of gulp is convert a bunch of files in some way as required. It's really a framework where you plug-in the tools you want. The example fragment is a task I've called buildMinJs that is called by watchers set to keep an eye on any file changes.

Interactive test runner

There is a lot of complexity and detail in the 5000+ lines of code. There are many wrinkles and things that don't work or need a return to the drawing-board. This is where a test runner that can be set to run on a single 'hard-nut' script line is important.

The basic usage is to run a whole script file and home in on the 'red' lines. A point to watch is you need a class of script line that will always get executed. Suppose line 10 changes a configuration setting and line 15 is the one you're looking at, then you still have to process the lines before 15 so that permanent state changes are executed.

Reload the page dammit!

Bloody hell! How many times do I have to switch from editor to browser then reload. Not any more hurrah! It's a really simple trick.
Gulp fileTest console code
var flagTestbedReload = function() { require('fs').writeFile('tests/nonce.txt', new Date()); };

All we do is change a flag file when we've 'recompiled'. The browser checks every few seconds to see if it has changed, and if so saves any state to localStorage and reloads itself. On reload it will read the just saved state and it's like nothing has happened except now it's 100% up to date.

// set the watch for server-side changes (during page ready) __localNonce = undefined; window.setInterval(TestNonce,5000);
function TestNonce(){ $.ajax({url:'nonce.txt',cache:false,timeout:1000,dataType:'text', success : function(Data,S,X){ if(__localNonce){ if(__localNonce != Data){ // compare with previous // save any current state in localStorage (not shown) location.reload(true);} }else{ __localNonce = Data;} // set first time }});};

TODO Batch testing in node
TODO Trivia. (Colour bar) TODO Other tools (Function ref.) * file listing * function reference prog * interactive testing (browser) * batch testing (node) * trivia colour bar