Monday, May 31, 2010

Cross-browser compatibility

My first true deep dive into heavy JavaScript programming was working on a project that required adding Firefox support for a web application that only targeted Internet Explorer. The web application was originally written for IE6 and therefore contained some hairy JavaScript.

I experienced firsthand the same rite of passage as millions of web developers around the world. Fortunately, adding support for Firefox also meant that the web app no longer needed to support IE6 (only IE7+) so fixing the JavaScript to be cross-browser compatible became a bit more reasonable and sane.

Now, some snippets collected for these changes in no particular order:

# 1

Replacing all references to the DOM Level 1 function
top.frames['myId']
, which gave Firefox much trouble, with the DOM Level 2 function:
document.getByElementId('myId')


# 2

Replace the function 'removeNode':
if (x)
{
    x.removeNode();
}
with a conditional check that uses the Firefox friendly function 'removeChild' if parentNode is defined (otherwise fallback on 'removeNode'):
if (x)
{                  
    if (x.parentNode) 
        x.parentNode.removeChild(x);  // Firefox                  
    else
        x.removeNode(); // IE
}

# 3

The 'innerText' property does not work in Firefox but it does in IE. Instead, Firefox does recognize 'textContent' serving the same purpose. Again, use another if-else statement checking if the target element exists:
function setText(elem, textValue)
{
    if (elem.textContent || elem.textContent == "")
    {
         elem.textContent = textValue; // Firefox
    }
    else 
    {
         elem.innerText = textValue; // IE
    }
}

# 4

Dot notation for defining functions is another area where JavaScript errors emerge in Firefox:

"...missing ( before formal parameters..."
function window.onDoSomething()
{
 // do some stuff
}
To fix is to swap around the 'function' keyword:
window.onDoSomething = function()
{
 // do some stuff
}

# 5

One of the web pages had a character that used the Webdings (TrueType dingbat) font for an expressive, functional symbol. It rendered incorrectly (and confusingly) in Firefox. Substituting the equivalent Unicode character resolved the discrepancy.


# 6

All AJAX calls involved the following IE6 object:
var xmlHttp = new ActiveXObject('Microsoft.XMLHTTP');
As mentioned, with no need for supporting IE6 (something web developers dream of someday being true for all the of internet) every instance of the previous line of code is fully replaced with:
var xmlHttp = new XMLHttpRequest();
Certainly, one of the more satisfying cross-browser changes.


# 7

Firefox does not support referencing global event objects specifically 'window.event' and when encounters JavaScript code that attempts to do so it responds with this error message:

window.event is not defined.

Instead, it is necessary to pass the event object as an argument via a function's parameter:

    function myFunction(e) // <-- add 'e' as a parameter for the global event object
    {
        if(!e) e = window.event // if e is undefined then set e using IE event object
        // other code
    }

    <button onclick="myFunction(event);">test events</button>

# 8

A web page had a file browser functionality to attach a file for upload to the server. The 'onclick' event handler for this element's tag and type
<input type="file" ...
was coded to be programmatically triggered. The reason for this was to allow for the user to type in and edit the free form text of the file path and then the JavaScript would create the 'input' and then fire the event on the user's behalf.

Firefox does not allow this requiring the user to manually click on the tag since the file path text is read-only and can not be edited. It is considered a potential security flaw hence the restriction. The code needed to be rewritten to have the user directly fire the event and open the file browser.


# 9

An html table on a page contained rows with hidden nested rows functioning as a tree-like grid. These top level rows when clicked toggled between the style of
display:none
when hiding its children rows and then used
display:block 
when showing the rows.

In Firefox, the rows do not align properly when made visible with 'block' display style. The premature solution was to replace 'block' with
display:table-row
which worked for both IE8 and Firefox.

However, I later discovered that this type of display was not supported in IE7. Instead, substituted the equivalent of no display style at all using empty string
rowObject.style.display = ''
to show hidden rows. Apparently, each browser knows by default how to appropriately render the rows without the need to be specific in the html.


# 10

Consider a deeply nested event firing and then needing to prevent it from triggering other event handlers further up the DOM hierarchy. In IE,
window.event.cancelbubble
should take care of this. Firefox, of course, does not recognize this command. Instead, one must use the
event.stopPropagation
function to exercise the same control over the scope of an event.

To ensure coverage across the different browsers, do this:
function doSomething(e)
{
 if (!e) var e = window.event;
 e.cancelBubble = true;
 if (e.stopPropagation) e.stopPropagation();
}

# 11

Some JavaScript code was not executing at all. No clear indications why the function was not defined. It turns out that using the term 'jscript' as part of the 'type' attribute in the 'script' tag:
<script language="javascript" type="text/jscript" ...
is , not surprisingly, recognized only in IE and not in Firefox. All instances of 'jscript' were replaced with 'javascript':
<script type="text/javascript" ...
This cross-browser issue caused so much grief for such a simple fix. I spent way too much time figuring it out. When I read:

"...Nevermind, I think I found it. I inherited the code and just noticed that the original programmer had specified JScript rather than Javascript as the script language..."

I glanced over to my aspx page and my eyes immediately saw that exact error. Unbelievable.


# 12

Another function not defined in Firefox but used in IE:
window.attachEvent
In Firefox, use this instead:
window.addEventListener
The cross-browser code might look like this:
eventName = 'load';

if (window.addEventListener) // Firefox
{
  window.addEventListener(eventName, myFunction, false);
} 
else if (window.attachEvent) // IE
{
  window.attachEvent('on' + eventName, myFunction);
}
(Note that IE requires the prefix "on" for the event name while Firefox does not.)

All of the above applies to IE's
window.detachEvent
For Firefox, use
window.removeEventListener


# 13

Some image icons when hovering over with the mouse were expected to show tooltip text. However, no tooltips shown in Firefox with the following:
<input type="image" disabled="disabled" text="Hi there"...
Instead, replaced the 'input' element tag with an 'img' element:
<img text="Hi there" src="disabled.gif"

# 14

An html table needed to be dynamically resized by changing it's style. The original IE-only code:
tableObject.style.left=400
    tableObject.style.top=400
No effect in Firefox (the size remained the same), so needed to explicitly add the unit of measurement "px":
tableObject.style.left=400 + "px"
    tableObject.style.top=400 + "px"
Also applies to 'height' and 'width':
tableObject.style.height="55px"
    tableObject.style.width="33px"


# 15
Dynamically adding some new html into a page relied on 'insertAdjacentHtml':
document.body.insertAdjacentHTML('AfterBegin', '<div>foo</div>')
Worthless in Firefox (at least until HTML5 is supported) so fall back on 'insertBefore':
elementHtml = '<div>foo</div>';

if (document.body.insertAdjacentHTML)
{
     document.body.insertAdjacentHTML('AfterBegin', elementHtml)
}
else
{
    element = document.createElement("div");
    element.innerHTML = elementHtml;
    document.body.parentNode.insertBefore(elem, document.body);    
}
(Orignally, used
document.body.insertBefore(element, document.body.childNodes[0])
but that seemed to cause the event (specifically the 'onload' event of an image) to fire repeatedly in Firefox so changed it to use the one listed above.)


# 16

The mouse's position was necessary to figure out where to render a dynamically injected image. In IE, to determine the X and Y coordinates relative to the web page document:
window.event.clientX + document.body.scrollLeft + document.documentElement.scrollLeft
    window.event.clientY + document.body.scrollTop + document.documentElement.scrollTop
Functions 'client<X|Y>' tell you the "viewport" position of the mouse which is a smaller, overlapping portion of the entire document but not a true subset of it. To obtain the document's actual mouse position, you need to add to these position values the scroll values by using the other functions shown above. Specifically,
document.body.scroll<Left|Top>
are the older (i.e. quirksmode) DOM syntax to retrieve the scroll values while
document.documentElement.scroll<Left|Top>
are the more modern standard approach. Depending on the browser only one of these will have the actual value while the other will equal zero. Therefore, it's safer and relatively harmless to include both.

In stark contrast, Firefox simply uses these:
e.pageX
    e.pageY
For a comprehensive code snippet that works across most major modern browsers:
function doSomething(e) {
 var posx = 0;
 var posy = 0;
 if (!e) var e = window.event;
 if (e.pageX || e.pageY)  {
  posx = e.pageX;
  posy = e.pageY;
 }
 else if (e.clientX || e.clientY)  {
  posx = e.clientX + document.body.scrollLeft
   + document.documentElement.scrollLeft;
  posy = e.clientY + document.body.scrollTop
   + document.documentElement.scrollTop;
 }
 // posx and posy contain the mouse position relative to the document
 // Do something with this information
}


# 17

To cancel an event just for the local scope only but not stop the event from bubbling up to the rest of DOM tree, in IE set
e.returnValue
to false.

For Firefox, use:
e.preventDefault
Cross-browser function:
if(e.preventDefault)
  {
    e.preventDefault();    // Firefox
  } 
  else
  {
    e.returnValue = false;    // IE
  }

One last (thought) snippet

While extremely beneficial being exposed to JavaScript's historical client-side scripting messiness in different browsers, next time when faced with cross-browser quirkiness I'd use a library like jQuery for simplified and easier web development.

Friday, April 30, 2010

Running Mono C# REPL on Windows

Being a big fan of programming language REPLs and command-line consoles, I have awaited one for C# that I can easily install and comfortably use for lightweight validation of C# syntax and for exploring its language features. Originally, Snippet Compiler had filled that role and then replaced with LINQPad but neither one is a REPL.

Fortunately, Mono, the open source project for .NET and C# that offers cross OS platform support especially for Linux and Mac, does have an interactive shell (CsharpRepl) for evaluating C# statements and expressions. Since Mono also runs on Windows, I downloaded and installed the latest Mono release 2.6 on my Windows development machine.

Once installed, to run the C# REPL simply:

  1. Select Start ->; All Programs -> Mono 2.6.1 for Windows -> Mono-2.6.1 Command Prompt
  2. At the command prompt, type "csharp"

The first step just adds Mono's bin directory to your environment PATH for the current shell session. (Alternatively, you can navigate via the command line to the Mono bin directory and directly run the file, csharp.bat.)

The C# REPL works if used within the Windows shell, cmd.exe (a.k.a. "command prompt") but with some caveats. The first immediate one is the command prompt text "csharp >" is not displayed making it a bit disorienting to use. It's difficult to distinguish between the input and the output of your expressions. [Update: This bug was subsequently fixed and it is now available with the version 2.6.7 release.]

Another is no autocomplete functionality but this feature is only available in Mono's GUI command console, 'gsharp'. GSharp requires an additional install of the mono-tools package available on the Windows platform download page under the link named "Gtk# for .NET". Once installed, to launch gsharp:

  1. Start -> All Programs -> Mono 2.6.1 for Windows -> Mono-2.6.1 Command Prompt
  2. c:\> gsharp

To activate autocomplete, type in part of a word and then hit <TAB>. Sometimes auto-completing words is slow in gsharp particularly if it has to search a large set of libraries. For example, typing the text "using Sys" and then <TAB> causes hanging since 'System' is the most top level .NET lib.

I decided to stick with using gsharp (a.k.a. the "C# InteractiveBase Shell"?) over csharp plus cmd.exe combo not only because of the autocomplete feature but because it also does display a "csharp >" prompt.

With those two issues resolved in gsharp, I continued to explore how the csharp REPL performed and behaved. The most striking deficiency I then encountered was when typing a statement containing invalid syntax, it did not show any output results. This was surprising since it goes against what I consider to be one of the hallmarks of a good REPL: immediate feedback not just on command/code that evaluated successfully but on things that failed to evaluate properly. Strange the lack of output...almost as if it was missing the 'Print' in REPL.

With continued use, I noticed in the examples found in the Mono REPL documentation that each expression or statement requires the ";" character at the end of it to produce any visible output. Otherwise, it gets ignored. I was expecting the same behavior found in the Visual Studio's Immediate Window where ";" is not always required. Not sure what the advantage of having to always type ";" (other than as a constant reminder that you are using a C based language in a REPL). Just seems like an extra unneeded keystroke.

However, reading more of the REPL docs, it implies multiple declarations can be made on a single line using ";" as a delimiter:

csharp> var a  = "why so many semi-colons?"; 5; "more stuff!";
"more stuff!"
csharp> a;
"why so many semi-colons?"
csharp>

All three statements get evaluated but only the last one ("more stuff!") is printed to the screen.

The docs also explicitly state the inverse: one declaration ending with ";" can be spread across multiple lines:
"...Statements and expression can take multiple lines, for example, consider this LINQ query that displays all the files modified in the /etc directory in the last week. The prompt changes from "csharp" to " >" to indicate that new input is expected..."
and
"...Multi-line input...If your code does not fit in a single line, you can enter expressions in multiple lines. The shell will not execute the code until a valid expression has been entered or a syntax error is flagged. A special prompt is shown to indicate that ics is waiting for input..."

Although the docs states that multi-line input is supported, it does not seem to be true of my current install on Windows. On Linux, I can do this:

csharp> var list = new int [] {1,2,3};
csharp> var b = from x in list
   >    where x > 1
   >    select x;
csharp> b;

On Windows, however, the special indented continuation prompt "  >" never appears when a new line is returned if the ";" character is not included. This is unfortunately a notable flaw in the REPL tool on the Windows platform.

In addition to the online documentation, another source describing the more common commands is available in the REPL itself. Just type "help;":
"Static methods:
Describe(obj)      - Describes the object's type
LoadPackage (pkg); - Loads the given Package (like -pkg:FILE)
LoadAssembly (ass) - Loads the given assembly (like -r:ASS)
ShowVars ();       - Shows defined local variables.
ShowUsing ();      - Show active using decltions.
Prompt             - The prompt used by the C# shell
ContinuationPrompt - The prompt for partial input
Time(() -> { })    - Times the specified code
quit;
help;
TabAtStartCompletes - Whether tab will complete even on emtpy lines
"
Of course, a couple of these commands exhibit some quirks. If type 'ShowUsing();' it does not display anything in the console although it is expected to do so (it behaves like this on Linux). After trying the command several times, I looked at the original command prompt window from which gsharp launched and saw the results of the command showing in there. The same was true with 'ShowVars()'. Recommend keeping the command prompt console in view while using gsharp to see any output piped outside of it. (The issue appears to been logged at the mono project as a bug.)

Perhaps to truly escape all of these OS specific limitations, I might be better off running Mono's REPL within a Linux VM on Windows. However, this approach negates the benefits of a cross-platform framework that Mono aspires to be.

Despite these minor difficulties, Mono's C# REPL has been a nice addition to my .NET development toolbox. It has proved useful when I needed a deeper understanding of how delegates and lexical closures behave in C# and when figuring out how to do list comprehensions in C# using List<T>.ConvertAll instead of LINQ. It provides a quick, frictionless way of observing and interacting with the functionalities of C# and the .NET libraries.

Tuesday, March 30, 2010

Database schema changes unit tested using TSQLUnit

How does one test the existence of a primary key (PK) constraint belonging to a table in a database? Simple, right? Just write a test that intentionally violates that constraint. One's initial impulse would be to write a test inspecting the data contained within the PK's table. This approach is sensible and can be viewed as the conventional state-based style of testing.

Let's try it. The following test using TSQLUnit attempts to insert in a table a new row with the same value for its primary key as another existing row's PK:
create procedure dbo.ut_SubProduct_PkConstraint
as
begin
    declare @failMessage        varchar(200),
            @err                int,
            @primaryKeyError    int

    set @primaryKeyError = 2627
           
    /* RUN TEST */
-- grab row already in table and try to re-insert it.
    insert into dbo.SubProduct    
        select top 1 sp.*
        from dbo.SubProduct sp
   
    set @err = @@error       

    /* ASSERT TEST */
    if  @err <> @primaryKeyError
        begin
            set @failMessage = 'PK constraint not defined.'
            exec tsu_failure @failMessage
        end
end
The above code will not work as a good test because error messages regarding the failure to insert a row with a duplicate PK will occur that can not be suppressed in TSQLUnit's test runner (as viewed in the 'Results' window in Sql Server Management Studio). (Even with some not-too-in-depth research, I could not find a means to hide or suppress these error messages using TSQL.)

Instead, it can be tested more cleanly using SQL Server's info schema views:
create procedure dbo.ut_Product_PrimaryKey
as
begin
    declare @failMessage        varchar(200),
            @tableName          varchar(40),
            @primaryKeyColumn   varchar(40)

    set @tableName           = 'Product'
    set @primaryKeyColumn    = 'ProductID'
           
    if not exists
    (
        select      c.table_name, c.column_name, c.data_type
        from        information_schema.columns c
            inner join information_schema.key_column_usage kcu
                    on c.table_name = kcu.table_name and c.column_name =
                       kcu.column_name
            inner join information_schema.table_constraints tc
                    on tc.table_name = kcu.table_name and tc.constraint_name =
                       kcu.constraint_name and tc.constraint_type = 'PRIMARY KEY'
        where       c.table_name = @tableName and kcu.column_name = @primaryKeyColumn
    )
    begin
        set @failMessage = 'PK constraint not defined for ''' + @tableName + '.' + @primaryKeyColumn + ''''
        exec tsu_failure @failMessage
    end
end
That's it. You now have a test that handles primary key constraints.

Of course, this is not the last and only time where primary keys will require testing. While the above sproc does a satisfactory job, it is not reusable for any other tables. Let's refactor it into a more generic "Assert" sproc in an xUnit style:
create procedure dbo.tsux_AssertPrimaryKeyExists
/* This is an extension to the TSQLUnit framework. */
(  
    @tableName            varchar(40),
    @keyColumn            varchar(40),   
    @failMessage          varchar(255)=null   
)
as
begin
    if not exists
    (
        select      c.table_name, c.column_name, c.data_type
        from        information_schema.columns c
            inner join information_schema.key_column_usage kcu
                    on c.table_name = kcu.table_name and c.column_name =
                       kcu.column_name
            inner join information_schema.table_constraints tc
                    on tc.table_name = kcu.table_name and tc.constraint_name =
                       kcu.constraint_name and tc.constraint_type = 'PRIMARY KEY'
        where       c.table_name = @tableName and kcu.column_name = @keyColumn
    )
    begin
        if @failMessage is null
            set @failMessage = ' PK constraint not defined for ''' + @tableName + '.' + @keyColumn + ''''
        exec tsu_failure @failMessage
    end
end
Your actual test becomes more compact and easier to understand:
create proc dbo.ut_Product_PrimaryKey
as 
begin
    exec dbo.tsux_AssertPrimaryKeyExists
        @tableName                  = 'Product',
        @primaryKeyColumn           = 'ProductID',
end
Any verification of pure schema changes such as the creation of new tables, columns, constraints, etc. via unit tests is better served using SQL Server's system tables and views to query the necessary meta information. This has proven to be more preferable than performing data centric state based tests (This technique reminds me a little of using reflection in .NET to do testing)

The earlier test for primary keys can be also applied to foreign key constraints as well. However, it requires some additional pieces to validate including the table and column being referenced. After (once again) finding the suitable tsql needed via an online search, here is the test:
create procedure dbo.tsux_AssertForeignKeyExists
/* This is an extension to the TSQLUnit framework. */
(
    @tableName            varchar(40),
    @foreignKeyColumn     varchar(40),
    @referenceTable       varchar(40),
    @referenceColumn      varchar(40),   
    @failMessage          varchar(255)=null   
)
as
begin
    if not exists
    (
        /*
            This is modified version of the tsql query used to retrieve foreign key info
            courtesy of:
                http://www.experts-exchange.com/Microsoft/Development/MS-SQL-Server/Q_22952666.html
        */

        select
            object_name(fkeyid)     as TableName,
            a.name                  as FKColumn,
            object_name(constid)    as FKConstraint,
            object_name(rkeyid)     as ReferenceTable,
            b.name                  as ReferencedColumn
        from sysforeignkeys f
            inner join syscolumns a on a.id = f.fkeyid and a.colid = f.fkey
            inner join syscolumns b on b.id = f.rkeyid and b.colid = f.rkey
        where
            fkeyid                        = object_id( @tableName )
            and a.name                    = @foreignKeyColumn
            and object_name(rkeyid)       = @referenceTable
            and b.name                    = @referenceColumn
    )
        begin
            set @failMessage = 'FOREIGN KEY does not exist for ''' + @tableName + '.' + @foreignKeyColumn  +  ''''
            exec tsu_failure @failMessage
        end
end
Now the new unit test would plainly look like this:
create procedure dbo.ut_Product_ForeignKeys
as
begin
    exec dbo.tsux_AssertForeignKeyExists
        @tableName            ='Product',
        @foreignKeyColumn     ='CategoryID',
        @referenceTable       ='Category',
        @referenceColumn      ='CategoryID'   

end
Database Testing: How to Regression Test a Relational Database, which details areas of any database that should be tested, had been an influence on developing these types of data definition language (DDL) tests. As an example, referential integrity is mentioned as being important area to test. On the surface, it seems a bit overkill to set up tests for PKs and FKs. However, even superfically minor data errors can be costly.

I was once tasked with expanding the size of a core primary key column that had multiple dependencies to other tables (including views and sprocs). As expected, to make the change required temporarily dropping the PK and FK constraints on those other tables and then add them back after applying the change.

Herein lies the risk. What if the "adding back" part was accidentally forgotten and not included in the change script? What if it was temporarily commented out with the intention to uncomment it later but overlooked? Allowing a deficient, regression-inducing script to rollout into a live production environment would be poor software development.

What automated unit tests provide in this situation is the insurance and safety net that the constraints are less likely to be missed or forgotten. They enforce the existence of those key constraints and firmly establish them as requirements for the database. It further instills greater confidence to make these sorts of changes by providing immediate feedback during development (and not much later) if the constraints are not set.

Another situation where unit testing using meta data came in handy was increasing the data type length of a column. Initially, when the size of varchar for a few columns needed to increase, I had some data heavy tests performing row insertions. These tests were fragile since they'd break the TSQLUnit test runner itself if a test inserted data larger than the expected size. Instead, I created a generic assert stored procedure that used the system sproc, 'COL_LENGTH':
create procedure dbo.tsux_AssertColumnLength
/* This is an extension to the TSQLUnit framework. */
(
    @tableName            varchar(40),
    @columnName           varchar(40),
    @expectedLength       smallint,
    @failMessage          varchar(200)=null
)
as
begin
    declare @actualLength        smallint

    select @actualLength = COL_LENGTH(@tableName, @columnName)

    if @expectedLength != @actualLength
        begin           
            set @failMessage = 'Column length for ''' + @tableName + '.' + @columnName  +  ''' does not match expected value.'
            exec tsu_failure @failMessage
            print 'Expected: '    + cast(@expectedLength as varchar(3))
            print 'Actual: '    + cast(@actualLength as varchar(3))
        end
end

Again, reusable and reliable code which avoids relying on sql compiler errors to indicate test failure.

Other examples of generic asserts for columns aside from length:

* tsux_AssertColumnExists
* tsux_AssertColumnDataType

Also, some other areas using test assertions:

* permissions on an object (quite important and often overlooked until too late)
* stored procedure parameter lengths
* existence of tables, views, and other similar database objects
* schemabinding on views

What's more is that these types of tests can be easily code generated. For example, if database changes included adding new columns, then unit tests can be generated by extracting the columns' meta data (e.g. name, datatype, length, etc.) as defined in a sql script (or even from an xml file or spreadsheet). The same can be done with existing data objects and structures requiring DDL changes but the meta data can be pulled from the database's system tables/views. In this situation, you gain some automatic test coverage for your current schema before attacking it with alterations.

Not sure if any of the techniques detailed earlier could be applied to check constraints (e.g. inserted/updated datetime value should not be greater than today's date, etc.). Perhaps, check constratins can be sufficiently managed with simple, direct unit tests using data rather than using meta data. (Although it could be argued that the logic for most check constraints should exist in the application code and not in the database. However, in practise, this is not always the case.) For now, without an immediate need, it will remain speculative.

This is likely my last post on TSQLUnit. In the future, my data access tests will probably be created in and executed from the application code rather than on the SQL Server side. However, if I do find myself in a scenario where database-only unit tests are needed, I'd probably try out T.S.T. the T-SQL Test Tool since it has built-in assert functions.

Wednesday, February 24, 2010

Software developer: an asset not a cost

I was quoted last year in a post, Convincing the Boss to Pay for Developer Training:

"...For example, software developer Ray Vega points out, "The 'culture' of the company is dedicated (sic) by how it makes money and who is responsible for helping to making that money." In general, he says, software companies whose product is technology-based tend to be better at providing and paying for skill improvement resources for tech employees. When the technology workers training is closely related to company revenue, it's easier to get the boss to listen. However, Vega adds, "If you work on an application that has no direct association with how the company makes money (for example, an internal time tracking application for an insurance company) then it will certainly be an uphill battle."


My response was based on not just my own work experience but also on an old Joel Spolsky article Five Worlds (which was provided to the author in my original response to her research on the topic). The "world" you write code for makes a significant difference in the overall health of your professional career beyond just training costs.

Most good programmers probably don't need formal "training" focused on a vendor specific technology, platform, or framework with a potentially short shelf life. They'd more likely learn on their own by creating and working on a side project or on a simple prototype specifically for that purpose. However, one exception is if the training class included like-minded individuals with whom one can collaborate and reciprocally learn from. Sadly, these are rare to find and difficult to vet prior to investing one's time in a chosen course.

That said, sometimes it doesn't hurt to be exposed to informational seminars, conferences, or coursework that cover the enduring fundamentals of software development (or even computer science) that people tend to forget or simply don't know.

Friday, February 27, 2009

It's True (And Not False): Assert Equality in TSQLUnit

When working with xUnit style frameworks like NUnit, it is generally expected to find support for assertions. Asserts are an indispensable tool in any testing process. Unit testing at its core is simply verifying whether something has occurred or not occurred i.e. checking if the state has changed marking it as either true or false. Built-in assert syntax provides programmers a means to perform this level of testing repeatedly and consistently. One of the more common assertions is to compare the values of two items, such as variables or objects, sometimes referred to as equality asserts. As an example, here is C# test code using NUnit to test a method that simply adds two arguments:
// some static class
public static int Add(int firstItem, int secondItem)
{
return firstItem + secondItem;
}

[Test]
public void Verify_sum_of_5_and_3_equals_8()
{
Assert.AreEqual(Add(5,3), 8);
}
Imagine my disappointment to discover the absence of equality assert functionality in TSQLUnit. The framework inexplicably does not have any native support for them. (Perhaps this might be because the project has not been active for a very long time.) Instead, you have to construct your own tailor-made assertions outside of the framework itself.

While using TSQLUnit without the aid of asserts, a definite pattern emerged as tsql plumbing code rapidly began to replicate in numerous tests. Here is an example of a unit test, plagued with the same identical code found in multiple places, whose purpose is to verify whether a 'ProductNumber' column has been successfully updated:
create procedure dbo.ut_CanUpdateProductNumber
/* UNIT TEST */
as
begin
declare @expectedProductNumber varchar(40),
@actualProductNumber varchar(40),
@currentAccountID varchar(16)

set @expectedProductNumber = '1234567890'

-- find a account to test
select top 1 @currentAccountID=AccountNbr
from dbo.Account

/* RUN TEST */
-- update product number
update dbo.Account
set ProductNumber = @expectedProductNumber
where AccountNbr = @currentAccountID

-- get updated product number
select @actualProductNumber=l.ProductNumber
from dbo.Account l
where l.AccountNbr = @currentAccountID

/* ASSERT TEST */
if @actualProductNumber != @expectedProductNumber
begin
exec tsu_failure 'The product number is not the same.'
print 'Expected: ' + @expectedProductNumber
print 'Actual: ' + @actualProductNumber
end

end
The duplicated code is the conditional 'if' block at the end of the stored procedure where the assertion is executed using 'tsu_failure', the obligatory proc call to the TSQLUnit framework that transforms your tsql code into a real live unit test. All test sprocs are built around this critical function. Unfortunately, 'tsu_failure' does not handle actual comparisons between two values, but only after the comparison has been made. It was not designed to recognize when value comparisons are useful or required. Instead, the surrounding test code is responsible for making that evaluation, in this case, using a custom conditional statement not originating from any TSQLUnit function.

In addition, being accustomed to seeing in NUnit test messages that display the detailed results of value comparisons (i.e. expected against actual), print statements were added to the test procs to simulate that same text. For example, the following is what would be shown if the aforementioned test were to fail:
The product number is not the same.
Expected: 1234567890
Actual: 0987654321
Although a welcomed improvement in the feedback provided by the test runner's results, I found myself repeatedly injecting that same structure over and over in numerous other tests.

When this form of repetition occurs, a strategy can be adopted of either (1) continuing copying and pasting code, (2) using code generation, or (3) formulating and developing some reusable code component to manage the duplication. With # 1 and # 2 being obvious maintenance sinkholes draining away any value earned from the test code, implementing # 3 was a more sensible choice.

To fight off test code rot, the 'if' block was refactored and encapsulated into a separate, shareable stored procedure (think Extract Method). Nothing extravagant but quite effective:
create procedure dbo.tsux_AssertAreEqual
/* This is an extension to the TSQLUnit framework. */
(
@expected varchar(8000),
@actual varchar(8000),
@failMessage varchar(255)
)
as
begin
if @actual != @expected
begin
exec tsu_failure @failMessage
print 'Expected: ' + @expected
print 'Actual: ' + @actual
end
end
Calling this new proc provides the familiar, sought-after "Assert.AreEqual" functionality found in NUnit and in a lot of other test frameworks. The old 'if' block in the original test was subsequently replaced with the new assert proc:
create procedure dbo.ut_CanUpdateProductNumber
/* UNIT TEST */
as
begin
/*...unaltered code...*/

/* ASSERT TEST */
exec dbo.tsux_AssertAreEqual
@expectedProductNumber,
@actualProductNumber,
'The product number is not the same.'

end
Now that we have a general utility assert sproc for the varchar type, we still have other data types, including int and datetime, that can also benefit from assertions of their own. Since TSQL does not support a flexible language feature like C# type generics for its stored procedures, creating sprocs for each data type is the only clear option to expand this functionality beyond vachar:
tsux_AssertDatesAreEqual
tsux_AssertIntsAreEqual
I can understand why the creator of TSQLUnit might have not initially built asserts into the framework since it requires building one for each every kind of data type in TSQL. Therefore, in its place, the burden falls on the user (i.e. me) to add additional asserts to the test code base as the need arises.

One kind of assert involving condition testing that might be interesting to implement but I am uncertain if it is remotely doable is this:
set @sqlConditionToEvaluate = (@expectedColor = 'BLUE'
AND @expectedSize = 23 OR StartDate between '1/1/11' and '2/2/22')
exec tsux_AssertIsTrue(@sqlConditionToEvaluate)
or more concisely:
exec tsux_AssertIsTrue(@expectedColor = 'BLUE'
AND @expectedSize = 23 OR StartDate between '1/1/11' and '2/2/22')
Maybe this could be achieved using dynamic sql and with storing each condition to verify within a 'TABLE' data type variable (functioning as an array) that can be looped checking each one to be true or false. However, implementing complex asserts to this extreme extent is a strong indication that TSQLUnit might no longer conceivably be the appropriate tool to write unit tests. It might be preferable to consider alternate unit testing frameworks that operate entirely outside of the database using a language other than TSQL that is better equipped for elaborate conditions and logic flow.

Friday, January 30, 2009

Paging in ASP.NET using NHiberate

NHibernate, the object-relational mapping (ORM) framework for .NET, supports custom pagination for collections. It provides a potential alternative to the built-in paging mechanism and native support found in ASP.NET GridView web controls. NHibernate exposes in the API for IQuery and ICriteria two methods, SetFirstResult and SetMaxResult, that can be used to enable paging:

Collections are pageable by using the IQuery interface with a filter:

IQuery q = s.CreateFilter( collection, "" ); // the trivial filter
q.setMaxResults(PageSize);
q.setFirstResult(PageSize * pageNumber);
IList page = q.List();

Adding Paging to the Base DAO

The design and architecture of my project, where NHibernate style pagination shall be introduced, was deeply influenced by Billy McCafferty's NHibernate Best Practices with ASP.NET resulting in the proliferation of Data Access Objects (DAO) throughout the guts of the application. Each DAO maps one-to-one to a single matching table in the database.

For example, a 'Task' table would have a corresponding 'TaskDao' in the data access layer (DAL) of the application. All of these DAOs inherit from the same base class 'GenericNHibernateDao' responsible for containing commonly shared code including managing NHibernate sessions and providing generic methods for summoning a specific persisted instance by ID and for saving/deleting an existing instance.

The initial step to implement paging was adding the following members to the aforementioned base class 'GenericNHibernateDao':

public abstract class GenericNHibernateDao : IGenericDao
{
public int PageSize
{
get { return _pageSize; } set { _pageSize = value; }
}

public int PageNumber
{
get { return _pageNumber; } set { _pageNumber = value ; }
}

protected int GetFirstResultPosition()
{
return _pageSize * (_pageNumber - 1);
}

protected void SetPagingFor(IQuery query)
{
query.SetFirstResult(GetFirstResultPosition());
query.SetMaxResults(_pageSize);
}

protected void SetPagingFor(ICriteria criteria)
{
criteria.SetFirstResult(GetFirstResultPosition());
criteria.SetMaxResults(_pageSize);
}

/*
other non-paging related members...
*/
}

The property 'PageSize' gets/sets the number of instances expected to be displayed on a web page for a specific strongly typed collection. Essentially, it handles how many rows to return for a embedded control (such as a GridView) on the page. This property is intended to be internally consumed by NHibernate's 'SetMaxResults' method belonging to IQuery or ICriteria. For example:

IQuery query = Session.CreateQuery();
query.SetMaxResults(_pageSize);

'PageNumber' specifies the set of multiple instances to be returned and displayed on a web page as identified and grouped by a page sequence numeric value (This is the equivalent of 'PageIndex' property for GridViews). For example, if you had a total of five pages worth of data rows then displaying the second page would require setting the page number value to '2' (e.g. _dao.PageNumber = 2). Just as with 'PageSize', this property is intended to be used by the method 'GetFirstResultPosition' as will be explained next.

The protected method 'GetFirstResultPosition' calculates the actual row at which to start paging based on the values provided by 'PageSize' and 'PageNumber'. This method's return value is expected to be passed to NHibernate's 'SetFirstResult' method, once again, part of IQuery or ICriteria. For example:

IQuery query = Session.CreateQuery();
query.SetFirstResult(GetFirstResultPosition());

The overloaded method named 'SetPagingFor' performs the actual paging functionality via the 'SetFirstResult' and 'SetMaxResult' methods of IQuery and ICriteria. (As an aside, the use of IQuery to build and execute HQL is much more common on this project in comparison to the almost non-existent use of ICriteria).

The base class 'GenericNHibernateDao' was also modified to have one of its existing methods 'GetAll' call 'SetPagingFor'. (The 'GetAll' method simply returns a list of all strongly type objects in the database without any specified criteria or filtering):

// GenericNHibernateDao class
public IList<T> GetAll()
{
ICriteria criteria = Session.CreateCriteria(persitentType);
SetPagingFor(criteria); // this is newly added!
return criteria.List();
}

As it shall be made clear later, this new line of code will optionally provide paging functionality when 'GetAll' is requested, if needed, but not required per se.

Applying Paging Functionality in the DAOs

Now that the base class 'GenericNHibernateDao' has been updated to manage paging, any of its derived DAO classes are instantly equipped to perform paging themselves. To actually invoke the paging functionality for any of the DAOs, simply set the appropriate values for page size and page number as shown in this example for the 'TaskDao' class:

// In context (such as a Presenter or Controller class
// part of an MVP/MVC applied framework)
TaskDao _taskDao = new TaskDao();
_taskDao.PageSize = 20;
_taskDao.PageNumber = 5;
IList<TaskDao> list = _taskDao.GetAll();

Without paging, 'GetAll' would have returned something in the neighborhood of 100 or so rows. With paging, the returned list will instead be only 20 rows starting at row (i.e. position) # 80. While a hundred rows might not sound too substantial, larger sets of data can have a more noticeable effect on your application's day-to-day operations if your table contains tens or hundreds of thousands of rows. Your performance will be progressively impacted as your application scales with more data.

On the other hand, NHibernate's paging will significantly decrease the size of your result set. This is in stark contrast to the default behavior of the existing paging available in any GridView control. If this native functionality of the control is used, it will return all rows from the database and then page them in memory. As mentioned earlier, this can lead to slower performance as your data grows. More on this later.

If, for any reason, paging is not essential for a particular web page in possession of a control bound to a strongly typed list (for example,a very small static list of data) then setting the page size and number properties is not required at all. Simply call 'GetAll' by itself disregarding the 'PageSize' and 'PageNumber' properties and the DAO should return all rows found in the associated database table. What makes this possible is that the default values for those two paging properties are formally declared in the DAO base class to behave as expected for "non-pageable" collections:

public abstract class GenericNHibernateDao : IGenericDao
{
protected int _pageSize = -1; // default for unlimited page size
protected int _pageNumber = 1; // default for first item in collection

// more members...
}

Typically, a DAO might have other custom methods that return more narrowly focused (i.e. filtered) lists of typed objects than what 'GetAll' offers. For these other DAO methods, the same pattern can be followed by adding the one line of code calling 'SetPagingFor'. For example, the 'TaskDao' might have a method that returns tasks that were completed in 2006 excluding any other tasks not done within that same year:

// TaskDao class
public IList<TaskDao> GetTasksCompletedIn2006()
{
IQuery query = Session.CreateQuery(
"some HQL statement that filters tasks by 2006...");
SetPagingFor(query); // newly added!
return query.List();
}

It is generally good practice not to pass the values for page size and number directly via the parameter list for any of these custom filtered data access methods. A few reasons to avoid this: (a) it can quickly clutter the intent of the method, and (b) it would prevent the paging functionality from being optional and, as a result, become less flexible, less reusable, and more cumbersome to work with.

Consequently, while it would be tempting to write the method's signature as such:

IList<TaskDao> list = _taskDao.GetAllTasksWithSubtasks(
param1, param2, param3, ..., paramN, pageSize, pageNumber);

It is preferable to do the following instead:

_taskDao.PageSize = 20;
_taskDao.PageNumber = 5;
IList<TaskDao> list = _taskDao.GetAllTasksWithSubtasks(
param1, param2, param3, ..., paramN);

The Downside of NHibernate Paging with ASP.NET Controls

As indicated earlier, one weakness of NHibernate's paging when combined with ASP.NET's GridView control involves some loss of "out-of-the-box" functionality. Under more conventional circumstances, when a collection of objects are bound to a GridView control, one of the built-in paging features of that control is to automatically render on the web page the navigation hyperlinks for the pages. For example, you might see following below your control:

" 1 2 3 ... 10 "

The GridView's default behavior assumes that all data bound directly to its DataSource can be paged as long as the number of items of that data is greater than its PageSize property value. Hence, if that condition is met, the control will slice up and present the data as appropriate.

Conversely, this is not true when using NHibernate style pagination. When binding to an NHibernate paged list method, the GridView's PageSize value will usually be set to the same size as the paged data list, a mere subset of the total data found in the database (or data source). Behind the scenes, the PageIndex property of the GridView will reset to zero because the number of items bound to its DataSource is less than or equal to the PageSize. Therefore, the GridView is under the impression that the data it receives is not pageable and, in turn, disables any paging features, unaware that more items do indeed exist but were just not provided at that moment. The paging features lost include not just immobilizing navigation links but also removing the availability of the event handlers linked with changing the page index.

Without the convenience of auto-generating navigation links, two options emerge that might help to produce the same desired behavior:

  1. Inherit from the GridView control and attempt to confirm whether or not if any paging methods can be overridden some how or in some way
  2. Create a custom, reusable user control to implement the navigation features

Initially, the easier path was taken by developing a very rudimentary and simple implementation of option # 2. This entailed providing in extremely basic custom controls functionality for navigating between pages using homemade "Previous" and "Next" buttons. Currently, it is not implemented as a shareable user control nor is it able to display the pages counts. I intend on exploring option # 1 a bit more in the event that option # 2 evolves into something more elaborate and unwieldy. Until then, it is a work in progress.

A third option does exist involving the possible use of the ObjectDataSource control. However, that strategy can lead down a less than desirable path for the following reasons:

  1. loss of control of how the data access is managed
  2. ease of maintainability diminishes if any widespread changes were to emerge in the future within areas of the application relying on paging
  3. disrupts and conflicts with how the MCP/MVC methodology is currently applied on the project
  4. increased difficulties in writing and running reliable automated unit tests

All things considered, despite some trade offs and a little bit of work, leveraging NHibernate's ability to do paging can be an area that could contribute significantly in optimizing and improving the performance of a data-intensive web application.

Thursday, January 15, 2009

Test Driven Blogging

Years ago when I first started learning and practicing Test Driven Development (TDD), I became extremely overzealous blindly believing all code needed to be unit tested (one of the more extreme cases were my over-the-top uses of TSQLUnit which included testing simple, low risk DDL changes that, in retrospect, was probably a bit too much with so little to gain.) This attitude was probably a common rookie mistake of becoming enamored with a new methodology (or language or framework or ....) by assuming this is how all software should be written. It is as if I had discovered the elusive magic bullet that a lot of programmers spend most of their careers searching for. Unit testing, along with its subset TDD, has steadily grown in popularity spanning the diverse programming communities spectrum. How could I possibly go wrong in my new found beliefs?

However, this past year, I have seriously reconsidered my views on unit testing. Instead, I have significantly curtailed the use of unit testing by being more selective as to when it should be applied in code. I have realized that while testing is an important tool, it is far from the "be all and end all" that others have proclaimed it to be. Instead, I readjusted my thinking to focus on what is genuinely the most important goal in programming which is to actually deliver good working software in an iterative manner.

On Stack Overflow (SO), Kent Beck an early pioneer of TDD and a creator of JUnit (the precursor for all modern xUnit style frameworks), provided a very interesting and perhaps unexpected response to the question: How deep are your unit tests?

I get paid for code that works, not for tests, so my philosophy is to test as little as possible to reach a given level of confidence.

The reality of what Beck wrote can not be ignored. (At least, I think it's him. The tone and content of his other responses on SO seem to indicate that it might just be.) Principally, working code is more important than the tests themselves. Tests can be just one of numerous methods to achieve the goal of delivering good quality software on a frequent basis but tests must certainly not overshadow this intent.

Along with Beck, it was unquestionably reassuring to read a recent post of Ayende (a notable .NET blogger/developer) that also tackles head on this very topic regarding software delivery and testing:

I want to make it explicit, and understood. What I am riling against isn't testing. I think that they are very valuable, but I think that some people are focusing on that too much. For myself, I have a single metric for creating successful software:

Ship it, often.

Keep in mind these statements are from the creator of Rhino Mocks, one of more popular mock testing frameworks for .NET. Subsequently, it carries a lot weight as it was written by someone who does truly and thoroughly understand the virtues of unit testing and TDD. Ayende is someone who I most certainly admire being as close to the ideal model of a 10x programmer in not just the realm of .NET but in programming in general (some of that admiration stems from being in complete awe of his unearthly prolific blogging). Having him confirm something that I myself have come to realize on my own this past year definitely does help to validate my current views. Quite simply, I have substantially toned down my TDD rhetoric and restrained my testing impulses in favor of renewing my true objectives in programming.

My learning TDD coincided with my learning of .NET and C#. At that point in time, I compulsively consumed the writings of a somewhat noted blogger in the .NET community who relentlessly championed unit testing and TDD. Practically treating this person's words as pure gospel, I considered this individual to be quite representative of the ALT.NET community by serving as one of the leading voices for all that it embodies. This is a community deeply immersed in the ways of unit testing and TDD.

However, with my now newly reformed outlook on software development, I have become much more wary of that blogger's "best practices" crusades. The blog still continues to be obsessively fanatical over the non-negotiable importance of unit testing and TDD almost to the exclusion of any other competing methodologies or tools. Their dogmatic writings assuredly fall into the "focusing on that [testing] too much" camp as described earlier in Ayende's post. Originally, this programmer's "test driven blog" once held one of the few selective spots on my blog's "Blog List" links. But, since it no longer carries the same relevance to me as it used to, I decided to remove it from the list.

Nevertheless, I still read that blog from time to time because it does have great information and observations regarding software development and best practices in the .NET ecosystem. In addition, I still consider myself a TDD practitioner despite reducing the scope and influence of unit testing in relation to my programming style. I just now better grok what my priorities are.