History

The Niggle framework really dates back to the summer of 1999. I was working on the initial prototype of the Medimom site, for Reminders On-Line (ROL). At that stage, we were using a home-grown page template system that had a lot of limitations. I had become aware of the open source Freemarker library which I preferred. Vastly. It was far less awkward to use than the template system we were using. The problem was that we already had a fair bit of existing work, both on the java coding and graphical design front, based on the other template library. Switching over all of the existing page templates to Freemarker seemed like a daunting task, the kind of thing that typically never happens, because it requires a huge all-at-once expenditure of energy and it never quite seems worth it at any given moment.

Still, I was determined to make the switch. I started off by creating a higher-level API whereby the two template systems could coexist, albeit temporarily. I created a Page interface and the concrete implementations LegacyPage and FreemarkerPage and refactored the code so that it only used the higher-level Page interface. Once I created a PageFactory that vended the appropriate concrete instance, I could then gradually change the implementations from LegacyPage to FreemarkerPage one by one over a period of time, while still having a working system throughout. This worked out quite well. Once I had finally converted all of the page templates from the LegacyPage to FreemarkerPage, I got rid of the older implementation class, yet kept our application code using the more abstract Page/PageFactory API, in the full knowledge that, if we found a better presentation solution than Freemarker, we could switch over to that in a similar way. (As it turned out, we never did find a better presentation solution than Freemarker. Or at least, Freemarker was good enough that we never actively looked for anything any better.)

Actually, I would not say that this is the point at which Niggle was born. But it was gestating. Meanwhile, another front was opening up -- the data side. It was becoming increasingly evident that there was an ugly error-prone coding pattern that was constantly repeated throughout the code. A user would submit data in an HTML form and we would create a data record from that. However, since a user can put anything in a field, we would, at various places, be checking that the data was valid. The problem was that this validation code was dispersed (and repeated) all over the place throughout our dispatching logic.

The solution was to find some way of specifying a record's field metadata separately and then, the code could always just iterate over the fields and check whether the data was valid. If the data was invalid, it would simply redisplay the page with the error message. If we wanted a different constraint to apply to a given record, we would simply change the field constraint in the metadata and all of the right things would simply continue to happen.

The other thing that this scheme introduced was that, on the basis of the metadata, we could have code that automatically (we used the term automagically) a lot when discussing this) generated the appropriate wrapper objects that Freemarker needed to expose the data on the page.

I think that this core functionality was running pretty robustly by November of 1999. Now, the Niggle framework was born!

Niggle still went through a lot of growth from there. Initially, I developed a custom way of specifying the record metadata in java code. This was rather awkward in retrospect. It certainly was not particularly easy for newbies to learn and feel comfortable with. It was Lawrence Austen, CTO of Oilspace who suggested that it would be better to specify record metadata in an external XML file. The switch to an external XML schema was made by June of 2000. At that point, there was in existence something that very much resembled the current version.

Current Status

Niggle is definitely in a usable state. In fact, that is probably quite the understatement. The Niggle code is highly robust. Reminders On-line have been building all of their servlet-based software on top of Niggle for well over a year. ROL's java servlet-based reminder service, Medimom, runs on top of Niggle. Still, there are various issues that have caused me to hold off on classifying the current version as a 1.0 final release.

Of the things that Niggle lacks right now, the most important are: documentation, documentation and documentation. And in that order! Currently, the only documentation to help you get into Niggle is what is to be found in the examples hierarchy. That, and the generated javadoc.

There is also a separate web-based forum application that I recently wrote show off Niggle. This is a separate download. Unlike the examples in the ./examples directory of the main Niggle distro, this is not really a toy app. Also, it is not well documented now, but if you have gone through the examples that came with Niggle, and you want something more complex to sink your teeth into, feel free to download that code and hack it.

So, documentation is our greatest single priority. In my opinion, the best documentation consists of examples that are well-constructed and annotated. And that will be my emphasis over the next while on that front.

So, to sum up, Niggle is not alpha-level code, since it has been stress-tested extensively and is known to be highly robust. On the other hand, it is not really completely ready-for-primetime material because it is missing some important things that the final non-beta release will have, the most important of these being more extensive documentation. It is once the above issues are addressed that I intend to take the pre out of the release tags and call that release Niggle 1.0 final.

License and Copyright Issues

I wrote most of the Niggle code while working for Reminders On-Line in 1999 and 2000. From fairly early on, it was agreed that Niggle -- which was really the generally useful code not specific to ROL's application domain -- would be made available under a liberal open source license.

Reminders On-Line owns the original copyright. I have not been working for ROL since March of this year (2001) and hence, I own the copyright to subsequent modifications. In any case, both the original code and the subsequent modifications are licensed under the same BSD-style open source license.

Future

Niggle is an ongoing project and you can fully expect further enhancements and refinements.

There are really 3 different fronts on which I see progress happening from here on:

In terms of this last point, a current weakness of Niggle is that the XML schema it uses is really pretty much implementation-defined. There is no DTD and not really any documentation besides the source code and existing examples. This is not a crime, mind you, and does not prevent Niggle from being useful as is. But, I think that with a formalized DTD, there would probably be a lot of low-hanging fruit that could be plucked off fairly easily. It would allow one to generate custom editors for the data specifications that would prompt you for possible options/completions and would make sure that the XML was valid (i.e. conformed to the DTD) before saving. This would also allow the development of specialized data modeling tools that would allow people (who are that way inclined) to model all of their data relationships and constraints before beginning a project. (With a good schema printout, at least one could start off on the right foot with management!)


The Niggle Project page on Sourceforge

To stay informed about Niggle, we strongly encourage you to subscribe to the Niggle-related mailing lists.

Jonathan Revusky, 14 May 2001