Yafumato Development Log

In the beginning…

Yafumato 0.5

As I said, I just released Yafumato 0.5. It had actually been ready for about a week, but I delayed releasing it because I was studying for the GMAT exam. Now that the test is over (750, woo-hoo!), I can start getting to all the things I had put off.

In my initial post I described the versions leading up to the initial public release, so I’ll just take a second to talk about the 0.4 release. From an interface perspective, the changes were minor; a few dynamic updates were bolted on to the page, but they could only update elements that were already visible. The real effort was the server-side code to support the dynamic updates — to determine what changed and create the XML. The 0.5 release was well into development when I released 0.4, which I tested for about a week. Although the changes in the 0.5 release are much more dramatic, only a few files were changed from the 0.4 release.

So, what’s new in the 0.5 release? Well, although it may not look it, the messaging page was completely redesigned. I implemented a JavaScript object model that is patterned after the server-side architecture. The JSP actually writes very little HTML when generating the messaging page, instead populating the JavaScript object model that dynamically creates the HTML for the pager. The pager is now completely dynamic as a result. Contacts are dynamically shown or hidden based on their status (by default the page uses the AIM convention of a separate group for offline contacts, though this behavior can be changed in the JSP). Also, the actions of sending messages and expanding or collapsing groups now dynamically update the page rather than redirect to a new page. The actions that still redirect are updating status and activating a contact. If a contact is active when the page is loaded, however, that contact can be deactivated and reactivated dynamically. When the update interval is set to a low value (I like to use three seconds at home) the application is very responsive. Once you’ve activated the contacts that you want to chat with, you could chat for hours with only dynamic updates.

The next release will feature DHTML windows — I believe that this is the main feature that other web messengers have that Yafumato lacks. The Dojo toolkit has them. Unfortunately, the DHTML window widgets are undocumented, so I’ll have to do some experimentation.

TLV the 0×0013th

A little while ago I had a problem — I wasn’t receiving any messages on AIM. At least, I wasn’t receiving any messages from my friends; the moviefone and shoppingbuddy bots were working fine. At first I thought I was just a pariah, but I realized that I had a real problem when the problem continued on Monday the 13th. I turns out that DAIM was throwing exceptions while parsing the messages. The problem turned out to be TLV chain 0×0013:

/*
 * Possessed TLV type, type 0x0013.
 *
 * A TLV of type 0x0013 can only be 2 bytes, despite its
 * server-specified length. I have never seen this in action, but
 * having this here hasn't ever broken anything.
 */
if (type == TLVType.AIM_TLV_ATH_REGISTRATION_STATUS && length != 2) {
    len = 2;
}

Apparently this idiosyncrasy was fixed, as this TLV chain is now confirming to the server-specified length. Messages from the bots don’t have this TLV chain, so they worked. This made things easier to debug, as I was able to compare the DAIM hex dumps and identify the problem. I’ve patched daim.jar, and will be releasing this patch soon.

While debugging this issue, I contacted the developer of the DAIM API, David Walluck. I’m going to try to join the DAIM project as a developer since David is busy with grad school and looking for someone to take over the project.

This issue underscores an important point — neither API propagates exceptions up to the application. This is a flaw that is going to have to be fixed before Yafumato can move out of the alpha stage.

Bride of Windows

I’ve decided to use BrainJar.com’s DHTML window implementation rather than the Dojo toolkit. BrainJar.com’s implementation performs better when moving and resizing and comes with a thorough tutorial while Dojo’s widgets are undocumented. The window decoration is a bit spartan, but I’ve improved that in my own version. I’ve learned a lot about JavaScript reading BrainJar.com and modifying the code; I intend to carry over that knowledge into a re-write of the messaging JavaScript object model. I’ve created a mock-up of the next version of Yafumato using DHTML windows that I’ll post here soon.

Yafumato 0.5.1

I have released Yafumato 0.5.1 to fix the aforementioned problem with AIM/OSCAR and TLV chain 0×0013.

Code Names are Cool

I’ve decided, starting with the next release, to use code names in addition to version numbers. Why? Because code names are cool, and I like being cryptic. As such, the next release — Yafumato 0.6 — will be designated “Kidd.”

Put Me to the Test

I’ve completed development of the next Yafumato build, 0.6 aka Kidd. I am currently testing it and plan on releasing it on April 7th. I generally like to test a release for about a week before releasing it. This release is particularly tricky, though, because it is much more complex. Kidd has about twice as much JavaScript code as 0.5, and it is now particularly vulnerable to browser-specific problems (more on that later).

I’ve been using junit for unit testing Yafumato’s Java code since the beginning. It was especially helpful when developing the utilities that perform the grunt work in Yafumato, such as managing properties and database connections. As useful as junit is, debugging Java code is much, much easier than debugging JavaScript code. Fortunately, I discovered script.aculo.us.

script.aculo.us is a Prototype-based JavaScript library. I was originally interested in script.aculo.us for the visual effects, but script.aculo.us also includes a JavaScript unit testing framework; what was originally a neat toy has become an invaluable tool. Since JavaScript is an interpreted language, there’s no compiler to catch typos. A typo that would be caught by the compiler in Java can cause totally non-deterministic behavior in JavaScript. The script.aculo.us unit test framework has become indispensable for my JavaScript testing, especially since the code is still maturing.

Yafumato 0.6 <Kidd>

I have released Yafumato 0.6 <Kidd>, featuring dynamic HTML windows to contain the pager and messages. Status messages and new message alerts have also been replaced with icons. Activating a contact (e.g. opening a chat window) is now dynamic.

The DHTML window implementation has been tested in Microsoft Internet Explorer 6 and Mozilla Firefox 1.5; I recommend Firefox, however, because there are currently a few graphical glitches in IE. The next release will likely be a maintenance release to clean up the DHTML windowing code without any significant functionality added.

Here There Be Object-Oriented JavaScript

I am not a JavaScript programmer. Before I undertook converting Yafumato to AJAX, I had only a passing familiarity with the language — I’d used it for a few simple effects years ago, mainly by adapting examples from the web. It isn’t much of an exaggeration to say that I didn’t know JavaScript at all. Allow me to be immodest for a moment and say that this release of Yafumato finally feels like a true AJAX application. I’m not sure which more odd though, the fact that I’ve written an AJAX web application when I didn’t even know JavaScript three months ago, or the fact that in spite of having written an AJAX web application I still wouldn’t say that I know JavaScript.

In this release I wanted to refine Yafumato’s JavaScript object model and include more object-oriented techniques, but I wasted a lot of time trying to decide on the best way to implement inheritance and polymorphism in JavaScript. While I’m happy with the results, allow me share the source of my frustration…

In JavaScript, there are several ways to define an object’s properties and functions. One way to define objects is to simply define all properties and functions in the constructor:

function Object(property) {
    this.property = property;
    function function(parameter) { ... }
    this.function = function;
}

This is how I defined objects in Yafumato 0.5. I don’t care for this technique though, especially the fact that you have to assign a function to a property in order to access it (I forgot to do this many many times). I also don’t like the fact that function declarations and function calls are mixed inside the constructor; as a result I tended to avoid making function calls inside constructors. Most importantly, this technique doesn’t address inheritance at all.

Another technique is to assign functions in the object’s prototype:

function Object(property) {
    this.property = property;
}
Object.prototype = {
    function: function (parameter) { ... }
};

Prototypes can be used to implement inheritance by assigning the prototype of the derived object to be an instance of the base object: DerivedObject.prototype = new BaseObject(). In this case the derived object does not inherit the constructor, though, so the derived object will need to explicitly set the properties inherited from the base object in its constructor.

Finally, there’s the Prototype library. Prototype purports to support “class-style OO”, and provides the Object.extend function, which copies the properties of one object to another. However, a limitation of this as well as the other techniques is that it doesn’t override functions, it overwrites them. So, you can see how this would be confusing to a JavaScript neophyte…

In the end the key was function.call, which allows an object to call a function as if the function were a member of the object. Using function.call I could call a function on the base object’s prototype as if it were a member of the derived object, finally allowing the derived object to override functions without losing the functionality already coded into the base object. Thus emboldened, I combined Prototype’s Object.extend function with prototypes to implement inheritance and polymorphism like this:

function BaseObject(baseProperty) {
    this.baseProperty = baseProperty;
}
BaseObject.prototype = {
    overriddenFunction: function(parameter) { ... }
}

function DerivedObject(baseProperty, derivedProperty) {
    BaseObject.call(this, baseProperty);
    this.derivedProperty = derivedProperty;
}
DerivedObject.prototype = new BaseObject();
Object.extend(DerivedObject.prototype, {
    function overriddenFunction: function(parameter) {
        BaseObject.prototype.overriddenFunction.call(this, parameter);
        ...
    }
});

Internet Explorer Sux0rs

Sorry for the inflammatory post title but, given the number of problems that Internet Explorer has caused me, I think that it is justified. During the testing the Kidd release I came across many many Internet Explorer glitches, some were fixed before release while a few others are still pending. The issues are annoyances rather than showstoppers, and this release is such a big improvement that I didn’t want to delay it anymore. I am not an expert web developer by any means, so I can’t rule out my code as source of the problem, but I think the fact that my code works beautifully in Firefox yet has so many glitches in Internet Explorer is telling.

The first problem that I discovered was with the scroll bar for the text area where you enter messages. If the CSS width of the text area is defined as a percentage rather than a fixed value then the scroll bar is offset to the right by the amount of the border and padding of the containing element when text is entered. At first the scroll bar is drawn correctly, and if the text area is resized then the scroll bar is again correctly positioned until more text is entered. This effect occurs even when the scroll bar is not visible, instead offsetting the window background. The image below shows the problem, and I’ve also created a simple test case.

I was able to fix the problem using a unique feature of Internet Explorer — CSS expressions. I added the following code to the stylesheet:

textarea.messageFormBody {
	width: expression(this.parentNode.parentNode.parentNode.offsetWidth - 19);
}

All’s well that ends well? Not quite. First, you can see that I had to hardcode the offset. If the padding of any of the containing elements changes, this value will need to be updated. Worse yet, this code can crash Internet Explorer! If the offset is less than amount of padding between the containing elements and the text area, IE resizes the containing elements to fully contain the text area causing an infinite loop as IE then resizes the text area again, and so on. Finally, this code causes a slight glitch when opening a chat window for the first time as IE initially sizes the text area, which you can see in the image below. I decided this was a worthwhile tradeoff, but my Internet Explorer woes were just beginning…

Another annoyance was IE adding extra padding to the first group in an identity, and the first contact in a group. In the image below you can see that there is extra space between the name of the AIM Bots group and the first contact in the group, while the first contact in the Buddies group appears immediately below the name. Also, there is more space between the first contact in the AIM Bots group than there is between the second contact and the next pager node.

After creating another simple test case I determined that this was because I was defining the font properties in the block-level elements and letting the inline elements inherit rather than setting the properties directly on the inline elements. IE still adds two pixels of extra padding even after updating the stylesheet, but at least now it is uniform. In the next release I may take advantage of IE’s ability to use conditional stylesheets to remove the padding from these elements, instead relying on IE’s extra whitespace to provide the proper spacing (since IE clearly knows padding better than I do), but I’m not sure that I want to stoop to IE’s level by using such a hack.

While these glitches were mere annoyances, the next issue was a major problem. If IE’s Temporary Internet Files settings are set to “Every visit to the page” then IE will request background images from the server every time a dynamic action is taken on the page! This causes the images to flicker, but more importantly it creates a ton of extra requests to the server. What at first appeared to be an annoyance turned out to be a threat to my design principles! Although this is a browser issue, it can be fixed with some server-side tweaks. I have incorporated those settings into the Apache HTTP Server configuration snippet included in the distribution, but Apache HTTP Server is an optional package; the issue remains when serving Yafumato directly from Apache Tomcat. I have noted this issue in the release notes with instructions for ensuring that the IE is properly configured to avoid this problem. More information about this issue is available at http://www.fivesevensix.com/studies/ie6flicker/.

Next, IE doesn’t create horizontal scroll bars for my DHTML windows’ contents. Instead it resizes the div to hold the contents without side-scrolling and refuses to resize the element any smaller than that. The windowing code sets the minimum resize width to be the width of the window contents, but that is set when the window is created; dynamically added content can still be a problem. This is still an open issue.

Finally, there are glitches with the script.aculo.us combination effects. Adjacent elements are briefly offset when using effects such as “SlideUp” or “Squish” to hide an element. Also, when using an expanding effect to make a large element appear, the portions of the expanding element that are outside of a DHTML window are briefly visible; the element is only clipped when the effect completes.