Yafumato Development Log

Put Me to the Test

I’ve completed development of the next Yafumato build, 0.6 aka Kidd. I am currently testing it and plan on releasing it on April 7th. I generally like to test a release for about a week before releasing it. This release is particularly tricky, though, because it is much more complex. Kidd has about twice as much JavaScript code as 0.5, and it is now particularly vulnerable to browser-specific problems (more on that later).

I’ve been using junit for unit testing Yafumato’s Java code since the beginning. It was especially helpful when developing the utilities that perform the grunt work in Yafumato, such as managing properties and database connections. As useful as junit is, debugging Java code is much, much easier than debugging JavaScript code. Fortunately, I discovered script.aculo.us.

script.aculo.us is a Prototype-based JavaScript library. I was originally interested in script.aculo.us for the visual effects, but script.aculo.us also includes a JavaScript unit testing framework; what was originally a neat toy has become an invaluable tool. Since JavaScript is an interpreted language, there’s no compiler to catch typos. A typo that would be caught by the compiler in Java can cause totally non-deterministic behavior in JavaScript. The script.aculo.us unit test framework has become indispensable for my JavaScript testing, especially since the code is still maturing.

Yafumato 0.6 <Kidd>

I have released Yafumato 0.6 <Kidd>, featuring dynamic HTML windows to contain the pager and messages. Status messages and new message alerts have also been replaced with icons. Activating a contact (e.g. opening a chat window) is now dynamic.

The DHTML window implementation has been tested in Microsoft Internet Explorer 6 and Mozilla Firefox 1.5; I recommend Firefox, however, because there are currently a few graphical glitches in IE. The next release will likely be a maintenance release to clean up the DHTML windowing code without any significant functionality added.

Here There Be Object-Oriented JavaScript

I am not a JavaScript programmer. Before I undertook converting Yafumato to AJAX, I had only a passing familiarity with the language — I’d used it for a few simple effects years ago, mainly by adapting examples from the web. It isn’t much of an exaggeration to say that I didn’t know JavaScript at all. Allow me to be immodest for a moment and say that this release of Yafumato finally feels like a true AJAX application. I’m not sure which more odd though, the fact that I’ve written an AJAX web application when I didn’t even know JavaScript three months ago, or the fact that in spite of having written an AJAX web application I still wouldn’t say that I know JavaScript.

In this release I wanted to refine Yafumato’s JavaScript object model and include more object-oriented techniques, but I wasted a lot of time trying to decide on the best way to implement inheritance and polymorphism in JavaScript. While I’m happy with the results, allow me share the source of my frustration…

In JavaScript, there are several ways to define an object’s properties and functions. One way to define objects is to simply define all properties and functions in the constructor:

function Object(property) {
    this.property = property;
    function function(parameter) { ... }
    this.function = function;
}

This is how I defined objects in Yafumato 0.5. I don’t care for this technique though, especially the fact that you have to assign a function to a property in order to access it (I forgot to do this many many times). I also don’t like the fact that function declarations and function calls are mixed inside the constructor; as a result I tended to avoid making function calls inside constructors. Most importantly, this technique doesn’t address inheritance at all.

Another technique is to assign functions in the object’s prototype:

function Object(property) {
    this.property = property;
}
Object.prototype = {
    function: function (parameter) { ... }
};

Prototypes can be used to implement inheritance by assigning the prototype of the derived object to be an instance of the base object: DerivedObject.prototype = new BaseObject(). In this case the derived object does not inherit the constructor, though, so the derived object will need to explicitly set the properties inherited from the base object in its constructor.

Finally, there’s the Prototype library. Prototype purports to support “class-style OO”, and provides the Object.extend function, which copies the properties of one object to another. However, a limitation of this as well as the other techniques is that it doesn’t override functions, it overwrites them. So, you can see how this would be confusing to a JavaScript neophyte…

In the end the key was function.call, which allows an object to call a function as if the function were a member of the object. Using function.call I could call a function on the base object’s prototype as if it were a member of the derived object, finally allowing the derived object to override functions without losing the functionality already coded into the base object. Thus emboldened, I combined Prototype’s Object.extend function with prototypes to implement inheritance and polymorphism like this:

function BaseObject(baseProperty) {
    this.baseProperty = baseProperty;
}
BaseObject.prototype = {
    overriddenFunction: function(parameter) { ... }
}

function DerivedObject(baseProperty, derivedProperty) {
    BaseObject.call(this, baseProperty);
    this.derivedProperty = derivedProperty;
}
DerivedObject.prototype = new BaseObject();
Object.extend(DerivedObject.prototype, {
    function overriddenFunction: function(parameter) {
        BaseObject.prototype.overriddenFunction.call(this, parameter);
        ...
    }
});

Internet Explorer Sux0rs

Sorry for the inflammatory post title but, given the number of problems that Internet Explorer has caused me, I think that it is justified. During the testing the Kidd release I came across many many Internet Explorer glitches, some were fixed before release while a few others are still pending. The issues are annoyances rather than showstoppers, and this release is such a big improvement that I didn’t want to delay it anymore. I am not an expert web developer by any means, so I can’t rule out my code as source of the problem, but I think the fact that my code works beautifully in Firefox yet has so many glitches in Internet Explorer is telling.

The first problem that I discovered was with the scroll bar for the text area where you enter messages. If the CSS width of the text area is defined as a percentage rather than a fixed value then the scroll bar is offset to the right by the amount of the border and padding of the containing element when text is entered. At first the scroll bar is drawn correctly, and if the text area is resized then the scroll bar is again correctly positioned until more text is entered. This effect occurs even when the scroll bar is not visible, instead offsetting the window background. The image below shows the problem, and I’ve also created a simple test case.

I was able to fix the problem using a unique feature of Internet Explorer — CSS expressions. I added the following code to the stylesheet:

textarea.messageFormBody {
	width: expression(this.parentNode.parentNode.parentNode.offsetWidth - 19);
}

All’s well that ends well? Not quite. First, you can see that I had to hardcode the offset. If the padding of any of the containing elements changes, this value will need to be updated. Worse yet, this code can crash Internet Explorer! If the offset is less than amount of padding between the containing elements and the text area, IE resizes the containing elements to fully contain the text area causing an infinite loop as IE then resizes the text area again, and so on. Finally, this code causes a slight glitch when opening a chat window for the first time as IE initially sizes the text area, which you can see in the image below. I decided this was a worthwhile tradeoff, but my Internet Explorer woes were just beginning…

Another annoyance was IE adding extra padding to the first group in an identity, and the first contact in a group. In the image below you can see that there is extra space between the name of the AIM Bots group and the first contact in the group, while the first contact in the Buddies group appears immediately below the name. Also, there is more space between the first contact in the AIM Bots group than there is between the second contact and the next pager node.

After creating another simple test case I determined that this was because I was defining the font properties in the block-level elements and letting the inline elements inherit rather than setting the properties directly on the inline elements. IE still adds two pixels of extra padding even after updating the stylesheet, but at least now it is uniform. In the next release I may take advantage of IE’s ability to use conditional stylesheets to remove the padding from these elements, instead relying on IE’s extra whitespace to provide the proper spacing (since IE clearly knows padding better than I do), but I’m not sure that I want to stoop to IE’s level by using such a hack.

While these glitches were mere annoyances, the next issue was a major problem. If IE’s Temporary Internet Files settings are set to “Every visit to the page” then IE will request background images from the server every time a dynamic action is taken on the page! This causes the images to flicker, but more importantly it creates a ton of extra requests to the server. What at first appeared to be an annoyance turned out to be a threat to my design principles! Although this is a browser issue, it can be fixed with some server-side tweaks. I have incorporated those settings into the Apache HTTP Server configuration snippet included in the distribution, but Apache HTTP Server is an optional package; the issue remains when serving Yafumato directly from Apache Tomcat. I have noted this issue in the release notes with instructions for ensuring that the IE is properly configured to avoid this problem. More information about this issue is available at http://www.fivesevensix.com/studies/ie6flicker/.

Next, IE doesn’t create horizontal scroll bars for my DHTML windows’ contents. Instead it resizes the div to hold the contents without side-scrolling and refuses to resize the element any smaller than that. The windowing code sets the minimum resize width to be the width of the window contents, but that is set when the window is created; dynamically added content can still be a problem. This is still an open issue.

Finally, there are glitches with the script.aculo.us combination effects. Adjacent elements are briefly offset when using effects such as “SlideUp” or “Squish” to hide an element. Also, when using an expanding effect to make a large element appear, the portions of the expanding element that are outside of a DHTML window are briefly visible; the element is only clipped when the effect completes.

The Ties that Bind

Another aspect of my JavaScript implementation that I wrestled with was how to link an object to the HTML elements that visually represent it. An object needed to be able to update it’s visual representation when its state changed; conversely a page elements needed to be able identify its corresponding object when responding to an event. In Yafumato 0.5, each element that needed to be dynamically updated was given a unique id attribute value. The element displaying the name of the moviephone bot, for example, would be identified as pager.aim.AIM Bots.moviephone.name. When the contact object representing the moviephone bot was updated it would use document.getElementById to retrieve the elements that needed to be updated. Events were assigned by changing the innerHTML of an element.

I wasn’t particularly fond of this design, and I wanted to improve upon it. BrainJar.com’s Bride of Windows implementation showed me another way. In BrainJar’s code, the elements that display a window are assigned as properties on the window objects, and vice versa. I also learned a bit more about the JavaScript’s event handling system while working with the code.

Thus, I set about to assign the elements that visually represent an object to properties on that object, but what to name these properties? The most obvious variable names were already taken by the data properties themselves. I considered appending “Element” to the name of the data property in order to create the name of the element that displays it, e.g. nameElement for the element representing the name property, but that seemed unwieldy. The solution came to me while reading the Dojo JavaScript Style Guide:

Variables that are intended to be private, but cannot be based on the semantics of JavaScript, SHOULD pre-pended with a “_” (underscore) char.

This naming scheme made perfect sense for me — no other object should be manipulating an object’s visual representation, and it allowed me to essentially reuse the name of the data property.

I didn’t abandon using ids entirely, though. Rather than make these cross-assignments in the JavaScript that creates the objects and elements, I give the HTML elements ids so that they can be identified in order to bind them to the object that they represent using the object’s bindToDocument function. Why? First, there’s encapsulation; the logic to identify and bind the elements that represent an object resides within the object itself, rather than the code that creates the object. Another advantage to this method is flexibility – I can bind an object to elements that were statically created in the HTML document or elements that have been dynamically created in JavaScript.

Ye Olde Firefox Bugs

Internet Explorer has the lion’s share of browser-specific problems, but I have come across a couple of Firefox bugs while developing Yafumato. I’ve already posted about the IE problems that I’ve encountered, so in the interest of fairness I will now discuss the Firefox issues.

Under certain conditions Firefox will not resize a table when content is dynamically added to it (Bug 287620). This bug affected the old, tables-based version of the messaging page. I can’t venture to say what the conditions are, but a new row containing a message might get pushed outside of the enclosing table and even off the bottom of the screen when appended to the page. A screenshot of the problem is below, where you can see the messaging text overlapping the border of the table that contains it.

Bug 287620 is not longer an issue with the DHTML window interface, but now there is another. Bug 167801 details problems with text area cursors when a text area is above another element. When the text area of a chat window is positioned above the message content of another chat window, the text area cursor is not visible. The screenshot below demonstrates the problem – only the part of the cursor that is not above the message area of the chat window below it is can be seen. This is the case even if the chat window below is not open and therefore not visible. However, if the lower chat window is minimized (which explicitly hides the message content), the problem does not occur. I also find it strange that the cursor is visible if it is above the portion of a chat window containing the text area that belongs to that window.

Another quirk is that if a bottom of a chat window is above another window and the text area of the lower window has focus, the cursor shows through. This occurs in both Firefox and Internet Explorer. This would seem to be related to the oddity noted above.

So, there have been a few Firefox issues, but one thing that helps is that these bugs are documented in a publicly accessible bug database. Internet Explorer has no such database, at least not for current and past versions. There is a bug database for IE 7 Beta, but it requires a Passport account to view it.

In Space, No One Can Hear You Stream

The messaging page currently uses polling to get updates, but I recently read about an alternative called HTTP streaming, or Comet. HTTP streaming is an AJAX communication technique designed to reduce latency by keeping an HTTP connection open and having the server periodically flush data to the client over this connection rather than immediately returning data and closing the connection. This technique would reduce latency when receiving messages, since the server could send messages as soon as it receives them rather than having to wait until the client requests an update. Based on what I have read, true HTTP streaming — where the server periodically sends data to the client while the client monitors the data that it has received without closing the connection — is not supported in Internet Explorer. This is because IE does not make the data that has been received from the server available until the connection has been closed. However, there is a variant of this technique that can be used in IE. Rather than keep the connection open and periodically send data, the server only keeps the connection open until it has data to send, at which point the server sends the data and closes the connection. The client would then be responsible for opening a new connection. This pseudo-streaming technique would still reduce latency, though not necessarily the number of server hits. My concern with this technique, though, is the length of time that the connection is open; it might draw undue attention in a monitored environment. Since I am not familiar with web monitoring software I don’t know which would draw more attention — a larger number of quick connections or a smaller number of connections that remain open for some time.

Where Do We Go Now?

After releasing Kidd, I wasn’t quite sure what to do next, but now I have made a few decisions regarding the functionality that I want to implement in the next release. I currently have three goals:

A> Fix Internet Explorer-specific issues. The major outstanding IE issue is horizontal scrolling of DHTML window contents. I may also add a resize callback to the windowing code to assist with the text area scroll bar problem. That issue is currently solved by an IE-only hack, but it causes a glitch when chat windows are first opened.

2> Decouple the database. I want to make MySQL an optional package to simplify the installation process. Single sign-on and message logging, however, require the database and would be unavailable without it. My plan is to make message logging optional and allow users to sign in directly with their messenger credentials.

D> Add HTTP streaming <Comet>. Given the uncertainty about web monitoring, I plan to implement streaming side-by-side with polling rather than as a complete replacement.

Talkin’ Smack

I’ve found a Jabber API for Java called Smack. Jabber is the protocol that Google Talk uses. Smack looks pretty robust and easy to use, so I will probably use it to integrate Google Talk into Yafumato once I have accomplished my current development goals. Adding Google Talk will give me a chance to work on the server-side Java code again, which could be a welcome change of pace since I have been working on the client-side scripting for the past several months.