Yafumato Development Log

Yafumato 0.7.1 <Strife>

Yafumato 0.7.1 <Strife> has been released. MySQL is now an optional package; you just need Java and Tomcat to get a Yafumato server up and running. Pre-built WAR files for both Tomcat 5.0 and Tomcat 5.5 are included in the distribution.

The existing single sign-on authentication method requires the database, but now you can also sign in to messenger services individually using your messenger credentials.

Messenger session handling has also been improved; messenger sessions are now saved across HTTP sessions. Active messenger sessions, those that have been successfully connected to the messenger service but not yet disconnected by you, will be recovered during sign in. Additionally, unstable messenger sessions can be disconnected and replaced using the new Reset Messenger link on the status page.

The next release, Aurion, will feature HTTP streaming for more responsive messaging. With my current schedule of teaching and business school applications, however, I won’t be able to begin working on the next release until at least late November.

DHTML Windowing Toolkit 0.2

I have released the JavaScript windowing code used in Yafumato as a separate library called the DHTML Windowing Toolkit (DWT).

The DHTML Windowing Toolkit is a cross-browser JavaScript/CSS library for the creation of inline DHTML windows to contain content. These windows can be opened, closed, moved, and resized much like native windows. DWT 0.2 is now available. I consider the DWT included in Yafumato 0.6 <Kidd> to be version 0.1, while the updated version included in Yafumato 0.6.1 <Kidd A> is the current, 0.2 release. DWT 0.2 provides an object model for creating windows from existing elements, but cannot create the requisite elements for you. The demo included with the distribution includes examples of both static HTML and dynamic JavaScript window elements. DWT 0.2 supports Microsoft Internet Explorer, Mozilla Firefox, Opera, and Konqueror (see release notes for browser specific notes).

I wanted to split the DWT out into a separate project because I think it is a very slick piece of code that other web developers may want to use. The DWT is based on BrainJar.com’s Bride of Windows, but has been heavily modified. In particular, I have tried to allow more freedom with the window decoration. Also, the DWT code is much more object-oriented. Prototype is used for cross-browser event handling, and optional script.aculo.us effects are integrated.

The most challenging feature to implement (a feature that isn’t even used in Yafumato!) was the ability to specify multiple north-south resize targets. I had considerable difficulty distributing the pixels evenly, especially once one resize target had reached its minimum size. Originally, if there was a remainder after dividing the number of pixels that the mouse had been moved by the number of targets, each target whose index was lower than the remainder got a remainder pixel. For example, if there were two targets and the mouse was moved by three pixels, the first target would two pixels (3 / 2 + 3 % 2) while the second target would simply get one (3 / 2). This worked fairly well, but what if the user constantly resized by only one pixel? In that case, the first target would always get a remainder pixel and the second target would never get one. So, I decided to add a Boolean value to each resize target to determine whether or not it had received a remainder pixel from the last mouse move; this way the remainder pixel would alternate between the two resize targets. Nice, but what if there were three resize targets? Only the first two resize targets would get remainder pixels in this manner; the third target would never get any pixels because one of the two above it would not have been resized after the last mouse move and thus would get the pixel. The technique that I finally settled on was to give each resize target a counter that to keep track of how many times that target had been given a remainder pixel; the resize target with the lowest number of remainder pixels would get it.

So now the remainder pixels were evenly distributed, but what if the user was shrinking a window and one resize target reached its minimum size? At first, a target that reached its minimum size continued to reduce the number of pixels distributed to the other resize targets, resulting in a discrepancy between the number of pixels the mouse had been moved and the number of pixels the window had been resized. This effect was quite annoying. Thus, I changed the code so that a resize target that had reached its minimum size was no longer given any pixels when shrinking a window. However, the decision as to whether the window was shrinking or enlarging was made based on the window size when the resize drag was started. In the event that a user shrunk the window so that a resize target was at its minimum size and then started enlarging the window, the target at minimum size would not get any more pixels until the window became larger than it was when the resize was started; at that point the target would “pop” back to original size. To solve this, I had to determine whether the window was shrinking or enlarging relative to the previous mouse move rather than the original window size, which I did by saving the size after each mouse move.

Finally, resizing from the top of the window, which requires moving the window as well as resizing the contents, was also a challenge. In the end I am quite happy with the results, even if I don’t make use of them myself!

Yafumato 0.7 <ZEX>

I haven’t posted in a while — work, training, holidays, birthdays, and general summer fun have kept me too busy to post — so Yafumato 0.7 <ZEX>, has been available for some time now. This version adds Google Talk (via the aforementioned Smack library) and MSN Messenger (via the JML library) to the collection of supported messengers. There was nothing particularly noteworthy about the development of this release; it was simply a matter of creating additional implementations of my generic messenger interface.

I am currently testing the next release, <Strife>. When this version is released MySQL will no longer be required to run Yafumato. Single sign-on and message logging will still require the database, but you will also be able to sign in using your individual messenger credentials. This should make it easier to get Yafumato up and running since fewer software packages and configuration steps will be required. Additionally, messenger sessions will be saved across HTTP sessions. When you sign in active messenger sessions will be restored into the new HTTP session rather than simply creating new messenger sessions and signing into the messengers again. I believe that, with these changes, Yafumato will be viable as a public server; I am currently investigating hosting options.

Yafumato 0.6.1 <Kidd A>

Yafumato 0.6.1 <Kidd A> is now available. I’ve cleaned up the windowing code a bit, fixing the horizontal scrolling and text area sizing issues in Internet Explorer. The Opera browser is now supported, though it has issues of its own (see release notes for details).

Talkin’ Smack

I’ve found a Jabber API for Java called Smack. Jabber is the protocol that Google Talk uses. Smack looks pretty robust and easy to use, so I will probably use it to integrate Google Talk into Yafumato once I have accomplished my current development goals. Adding Google Talk will give me a chance to work on the server-side Java code again, which could be a welcome change of pace since I have been working on the client-side scripting for the past several months.

Where Do We Go Now?

After releasing Kidd, I wasn’t quite sure what to do next, but now I have made a few decisions regarding the functionality that I want to implement in the next release. I currently have three goals:

A> Fix Internet Explorer-specific issues. The major outstanding IE issue is horizontal scrolling of DHTML window contents. I may also add a resize callback to the windowing code to assist with the text area scroll bar problem. That issue is currently solved by an IE-only hack, but it causes a glitch when chat windows are first opened.

2> Decouple the database. I want to make MySQL an optional package to simplify the installation process. Single sign-on and message logging, however, require the database and would be unavailable without it. My plan is to make message logging optional and allow users to sign in directly with their messenger credentials.

D> Add HTTP streaming <Comet>. Given the uncertainty about web monitoring, I plan to implement streaming side-by-side with polling rather than as a complete replacement.

In Space, No One Can Hear You Stream

The messaging page currently uses polling to get updates, but I recently read about an alternative called HTTP streaming, or Comet. HTTP streaming is an AJAX communication technique designed to reduce latency by keeping an HTTP connection open and having the server periodically flush data to the client over this connection rather than immediately returning data and closing the connection. This technique would reduce latency when receiving messages, since the server could send messages as soon as it receives them rather than having to wait until the client requests an update. Based on what I have read, true HTTP streaming — where the server periodically sends data to the client while the client monitors the data that it has received without closing the connection — is not supported in Internet Explorer. This is because IE does not make the data that has been received from the server available until the connection has been closed. However, there is a variant of this technique that can be used in IE. Rather than keep the connection open and periodically send data, the server only keeps the connection open until it has data to send, at which point the server sends the data and closes the connection. The client would then be responsible for opening a new connection. This pseudo-streaming technique would still reduce latency, though not necessarily the number of server hits. My concern with this technique, though, is the length of time that the connection is open; it might draw undue attention in a monitored environment. Since I am not familiar with web monitoring software I don’t know which would draw more attention — a larger number of quick connections or a smaller number of connections that remain open for some time.

Ye Olde Firefox Bugs

Internet Explorer has the lion’s share of browser-specific problems, but I have come across a couple of Firefox bugs while developing Yafumato. I’ve already posted about the IE problems that I’ve encountered, so in the interest of fairness I will now discuss the Firefox issues.

Under certain conditions Firefox will not resize a table when content is dynamically added to it (Bug 287620). This bug affected the old, tables-based version of the messaging page. I can’t venture to say what the conditions are, but a new row containing a message might get pushed outside of the enclosing table and even off the bottom of the screen when appended to the page. A screenshot of the problem is below, where you can see the messaging text overlapping the border of the table that contains it.

Bug 287620 is not longer an issue with the DHTML window interface, but now there is another. Bug 167801 details problems with text area cursors when a text area is above another element. When the text area of a chat window is positioned above the message content of another chat window, the text area cursor is not visible. The screenshot below demonstrates the problem – only the part of the cursor that is not above the message area of the chat window below it is can be seen. This is the case even if the chat window below is not open and therefore not visible. However, if the lower chat window is minimized (which explicitly hides the message content), the problem does not occur. I also find it strange that the cursor is visible if it is above the portion of a chat window containing the text area that belongs to that window.

Another quirk is that if a bottom of a chat window is above another window and the text area of the lower window has focus, the cursor shows through. This occurs in both Firefox and Internet Explorer. This would seem to be related to the oddity noted above.

So, there have been a few Firefox issues, but one thing that helps is that these bugs are documented in a publicly accessible bug database. Internet Explorer has no such database, at least not for current and past versions. There is a bug database for IE 7 Beta, but it requires a Passport account to view it.

The Ties that Bind

Another aspect of my JavaScript implementation that I wrestled with was how to link an object to the HTML elements that visually represent it. An object needed to be able to update it’s visual representation when its state changed; conversely a page elements needed to be able identify its corresponding object when responding to an event. In Yafumato 0.5, each element that needed to be dynamically updated was given a unique id attribute value. The element displaying the name of the moviephone bot, for example, would be identified as pager.aim.AIM Bots.moviephone.name. When the contact object representing the moviephone bot was updated it would use document.getElementById to retrieve the elements that needed to be updated. Events were assigned by changing the innerHTML of an element.

I wasn’t particularly fond of this design, and I wanted to improve upon it. BrainJar.com’s Bride of Windows implementation showed me another way. In BrainJar’s code, the elements that display a window are assigned as properties on the window objects, and vice versa. I also learned a bit more about the JavaScript’s event handling system while working with the code.

Thus, I set about to assign the elements that visually represent an object to properties on that object, but what to name these properties? The most obvious variable names were already taken by the data properties themselves. I considered appending “Element” to the name of the data property in order to create the name of the element that displays it, e.g. nameElement for the element representing the name property, but that seemed unwieldy. The solution came to me while reading the Dojo JavaScript Style Guide:

Variables that are intended to be private, but cannot be based on the semantics of JavaScript, SHOULD pre-pended with a “_” (underscore) char.

This naming scheme made perfect sense for me — no other object should be manipulating an object’s visual representation, and it allowed me to essentially reuse the name of the data property.

I didn’t abandon using ids entirely, though. Rather than make these cross-assignments in the JavaScript that creates the objects and elements, I give the HTML elements ids so that they can be identified in order to bind them to the object that they represent using the object’s bindToDocument function. Why? First, there’s encapsulation; the logic to identify and bind the elements that represent an object resides within the object itself, rather than the code that creates the object. Another advantage to this method is flexibility – I can bind an object to elements that were statically created in the HTML document or elements that have been dynamically created in JavaScript.

Internet Explorer Sux0rs

Sorry for the inflammatory post title but, given the number of problems that Internet Explorer has caused me, I think that it is justified. During the testing the Kidd release I came across many many Internet Explorer glitches, some were fixed before release while a few others are still pending. The issues are annoyances rather than showstoppers, and this release is such a big improvement that I didn’t want to delay it anymore. I am not an expert web developer by any means, so I can’t rule out my code as source of the problem, but I think the fact that my code works beautifully in Firefox yet has so many glitches in Internet Explorer is telling.

The first problem that I discovered was with the scroll bar for the text area where you enter messages. If the CSS width of the text area is defined as a percentage rather than a fixed value then the scroll bar is offset to the right by the amount of the border and padding of the containing element when text is entered. At first the scroll bar is drawn correctly, and if the text area is resized then the scroll bar is again correctly positioned until more text is entered. This effect occurs even when the scroll bar is not visible, instead offsetting the window background. The image below shows the problem, and I’ve also created a simple test case.

I was able to fix the problem using a unique feature of Internet Explorer — CSS expressions. I added the following code to the stylesheet:

textarea.messageFormBody {
	width: expression(this.parentNode.parentNode.parentNode.offsetWidth - 19);
}

All’s well that ends well? Not quite. First, you can see that I had to hardcode the offset. If the padding of any of the containing elements changes, this value will need to be updated. Worse yet, this code can crash Internet Explorer! If the offset is less than amount of padding between the containing elements and the text area, IE resizes the containing elements to fully contain the text area causing an infinite loop as IE then resizes the text area again, and so on. Finally, this code causes a slight glitch when opening a chat window for the first time as IE initially sizes the text area, which you can see in the image below. I decided this was a worthwhile tradeoff, but my Internet Explorer woes were just beginning…

Another annoyance was IE adding extra padding to the first group in an identity, and the first contact in a group. In the image below you can see that there is extra space between the name of the AIM Bots group and the first contact in the group, while the first contact in the Buddies group appears immediately below the name. Also, there is more space between the first contact in the AIM Bots group than there is between the second contact and the next pager node.

After creating another simple test case I determined that this was because I was defining the font properties in the block-level elements and letting the inline elements inherit rather than setting the properties directly on the inline elements. IE still adds two pixels of extra padding even after updating the stylesheet, but at least now it is uniform. In the next release I may take advantage of IE’s ability to use conditional stylesheets to remove the padding from these elements, instead relying on IE’s extra whitespace to provide the proper spacing (since IE clearly knows padding better than I do), but I’m not sure that I want to stoop to IE’s level by using such a hack.

While these glitches were mere annoyances, the next issue was a major problem. If IE’s Temporary Internet Files settings are set to “Every visit to the page” then IE will request background images from the server every time a dynamic action is taken on the page! This causes the images to flicker, but more importantly it creates a ton of extra requests to the server. What at first appeared to be an annoyance turned out to be a threat to my design principles! Although this is a browser issue, it can be fixed with some server-side tweaks. I have incorporated those settings into the Apache HTTP Server configuration snippet included in the distribution, but Apache HTTP Server is an optional package; the issue remains when serving Yafumato directly from Apache Tomcat. I have noted this issue in the release notes with instructions for ensuring that the IE is properly configured to avoid this problem. More information about this issue is available at http://www.fivesevensix.com/studies/ie6flicker/.

Next, IE doesn’t create horizontal scroll bars for my DHTML windows’ contents. Instead it resizes the div to hold the contents without side-scrolling and refuses to resize the element any smaller than that. The windowing code sets the minimum resize width to be the width of the window contents, but that is set when the window is created; dynamically added content can still be a problem. This is still an open issue.

Finally, there are glitches with the script.aculo.us combination effects. Adjacent elements are briefly offset when using effects such as “SlideUp” or “Squish” to hide an element. Also, when using an expanding effect to make a large element appear, the portions of the expanding element that are outside of a DHTML window are briefly visible; the element is only clipped when the effect completes.