Monday, October 29, 2007

A disk-based XForms state store for Orbeon Forms

Rotor

We just committed changes in Orbeon Forms to handle a disk-based XForms state store. What is this about? As discussed recently, Orbeon Forms needs to store XForms state information somewhere. One option is client-side, but this has drawbacks in terms of bandwidth and latency. So the default since the beginning has been to store state on the server.

Until this last round of changes, Orbeon Forms would keep some of the information in a session LRU memory store and then discard it completely. You had to discard information, because RAM is limited. But this approach has proven problematic because, however good the expiration heuristic, in many cases the state information would be lost completely, leaving the user in the dust. This includes cases such as the user going back to a page after a while, or accessing multiple large pages within a same session.

So we had to fix this, and you can say welcome to the brand new disk-based session store! With this mechanism, state information which expires from RAM is instead migrated to a disk store. Since disk is cheaper than RAM (you can maybe allocate a few tens of MB of state store in RAM, but probably several GB of state store on disk, at least two orders of magnitude more), memory is no longer really a constraint. (This can work because disk access is expected to be rare, following the principle of locality of access, which is central to all caching systems.)

Besides making the user experience better, we hope that this larger and more reliable store will enable Orbeon forms to support very cool features soon, including built-in undo and history checkpoints in forms. This should take Orbeon Forms where few Ajax platforms have already been.

Even with a disk-based store, you need to expire data otherwise it will grow indefinitely. In this first iteration of the store, we implement a fairly simple expiration stategy: stuff expires from the database when the user session is terminated. This is not perfect, but will cover a large number of uses cases.

Technically, we have made a fairly bold step and decided to use the embedded eXist database, included with Orbeon Forms, as the persistence layer for XForms state information. One reason is that we did not want to deal with writing our own persistence layer. Another reason is that this will give us the power of XQuery to implement expiration strategies in the future.

This feature is now available in the Orbeon 3.6 beta nightly builds, and will be enabled by default in Orbeon Forms 3.6.

No comments:

Post a Comment