Wednesday, April 23, 2014

Smarter dates and times

Orbeon Forms prior to version 4.4 used to display full dates and times in the Summary page:

Full dates
Full dates

We didn’t like this because the resulting dates and times took a lot of precious space on screen.

One way to solve this is to indicate relative dates such as “3 hours ago”, “20 days ago”, or “1 year ago”. [1] But we don’t like this either because the resulting information is often rounded to the point of being meaningless.

Instead, we chose a very simple method inspired by applications such as Google Drive. [2] We use exactly 3 date and time formats:

  • if the instant is at most one day in the past, show hours and minutes only
  • if the year is the same as the current year, show the month and day only
  • otherwise, show year, month and day in a compact format

The result is as follows:

Short dates
Short dates

These shorter dates present enough meaningful information and always in a compact way. Of course this remains a tradeoff: for example if you want to save a screenshot for further reference, it would better to have full dates and times. But for most common uses, we think this is a pretty good approach. Let us know what you think!

This change is available in Orbeon Forms 4.4 and newer.


  1. As GitHub does, for example.  ↩

  2. Formerly Google Docs.  ↩

Friday, April 18, 2014

Solving the observer problem

Abstract

As most sophisticated web apps, Orbeon Forms has to deal with lots of asynchronous events in the browser, like Ajax requests, timers, and user interactions. We'll go over a real-life example of logic involving several asynchronous events, see the difficulty involved in implementing it with observers, and explore 3 alternatives: state machines, await, and Functional Reactive Programming (FRP). We'll see why, based on our experience, we don't consider state machines to be a good solution, and through an example explore the two remaining alternatives, await and FRP.

Example of the loading indicator

Orbeon Forms can show on the page a "loading indicator" to inform users that something is happening: data was sent to the server, and we're waiting for a response, maybe because we need to validate user input, save data, or call a service. Its logic works is as follows:
  1. In essence, when a request is in progress, we want the loading indicator to show.
  2. However, if the request takes very little time, we don't want to show the loading indicator.
  3. Also, when a request comes back, we don't want to hide the indicator if we already know we'll right away send another request to the server.
Stated in plain English, this is quite simple.

The problem with observers

The browser gives us events, and we can run code when those events happen. Code running upon some event is called an observer. For the loading indicator, we would have an observer when an Ajax request starts, and another one when it ends. Since we don't want to show the indicator right away, the observer on the "Ajax request started" starts a timer for say 200 ms. So we would have an observer on that timer, which on completion shows the indicator if the request is still in progress. But to know that, we need to have state (isRequestInProgress), shared between the two observers.
But a boolean isn't enough. Say you start a first request at t = 0 ms. At that point, you also start a timer for 200 ms. Say that first request ends at t = 100 ms. Now say a second request starts at t = 150 ms. At t = 200 ms, the first timer ends: a request is still in progress, so it shows the loading indicator. Bug! At that point the request has been going on for only 50 ms, so we shouldn't show the loading indicator just yet. One way to solve this problem is to use a counter, instead of a boolean, an increment/decrement that counter at the right time.
I won't go here through all the details of the implementation, but the end result is that the above logic is spread across a number of observers that share and update some global state. Logic written this way is surprisingly hard to maintain, and subtle bugs can lead to a loading indicator that stays "stuck", i.e. shown even if there no Ajax request in progress, or that doesn't show anymore.

Possible solutions

We're aware of 3 alternatives to observers:
  1. State machines
  2. await
  3. Functional reactive programming (FRP)

State machines

In Form Builder, we started using a state machine about 2 years ago, and had a mixed experience with that approach. A state machine solves part of the observer problem, by putting the high level logic in a single place, in a fairly declarative form. However, we found that complex state machines are hard to debug, as you can't debug the declaration of the state machine itself, but instead have to debug the generic implementation of the state machine.

Experimenting with await and FRP

Before going for await or FRP in production code, we wanted to do some experimentation. For this, we took the example described in the excellent paper Deprecating the Observer Pattern. The example uses the canvas and the mousedown, mousemove, and mouseup events to let the user draw on the canvas. You'll find below the CoffeeScript code for 3 implementations:
  1. With observers, using plain jQuery.
  2. With await, using IcedCoffeeScript.
  3. With FRP, using Bacon.js.
For each case, you can click on the link after the code to view the full source and run that example.

With observers

isDown = false

canvas.on 'mousedown', (e) -> 
    isDown = true
    startLine(e)

canvas.on 'mousemove', (e) -> 
    if isDown
        continueLine(e)

canvas.on 'mouseup', (e) -> 
    isDown = false
See and run the code

With await

while true

  await canvas.one('mousedown', defer(e))
  startLine(e)

  until e.type == 'mouseup'
      rv = new iced.Rendezvous
      canvas.one('mousemove', rv.id().defer(e))
      canvas.one('mouseup'  , rv.id().defer(e))
      await rv.wait(defer())
      continueLine(e)
See and run the code

With FRP

downStream         = canvas.asEventStream('mousedown')
upStream           = canvas.asEventStream('mouseup')
moveStream         = canvas.asEventStream('mousemove')

isDownProp         = downStream.map(true)
                               .merge(upStream.map(false))
                               .toProperty(false)
moveWhenDownStream = moveStream.filter(isDownProp)

downStream         .onValue(startLine)
moveWhenDownStream .onValue(continueLine)
See and run the code

Conclusion

We believe that both await and FRP are significant improvement over observers or a state machine. At this point, we have a slight preference for await, as we think that:
  • Most developers will find it easier to understand code using await than code using FRP, and we believe in Erik Meijer's mantra: "write baby code".
  • We think that code using await will be easier to debug than code using FRP, as it will be easier to put breakpoints in the right place, and follow the code execution.

Monday, April 7, 2014

Orbeon Forms 4.5

Today we released Orbeon Forms 4.5!

Major features

This release is rich in new features and enhancements:
  • IE11 support. Both Form Builder and deployed forms now work with with IE11.
  • Versioning of form definitions. Publishing a form definition doesn't overwrite the previously published form definition, but creates a new version of it. Orbeon Forms associates the version of the form definition used for creating and editing data. This is supported with relational databases (Oracle, MySQL, and DB2 at this time). (blog post)
  • Improved fields, checkboxes and radio buttons in preview and PDF. Automatic PDF output now shows all radio buttons and checkboxes, nicely highlights fields, supports optional page breaks before sections, and shows the form's title in the page's header and footer. (blog post)
  • Inserting and reordering of grid rows. Repeated grids now have the ability to reorder grid rows and to specify exactly where to insert a new row. (blog post)
  • Repeated sections. Form Builder and Form Runner now support repeated section content. This is very similar to repeating a group of grid rows, except that the entire content of a section can be repeated, including nested grids and subsections. (blog post)
  • Improved help messages appearance and positioning. Help messages have a fresher, more modern look, and are now more lightweight and better positioned. (blog post)
  • Hints for checkboxes and radio buttons. You can now associate hint messages with specific radio button or checkboxes. They appear as tooltips upon hover. (blog post)
  • Form definitions and form data duplication. The summary pages for both Form Builder and your own forms now can have a Duplicate button which allows you to duplicate form definitions (in Form Builder) and form data, including attachments. (blog post)
  • Improved dropdown menus. Dropdown menus are now a bit smarter! (blog post)
  • Improved Form Builder Choices editor. The Form Builder Choices editor allows you to reorder items and to directly localize labels and hints. (blog post)
  • Improved Form Builder variable resolution. We have fixed a long-standing issue which prevented the use of variables in repeated grids. (doc)
  • Detecting login pages in Ajax requests. Administrators can now tell Orbeon Forms which pages are login pages, to help with error recovery. (blog post)
Internationalization:
  • This version features full Spanish localization, courtesy of Bruno Buzzi of AGESIC.
  • This version also features Swedish localization for Form Runner.
See Localizing Orbeon Forms for more details about supported languages.
Other bug-fixes and features

Including the major features and enhancements above, we closed over 110 issues since Orbeon Forms 4.4. We should mention these notable bug-fixes and features:
  • XForms 2.0 bind() function (#1423)
  • Service request or response cannot target control in subsection (#1465)
  • Support repeats in actions (#1105)
  • PDF: Option to output a page break before a section (#1556)
  • FB: UI to edit control, grid and section classes (#1555)
  • PDF: Ability to set form title in header and/or footer (#1558)
  • Processes: reusable confirmation dialog (#1570)
  • XML Schema xs:import doesn't support relative paths (#1340)
Current browser support
  • Form Builder (creating forms):
    • Chrome (latest versions, both in the stable and dev channels)
    • Firefox (latest released version and current Firefox ESR)
    • IE10 and IE11
    • Additional support for Form Runner (accessing form):
      • IE7 (deprecated), IE8, and IE9
      • Safari Mobile on iOS 6 and iOS 7
      • Chrome for Android (stable channel)
    Compatibility notes
    • Date picker. The Date Picker control has been removed from Form Builder. It was similar to the normal Date control, except the date was shown as text in the page, not in a text field. The Date Picker didn't have much of a benefit over the regular Date control, but had a big drawback: it wasn't usable on mobile devices, which expect a text field to enable the native date picker. Note that while the control isn't available in the Form Builder sidebar anymore, the underlying implementation is still there, so your existing forms that use a Date Picker can still be edited in Form Builder, and will still function properly. (#1364)
    • Datatable. The fr:datatable component, deprecated in 4.0, has been completely removed.
    • Deployment. Integrated deployment, consisting in combining your Java/JSP applications with Orbeon Forms within the same WAR, is now deprecated. We recommend using separate deployment instead.
    • Oracle flat view. The truncation of section/control column names has changed. If the section or control name is 14 characters or less, it is kept as is, and the other part is truncated if needed. A numerical suffix is used instead for those columns which would introduce duplicates. (doc)
    • Tamino support. Unmaintained support for the Tamino XML database has been removed.
    You can download the latest version of Orbeon Forms from the downloads page.
      Don't forget to grab a trial license for the PE version.

      Please send feedback:
      We hope you enjoy this release!

      Friday, April 4, 2014

      Configure the URL of your services in properties

      Rationale

      A few weeks ago, we've seen how you can store Orbeon Forms configurations, in particular the properties-local.xml, outside of the Orbeon Forms war file. We've seen how this allows you to have a single war you deploy on all your environments, and keep differences in configurations between these environments outside of the war file.

      This is all good for the Orbeon Forms configurations. Here we'll see that you can use the same mechanism to define custom properties to specify which on what server your services run, or to what server you want data to be posted on submission.

      The xxf:property() XPath function

      Say you have forms you created with Form Builder, those forms use services, and in the service URL you want a different host name depending on the environment you're in. But of course, you don't want to change the forms as you run them on different environments. You can do this by defining a property, say:
      <property as="xs:string"  
                name="com.example.service.host"
                value="prod.example.com"/>
      
      Then, in Form Builder, in the HTTP Service Editor dialog, you refer to the value of the property with:
      http://{xxf:property('com.example.service.host')}/your/service
      

      Submission server

      When users submit a form, you can setup Orbeon Forms to POST the data captured by the form to one of your servers. You do this with the send action, e.g.:
      send(uri = "http://prod.example.com/your/service")
      
      However, if you'd like, you can also move the server URL to its own property:
      <property as="xs:anyURI"
                name="com.example.send.uri"
                value="http://prod.example.com/your/service"/>
      
      And refer to that property in the send action (note that there is .uri at the end of the value passed to send, as the parameter to send is only a prefix, to which send can add different suffixes; for more on this, see the send action):
      send(property = "com.example.send")