Wednesday, November 28, 2012

Simplify your queries with Oracle's dense_rank

Data persistence for forms created with Form Builder is done through a REST API. You can implement that API, or use one of the provided implementations. Orbeon's implementation for relational databases, for now specifically MySQL and Oracle, always uses inserts instead of doing updates or deletes1. For this, tables have a timestamp column, and when data is "updated", a new row is added with the current timestamp. So to retrieve data, we need to get the row matching a condition that has the latest timestamp.

Let's transpose this problem to an equivalent one on the emp table, to make it easier for you to run the queries below in your own database2. On the emp table, a similar problem is to find for each job, the employee with the most recent hire date3. This can be done with nested queries:
select e1.* 
  from emp e1,
       (  select job, max(hiredate) hiredate 
            from emp 
        group by job ) e2
 where e1.job = e2.job
       and e1.hiredate = e2.hiredate ;
A bit complicated isn't it? This is where Oracle's dense_rankcomes in. This analytic function allows you to rank data. The following adds a rank column, and for each job, the employee with the latest hire date will get a rank of 1, the employee hire just before that a rank of 2, and so on.
select e.*,
    dense_rank() over (partition by job order by hiredate desc) rank
from emp e;
From here, we just need to keep only the employee with rank = 1 to get the expected result:
select * from
    (select e.*,
            dense_rank() over
            (partition by job order by hiredate desc) rank
       from emp e)
 where rank = 1 ;

1 This allows auditing and allows an admin to retrieve older version of the data if necessary.
2 In Oracle, this table is normally found in the scott schema, and you don't have it, or would like to import it in another database, you can download the SQL to create the table and add data.
3 For the purpose of this discussion, we'll assume there is only one employee for each job with the most recent hire date.

Orbeon Forms 4.0 M17

Today we released Orbeon Forms 4.0 M17 (Milestone 17). Like 4.0 M16 and the previous milestones this is not a final release.

Due to the Thanksgiving break, M17 is a rather small update, but it happens to contain two important new bits:
  • a new landing page for the Form Runner and Orbeon Forms  examples (#165)
  • a new "wizard" mode which allows showing forms one section at a time
In addition, the following issues were addressed:
  • Form Builder
    • No default button set in some dialogs (#571)
  • Form Runner
    • fr:alert-dialog with static label crashes (#607)
    • fr:error-summary: improved recalculate and use xxforms-visited/unvisited (ab3ff7f890)
  • XForms engine
    • xxforms:value() function (1021d5eade)
    • New `focusable` mode for XBL components (#526)
    • TinyMCE with required content doesn't reliably show error (#301)
    • Change of visited state requires refresh (84a13e93c8)
    • New xxforms-visited/unvisited events (f6c02b4377)
    • Add deferred flag to xf:recalculate action (3339a5c2c2)
More information is available in the in-progress release notes for 4.0.

You can download the builds using these links:
Don't forget to grab a trial license for the PE version.

Please send feedback:

Monday, November 19, 2012

Orbeon Forms 4.0 M16

Today we released Orbeon Forms 4.0 M16 (Milestone 16). Like 4.0 M15 and the previous milestones this is not a final release.

In M16, we addressed the following issues:
  • Form Builder
    • FB: cut to clipboard fails if fr-form-binds id is missing (#579)
    • FB: tabbing out of field reaches delete icon (#580)
    • FB: don't run custom event handlers at design time (#583)
    • Fix crash when section template is non-existing (91cee9a750)
    • Section editor is in wrong place (#572)
    • On Firefox, in the section label editor, pressing enter does not close the editor (#585)
    • FB: cut/paste control with bind/@nodeset adds bind/@ref attribute (#588)
  • Form Runner
    • Improvements to fr:number
      • Add fr:number field to Controls form (#399)
      • fr:number: decimal formatting not working (#547)
      • fr:number: support blank thousands separator (#587)
    • Value of components in section templates are not saved (#596)
    • CE: DMV-14 PDF returns XML document (#253)
    • fr:section doesn't output grouping span elements
    • fr:grid: exclude +/- from tab navigation
    • Move forms to eXist
      • Controls form
      • Bookshelf form
    • Hide help and hints in PDF
    • Regression: With MySQL, Form Builder summary page doesn't show app/form names (#598)
  • XForms engine
    • Page crashes if xforms:output refers to not found resource (#582)
    • Document implemented XBL modes and related code updates (#394)
    • Support XBL "value" mode (#437)
  • Other
    • User can download page flow XML (#564)
    • Upgrading jQuery from 1.7.1 to 1.8.2
    • Standardize usages
      • xforms:var (#369)
      • ref instead of nodeset
      • xforms:property instead of xxforms:context
      • targetid instead of target on xforms:dispatch
      • xforms and xhtml namespaces (#368)
      • xf:trigger instead of fr:button
More information is available in the in-progress release notes for 4.0.

You can download the builds using these links:
Don't forget to grab a trial license for the PE version.

Please send feedback:

Wednesday, November 14, 2012

Variables in XForms 2

Orbeon Forms has long supported exposing XPath variables with the xxforms:variable extension [1]:
<xxforms:variable name="answer" value="42"/>
So it’s great that XForms 2 now standardizes variables with the new xforms:var element! The above example becomes:
<xforms:var name="answer" value="42"/>
The working group went with var for concision (not to mention that the name will be familiar to JavaScript programmers!).[2]
With XForms 2, you can use variables[3]:
  • interleaved with controls
  • within actions
Immutability is a good thing in programming, and XForms variables were designed this to be immutable within a given context. In particular, you cannot set a variable value in XForms (or XSLT): the variable value updates “when needed” based on the expression in the value attribute, respectively when the control tree is refreshed, or when entering the scope of an action.[4]
You refer to a variable using the $ notation of XPath:
    value="if ($answer = 42)
           then 'Answer to the Ultimate Question'
           else 'Wrong!'"/>
Variables are also very useful within actions:
<action iterate="books/book">
    <var name="pos"  value="position()"/>
    <var name="book" value="."/>
    <setvalue ref="$book/@id" value="$pos"/>
You can do this kind of things with plain XForms 1.1, but then you would have needed to use the less clear context() function, for example.
Orbeon Forms 4.0 milestone builds support the standard xforms:var, so feel free to move your code away from good old xxforms:variable!

  1. See the documentation on Variables in Orbeon Forms.  ↩
  2. When Orbeon Forms introduced variables, it first used a select attribute to specify the variable value, partly inspired by XSLT. But select didn’t feel right in XForms, and of course using “value” is natural as in most programming languages, a variable has a name and a value!  ↩
  3. Orbeon Forms supports, in addition, placing variables directly under the xforms:model element. XForms 2 does not standardize this yet.  ↩
  4. This could have made a case for calling the element val instead of var, as the Scala programming language does for immutable variables.  ↩

Tuesday, November 13, 2012

Orbeon Forms 4.0 M15

Today we released Orbeon Forms 4.0 M15 (Milestone 15). Like 4.0 M14 and the previous milestones this is not a final release.

In M15, we have decided to address a number of smaller issues, including a lot of polishing:
  • Form Builder
    • Section help doesn't save (#552)
    • Don't show icons to add column when adding will have no effect (#550)
    • Don't show global templates when editing orbeon/library (#567)
    • Don't show acme app templates when editing acme/library (#573)
    • Build includes sample section templates (bee71ee5ca)
    • FB toolbox improvements (1e017fda6a)
      • improved grouping
      • a few improved icons
      • smaller legends
    • Toolbox and services/actions improvements (dd953f338a)
      • improve toolbox buttons width, padding, etc.
      • remove unneeded CSS
      • we don't properly rename services so disallow editing name
      • Delete button is now in dialogs
      • fixed consistency issues between dialogs
      • don't use icons in dialog buttons
    • I18n New Repeat label in toolbox (041137f750)
    • Disable Save As button in Form Builder (#556)
    • Increase minimal Internet Explorer version for Form Builder to IE 9
  • Form Runner
    • FR: noscript: link buttons at top are not right (#476)
    • FR: noscript: section titles have wrong color (#475)
    • Make XML Schema service a PE feature (#450)
    • Review mode: multiple-row repeated grid appears funny (#206)
    • Regression: ClassNotFound in Form Runner server with proxy portlet (#558)
  • XForms engine
    • Absolute ids should not accidentally leak to users (#551)
    • Autocomplete: selecting a value with a single quote in it populates the field with &quot; (#557)
  • Other
    • Exceptions don't show in UI in dev mode (#554)
More information is available in the in-progress release notes for 4.0.

You can download the builds using these links:
Don't forget to grab a trial license for the PE version.

Please send feedback:

Friday, November 9, 2012

Simplifying and improving XForms in Form Builder

Behind the scene, Form Builder manipulates XForms. Hence, to a great extend, the features provided by the builder are heavily influence by how things are done in XForms, and there often is a direct mapping from a Form Builder feature to an XForms capability.

Along the way, we took the liberty to rename a few concepts. For instance, in XForms, a control can be relevant or not. In Orbeon Forms, non-relevant controls aren't shown. So, in Form Builder, instead of talking of relevance, we use the term visibility. For every control, form authors can define whether the field is visible, and whether it is read-only. With this, controls can be in one of the 3 following states: read-write, read-only, invisible1.

We also brought what we consider to be a simplification to how controls type and required properties are handled. In XForms, you can specify the type of a field (typically for text fields), say xs:integer. Independently, you can define whether entering a value is required or not. Required fields are often styled with a red star showing next to the label, and to be valid a field must conform to both constraints: the value must be of the right type and it must not be empty if required. XForms authors often find the situation where they set the field to be non-required but use a type that doesn't accept empty values confusing, and this in two ways:
  • Some are surprised to see that empty values are not accepted in a field of type xs:integer which they set to be non-required2.
  • Some are surprised to see the field isn't styled with a red star, to indicate a value is required, since users can't leave the field empty3.
Form Builder solves this problem by picking a variant of the type you specify based on whether you set the field to be required or not4. So you will never have a field which you marked non-required, but that considers empty values as non-valid, and required fields will always be styled with a red star next to the label.

1 Some of those states might not make sense for some controls. For instance, a control that just shows text, like the text output control, is in effect always read-only.
2 Since an empty string isn't an xs:integer, XForms considers the field invalid, whether required is set to true or false. To get the expected behavior, those XForms authors will want to use the xforms:integer type instead, which accepts empty values.
3 For this, XForms authors will want to make sure to also set required to true.
4 Form Builder does this by picking for you the variant of the type that matches radio button you selected for required. For instance, for an integer, if required, the XForms type is set to xs:integer ; if not required, the type is set to xforms:integer.

Tuesday, November 6, 2012

Orbeon Forms 4.0 M14

Today we released Orbeon Forms 4.0 M14 (Milestone 14). Like 4.0 M13 and the previous milestones this is not a final release.

M14 addresses the following issues:
  • Form Builder
    • Rename control in repeat: instance other subsequent iterations disappear (#541)
    • Required fields don't show as required in Edit Validation Properties dialog (#542)
    • User can add any number of grid columns (#545)
    • Fix crash when loading form using `fr:body` in repeats or grids
  • Form Runner
    • Paging in the summary page taking ~1s in the XForms engine (#521)
    • IE7: checkboxes are truncated in summary page (#210)
    • IE7: section titles show in blue (#529)
    • IE7: controls cut on the right (#527)
    • RFE: Property for page to go to on close (#530)
    • Protect access to eXist out of the box (#451)
    • Repeated item with "Show in summary" doesn't show in summary (#287)
    • Regression: Summary page shows app name in form name, title, and description columns (#535)
    • XForms inspector doesn't show properly with Bootstrap (#540)
      • XForms engine
        • XForms engine can leak threads (#534)
        • Server sends itemset update even though positions are identical (#531)
        • JavaScript error when an iteration is added to an empty  (#151)
        • Improve look of autocomplete button with Bootstrap (#533)
        • fr:autocomplete: selection sometimes has no effect (#436)
        • JavaScript error with nested repeats in portlet (#242)
        • Optimized event dispatch (see blog entry)
      More information is available in the in-progress release notes for 4.0.

      You can download the builds using these links:
      Don't forget to grab a trial license for the PE version.

      Please send feedback:

      Monday, November 5, 2012

      Optimizing event handling

      Recently, while doing a round of profiling, we found out that event processing on the Form Runner summary page when paging and searching was taking a long time to execute (in the order of hundreds of milliseconds). So we set to improve this.

      In XForms as in plain HTML, event dispatch follows the processing model defined by the DOM Events specification. An event conceptually travels from the root of the document to the target, and then back to the root of the document, in a capture phase, a target phase, and a bubbling phase. It also supports features such as event cancellation, stopping event propagation, and preventing the execution of the default action. And of course, an important part is to figure out the events handlers to run during each phase. So it's potentially a lot of work, as shown on this diagram from the DOM Events spec:

      A great benefit of XForms is that the installation of event handlers is primarily done declaratively, thanks to XML Events. (XForms doesn't say that you cannot support other ways, with script for example. But in Orbeon Forms we exclusively support the declarative way.) The declarative aspect means that all event handlers are known in advance and can be analyzed statically, and so we just implemented such a static analysis!

      Here is how it works. For an event with a given event type (or name) dispatched to a control, we determine whether in general there can be event handlers to run, and if so during which phase and for which event observers. In other words, we gather the maximum amount of information possible so that when an event of this type is concretely dispatched, we quickly know what to do.

      For example, let's say an xforms-enabled event is dispatched to a input control. If we find that there are no event handlers for that event, we remember that fact so that the next time xforms-enabled is dispatched to that same input control, we know we can just skip all the processing associated with event handlers.

      On the other hand, let's say xforms-value-changed is dispatched to that same control, and there is a single event handler for it: we store that information alongside the phase and the id of the observer to which the handler is attached. This way, the next time that event is dispatched, we can very quickly just run that single event handler associated with that observer.

      Things are not quite that simple in practice (we must handle XForms repeats, for example), so there is some work to do each time an event is dispatched, but it's much smaller than it used to be.

      Event handler information is gathered lazily into a cache as events are dispatched. This has the benefit of doing the work only for events that are actually dispatched. (Also, in XForms, the name and target of events can be dynamic, which means that it isn't possible to analyze everything in advance anyway in all cases. So laziness is a good idea.) The cache is shared between form instances that use the same form definition. So if 100 users visit a given form, they all benefit from the cache.

      The result is not only a faster Form Runner summary page, but also faster event processing in the XForms engine in general.