Wednesday, March 18, 2015

New wizard validated mode

The Form Runner wizard view is widely used by Orbeon Forms users: it presents a simple, navigable view of a form’s top-level sections, shown as separate pages.

Until recently, the wizard view only supported free navigation. In this mode, you can freely navigate between pages, independently from whether there are validation errors.

In Orbeon Forms 4.9, we are introducing a new wizard mode, called the validated mode. [1]

Form Runner wizard in validated mode
Form Runner wizard in validated mode

The main idea of this new mode is to prevent the user from navigating to subsequent pages until all the data entered so far is valid:

  • You can always navigate back in the wizard, even if you have errors on the current page.
  • But you can only navigate forward if there are no errors on all preceding pages as well as the current page.

This helps ensure that the user follows (and hopefully pays more attention to) specific steps when filling out a form.

There is a little twist: once you have visited a page, you can freely navigate again to it, whether with the Next button or the table of contents.[2] The idea behind this is that it can be frustrating to enter information on a page, only to be prevented by the wizard to reach that information at a later time.

As an aside, this improvement also fixes an issue related to incorrect highlighting of errors within wizard pages. [3]

The complete documentation is available here. We hope you will enjoy this feature!

  1. The free mode is still the default, and you enable the validated mode with a property.  ↩

  2. Here is an example of back-and-forth:  ↩

    • You are on the first page.
    • You press the Next button, but you have errors on the page. All the errors show in the error summary and forward navigation is prevented.
    • You fix the errors, and press the Next button again. This time navigation succeeds and you reach the second page.
    • You press the Back button to the first page again, and make a field invalid.
    • Now, even with that invalid field, you can still reach the second page, because you have already visited it.
  3. See this issue for details.  ↩

Monday, February 2, 2015

Orbeon Forms 4.8.1

Today we released Orbeon Forms 4.8.1 PE. This update to Orbeon Forms 4.8 PE contains bug-fixes and is recommended for all Orbeon Forms 4.8 PE users.

Specifically, this release addresses the following issues:
  • Form Builder
    • Incorrect label edited upon insert of XBL with nested labels (#2075)
  • Form Runner
    • Error when producing PDF in parallel with Ajax request (#2071)
    • Crash when paging on Summary page (6af5bed6c5)
    • fr:number: incorrect rounding when using decimal separator other than `.` (#2076)
    • Custom mips are not propagated to section templates (#2065)
  • Other
    • Ajax retry doesn't work with Firefox (#2074)
    • Object cache statistics are not used but can be costly (#2072)
    • NPE using xxf:evaluate-bind-property() (#2048)
You can download the latest version of Orbeon Forms from the downloads page.
    Don't forget to grab a trial license for the PE version.

    Please send feedback:
    We hope you enjoy this release!

    Thursday, January 22, 2015

    Saying goodbye to internal HTTP connections

    Orbeon Forms often needs to perform calls to itself, for example to:

    • call persistence layer implementations,
    • load internationalized resources,
    • navigate between pages,
    • proxy requests for dynamic XForms resources,
    • and request CSS, JavaScript and image assets when producing PDFs.

    In previous versions, Orbeon Forms performed actual HTTP connections on itself. This caused problems:

    • Configuration: Orbeon Forms needed a host and port to connect. In some cases, this could be guessed, but in others, for example when a front-end was placed in front of Orbeon Forms, you had to override a configuration property. There were also reported cases of interference with some servlet filters installed by our users.
    • Authentication and authorization: Internal services typically need the same credentials as the incoming request. To do this we were forwarding cookies and other headers, which could fail in some corner cases and was sometimes difficult to configure.
    • Thread usage: This required more servlet container threads than needed. In some cases, a external request to Orbeon could cause 3 threads to be used, 2 of them just waiting on HTTP connections.
    • Deadlocks: When the pool of threads provided by Tomcat was exhausted, deadlocks could occur, especially since we recommended a small pool in order to avoid overloading the forms processor. [1]
    • Performance: Page and service calls were slower than needed, as HTTP connections needed to be established, and communicating over HTTP has a certain cost.

    In short, change was needed! So with Orbeon Forms 4.7, we got on this and entirely removed of the use of HTTP for internal calls!

    How did we do this?

    We were already using an abstraction to perform requests within Orbeon Forms. This abstraction unifies how we access internal resources stored in the Orbeon application itself, as well as calls such as HTTP calls.

    So we built on this [2] to provide an additional HTTP abstraction. We now have an ApacheHttpClient, which uses HTTP proper, and a new InternalHttpClient, which behaves like the real HTTP client from the perspective of the caller but in fact does its work entirely internally, without any network access.

    From the perspective of the page or service called, it is as if the request was coming from an external HTTP client (with some minor twists).

    This in one shot addresses all the issues listed above. So we are quite happy with this change!

    1. Orbeon Forms 4.8 introduces a new limiter filter to deal with this in a much better way.  ↩

    2. Including some fairly massive refactoring along the way!  ↩

    Wednesday, January 14, 2015

    Choosing the best versioning option when publishing a form

    Orbeon Forms supports versioning of form definitions since Orbeon Forms 4.5 [1].

    Since Orbeon Forms 4.6 [2], Form Builder also allows you to control whether a new form definition version must be created:

    Creating a new version
    Creating a new version

    Or whether the last existing form definition version must be overwritten:

    Overwriting an existing version
    Overwriting an existing version

    You notice that there are warnings associated with each option! Ideally, we wouldn’t have a need for them, but the use of each option has different consequences on published forms.

    So which option should you chose when you republish a new form definition?

    1. If there is no data associated with the form definition in the database, it is safe to overwrite the latest form definition, unless you really want to keep the older version published.
    2. If there is data associated with the form definition in the database and the changes you have made to the form definition
      1. are not structural, it is also safe to overwrite the latest form definition.
      2. are structural, it is not safe to overwrite and you should create a new version.

    The following operations count as structural changes:

    • adding controls, grids, or sections
    • removing controls, grids, or sections
    • moving controls, grids, or sections
    • renaming controls, grids, or sections

    The following operations are not structural:

    • changing the form’s title or description
    • changing a control’s label, hint, help, or alert message
    • changing a section’s title or help
    • adding or removing repeated grid or section iterations
    • adding or removing languages
    • changing the form’s permissions
    • adding, changing, or removing the form’s PDF template [3]

    In general, the following non-structural operations are safe, but can have consequences on data validation and access:

    • adding or removing services and actions
    • adding, changing, or removing validations or formulas
    • adding, changing, or removing an XML Schema

    For example if validation rules change, the situation can arise where data stored in the database was valid at the time it was saved, but the new form definition considers them invalid. So viewing the data will work, but attempting to save without changes will not satisfy the new validation rules and will fail. It is important to be aware of this when making validation rules changes.

    Similarly, changing actions or visibility rules can make the form behave differently when editing or viewing existing data. Here again, you must be cautious.

    Now you might ask, what is the drawback of always creating a new form version when publishing?

    Existing data in the database is associated with a specific form definition version. This means that existing data cannot be viewed or edited with a different form definition version. So a fix for a typo in a control label, for example, wouldn’t show when editing or viewing existing data. This may or may not be acceptable depending on your use case.

    As discussed in the original post about versioning, we hope to enable more scenarios with basic data migration in the future.

    1. As of Orbeon Forms 4.8, versioning is supported with relational databases, but not with eXist. See also database support.  ↩

    2. That’s a couple of versions ago, since as we write Orbeon Forms 4.8 has just been released, but we didn’t get to talk about this feature until now!  ↩

    3. Note that changing the PDF template can cause incorrect PDF production for existing data if the PDF template is incorrectly setup.  ↩

    Thursday, January 8, 2015

    Orbeon Forms 4.8

    Today we released Orbeon Forms 4.8!

    Major features and enhancements
    • PostgreSQL support. Thanks in good part to an external contribution, Orbeon Forms supports PostgreSQL in addition to Oracle, MySQL, SQLServer, DB2 and eXist. (blog, #1941)
    • Better management of server resources. Under load, Orbeon Forms limits the number of concurrent form processing requests in order to reduce the likelihood of the server running out of memory (doc, #1971).
    • Support for repeated grid visibility. It is possible to completely hide (including the header row) or mark as readonly a repeated grid (doc#597 / #635).
    • Owner and group-based permissions eXist support. Owner and group-based permissions were only supported with relational databases. This release adds eXist support. See also Database Support. (doc, #1341)
    • Singleton forms. A singleton form is a form with only a single instance of form data visible to a user. Form Builder allows you to mark a form as such, and Form Runner then uses this information to prevent the creation of more than one instance of form data. (doc, #2009)
    • Improved XForms 2.0 support. 
      • The caseref attribute on xf:case is implemented (#1089).
      • The xf:case() function is implemented (#1951). It was already available as an extension function with xxf:case().

    See Localizing Orbeon Forms for the latest localization support. Localization depends on volunteers, so please let us know if you want to help!

    Other new features and bug-fixes

    Including the major features and enhancements above, we closed over 30 issues since Orbeon Forms 4.7.1 (and over 60 since 4.7).
    Current browser support
    • Form Builder (creating forms):
      • Chrome 39 (latest stable version) and Chrome 41 dev (current dev channel)
      • Firefox 34 (latest  stable version) and the current Firefox ESR
      • IE 11
      • Safari 8
      • Form Runner (accessing form):
        • All browsers supported by Form Builder (see above)
        • IE8, IE9 and IE10
        • Safari Mobile on iOS 6, iOS 7 and iOS 8
        • Chrome for Android (stable channel)
      Compatibility notes
      • The internal data format for repeated grids has changed following #635. This shouldn't affect users and form author much:
        • Form definitions are upgraded automatically when loaded in Form Builder, but published forms are not affected until republished, whether from Form Builder or from the Home page.
        • The change is visible when looking at the form definition source code in Form Builder.
        • The external data format, that is the data saved to the persistence layer, is unchanged, therefore existing form data in the database doesn't need migration.
        • The send action also uses the "old" format by default (this can be overridden). (doc)
        • POSTing initial data to a form also uses the "old" format by default (this can be overridden). (doc)
        • However initial data obtained from a service can only be in the "old" format at this time.
      • Relevance and readonly for repeated grids
        • The "Visibility" and "Readonly" formulas in Form Builder  apply to the entire grid, whereas they used to apply to individual iterations only (although this was not an intended feature). In the future we might add separate "Visibility" and "Readonly" formulas for iterations.
      • fr:grid component (doc)
        • The repeat="true" attribute value is deprecated. Instead use repeat="content" which expect different data layout.
        • The relevant and readonly MIPs apply to the entire grid with repeat="content".
        • The evaluation context for the min and max AVTs is properly specified and documented.
      • The xxf:tree and xxf:menu appearances on xf:select and xf:select1 have been removed. The way these appearances were implemented is considered obsolete and not showcased by Form Runner and Form Builder anyway.
      • The default session duration used to be 12 hours. It is reduced to 1 hour. The session heartbeat feature helps keeping the session alive.
      You can download the latest version of Orbeon Forms from the downloads page.
        Don't forget to grab a trial license for the PE version.

        Please send feedback:
        We hope you enjoy this release!

        Monday, December 1, 2014

        Orbeon Forms 4.7.1

        Today we released Orbeon Forms 4.7.1 PE. This update to Orbeon Forms 4.7 PE contains bug-fixes, security fixes, performance improvements, and localization improvements. We recommend this update for all Orbeon Forms 4.7 PE users.

        Specifically, this release addresses the following issues:
        • Form Builder
          • Renaming an action duplicates it (#1913)
          • FB user sees all forms in summary page if role is not found (#1963)
          • Test mode: Validate button shows instead of inner buttons (#2002)
          • Publish: document parameter value is not encoded (#1985)
          • Exception when calling dataMaybeMigratedTo() (#1986)
          • FB: Keyboard focus is not on first dropdown when present upon new (#2010)
        • Form Runner
          • Automatic PDF performance improvements
            • Requests for resources must be internal (#1995)
            • Don't store XForms document in cache (#1999)
            • Automatic PDF output should cache images (#1996)
          • Other automatic PDF improvements
            • Grid rows without visible controls must not take space (#2004)
            • Page number/count loses alignment if form title is too long (#2005)
            • Don't let long title wrap in header or footer (#2006)
            • Show form logo on PDF header (#2007)
            • Textarea content must wrap (#2008)
          • PDF templates improvements
            • Automatically decide whether to use label or value (#736)
            • Non-boolean checkboxes don't work (#1987)
            • fr:number is always blank (#1955)
            • fr:xforms-inspector causes PDF template error (#1949)
          • XVT not evaluated for `uri` in `send` action (#1972)
          • xxf:itemset() not working anymore now that we use fr:dropdown-select1 (#1856)
          • Add a few missing Swedish resources
        • XForms
          • Incorrect check on headers in Connection (#1933)
          • Error dialog not working (#1938)
          • Don't log full stack trace of session expiration messages (#1979)
          • Serializer not to filter elements in no namespace (#1981)
          • Default value for Dynamic Data Dropdown is lost when pasting form source (#1962)
        You can download the latest version of Orbeon Forms from the downloads page.
          Don't forget to grab a trial license for the PE version.

          Please send feedback:
          We hope you enjoy this release!

          PostgreSQL support in Orbeon Forms

          According to the DB-Engines database ranking by popularity, the top 5 relational databases are: Oracle, MySQL, SQL Server, PostgreSQL, and DB2. In 4.6 we added support for SQL Server. The upcoming 4.8 adds support for PostgreSQL, thus completing Orbeon Forms support for the top-5 relational databases.

          Past the top-5, we find Access and SQLite, which are typically not used in the type of server-side environments our customers have in place. The next candidate would be Sybase, but we're then getting pretty low on the popularity curve, and we've yet to encounter a customer who wouldn't be able to use Orbeon Forms without Sybase support. So it is with a certain sense of accomplishment that we're adding the support for PostgreSQL, thus allowing Orbeon Forms to reasonably claim it "supports, out-of-the-box, all the top relational databases".

          And for this, we have to thank Aaron Spike, from the Martin Luther College, who implemented and contributed the implementation for PostgreSQL. Thank you, Aaron!