Project Description

Some years ago, I was working for a Portuguese bank and I was challenged to create a new forms solution that would replace the current solution, keeping some additional requirements in mind. There were 800 forms, what demonstrate the importance of that application in my client activity.

The main objective of this solution was to build an application that would empower the bank to easily create, publish, fill and archive forms necessary to support business transactions executed by almost all the bank’s applications used by the bank users from the bank’s branches or form the bank’s online presence. For instance, if you, as a client, go to a bank branch to subscribe a financial product, you will have to read and sign some documents related to that subscription. Those documents will be created by this application. The documents will have prefilled data, if you are already a bank client, and you may even be able to fill additional data. The resulting document will then be generated and printed or simply archived for later use.

Along the way, a lot of work needed to be done. A template of the document was created (designed) and published so that the bank applications and its users may use it. But, to easily create templates, a forms framework was built first. Also, a set of web services were created so that the bank applications could easily “talk” with this new forms solution.

In this article, I will explain how my client moved from an AS-IS solution to a TO-BE solution using the Adobe LiveCycle Enterprise Suite.

Adobe LiveCycle logo

The Business and Technical Requirements

The requirements are the cause of the solution. So, in order to understand the solution, it is better to start understanding the requirements. The following list presents the main requirements for the new forms solution, with a brief explanation of each one.

  • Enable the bank to create and deploy new or modified forms on the same day the form is needed in the market.
The financial market is very dynamic. New products are always emerging, and updates to existing products are sometimes needed. The aim with this requirement was to enable the bank to leverage its time to market regarding financial products.

The delivery of a new financial product to the market is a hard work, with many people with different skills involved, like business experts, lawyers and IT technicians. It would be a shame to do the most difficult and complex part of the process in time and then delay the product delivery because the right documents to support the product are not available.

It seems ridiculous, right? But it happens, and that proves the importance of the documents in the business world, either in paper form or in electronic form. Without it, there is no business.

  • Empower business users to create the forms by themselves
The aim here was to avoid the need for developers to change or create new forms. This means that all forms’ features should be configured without coding the forms. Plus, the creation of the forms would be closer to the ones that actually understand the business.

  • The forms must follow a WYSIWYG (what you see is what you get) approach
In other words, the forms must be printed to paper looking exactly (or close to it) as you see them on the computer screen. This is important because most of these forms would be manual signed by the client and one copy would be given to the client. So, if you deliver something to the client, it should look great.

  • The new forms solution should be easily integrated with any of the bank’s business solutions that supports web services.
We could create a wonderful way for building forms, but these forms would be of almost no use if they were not integrated in the business solutions that actually support the bank products and services. Imagine that you have dozens of applications and each time a form is need for each solution, you would need to search somewhere which form should be used. After finding one, how would you integrate it in your application? This process should be almost transparent for all applications.

  • The forms must implement reach customized objects
The forms must be interactive, editable and have some out of the box functionalities like: counterpart specification on footer; pre-numeration codes; barcodes; different form views applied to the same form; filed totals and calculated fields (formulas).

  • The new solution should integrate easily with backend applications that owns business data (central system)
While the forms would be interactive and users would be able to fill in data, most data should be filled automatically without user interaction. This way it wouldn’t be necessary to fill all the time the same data. Thus, the time to fill a form would decrease significantly.

Now, let’s take a look into the AS_IS solution in order to understand the path chosen to build the new solution.

The Former Forms Solution (AS-IS)

The former solution was based on the nice Accelio Forms Server product. The Accelio Form Server was a client and server side solution based in a XML document description. Basically, Accelio created a XML representation of a document and then implemented it at client-side and server-side. At client-side, an ActiveX running in a browser render the XML representation of the form in an interactively printed document with merged data. At the server-side, its Form Server component was able to do the same, converting the XML representation of the form, merging the data with the form and presenting it on the screen as a document filled with data. So, the same XML structure was supported in both client-side and server-side.

This solution was very well implemented by my client, with some web services exposing Accelio Forms Server functionality for all the business applications in the bank. So, any application in the bank was able to use XML templates to present very nice forms to the end-users. Actually, these services were created by me when I started working at this client, because before that it worked like a monolithic solution where the users needed to access this application to generate the desired forms.

In 2002, Accelio was bought by Adobe. Adobe was then looking for a way to make its PDF format reaching the business users. The PDF file was already a huge success among personal users, but the business market was very attractive because they actually have the money to pay for solutions.

At that time, a PDF file was mainly a static file with some interactive features, known as AcroForms. You could create (and you can still create) AcroForms using Adobe Acrobat. While you could fill data in these forms, a must have feature for the business market, the forms lacked some interactiveness with the end-users and, most importantly, were extremely static. For instance, you couldn’t create sections of a document that would resize based on the user input. The following image shows the creation of an AcroForm using Adobe Professional.

AcroForm Creation with Acrobat Professional

AcroForm Creation with Adobe Acrobat Professional

These features that PDF didn’t have were found by Adobe in the Accelio solution, and that is why I believe that Adobe bought Accelio. Then, Adobe took the XML representation of Accelio forms (known as XFA forms) and put it inside PDF. Plus, in order to maintain compatibility with previous version of Adobe Reader and previous created AcroForms, they maintained the Acroform structure inside the PDF file. So, the new PDF files would have two kinds of structures inside: AcroForm and XFA form structures. Basically, a PDF file was no longer “just” a PDF file, it was something more, and if you would like to use that “something more” you would need to use a PDF reader with XFA capabilities like Adobe Reader. This is the current state of PDF technology.

Given this change in the market and the new bank requirements, a new solution was needed to replace the current Accelio forms solution, and that is what we will look into next.

The New Forms Solution (TO-BE)

Keeping in mind the technology already in use by the bank and the listed requirements, our objective was to build a new forms solution based on the Adobe LiveCycle ES Enterprise Suite. As you can see from the image bellow, the LiveCycle solution has a lot of components.

Adobe LiveCycle Solution Architecture

Adobe LiveCycle Solution Architecture

Based on the requirements, the following table lists the necessary Adobe LiveCycle components needed to build the new solution.

Adobe LiveCycle component Reasons to use the component
Adobe LiveCycle Forms Designer The Forms Designer is an editor for designing PDF with a XFA structure. Using this designer, a business user can create forms easily without any technology expertise.

The Forms Designer can be bought independently, but it is also available with Adobe Professional.

Adobe LiveCycle Workflow The workflow component can be used to create forms services using a design approach, instead of a bunch of programmatic code that is always difficult to understand and maintain.

While installing the remaining server-side LiveCycle components, almost all of them will include their services as workflows components so that their functionality may be easily used inside workflows designs.

Adobe LiveCycle Workbench This component is like an IDE where users can design forms and workflows at one unique place. All the assets used in the forms designer and in the workflows are saved in a central repository shared by all users. At that time, the shared repository was built with Alfresco.
Adobe LiveCycle Forms This is the main server component used to convert from XFA (XML representation of the PDF) into PDF and from PDF into XFA. While it was possible to create the solution without this component, using it simplified the final solution.

The development of the new forms solution can be decomposed in the developing of the following components:

  • A JavaScript PDF forms framework that would be the base to all forms templates and would support all the forms requirements. For instance, based on this framework, it would be easy to configure the companies supported by a form, the number of counterparts, form formulas or any other form features. This framework would be integrated inside each PDF template.
  • A set of services to support the publication and the availability of the forms templates for all the bank applications.

I will detail a little bit each of these custom components on the next sections.

The New Solution: the forms JavaScript Framework

The Adobe PDF forms (XFA forms) are forms that exposes a JavaScript API useful to control many aspects of the forms, including button events or objects rendering. Building forms with this technology is like building small windows applications, because each of them will have inside all they need to be used.

The following image shows a high level architecture of the forms framework that was built. Each form will be opened in a web page inside a browser window. The web page references a set of JavaScript functions called the external forms JavaScript framework. This includes JavaScript to enable the communication between the page and the form. There is also a XML data variable where the data to be displayed inside the form is stored. This means the web page is dynamically generated with a specific set of data, depending of the form being opened and of the specific data to be displayed on the form.

ToBe Forms Architecture

ToBe Forms Architecture

The Object element is a standard HTML object that instantiates the Adobe ActiveX used to read and present the PDF form on the page. This object also points to a specific form template that was dynamically set when the page was generated. So, when the object element is created, the Adobe ActiveX is instantiated and the form is read. When the form is read, the internal JavaScript framework runs, including a standard JavaScript library called form bridge. The form bridge is used by the form to communicate with the external JavaScript library referenced by the outsider page. Thus, the form can say to the page things like: “I´m loading”; “I finished loading”; “I was not able to load due to this error”. If the form loads without errors, the internal forms JavaScript frameworks start executing. One thing this framework will do is to set the form data in the right form fields. Plus, the internal JavaScript framework will execute all the business rules, validations and configurations set to the form by the form designer when the form was designed.

One thing we had in mind while developing the forms framework was that we and the client wanted to simplify the process of designing a form, and to achieve this, it was important to give the designer a simple way to set the rules, validations and further configurations of the form. What we thought was that it would be nice for the designer to set these configurations using the same tool used to design the form, the Livecycle Forms Designer. But, this tool is to design forms, not to set properties, so, we thought about using hidden pages where the designer could set those configurations. For that we used master pages. Basically we created standard master pages that each form template would have. In each master page we draw tables with labels and fields to set the forms specific data, like the logos; the copies descriptions; the companies the form may support (my client has different companies and the same form could fit more than one); fields validations; between many others.

Additionally, the internal framework was supported by a set of XFA objects easily shared and accessed by the designer through a custom object library. One example of these objects was the objects representing the logos of the different companies of my client. When the designer knew that a form should only support the company A and company B, the designer would drag and drop the needed logos, to the logo configuration field in the master pages. Also, there were objects representing the footer of the different companies. The process to use them was the same as the logos, the designer simply drags and drops the needed footers from the library to the corresponding configuration fields inside the master pages. Another important library objects was the ones representing the central transactions the form should use, if any. Whenever a form could get its data or part of it from the central system, the designer should add that transaction (or transactions) to the form, simply dragging and dropping them in the transaction configuration section on the master pages. Then, when a form was generated based on that template, the web service used to generate the form would know that the form wants to get data from a specific transaction and so, it would execute that transaction and load the output data to the form.

The configuration of the form set by the designer was used in different moments of the form´s live cycle. For instance, the transaction’s configuration was used by the web services that deliver the forms to the client application. In the workflows that support those services, the form data was inspected in order to understand if the form supports central transactions and what transactions were supported. Based on this information, the workflow would execute the transactions and set the form data xml accordingly. The same happens to dynamic logos. If, at designing time, the designer would set logos dynamically instead of embedding them using the respective logos object (from the custom object library), the workflow would get those logos from the repository and set them in the form data.

On the other hand, some configurations are only used when the form is opened in the browser. For instance, the number of copies configuration property is used to set the number of recipients of the form. Usually, the recipients are the bank and the customer (but we can have more), so the solution must print a copy of the form to the bank, and other to the customer. But, these copies are not completely equal. For example, the label identifying the recipient is different.

Also, some sections of the form may only be seen in one of the copies. Instead of creating a large form with almost the same content, we created small forms with only the content of one copy and then the framework generates additional copies on the fly, based on the configuration properties. Another good example of configurations being read at run time are the business rules and fields validations. At designing time, the designer sets formulas using an intuitive language (we created a custom short intuitive language), for instance, saying that a field is only mandatory if another filed is filled; or saying that a value of a specific field should be the sum of the values of other fields; and then at run-time, the forms framework runs these formulas as needed (regarding the events where the formulas were set to run).

This is only a small part of the form framework that was built. It is hard to explain because it is actually complex. The following video shows some of its functionalities. I bet that most of you might have never saw and never thought of a PDF like this, but as you can see, it is possible to do it.

The New Solution: The Forms Services

Building nice forms is not a big deal if no one uses them in the organization. So, it was necessary to make them available not only to users, but specially to the bank’s business applications. In order to achieve these, the following (web) services were created. It is important to note that all these services were created as workflows using Adobe LiveCycle Workflow. While creating a workflow with this component, we can easily expose it in different ways like as web services, just by configuring the workflow properties. No programming code is required for this.

Service name Service Description
FormPublishing This service is used to publish new or updated forms templates. This is a two way publishing system. First, the forms are saved in the document repository for versioning control. At the same time, the forms are saved to the file system, from where business applications will get them (it is faster to get them from the file system than to get them from the repository).

Also, a schema of the PDF template is generated automatically and saved in the file system. To generate the schema this way, a custom workflow component was developed that parses the form XFA structure and generates a XSD schema from it. Basically, this module knows how to transform each forms object in a XSD counterpart.

When a business application needs to use a specific form template, that application first gets the schema and imports it, and then invokes the GetForm service, passing the needed inputs as stated in the schema. The schema has everything the application needs to know about the form.

CreateForm This service receives application’s inputs and returns a filled PDF form. The application’s input includes the template ID of the form to generate some business data.

While executing, this service gets the PDF template from the file system and reads it (using its XFA structure), as the template has some important information about mainframe transactions that need to be executed in order to get the forms data and other configuration information like if the form has bar code and pre-numeration (in this case, it is invoked stored procedures that will generate new incremental numbers). Then, it adds the inputs from the service and generates a PDF form using Adobe Livecycle Workflow out of the box components.

Note that a schema validation is made at this time. If the client application sends a form input that doesn’t respect the form schema, an error will be thrown by the service.

GetForm The GetForm service is used to get an already created PDF form from the documental>
SaveDoc This service is invoked directly by the PDF forms. When a form is opened, most of the data will be already filled, but the user will be able to change some fields (others not). After the user has finished editing the form, the user should save the form as a new version of an existing form or as an entirely new version. To accomplish this, the user clicks in a form button, that will invoke the SaveDoc service passing the form data as input.

At server-side, this service will re-merge the data with the corresponding form template and thus re-generate the form. We implemented the save functionality in this why to avoid sending the whole form (PDF + data), because there was strong bandwidth restrictions in the bank.

Using the Forms

To understand how a form is used, let’s imagine that you have gone to a bank branch to subscribe a product. According to the product you are subscribing, the bank employee opens the corresponding application. The application will know what form to open because it was configured to know that. So, the application will call the service CreateDoc, telling the CreateDoc service the ID of the form it wants to get. The application also sets the form data as an input of the service. The application knows the data structure of the form because when the form was published, a XSD schema file was also published describing the form data.

The CreateDoc service will get the corresponding form template from the cache and inspect its embedding data elements (that was set by the designer in configuration step while designing the form). Regarding the inspected data, the CreateDoc service may execute central transactions and set the form XML data accordingly. It will also get barcode values and other identity data from the database and set it to the form data. Finally, the service will save the resulting form in a public cache and send the path to the calling application.

The calling application gets the form and presents it to the user inside an html page. Along with the external JavaScript framework, the xml data and the PDF object, the page also presents buttons to execute actions in the form. For instance, if the user clicks the Print button, the custom print dialog will open and the user will be able to print the document.

As an alternative to the buttons in the html page, we created a toolbar inside the forms. However, we didn’t use it in production environment, because the JavaScript of these toolbars would need to be deployed in each user machine due to security restrictions and that was not acceptable. Instead of deploying the code to each user machine, we could change the security restrictions in each user’s Adobe Reader, but that was not acceptable also. Another disadvantage of these toolbars was related to its user-experience, more concretely, the users could close the toolbar and then have some troubles opening it again. This toolbar is presented in the next video, but keep in mind that it was not used.

It is also important to realize that although the forms are opened inside a web page, actually what it is being used is the Adobe Reader’s ActiveX component. If we add to this that the PDF form being opened is actually a XFA Form and not an AcroForm, this means, almost for sure, that only Adobe Reader and some browsers would be able to open these forms.

This was not an issue for the customer because, at that time, the forms would only be used inside the customer’s Intranet. While opening the solution to the Internet so that users could use them in their home banking solution, the best approach would be to flatten the XFA forms. This was accomplished in a second phase of the project with Adobe LiveCycle Output Server.

Closing up

This project was one of the most complex projects I did so far. It took almost one year to complete starting from the pre-analysis phase, and it included not only the forms solution I explained here, but also the creation of workflows to dynamically generate business drafts and contracts that I will try to present in a future article. It also included a web solution to search the repository using Google search and a web portal to access the forms functionality, but I was not directly involved in that part of the solution.

I was the forms (and contracts) solution architect and team leader. The team composed of me and three developers. Additionally, 4 form designers where added to the project to help migrate some of the older forms (about 200 of 800) to the new technology. It was a hard job but, at the end, we “killed” the chiken… 🙂