Return to Homepage Return to SOA Industry Papers

The SOA Gateway & Data Aggregation Projects

This document is intended to give technical architects and project managers a detailed view of how the SOA Gateway can help them where data (aggregation) from multiple sources is required. This document is distributed for information purposes only and does not form part of or constitute an agreement with Risaris Ltd. Although Risaris Ltd. uses reasonable efforts to include accurate and up-to-date information in this document, Risaris makes no warranties or representations as to its accuracy. Risaris Ltd. may also make improvements and/or changes to this document at any time without notice. The various approaches outlined in this document are put forward in good faith, but it remains possible that individual results may vary. For that reason and in accordance with standard practice, readers are encouraged to test any materials developed on the basis of this paper before putting them into productive use.

INDEX

Introduction ........................................................................................................3
The Business Case ............................................................................................... 3
Using the Traditional Approach .......................................................................3
Using the SOA Gateway ................................................................................. 5
The Benefits to this and future projects ............................................................6
Implementation and Using the SOA Gateway ........................................................... 7
Installation of the SOA Gateway......................................................................7
Registering to use the SOA Gateway ........................................................7
After the download ................................................................................ 9
Installing and Configuring the SOA Gateway Server .................................... 9
Windows Installation ............................................................................................ 9
Other Platform Installation .................................................................................. 11
Common Configuration Steps............................................................................... 12
Multiple SOA Gateway Servers ....................................
Error! Bookmark not defined.
3.2. Creating the Services .................................................................................. 12
Creating Database Services................................................................... 13
Creating Business Logic Services ........................................................... 14
Registering Services in a UDDI Server .................................................... 15
Using the Services in your project ................................................................. 16
Using the Services Directly (without a UDDI Server) ................................. 16
Using the Services from a UDDI Server) ................................................. 16


1. Introduction

The explosion in the amount of data that is being maintained in various databases across organizations has resulted in an eclectic mix of technologies where this data resides. This makes it difficult when a single view of that data is required for specific purposes as the data may be contained in different database technologies using different data formats. When the data spans multiple technologies, platforms and even multiple organizations, it is practically impossible to cost effectively get at this data in a standard way using traditional tools and technologies. This leads to short term solutions which are not optimum but help the project to get the job done while leaving a difficult environment to maintain into the future. The SOA Gateway technology is a tool that can help an organization to avoid such implementations by making the data from multiple platforms and databases available in a standard way, quickly, simply and in a cost effective manner.

2. The Business Case

Very few projects are approved and implemented without a cost benefit analysis and a view of the return on investment (ROI) from the project. This leaves many projects extremely sensitive to the cost of implementation. There are many new platforms, languages and technologies that allow for the cost effective implementation and execution of such projects, however, getting at existing data is still the Achilles heel of most projects. Ultimately the challenge is illustrated in the following diagram: how can new applications, which need to get at multiple types of data, get to this data?
This section illustrates how the SOA Gateway can now make it possible to use a fully integrated approach for projects which need to access data on multiple databases in a single, standard way.

2.1. Using the Traditional Approach

Each database generally has its own mechanism for remotely getting at the data if the user is not local to the data. For each different type of database, or even different version of a database, the mechanism for getting at the data can differ and require different approaches from the application that needs to see the data. We end up with architecture like the following:
This leads to the following issues with this approach:
  1. Each different type of access requires installation and configuration initially along with a maintenance cycle to keep the software up to date.
  2. The format of the data may be different on each database. For example, a person’s nationality may be marked as ‘IR’, ‘UK’ or ‘DE’ on one database whereas it will be ‘Ireland’, ‘United Kingdom’ or ‘Germany’ or even ‘1’, ‘2’ or ‘3’ on another database. This means that the application developer must have knowledge about the database being accessed and the schema in use on that database.
  3. Securing architectures such as these is extremely difficult as it adds to the configuration and maintenance to ensure that only those authorized to get the data actually get to see it.
  4. Each mechanism is different so the application developers need to learn different ways to access different systems. As the number of different databases grows, the problem to access them grows exponentially as different databases can give different results for what looks like the same question.
  5. It is quite clear that having a single, standard view of all databases would alleviate all of these problems when it can be done in a cost effective manner.

2.2. Using the SOA Gateway

The SOA Gateway can resolve all of the issues outlined in the previous section. The following illustrates the architecture:
By exposing existing data using proven industry standards, the SOA Gateway provides a cost effective way to get at the data on different databases in a standard way as follows:
  1. Once the SOA Gateway is installed, it literally takes minutes to define the services within the SOA Gateway to make your existing data available as services.
  2. Once a service is available, it can be used again and again from any number of systems at no additional cost.
  3. Once the service is available, it can be used as a SOAP based Web Service or via a URL (REST) based access which means that the service can be accessed by any language or technology available today such as Excel, Word, InfoPath, Java, vb.net, c#.net and so on.
  4. The SOA Gateway requires no software installation on the machine that needs access to the database which saves time in terms of installation and maintenance going forward.
  5. The application programmer will use one standard way to get at the data. Once they have accessed one service, each other service will be identical (except of course for the data) and thus the programmer needs to know nothing about what database or structure has been implemented in the database itself.
  6. The SOA Gateway uses standard technologies and can thus use the most up to date security standards available today. This means that data can be secured using the Secure Sockets Layer (SSL) technology which is the standard in this area.
    1. The SOA Gateway interfaces with existing security systems. This ensures that existing security rules, which have been built up over many years, are still enforced for requests
    2. coming through the SOA Gateway. In effect, the SOA Gateway maps an ‘internet credential’ to something the local system where the database is running can understand
  7. The SOA Gateway can map database internal types to a well defined external type. This means that the data can be normalized so that application developers are dealing with the same type of data no matter what database. For example, taking the nationality field again, the SOA Gateway can map the internal database representation into an agreed external format and ensure that the nationality field from each different database uses the same external format no matter how the internal format looks.
  8. Once created, a service may then be registered with a UDDI server. Examples of such servers are CentraSite from Software AG, the Systinet registry from HP and the OpenUDDI server available as open source. This means that a central repository of all available services can be maintained electronically and future requirements for these services may be satisfied using that repository.
  9. The SOA Gateway license model is based on usage so even though the full power and facilities of the technology are available, you only pay for what you use. It’s possible to pay for units of 5 services per year. For example for a cost of 400 Euro per year, it would be possible to expose 5 database tables as data services.

2.3. The Benefits to this and future projects

It is clear that projects requiring access to multiple data sources can benefit through the use of the SOA Gateway due to the standards based access to the data it provides along with the normalization capability it offers.
Once completed, the organization can continue to get benefit from the infrastructure as can be seen from the following architecture: The services created for an initial project can be reused again and again in later projects no matter what language they are being developed in or what technology is being used. The services are future proofed in that they will interface with any technology on the horizon at the moment. As the infrastructure is 100% based on standards, your installation will also benefit from run time governance, policy driven processing and standards based security out of the box. As new standards or best practices emerge, these will be added to the SOA Gateway without any changes to your applications. In addition, the license model ensures that licensed customers are entitled to all of the new developments and improvements that are made in later versions of the SOA Gateway. This means that the full power of the technology is available if you continue to use it with a small number of services or if it is used on an extensive basis in your projects.

3. Implementation and Using the SOA Gateway

The SOA Gateway has been designed to be as simple as possible to license, install and use in your projects. The following steps can generally be completed in a half a day or less at which point, it is possible to continue creating services from your existing core assets in minutes. In general, the following time is required to start working with the SOA Gateway:
  • Registration and download: 30 minutes (depending on the speed of your connection to the Internet)
  • Installation of the SOA Gateway Control Centre and one SOA Gateway Server: 1 hour (depending on target platform and speed of the link to that target platform)
  • Creation of services: <1 Minute per service
3.1. Installation of the SOA Gateway
The SOA Gateway consists of two distinct pieces of software.
  1. The SOA Gateway Control Centre is an Eclipse plug-in that runs in Eclipse. Eclipse and this plug-in are contained in the package downloaded after you register.
  2. The SOA Gateway Server is installed as a stand alone component on the platform where the data or business logic that you wish to expose is running. The server implementation for all available platforms is also included in the package downloaded after you register. The SOA Gateway Server can then be deployed to each target server using the SOA Gateway Control Centre.
The installation of the SOA Gateway requires that you register for a license for the SOA gateway and download the installation materials. There are free licenses available for most uses with some limitations on commercial usage.

3.1.1. Registering to use the SOA Gateway

You must register to use the SOA Gateway here as illustrated by the following screenshot.
Once you have filled the details in on the above screen and hit the ‘Register’ button, a confirmation email will be sent to the email account with which you registered. When you have received that email and confirmed the email address, a second email will be sent with a link to continue the process and a license file attached. This link will give you further information about the installation process and will start the download of the SOA Gateway installation materials to your local PC. This download is approximately 250 Meg and how quickly this downloads will depend on the speed of your link to the Internet.

3.1.2. After the download

Once you have downloaded the package, you are ready to begin installing the SOA Gateway. The next steps are documented in the email sent to you after you have confirmed your registration. Once you have completed those steps, your configuration will look like the following:

3.1.3. Installing and Configuring the SOA Gateway Server

Once the SOA Gateway Control Centre has been started in Eclipse, you will need to install the SOA Gateway on the target platform. This is a little different depending on your target operating system:

Windows Installation

On Windows, the SOA Gateway Control Centre must be installed on the Windows system where you wish to install the SOA Gateway server. When you select that you wish to install the SOA Gateway server on Windows, the Control Centre will launch the Windows Setup program on the local machine and the SOA Gateway server installation and configuration steps are managed by that setup script.
Once the setup script has completed, you will be returned to the Control Centre Deployment Wizard. Note that the Eclipse running on the Windows server is only required for installation. This server can subsequently be managed and configured from a remote administrators PC in the same way as other platforms.

Other Platform Installation

On all other platforms, the SOA Gateway Control Centre will FTP the required installation materials to the target system.

Once the FTP has been completed, you must logon to the target system to run a short script or a number of jobs to complete the installation process. These are documented in the installation documentation for the platform where you are installing the SOA Gateway Server. Once this has been completed, return to the control centre to complete the process.

Common Configuration Steps

Once the SOA Gateway Server is running, it is possible to test if the Control Centre can communicate with it from the current screen in the deployment wizard. Once this communication is ok, you just need to hit the configure button. This will install and configure each of the licensed SOA Gateway data source drivers in your server environment. Where additional installation specific information is required, this will be requested during this process. Please refer to the documentation for more details on what may be required for each of the data source drivers.

Once the configuration step has been completed, you are ready to start creating services.

Supporting Multiple Platforms and Operating Systems

It is possible to deploy and install the SOA Gateway server on multiple machines where access is required while monitoring these from the one administrators PC as per the following architecture:

3.2. Creating the Services

The creation of services is achieved from the SOA Gateway Control Centre and is a 3 step process.

  1. The SOA Gateway can discover what resources are available for a specific data source for which services can be created.

  2. The Meta data for those resources is identified and used to create what is required by the SOA Gateway.

  3. The service definition is deployed to the SOA Gateway Server and is ready for use.

Risaris Limited also recommend an additional step which is the registration of the WSDL in a UDDI server such as CentraSite from Software AG, the Systinet registry from HP or the OpenUDDI server available as open source, however, this is not required to use the service.

3.2.1. Creating Database Services

Database services are created by simply following these steps:

  1. Identify the database from which you wish to create services.

  2. Provide the database name.

  3. The SOA Gateway Server will return a list of tables or files available on that database as per the following:

  4. Select the tables for which you wish services to be created and hit the continue button.

  5. The wizard will create services for each of the tables you selected and deploy these services to the SOA Gateway Server. These services are now available for use.

3.2.2. Creating Business Logic Services

Business logic services are created by simply following these steps:

  1. Identify where the application which provides the business logic is implemented (e.g. Natural program, CICS application, Windows DLL etc.)

  2. Provide the location of the application.

  3. The SOA Gateway Server will return a list of applications available at that location.

  4. Select the applications for which you wish services to be created and hit the continue button.

  5. Provide the source file for each service to be created.

  6. The wizard will create services for each of the applications you selected and deploy them to the SOA Gateway Server.

  7. Risaris recommend that you review and modify the input only, output only and input/output fields for each service based on your knowledge of the application requirements.

  8. These services are now available.

3.2.3. Registering Services in a UDDI Server

The services you have been created can be registered with a UDDI server as follows:

  1. Define your UDDI server to the SOA Gateway Control Centre (must only be done once).

  2. Select the SOA Gateway services to be registered.

    3. Provide the information required by the control centre Wizard until the registration is

3.3. Using the Services in your project

How the services are used will depend on whether you have used a UDDI server or not.

3.3.1. Using the Services Directly (without a UDDI Server)

  1. The application developer is provided with a WSDL location.

  1. The application developer will import the WSDL from the SOA Gateway server into their

  2. The service is now available for use from the project.

3.3.2. Using the Services from a UDDI Server)

  1. The application developer is provided with the location of the UDDI server.

  1. The application developer selects from the UDDI Server the service that they wish to use.

  2. The UDDI server provides the location of the WSDL.

  3. The application developer will import the WSDL from the SOA Gateway server into their project.