Copyright Notice

Some material in this lecture is adapted from the teaching materials of the book “Web Services: Concepts, Architectures and Applications” and is thus Copyright © 2003 Gustavo Alonso, ETH Zürich and/or Copyright © 2004 Springer-Verlag Berlin Heidelberg.

All other material is Copyright © 2006 Kenneth M. Anderson

Enterprise Application Integration

Initially, middleware was used to construct new systems… Later, it was used to distribute [application logic] and to integrate [servers of 3-tier architectures]… When these systems are very different in nature and functionality, using conventional middleware to integrate them becomes rather cumbersome, and in some cases simply infeasible. EAI can be seen as a step forward in the evolution of middleware, extending its capabilities to cope with application integration…
    — Introduction to Chapter 3 of our Textbook

• The Problem of Enterprise Application Integration (EAI)
• Message Brokers
• Workflow Management Systems

From Middleware to Application Integration

• The difference between conventional middleware and EAI is subtle.
• To understand the differences, we need to differentiate between application development and application integration
• Conventional middleware was used to create applications on top of a heterogeneous set of resources (e.g. databases)
  • developers focused on creating application logic in the middleware that could access the required information in the resources and prepare it for delivery to the presentation layer
  • this process was aided by various standardization efforts that made it easy to access a particular class of resources (e.g. JDBC for databases)
• The functionality created by this type of application development can be exported as a new service, which can be used to create higher-level services.
• This in turn leads to a proliferation of services that need to be integrated, but its harder to integrate separate services because
  • there is no agreement on a generic service interface
  • a company may find that it needs to integrate services from two (or more) different middleware platforms, and this is much more difficult than integrating multiple services and resources that run under a single middleware platform
• In summary, enterprise application integration refers to the problem of integrating services hosted by a heterogeneous set of middleware platforms, often owned and operated by different parts of the same companies, or by different companies altogether

EAI Example: Automating a Supply Chain

• A supply chain is the set of operations carried out to fulfill a customer's request for products and services.
• A basic supply chain consists of the following steps: quotation, order processing, and order fulfillment
• Quotation involves processing a RFQ from a customer.
  • In a RFQ, the customer asks about the availability and price of various types of goods.
  • If the returned quote is satisfactory, the customer will submit a purchase order to the company.
• Order processing involves analyzing a purchase order to verify that it matches its associated RFQ and that it can be fulfilled under the conditions specified by the customer. It also involves placing the order internally to begin the production process.
• Order fulfillment involves procurement, shipment, and financial aspects (such as invoicing the customer, paying suppliers, etc.)

Characteristics of a Supply Chain before EAI

• Most supply chains are much more complex than what was presented on the previous slide!
• Each step of the process is likely to be supported by a different information system
  • Companies maintain extensive customer, product, and supplier databases
  • Processing an RFQ may involve multiple queries to each of these systems, checking inventories, coordinating with suppliers, etc.
  • Each of these databases may live in separate systems that may reside in separate geographic locations.
• These systems will be heterogeneous with some being home grown, others based on off-the-shelf packages, and others being the results of previous EAI efforts. Other characteristics include
  • Each system may run on a different operating system
  • Each system may support different interfaces and functionality (e.g. some may support transactions, others may not)
  • Each system may use different data formats
  • Each system may have different security requirements
  • Each system may use a different infrastructure (e.g. middleware system) as well as different interaction models and protocols
• Note: these are just TECHNICAL issues; to make matters worse, an EAI effort may encounter POLITICAL issues, such as systems being owned by different departments or companies with different needs and goals

Benefits of EAI

• Given all the issues, is it worth attempting EAI in this situation? YES!
• If a supply chain is not automated, it means that transitions between steps are handled manually
• This involves a lot of repetitive steps, exchanging paper documents, with all sorts of opportunities for the introduction of errors.
• Order are difficult to monitor and track.
• Its difficult to get an overview of the entire supply chain, or even to get status on a single order
• Tracking and monitoring of an order means following the paper trail (which may not all be located in a single physical location)
• These inefficiencies strongly affect the quality of the supply chain

Message Brokers

• To address the difficulties of EAI, MOM systems were evolved into message brokers
  • Traditional MOM systems have a problem in that they require the sender of a message to specify the recipient (in particular, the name of a specific queue in the MOM system)
  • This “point-to-point” addressing scheme becomes increasingly complex to manage as the number of participants increase and as the environment becomes more dynamic
  • In particular, if a new application is added to an existing MOM-based system that requires a particular type of message, all other applications that generate that message need to be modified to send that message to the new application

• Message brokers extend MOM functionality in the following ways:
  • provides the capability of attaching logic to messages and processing messages within the middleware level
  • filters and routes messages automatically; the sender of a message does NOT have to specify a recipient (known as content-based routing)
  • provide adapters that hide heterogeneity and make it possible to access all systems with the same programming model and data exchange format

Basic Architecture of a Message Broker

• In order to process messages, message brokers allow developers to specify logic at the middleware level
• One advantage to this approach is that the logic is centralized within the middleware and not distributed across multiple applications
  • if the logic has to change, we only have to modify the module in the middleware
• A second advantage to this approach is that senders and receivers are decoupled.
• Routing logic:
  • Routing logic can be based on the sender's identity, on the message type, or on the message content
  • Routing logic is typically written in a rule-based language; if a rule's condition evaluates to true for a given message, its action indicates to which queues the message should be delivered.
  • Routing logic can be defined at a global level or for a specific queue. At the global level, it is applied to all messages; At the queue level, the rule is used to indicate the type of messages that it wants to receive
• Other types of logic can be placed within a message broker including filtering logic, which indicates messages that should not be processed at all, and content transformation rules which can be used to handle differences in data formats between applications.
  • An example of a content transformation rule might be to convert a value generated with one type of units (e.g. pounds) into another type of units (e.g. kilograms)

Publish/Subscribe Interaction Model

• Publish/Subscribe (aka Pub/Sub) is one of the best known interaction models that can be supported by message brokers.
  • A lot of research has been done on this style of interaction, sometimes going under the name of “event notification systems”. One example of a WAN-based event notification system is Siena, developed at CU Boulder under the direction of Antonio Carzaniga.
• Pub/Sub is a message exchange interaction style in which publishers publish messages (aka events) which are typed attribute-value sets. Publishers do not know (and should not care) where their messages go or who receives them. If an application is interested in receiving a particular class of message it must subscribe its interest with the message broker. Whenever a message is published, the message broker examines its current set of subscriptions and delivers a copy of the message to only those applications that subscribed for it.

• Subscriptions take one of two forms
  • the name of a message type (which can often live in a hierarchical name space)
  • or, a set of boolean conditions that reference the attribute names that appear in messages
    • Example: sub(type = "openDocument", app = "MS Word", user = "kena") would subscribe to all "openDocument" events that involved user kena opening Microsoft Word documents.
• The Pub/Sub model makes it easy to integrate multiple message broker systems. (Messages are a universal data type!) You simply have one message broker register subscriptions for its applications with the external message broker. When a relevant event occurs in the external message broker, it is delivered to the original message broker which in turn forwards it to relevant subscribers.

EAI Architecture

• EAI platforms are based on two basic principles
  • Adapters: programs that map heterogeneous data formats, interfaces, and protocols into a common model and format. The purpose of adapters is to hide heterogeneity; a different adapter is needed for each type of application that needs to be integrated.
  • Message Broker: the message broker facilitates the interaction among adapters and a program that contains the integration logic

• With this in place, EAI-based application integration amounts to:
  • Developing a program that contains the integration logic (what messages need to be sent and in what order)
  • Configuring the adapters so that they subscribe to the appropriate messages and perform the appropriate actions on their associated back-end system

Returning to the Supply Chain…

Assessment of Message Brokers as EAI Platforms

• Advantages
  • Lower development costs: integration is relatively simple and can be done more quickly in comparison to other middleware platforms
  • Lower opportunity costs: since integration can occur more quickly, automation can be achieved more quickly; this increases an organization's ability to deploy new services
  • Lower maintenance effort: The use of adapters modularizes the system nicely; if an integrated system changes, only its adapter needs to change
• Disadvantages
  • Software Licenses are extremely expensive; an EAI message broker and associated tools can cost in the hundreds of thousands to millions of dollars.
  • EAI vendors also charge ADDITIONAL money for standard adapters
  • EAI systems requires investments in the training of IT personnel and may require the acquisition of IT resources (hardware, networks, etc.)
  • If you have an application that is not covered by a vendor's adapters, your organization will have to develop a new adapter from scratch

Workflow Management Systems

• Relevant to the context of EAI because they are often used to specify the integration logic of an EAI system (back on this slide, the integration logic was simply a black box)
• Originally used in office automation systems to automate administrative workflows (such as the ever popular travel authorization process), workflow management systems are now being used to specify the business logic necessary to coordinate heterogeneous and distributed systems (aka production workflows).
• One advantage is that some of the business logic can be expressed in the graphical languages used by these systems to construct workflow, and these graphs are much more intuitive to end users than code written in standard programming languages
• Most EAI vendors today offer workflow management systems that can be used to interact with their message broker and adapter products

Workflow Specifications

• Workflows are a directed graph where nodes can be one of the following types:
  • Work node: represent work items to be performed by a human or automated resource
  • Routing node: define the order in which work items should be executed and allows the definition of parallel or conditional activation of work nodes
  • Start and Completion nodes: indicate respectively the start and the end of the workflow process

Workflow Execution

• Workflows are executed by a workflow engine with the following generic architecture

• In the general case, a workflow engine:
  • 1) receives notification that a work item was complete
  • 2) examines the workflow specification to determine what work node needs to be activated
  • 3 & 4) consults with a resource broker to find a resource for the activated work node
  • 5) assigns the work node's task to the selected resource's work queue
• A resource will occasionally check its queue (especially if that resource is human) and if a work item is there, will begin to work on it. Eventually the task will be complete and it will be sent to the workflow engine's completed work queue.

Assessment of Workflow Management Systems

• Advantages
  • Rapid process design and maintenance
  • Failure and Exception Handling
  • Workflow design is supported via a graphical development environment
• Disadvantages
  • Expensive Software Licenses
  • Complex Installation and Operation
  • Long development cycles to actually automate real business processes

Coming up Next

• In lecture 6, we will move onto chapter 4 of the textbook and discuss a variety of Web Technologies.
• Note: by Thursday of next week, I will need volunteers to talk about the “big three” of Web services: SOAP, WSDL, and UDDI