Filters and pipes

Pipes and Filters is a design pattern that is responsible for moving the output of one system into another. This design pattern is useful when trying to connect two different systems with different web services schema or different formats. This often occurs during updating old software, so that the new one can use portions of the old one. Integrating such systems involve a whole series of transformations, like transforming XML-based service communications another XML format or Binary to XML based.

A solution for this problem is to implement a sequence of filter components, where filters transform the received messages and forward the to the next component. The interchanged data between filters and other components are transported through pipes.

When a system needs more than one filter, the pipes carry the data from the first filter to another until its fully transformed into a compatible format. Pipes also buffer the carried data until the target filter is ready to transform them. After applying all needed transformations, the pipe transfer the data to the targeted application which consumes it.

This pattern needs the output data to be compatible with the input of the next filer or application. In general the output of one component should be compatible with the input of the next component in row.

The picture below shows an example of using filers and pipes:

SOA: The Definition

I have decided to write a series of posts regarding the topic that recently became a hot one. I will write a series about Service Oriented Architecture (SOA), here you have the first part, which will introduce you to the concept of what is SOA…

First of all before diving into technical aspects of this topic, you should have an understanding of what Service Oriented Architecture is. Let’s take a look at your city. It is already full of service-oriented businesses. Individual companies are service-oriented in that each provides a distinct service that can be used by multiple customers. Collectively, these businesses comprise a business community. It makes sense for a business community not to be served by a single business outlet providing all services. By decomposing the community into specialized, individual outlets, we achieve an environment in which these outlets can be distributed.

When coupled with “architecture”, service-orientation takes on a technical connotation. “Service Oriented Architecture” is a term that represents a model in which automation logic is decomposed into smaller, distinct units of logic. Collectively, these units comprise a larger piece of business automation logic. Individually these units can be distributed.

Distributing automation logic into separate units is nothing new. What is it then that makes service-oriented separation so different? Keep on reading…

Even in a distributed business community, if we impose overbearing dependencies, we could inhabit the potential of individual businesses. Although we want to allow outlets to interact and leverage each other’s services, we want to avoid a model in which outlets from tight connections that result in constrictive inter-dependencies. By empowering business to self-govern their individual services, we allow them to evolve and grow relatively independent from each other.

Though we encourage independence within our business outlets, we must still ensure that they agree to adhere to certain business conventions – for example, a common currency for the exchange of goods and services, a building code that requires signage to confirm to certain parameters or perhaps a requirement that all employees speak the same language as the native consumers. These conventions standardize key aspects of each business for the benefit of the consumers without significantly imposing on the individual business’s ability to exercise self-governance.

Similarly, service oriented architecture (SOA) encourages individual units of logic to exist autonomously yet not isolated from each other. Units of logic are still required to conform to a set of principles that allow them to evolve independently, while still maintaining a sufficient amount of commonality and standardization. Within SOA, these units of logic are known as services.

Service Oriented Architecture: Common Characteristics

This is the second post in my series about Service Oriented Architecture. After defining what SOA is in my first post, I will discuss the common characteristics of service-oriented solutions:

 

SOA is fundamentally autonomous

The service-oriented principle of autonomy requires that individual services be as independent and self-contained as possible with respect to the control they maintain over their underlying logic. This is further realized through message-level autonomy where messages passed between services are sufficiently intelligence-heavy that they can control the manner in which they are processed by recipient service.

SOA builds upon and expands this principle by promoting the concepts of autonomy throughout solution environments and the enterprise. Applications comprised of autonomous services, for example, can themselves be viewed as composite, self-reliant services that exercise their own self-governance within service oriented integration environments.

 

SOA is based on open standards

Perhaps the most significant characteristic of Web Services is the fact that the data exchange is governed by open standards. After a message is sent from one Web Service to another it travels via a set of protocols that is globally standardized and accepted.

Further, the message itself is standardized, both in format and how it represents its payload. The use of SOAP, WSDL, XML and XML Schema allow for messages to be fully self-contained and support the underlying agreement that to communicate, services require nothing more than knowledge of each other’s service description. The use of open, standardized messaging model eliminates the need for the underlying service logic to share dependencies (such as type systems) and support the loosely coupled paradigm.

 

SOA supports vendor diversity

The open communications standards just explained not only has significant impact for bridging much of the heterogeneity within (and between) corporations, but it also allows organizations to choose best-of-breed environments for specific applications.

For example, regardless of how proprietary a development environment is, as long as it supports the creation of standard Web Services, it can be used to create a non-proprietary service interface layer, opening up interoperability opportunities with other, service capable applications. This, incidentally, has changed the face of integration architectures, which now can encapsulate legacy logic through service adapters, and leverage middleware advancements based on Web Services.

Organizations can certainly continue building solutions with existing development tools and server products. In fact, it may make sense to do so, only to continue leveraging the skill sets of in-house resources. This option is made possible by the open technology provided by Web Services framework and is made more attainable through the standardization and principles introduced by SOA.

 

SOA promotes discovery

Even though the first generation of Web Services standards included UDDI, few of the early implementations actually used service registries as part of their environments. This may have to do with the fact that not enough Web Services were actually built to warrant a registry. However, another likely reason is that the concept of service discovery was simply not designed into the architecture. When utilized within traditional distributed architectures, Web Services were more often employed to facilitate point-to-point solutions. Therefore, discovery was not a common concern.

SOA supports and encourages the advertisement and discovery of services throughout the enterprise and beyond. A serious SOA will likely rely on some form of service registry or directory to manage service descriptions.

 

SOA fosters intrinsic interoperability

Further leveraging and supporting the required usage of open standards, a vendor diverse environment, and the availability of a discovery mechanism, is the concept of intrinsic interoperability. Regardless of whether an application actually has immediate integration requirements, design principles can be applied to outfit services with characteristic that naturally promote interoperability.

When building a service-oriented application from the ground up, services with intrinsic interoperability become potential integration endpoints. When properly standardized, this leads to service-oriented integration architectures wherein solutions themselves alleviate the cost and effort of fulfilling future cross-application integration requirements, and evolves the enterprise into one where traditional application boundaries begin to disappear.

 

SOA fosters inherent reusability

SOA establishes an environment that promotes reuse on many levels. For example, services designed according to service-orientation principles are encouraged to promote reuse, even if no immediate reuse requirements exist. Collections of services that form service compositions can themselves be reused by larger compositions.

The emphasis placed by SOA on the creation of services that are agnostic to both the business process and the automation solutions that utilize them leads to an environment in which reuse is naturally realized as a side benefit to delivering services for a given project.

 

SOA emphasizes extensibility

When expressing encapsulated functionality through a service description, SOA encourages you to think beyond immediate, point-to-point communication requirements. When service logic is properly partitioned via an appropriate level of interface granularity, the scope of functionality offered by a service can sometimes be extended without breaking the established interface.

Extensibility is also a characteristic that is promoted throughout SOA as a whole. Extending entire solutions can be accomplished by adding services or by merging with other service-oriented applications (which also, effectively, “adds services). Because the loosely coupled relationship fostered among all services minimize inter-service dependencies, extending logic can be achieved with significantly less impact.

SOA implements layers of abstraction
One of the characteristics that tend to evolve naturally through the application of service-orientation design principles is that of abstraction. Typically SOAs can introduce layers of abstraction by positioning services as the sole access points to a variety of resources and processing logic.

When applied through proper design, abstraction can be targeted at business and application logic. For example, by establishing a layer of endpoints that represent entire solutions and technology platforms, all of the proprietary details associated with these environments disappear. The only remaining concern is the functionality offered via the service interfaces.

 

SOA is a building block

Service-oriented application architecture will likely be one of several within an organization committed to SOA as the standard architectural platform. Organizations standardized on SOA work toward an ideal known as the service-oriented enterprise (SOE), where all business processes are composed of and exist as services, both logically and physically.

When viewed in the context of SOE, the functionality boundary of an SOA represents a part of this future-state environment, either as a standalone unit of business automation or as a service encapsulating some or all of the business automation. In responding to business model-level changes, SOAs can be augmented to change to nature of their automation, or they can be pulled into service-oriented integration architectures that require the participation of multiple applications.

What this all boils down to is that an individual service-oriented application can, in its entirety, be represented by and modeled as a single service. There are no limits to the scope of service encapsulation. An SOA consists of services within services within services, to the point that a solution based on SOA itself is one of many services within SOE.

 

SOA is an evolution

SOA defines an architecture that is related to but still distinct from its predecessors. It differs from traditional client-server and distributed environments in that it is heavily influenced by the concepts and principles associated with service-orientation and Web Services. It is similar to previous platforms in that it preserves the successful characteristics of its predecessors and builds upon them with distinct design patterns and a new technology set.

For example, SOA supports and promotes reuse and encapsulation, as well as the componentization and distribution of application logic. These and other established design principles that are commonplace in traditional distributed environments are still very much part of SOA.

SOA is still maturing

While the characteristics described so far are fundamental to SOA, this point is obviously more of a subjective statement of where SOA is at the moment. Even though SOA is being positioned as the next standard application computing platform, this transition is not yet complete. Despite the fact that Web Services are being used to implement a great deal of application functionality, the support for a number of features necessary for enterprise-level computing is not yet fully available.

Standards organizations and major software vendors have produced many specifications to address a variety of supplementary extensions. Additionally, the next generation of development tools and application servers promises to support a great deal of these new technologies. When SOA platforms and tools reach an adequate level of maturity, the utilization of Web Services can be extended to support the creation of enterprise solutions, making the ideal of a service-oriented enterprise attainable.

If you need to provide an accurate definition of SOA today, you would not be out of line to mention the state of its underlying technology. Considering the rate at which the IT industry as a whole is adopting and evolving the SOA platform, though, it should not be too long before such a statement would no longer be required.

Message exchange patterns

Every task automated by a Web Service can differ in both the nature of the application logic being executed and the role played by the service in the overall execution of the business task. Regardless of how complex a task is, almost all require the transmission of multiple messages. The challenge lies in coordinating these messages in a particular sequence so that the individual actions performed by the message are executed properly an in alignment with the overall business task.

Message exchange patterns (MEP) represent a set of templates that provide a group of already mapped out sequences for the exchange of messages. The most common example is a request and response pattern. Here the MEP states that upon successful delivery of a message from a service to another, the receiving service responds with a message back to the initial requestor.

The MEP currently in the second generation Web Services are divided into two categories: Primitive and Complex, in this post I will discuss the two most popular primitive MEPs:

 

Request-response

This is the most popular MEP in use among distributed application environments and the one pattern that defines synchronous communications (although this pattern also can be applied asynchronously).

The request-response MEP establishes a simple exchange in which a message is first transmitted from a source (service requestor) to a destination (service provider). Upon receiving the message, the destination (service provider) then responds with a message back to the source (service requestor).

 

Fire-and-forget

This simple asynchronous pattern is based on the unidirectional transmission of messages from a source to one or more destinations.

A number of variations of the fire-and-forget MEP exist, including:

- The single-destination pattern, where a source sends a message to one destination only.
- The multi-cast pattern, where a source sends message to a predefined set of destinations.
- The broadcast pattern, which is similar to the multi-cast pattern, except that the message is sent out to a boarder range of recipient destinations.

The fundamental characteristic of the fire-and-forget pattern is that a response to a transmitted message is not expected.

©2008 Karim Agha. All rights reserved.