INTRO

This article will attempt to act as an overview of common message broking architectures, the BizTalk architecture and application, BizTalk components and BizTalk capabilities as a message orchestrator and potential application as a human to human and human to system workflow engine.

The message broker patterns commonly used for system integration are primarily concerned about the transport of data from one system to another.

Figure 1: Message Broker Pattern

The pattern has 4 main components.

Transport Layer: As the source or destination of the data could be anything from a file to a mobile text message, it is the job of the transport layer to pick the data up from any supported proprietary location. The transport layer is not concerned about the structure of the data. It only knows how to speak with the location and the format of the data.

Transformation layer: Once the data’s been picked up, the transport layer will need to understand the structure of the data. Most message brokers will use this layer to transform the data into a common format for internal use. Let’s call this our message

Mapping layer: Sometimes combined with the transport layer, the data, now in a common format, can be remapped and manipulated if needed.

Filtering/Routing Layer: This is the core of the message broker. The idea is that the broker can now route the message out based on the content of the data inside the message. The message could have enough information for the routing engine to determine where it should go and what it should do. Alternatively some work may need to be done to figure this out. This could be achieved by manipulating the message, or by applying complex routing rules and decisions and conditions… from here, the process is exactly the same, but in reverse.

Using this pattern, the BizTalk architecture overview looks like this:

Figure 2: BizTalk Architecture

Want to talk?

BizTalk is primarily a technology for integrating applications. In other words, BizTalk acts as a "broker" between applications, to enable them to work together to achieve a goal, solve a problem or provide extra value.    By using a broker approach to application integration you provide a common connection point for these applications and avoid the complexity of custom integration, often significantly decreasing development time.   The broker services provided by BizTalk can offer message transport and transformation and increased support for diverse communication mechanisms such as HTTP, MSMQ, FTP, etc.  On the Windows platform, using BizTalk in this approach is often the best way to integrate with non-Windows applications, such as AS/400 and mainframes.  Various third party adapters are available for BizTalk.

Show me the message!

It’s all about the message. As a message broker, BizTalk is primarily concerned about the transport of messages. It is the main entity in any BizTalk application. By using either the supplier adapters or utilizing third party adapters, BizTalk can pick up messages from a wide variety of sources including FTP, HTTP, MSMQ, etc. The receive pipeline (BizTalk’s implementation of the transformation layer) then converts the message into a known structure (XML as preference). Also here a wide variety of industry standards are supported including flat files, EDI documents, XML, etc. The configuration of ports and the mapping of data in the various pipelines and the optional orchestration a business processes can be done code-free. Current support for different transport and message formats include SAP, Oracle, Siebel and WSS.

Other features include a reliably delivery architecture that ensures that messages don’t get lost.   Using a process of dehydration, processing times of messages in BizTalk can be seconds or years with no detrimental effect on the system.

So within this, what can it provide?

Some of the core capabilities are:

·         XML messaging through different open Internet standards

·         Message transformation/mapping

·         Message validation

·         Application/system connectivity through application adapters

·         Visual business-process orchestration

·         Business rules repository and engine

·         Support for secure and reliable messaging

Support for long-running transactions

Moving beyond just passing message from one point to another, BizTalk gives a user the ability to graphically define the logic that gets applied via orchestrations for its application integration within the filtering and routing layer. This means that a business process can be dictated inside BizTalk by means of receiving messages, and effectively routing them (or related messages) based on a business process. In BizTalk world this is called orchestration.  It can also provide state management and support for longer-running transactions.    Building in transformation and or orchestrations is done graphically in BizTalk, and for those trained and knowledgeable, this is a great feature and makes BizTalk powerful yet easy to use.

Is BizTalk workflow?

Define: ‘Workflow’: "The automation of a business process, in whole or part, during which documents, information or tasks are passed from one participant to another for action, according to a set of procedural rules." – WMC (Workflow Management Coalition)

Indeed messages flowing from system to system through BizTalk, with rules dictating the routing logic is workflow.   However, human-to-human and human-to-system workflow is not always message centric but normally state centric. During human workflow the routing is often based on variety of events and data that we collate into a case. The progression of the workflow normally represents a state change in the case as a whole. Together with that, human workflow requires facilities we as information workers need to effectively and easily interact with a business process.

So what makes human workflow different?

If the bulk of the process is system integration related (transforming, integrating, processing, chunking) and the level of human interaction (making decisions, escalating , displaying data, entering information, tasking, tracking etc) is minimal, a message brokering platform like BizTalk can be leveraged the best bet would be a message broking platform like BizTalk.  However, if the primary focuses of your processes are based on human interaction, then a tool set focused on human-to-human or human-to-system processes and workflow would be more applicable. Not doing so could increase the amount of code, time and level of knowledge required to bring these solutions to production and the level of effort required to maintain them.  Building processes and workflow with a human-focused platform often makes for easier development and deployment. Some items to keep in mind when considering human workflow the features and intricacies that often become important when building and deploying them:

Task management – How are people assigned tasks based on a business process and how are those tasks managed and escalated?

Task and/or work item notification – When someone is assigned a task, how are they notified? Is the notification mechanism flexible and does it conforms with common collaborative technologies like email or instant messaging?

Work item redirect – Can tasks be easily be re-assigned and routed?

Out of office – How are tasks and work items routed and escalated when the intended actor is not available? Is the actors availability based on some sort of presence notification technology?

Escalations – When work sits on one step, or with one individual, do you want the workflow system to take action, such as send reminder e-mails or escalate the work item to a manager in line with the business process?

Ad-hoc workflow – Informal human workflow and routing used today only have their routes decided at run time. How can the system support this?

Versioning – Human workflows tend to change. How will your system support these changes dynamically and allow for the different versions of the business process to exist at the same time?

Reporting – Human workflows often require the generation of reports and SLA’s based on the business process. This could be based on the meta data of the business case or the people involved in the business process.

Because a message broker platform like BizTalk does not natively address requirements placed on a human business process, accomplishing some of the above can be difficult and cumbersome.

What have we learned and how does that help me?

BizTalk is about applications, systems, messages, etc. Each of these can be called process or workflow, but don’t be confused: This is rarely about interaction with a person, or human, or role. The routing of messages is message centric while human workflow is case centric.  

For BizTalk evaluations ask:

·         Are my needs primarily message based?

·         Do I need to do complex transformations on messages?

·         Do I need application integration?

·         Are humans going to be involved in this process and what human workflow activities will be required?

Once I have this mapped out, I can start to make decisions on what I need to write (what must I control completely) and what I am going to buy.  BizTalk can help greatly in platform- and system- integration scenarios.  It excels when you need to connect various systems or applications, especially when they use different communication methods or protocols.  If you have simple integration needs, without the need for extensive transformations, and you do not anticipate many other needs of this type, then perhaps coding them manually is the way.

By considering the above, your options range from custom, point to point integration to implementing a robust, industry standard approach with added benefits of orchestration like BizTalk Server. For the automation of business processes that require human task driven execution, consider a human workflow platform like the K2. If a combination of the above is required make sure your human workflow engine and your system integration tool will happily speak to each other.

Where to now?

BizTalk is an information broker for disparate systems.  It does not provide data, it does not service data, and it does not expose data.  It moves data. So if you need a tool that can easily handle moving data from one format or system to another with built-in capabilities for atomic transactions, and planned roll backs, and the ability to orchestrate a message and system centric process around this, then BizTalk is a sound option.     

War stories…

One customer I talked to had custom .NET business objects/entities (like a customer object or a price change object) that they had developed.  They also had configured these to be serializable (so you could use them in a Web service where they're represented as XML). The benefit to this is that, in their BizTalk  maps, the customer could map the data to the XML schema for the object in question and then just write a single line or two of .NET code to “object.deserialize(xml).” Then they have a fully populated .NET object without doing much at all. This makes creating your Web services simpler, because instead of having to pass in the individual parts, you could just pass in your business object whose parts had been populated for you. This is a good example of a customer that LOVES BizTalk.

This customer says, “For us, BizTalk is a one-stop shop for moving data from one system or format to another with full system-side, non-human-interactive process development flow.”   For more human interactive workflow, the purpose of the tools is to better expose and use data  interactively — be it decision criteria, expense report data — and have humans interact with it, use it, change it or reply to it. 

I would like to take this opportunity to thank all those who helped contribute to this most specifically Gabriel Malherbe, Jeremy Ragan, Bob Maggio, and ChrisT