Category name:ESB

Building your own Adapter. Serious Pitfals / Adapter WOES !!

March 31, 2009 / Comments Off on Building your own Adapter. Serious Pitfals / Adapter WOES !!

This blog post is about several experiences I had when building my own Sybase adapter.

Normally i would suggest to the customer that they should buy an adapter, but in this case that was not possible since the only good working Adapter for Sybase was produced by TopXML and they vanished from this globe…. So I had to start building my own.

If you do a couple of searches on google you will probably find the BizTalk Adapter Wizard on codeplex. This is a good start cause it helps you a great deal in getting the framework four your adapter up and running in no time. However there are some dangers involved in using the wizard. One of the problems caused by the Adapter Wizard is the dreaded “The transport proxy method DeleteMessage() failed for adapter XYZ (Bug in Codeplex Adapter Wizard)” message wich I blogged about previously.

To describe this problem in the shortest possible form I could say, it copies ALL of the context from the message beeing send to the response message (request response port) so everything, messageID and stuff are all exactly the same. At a point in time BizTalk wants to delete a message with a specific ID and it finds two messages with that ID. This will probably result in several strange messages in the eventlog and sometimes even cycle your complete BizTalk Server. This is not the behaviour that you want.

Fortunately the solution is pretty simple, If you look at the generated code you will see a method named BuildResponseMessage. In this method the copying of the context is done. Simply remove the last statement from the method (btsResponse.Context = contextπŸ˜‰ and your adapter is cured from this obscure problem.

Then I thought everything would run well and I started to submit Loads of requests against my new adapter. And that’s when the problems really started. To explain a littlebit what happened i have to explain what I did as the first step in my Adapter.

The first thing my adapter would do is read the incoming stream (from the pipeline) and load that up into a XML document. And i could see that stap failing in high volume scenario’s. I would see this step fail with several errors in the eventlog. Errors i saw in the eventlog were:

  • There are multiple root elements
  • Root element is missing
  • Tag is missing
  • And many more variants

This was pretty obscure cause I was sending the same XML file (via passthrough pipelines) in a high volume. And all my XML files were correct. !! This made me believe there was something wrong with BizTalk cause my streams that should contain valid XML were all messed up…

So I had a look at the HTTP adapter that comes with the SDK and I compiled it and set it up so it would call a web service over HTTP. And for sure within an hour i was up and running and my webservice gave the correct response. Again all with passthrough pipelines so BizTalk should not touch the message at all. And When i ran a High Volume of messages over this new connectio i got EXACTLY the same problem. So now i knew it was not my code messing stuff up, but it was some more generic problem.

Since I am not the right developer to look at the inner workings of the adapterframework, I called MS for help. At first they were reulctant to help, (cause they were just samples). After some help from the guys from MS Netherland the issue was pushed through and and the guys from MS finally came through with a solution. It took quite some time for MS to solve this one and I guess they are probably the ony people on earth who could have solved the problem….

So here is what they changed: (Original First)

        public override IBaseMessage ProcessMessage(IBaseMessage message)
        {
                this.solicitMsg = message;
                HttpAdapterProperties props = new HttpAdapterProperties(message, this.propertyNamespace);
                IBaseMessage responseMsg = null;
                if (props.IsTwoWay)
                {
                    WebResponse response = SendHttpRequest(this.solicitMsg, props);
                    responseMsg = BuildResponseMessage(response);
                }
                else
                {
                    WebResponse response = SendHttpRequest(this.solicitMsg, props);
                    response.Close();
                }
                return responseMsg;
            }
        }

 They changed it into :

        public override IBaseMessage ProcessMessage(IBaseMessage message)
        {
                HttpAdapterProperties props = new HttpAdapterProperties(message, this.propertyNamespace);
                IBaseMessage responseMsg = null;
                if (props.IsTwoWay)
                {
                    WebResponse response = SendHttpRequest(message, props);
                    responseMsg = BuildResponseMessage(response);
                }
                else
                {
                    WebResponse response = SendHttpRequest(message, props);
                    response.Close();
                }
 
                return responseMsg;
        }

After applying this change the HTTP adapter would immediately work as expected. So I tried the change in my own Sybase adapter and BINGO…. everything worked as expected.

So I have learned a lot from building this adapter. Some TakeAway’s

  • Really stress test your adapters (With100 messages everything would run fine ..)
  • If you have done the first step, multiply the number of messages with 10 and see what happens…
  • Codeplex and stuff to make you more productive are nice, but they have bugs in them as well..
  • Samples from MS are not flawless either. (Expect an update from MS in the future)

So with this post I hope to help several people who have run into problems when creating custom BizTalk adapters….

The only thing I really don’t get is why this code change would have such a dramatic impact…..

 

Interesting post about Schema versioning in BizTalk

December 4, 2008 / Comments Off on Interesting post about Schema versioning in BizTalk

Today I found a very intersting post about schema vesrioning and BizTalk >>Here<<

Basically it says, if you have schema versioning, just increase the version number of your DLL with the schema.
BizTalk will then resolve automatically to the latest version.

Just one of those things that are easy to forget.

HowTo define a Custom Soap Header in BizTalk. Expose it, Consume it , even map them..

June 13, 2008 / Comments Off on HowTo define a Custom Soap Header in BizTalk. Expose it, Consume it , even map them..

 

1. Create the schema that defines your custom header. This schema should be a NORMAL schema. (Give it a decent rootnode name)

2. Create a PROPERTY schema with TargetNamespace : http://schemas.microsoft.com/BizTalk/2003/SOAPHeader.

3. Make sure you define a property in the PROPERTYSCHEMA with EXACTLY the same name as the ROOTNODE of the schema in step 1

4. Make sure you set the set the “Property schema base” to “MessageContextPropertyBase” !!!!!

5. Deploy.

That’s it…… I was really confused by other posts on the web that are just not clear enough… But the summary above is very short so I will explain a bit more.

First step.

Create a normal schema that represents your custom header. Below is a screenshot of my schema.

But I also wanted to send a custom header, so I had to create a second schema representing my outgoing envelope.

 

Second step.

Now I had to create a property schema. Taking care that my property names where EXACTLY the same as the Rootnodes in the schemas. I also had to set the namespace to the correct SOAPHeader namespace.

After setting  the “Property schema base” to “MessageContextPropertyBase” I deployed everything. And I was basically ready to go.

The orchestration

I created a very simple orchestration. It Receives a message, Maps it to the output and sends the output back. More on this orchestration later. I then generated a web service With the wizard that came with Biztalk. It is probably important to mention that when generating the service, I added extra SOAP headers. As the inbound header I specified the “inHeader” schema  and as the outbound header I specified the “outHeader” schema (So not the propertyschema). Then after some wizardry of the wizard, I fired up my trusted WebService Studio tool” and hit the GET button. I was presented with the following picture..

Everything I was expecting, was there. So it was very easy to send and receive headers.

Receiving

After receiving the message, the Adapter (it also works with PASSTHROUGH) will strip off the header and put it into the context property. Below is a screenshot of the received message. As you can see there is no messagetype set so it was received via a passthrough pipeline.

But how nice … the context property “inHeader” contains an XML document with the content of the SOAP header.

So if you want to get to a Specific SOAP Header in an Orchestration. The following code should do the trick.

 

Sending

For Sending messages with the header it works the other way around. Just make sure you have set the content of the outbound header context property to an XML document representing a valid header. I created mine with the mapper.

Then I threw in a shape where I put the contructed document into the context property…

And Presto….

 

 

 

 

BizTalk V-Next

October 29, 2007 / Comments Off on BizTalk V-Next

At last they have shed some light on what the next version will be :


Service Oriented Architecture









β€’

Deeper alignment with the .NET Framework – Building deeply on the investments made with Windows Communication Foundation and Windows Workflow Foundation by closer alignment of these technologies and BizTalk Server to enable customers to build richer composite applications and service oriented architectures than they can today.

β€’

Hosted Services – BizTalk Services, which represent hosted versions of a core set of platform services. While currently in incubation, BizTalk Services is an Internet Service Bus (ISB) that combines identity and access control, message routing (relay), and publish/subscribe event brokering (eventing). For more information, see BizTalk Labs.


Composite Applications









β€’

Broader Model Support – Building on the work with XLANG (BizTalk Server) and XAML (WF) by providing a richer set of domain-specific models

β€’

Runtime – Building on the host investments made with WF and BizTalk Server, to enhance the deployment and management experience of composite applications.


Business Process Management












β€’

Human Tasks – Integration with SharePoint Server to provide expanded support for human task-based scenarios.

β€’

Model Driven – Enhancements to existing visual modeling and design experiences that span roles and help foster team collaboration.

β€’

Complex Event Processing – Enhancements to BAM to provide broader monitoring support.


complete post, have a look here

Storing Context properties in Processing instructions

September 25, 2007 / Comments Off on Storing Context properties in Processing instructions

If you are working with BizTalk you have probably used context properties for routing etc. Sometimes you want to inspect a message and see what the context properties are.
Or if you have build a (kind of) ESB using the messagebox it would be nice to have a listener and spit all the messages that hit the ESB to a specific directory just for diagnostic purposes. But you want to be able to see the context properties together with the message.


I have build a very simple pipeline component to do just that. Below is the listing of the pipeline component that writes a context property to a processing instruction :



  public Microsoft.BizTalk.Message.Interop.IBaseMessage Execute(Microsoft.BizTalk.Component.Interop.IPipelineContext pc, Microsoft.BizTalk.Message.Interop.IBaseMessage inmsg)
  {
   MemoryStream  memStream;
   Stream originalStrm;
   originalStrm = null;
   memStream = new MemoryStream();
   // System.Diagnostics.Debugger.Launch();


   IBaseMessagePart bodyPart = inmsg.BodyPart;


   if (bodyPart!=null)
   {
    originalStrm = bodyPart.GetOriginalDataStream();


    if (originalStrm != null)
    {
     originalStrm.Seek(0,System.IO.SeekOrigin.Begin);
     // Create a new XML doc
     XmlDocument xDoc = new XmlDocument();
     xDoc.Load(originalStrm);


     // Now read the Context property
     IBaseMessageContext context = inmsg.Context;
     object obj = context.Read( ContextPropertyName , ContextPropertyNameSpace);
     if (null != obj)
     {
      // do some work with each value that was promoted
      string ProcessingInstruction = (string)obj;
      // create a new processing instruction
      XmlProcessingInstruction piNew = xDoc.CreateProcessingInstruction(ContextPropertyName, ProcessingInstruction ); 


      // Insert before the first node of the root
      xDoc.DocumentElement.InsertBefore(piNew,xDoc.DocumentElement.FirstChild  );
     }
     //Write XML to a memory stream
     xDoc.Save(memStream);
     memStream.Seek(0,SeekOrigin.Begin);


     //Change the data
     bodyPart.Data = memStream;


     //The memory stream object must persist after you’ve
     //left the function
     pc.ResourceTracker.AddResource( memStream );
    }
   }
   return inmsg;    
  }


And here is the code that writes a processing instruction to a context property. After adding a PI to the context we have to make sure we remove it from the original underlying XML otherwise if a message would be send again using these pipelines we would double the processing instructions.



  public Microsoft.BizTalk.Message.Interop.IBaseMessage Execute(Microsoft.BizTalk.Component.Interop.IPipelineContext pc, Microsoft.BizTalk.Message.Interop.IBaseMessage inmsg)
  {
   MemoryStream  memStream;
   Stream originalStrm;
   originalStrm = null;
   memStream = new MemoryStream();
   string PropertyValue = string.Empty ;
    


   IBaseMessagePart bodyPart = inmsg.BodyPart;


   if (bodyPart!=null)
   {
    originalStrm = bodyPart.GetOriginalDataStream();


    if (originalStrm != null)
    {
     originalStrm.Seek(0,System.IO.SeekOrigin.Begin);
     // Create a new XML doc
     XmlDocument xDoc = new XmlDocument();
     xDoc.Load(originalStrm);


     // Now get the desired processing instruction
     string PropertyName = “”” + ContextPropertyName  + “””;
     string SelectString = “//processing-instruction(” + PropertyName + “)”;
     XmlNode contextpropertyNode = xDoc.SelectSingleNode(SelectString);
     if (null != contextpropertyNode)
     {
      PropertyValue = (string)contextpropertyNode.Value;
      // Now remove the processing instruction
      xDoc.DocumentElement.RemoveChild(contextpropertyNode);
     }
     
     //Write XML to a memory stream
     xDoc.Save(memStream);
     memStream.Seek(0,SeekOrigin.Begin);


     //Change the data
     bodyPart.Data = memStream;


     //The memory stream object must persist after you’ve
     //left the function
     pc.ResourceTracker.AddResource( memStream );
    }
   }
   if (PropertyValue != string.Empty)
   {
    inmsg.Context.Promote(ContextPropertyName , ContextPropertyNameSpace , PropertyValue );
   }
   return inmsg;    
  }


Below are examples of how to configure them. In our solution we have several ESB properties so you just have to add multiple pipeline components. Below is the receive pipeline. This is PartyResolution pipeline component, so we already know it’s a valid known schema to BizTalk.


And inject context properties as processing instructions in a XML document. Again we have multiple Context properties so we have to add multiple pipeline components.



It’s all quite basic and probably using a XMLdocument isn’t the fastest solution. However, it still is very fast and works very well. Below is a sample xml that is written to a Listen directory.



The fun part is, with the Receive pipeline you can drop files with processing instructions and have the processing instructions written to the context of the received message so they can continue their execution inside the ESB.


There are probably more ways of doing this but for now this seems a nice solution that I wanted to share on my blog.

  • Recent Posts
  • Recent Comments
  • Archives
  • Categories
  • Meta