Caution required with BizTalk’s SB-Messaging adapter

BizTalk 2013 ships with a native adapter for the Windows Azure Service Bus – the SB-Messaging adapter.

The adapter provides a very easy to use approach to receiving and sending messages to and from a service bus queue/topic/subscription but, as it turns out, it might be that it is a little bit simplistic or – at the very least – some caution is required as I’ve learnt with a customer yesterday –

The customer uses BizTalk to read a message from a Service Bus queue using the adapter. In the pipeline they ‘debatch’ the incoming message into multiple messages that get published to BizTalk’s message box.

This all worked just fine during testing, but when the customer moved to performance testing they hit a point after which they got a lot of errors such as the follows –

The adapter "SB-Messaging" raised an error message. Details "System.ObjectDisposedException: Cannot access a disposed object.

Object name: ‘Microsoft.ServiceBus.Messaging.Sbmp.RedirectBindingElement+RedirectContainerChannelFactory`1[System.ServiceModel.Channels.IRequestSessionChannel]’.

The adapter "SB-Messaging" raised an error message. Details "Microsoft.ServiceBus.Messaging.MessageLockLostException: The lock supplied is invalid. Either the lock expired, or the message has already been removed from the queue.

When reading from the Service Bus the client can employ one of two strategies – a desructive read whereby the message is remove from there queue as soon as it is being read or a ‘peek lock’ strategy where the read message is locked and effectively hidden on the queue for a duration, allowing the client to process the message and then come back and either confirm that it had successfully processed the message or abandon the message, effectively  putting it back on the queue.

The peek-lock strategy is very useful to ensure no message loss and BizTalk’s SB-Messaging adapter makes a good use of it – until the received message is persisted in the message box it is not removed from the queue and hence zero-message loss can be guaranteed.

On the flip side, it means that a message may be delivered more than once and this has to be catered for.

In this customer’s case, the debatching they performed meant that, under load, it took BIzTalk too much time to complete the debatching and persisting all the messages to the message box. That meant that it did not go back to the queue in time to release the lock resulting with the error above. unfortunately this also caused a snow-ball effect as it meant that another BizTalk instance picked up the same message and the issue repeated itself, with the large number of messages being persisted to the message box increasing the load on the server and with it the duration of the receive pipeline processing….

Once identified they have found a quick workaround, at least in the first instance – remove the debatching from the service bus receive port and move it to a later stage in their BizTalk processing. This, I think, is a good appraoch, as it lets them ‘take ownership’ of the message in BizTalk, removing it from the Service Bus, before starting the heavy processing.

Another approach is to change the lock duration property on the queue/subscription in question. the default is 60 seconds. This could by time and with careful load-testing a reasonable value may be found, but – of course – it does not eliminate the risk, just pushes it out further.

Asking in the virtual corridors of Solidsoft it appears we have seen this issue with at least  two other customers. I think that the SB-Messaging adapter should support both strategies and allow the customer the choice, and perhaps flagging the need to set the lock duration value on the queue/subscription as required.

Advertisements

Reflections on BizTalk Services

I had the pleasure of spending some more time this week with some integration legends – Richard Seroter, Michael Stephenson, Steef-Jan Wiggers, Sam Vanhoutte and Saravana Kumar amongst others.

When my dear wife asked me, as I set off for our dinner, whether we’d spend all evening talking about BizTalk I was adamant we would not. I was right of course – we only talked about integration for around 70% of the time!

Anyhow – discussion flowed on many topics and there’s much to reflect on, the first thing I want to go back to is BizTalk Services

These days I’m asked at least once a week, if not twice,  to position BizTalk vs. Service Bus vs BizTalk Services. This very much reminds me the early days of Windows Server AppFabric. There’s a lot of confusion out there as to how these all fit together and what’s the direction. or – in short – is BizTalk dead? 🙂

I always thought that the story from Microsoft makes reasonable sense, but that it was a difficult one to communicate; Following our dinner, and Sam’s talk at the event the following day I think I’ve managed to distil this a little bit –

Thinking about BizTalk Services capabilities (ignoring tooling for the moment) I think there are three big ticket items missing; The first two are –

1. Better routing – ‘first match’ routing is too basic. A more comprehensive routing solution is needed which allows distributing messages to n endpoints, Ideally allowing adding/removing them outside of code. sound familiar?

2. Persistence. BizTalk Services is a light weight solution. it takes a message, performs some mediation and delivers it to a destination. It does not provide any persistence and so it does not provide any ability to handle any exceptions – there are not retries, no suspended queue, no backup transport, etc.

Thinking about these two it is easy to be tempted by concluding that both could be mitigated by finally adding integration with the Azure Service Bus on the receive side of BizTalk Services. the Service Bus obviously has persistence and a pretty good routing capability using Topics and Subscriptions. Is that the future answer?

Not quite – the service bus is passive and with a very thin administration layer. to handle the configuration of the routing and certainly to handle the end-to-end scenarios with retries etc one will have to build a solution on top of these two services. Not to mention technical log/monitoring and business activity monitoring if these were deemed important.

Which leads me to number 3 – error handling, monitoring and administration

Hold on – we’re building BizTalk all over again, aren’t we?

I truly don’t think that this is Microsoft’s intention. I equally don’t think that BizTalk Services is a bad direction, I just think they serve wholly different purposes. at least for the foreseeable future - 

BizTalk Server is a full blown integration-platform/Middleware / SOA / ESA / all that Jazz.     ‘nough said.

To me – BizTalk Services is emerging as a better way to connect an application with the outside world. But it’s probably still best viewed as a part of a specific solution rather than a generic messaging platform.

If you like – it’s a better way to do point to point integration.

Viewed this way it is the overall solution that takes responsibility for the end-to-end message delivery, including monitoring and error handling. BizTalk Services merely provides a ‘bridge’ (that term finally makes some sense!) between the application to the outside world, exposing easier ways(?) to deliver the mediation – enrichment/fransformation/adaptation.

Sure – there’s still some way to go for BizTalk Services, particularly around the tooling, and certainly there’s many more features that would be very nice to have, but when I think about it this way I am less annoyed that there’s no built in integration with the service bus or that there’s not much you can do by way of routing, etc.

If I’m deploying a web application to the cloud and need a light weight integration with a known third party, or indeed on-premises system, I might consider BizTalk Services for the job. if I don’t need, or can’t afford, a full blown integration platform, that is.

And of course – if BizTalk Services is an extension to a solution, providing mediation through the cloud, the solution itself could be BizTalk Server, either on-premises or in the cloud, too. At Solidsoft we see quite a few opportunities these days of using Windows Azure Service Bus in conjunction with BizTalk. BizTalk Services fits the very same pattern building on BizTalk’s persistence, configuration, error management, logging, etc. 

Final note – in many respects this is in par with my thinking of light-weight ESBs such as nServiceBus and the likes – they are very good frameworks/platforms (well – some are!) but they are light weight, and are typically suited as a component in a particular solution which will wrap them nicely.

Integrating BizTalk and BizTalk [Services]

Over the last few months I was constantly asked how does BizTalk Services (formerly known as Windows Azure Integration Services) compares to BizTalk and whether one can use one instead of the other…

Whilst, generally speaking, I like the fact that this had been renamed to BizTalk Services (even if it means that searching information online about it will be somewhat difficult, due to the raft of publications on BizTalk out there), I suspect that this will only add to the confusion.

In any case – my answer has been, and will be for the foreseeable future – BizTalk Services, for the most part, is a great complimentary component to BizTalk. Sure – in some cases it can be a component in a solution that does not involve BizTalk, but it is very far from doing everything that BizTalk does, and – as far as I know – there are no plans to make it so.

So – if the two are so complimentary – how does one integrate BizTalk and BizTalk services?

Going in one direction – BizTalk Services to BizTalk Server – is very easy – one can publish a web service on BizTalk and consume that from BizTalk Services, maybe leveraging the Relay Service, part of the Windows Azure Service Bus to traverse the firewall. In some case one might want to publish the message from BizTalk Services to a Queue or a Topic on the Windows Azure Service Bus and read that from BizTalk using the built in capability. There are plenty of options to choose from.

Going in the other direction, though, is less flexible. At the time of writing BizTalk Services only support 3 way to deliver messages to it – HTTP, FTP (ok – including FTP/S, so maybe 4) and SFTP. To my surprise there is no built in capability to reading messages from the Windows Azure Service Bus or the ability to publish a web service.

Considering all these potions, I suspected that the HTTP route is the best one for integrating with BizTalk Server, so I decided to give it a go. here’s how it went –

The first step was to use the MessageSender sample to confirm that my bridge is working correctly (baby steps are important!)

Next, I wanted to make sure BizTalk is working (and I still remember how to configure it!) so I created a simple, messaging only, scenario – I started with a simple File Receive Location to File Send Port.

Procrastination over it was time to get serious, so I went on to add a second send port, this time using the WCF-WebHttp adapter configured to my bridge’s address, which is easily found in it’s properties –

image

It is important to use HTTPs and not HTTP, I found (trying to be lazy). Using the latter, Iwill result in the following error –

<Error><Code>400</Code><Detail>Transport security is required to protect the security token.</Detail></Error>

BizTalk Services protects me from myself and prevents me from exchanging security tokens without ssl. good or bad – you decide…

 

At this point my port configuration looked like this –

image

– and I figured it’s time to give it a go. As I haven’t integrated ACS yet I don’t expect to get through to the bridge but it’s a useful experiment on its own right – firstly to see that indeed authentication is required (no real doubt there) but also to see the end-to-end behaviour

Sure enough dropping a message for the File receive adapter I quickly receive a suspended instance, with the following error –

image

Which, unsurprisingly, looks very much like what I get in a browser –

<Error>

<Code>401</Code>

<Detail>Manage claim is required for this operation..TrackingId:b44aa846-7105-4869-beb5-d7830eaeb2df_G0,TimeStamp:6/12/2013 9:02:58 PM</Detail>

</Error>

Good – so I’m failing consistently, that’s progress 🙂 now all I needed to do is add the required bits to obtain the ACS token and add it to the outgoing request…..easy 😦

To do this I needed a solution that will allow me to obtain the ACS’ token for this request and inject it as an HTTP header to the outgoing request.

At this point I was sure I’m going to need to implement a custom WCF behaviour that will get the ACS’ token for the request and manipulate the outgoing message’s http headers to include the token but then it hit me – BizTalk already knows how to do that, because it has built in support for Windows Azure Service bus integration, which uses exactly the same mechanism.

A quick look at the WCF-WebHttp adapter configuration revealed a little tick box and some properties behind it –

image image

Awesome – a big chunk of work I expected to have to do – create and configure a custom behaviour to call ACS and obtain a token and add it to every outgoing message – was all done for me!

At this point I had to try again, and it looked promising, alas I did get another error – this time about not being able to establish trust relationship for the SSL/TSL secure channel.

This makes sense, it is exactly the issue I had with deployment of my BizTalk Service project initially and of course it is because I’m using a self-signed certificate so I promptly added the certificate to the Trusted Root Certificate Authorities store on my BizTalk server and tried again.

Well – nearly – it looks like I did get to BizTalk services, as I’m getting back a tracking id, although I didn’t quite manage to see anything in the BizTalk Services tracking page. Unfortunately – the tracking id came with an error message –

The incoming request is not recognised as a namespace policy put request: Unsupported or missing Content-Type

Ah! –BizTalk to the rescue again – at this point I was happy with hardcoding everything, I just wanted to get this to work! 🙂 and it turns out the WCF-WebHttp Binding lets you set static Http headers you wish to add to the outgoing message –

image

That done and I’m ready for another attempt – message into BizTalk and……..nothing. No suspended instance….did it actually work? a quick check in the BizTalk Services Tracking reveals that it has! BizTalk had really made this easier than I expected and now I can integrate my on-premises BizTalk environment with BizTalk Services and through that…..the world! 🙂

Check what happens on failure on the BizTalk services side – do we still get an error on the BizTalk side? (so that we don’t lose the message)

 

Cross posted on the Solidsoft Blog

A hidden gem in BizTalk Services?

A fellow Solidsofter– Ian McQueen had pointed out a real (hidden?) gem with BizTalk services I hadn’t heard about.

At the off-chance it isn’t just me, here it is –

Did you know that you get a BizTalk Standard license to use on-premises with BizTalk Services Premium edition?

We spotted this here

image

At this point I could not find any more details on this, but it is very interesting indeed as it opens the capability for more advanced integration to on-premises systems in conjunction with the ease and cost-effectiveness of BizTalk Services

%d bloggers like this: