BizTalk Services and partitioned queues….

….don’t mix at the moment.

If you’re deploying a BizTalk Services project with a queue source, and are getting the error

Failed to connect to the ServiceBus using specified configuration

and/or if you have a queue destination and are getting an error that contains

This client is not supported for a partitioned entity

check whether you’re using queues that have been created with the default option to enable partitioning –

image

Advertisements

Reflections on BizTalk Services

I had the pleasure of spending some more time this week with some integration legends – Richard Seroter, Michael Stephenson, Steef-Jan Wiggers, Sam Vanhoutte and Saravana Kumar amongst others.

When my dear wife asked me, as I set off for our dinner, whether we’d spend all evening talking about BizTalk I was adamant we would not. I was right of course – we only talked about integration for around 70% of the time!

Anyhow – discussion flowed on many topics and there’s much to reflect on, the first thing I want to go back to is BizTalk Services

These days I’m asked at least once a week, if not twice,  to position BizTalk vs. Service Bus vs BizTalk Services. This very much reminds me the early days of Windows Server AppFabric. There’s a lot of confusion out there as to how these all fit together and what’s the direction. or – in short – is BizTalk dead? 🙂

I always thought that the story from Microsoft makes reasonable sense, but that it was a difficult one to communicate; Following our dinner, and Sam’s talk at the event the following day I think I’ve managed to distil this a little bit –

Thinking about BizTalk Services capabilities (ignoring tooling for the moment) I think there are three big ticket items missing; The first two are –

1. Better routing – ‘first match’ routing is too basic. A more comprehensive routing solution is needed which allows distributing messages to n endpoints, Ideally allowing adding/removing them outside of code. sound familiar?

2. Persistence. BizTalk Services is a light weight solution. it takes a message, performs some mediation and delivers it to a destination. It does not provide any persistence and so it does not provide any ability to handle any exceptions – there are not retries, no suspended queue, no backup transport, etc.

Thinking about these two it is easy to be tempted by concluding that both could be mitigated by finally adding integration with the Azure Service Bus on the receive side of BizTalk Services. the Service Bus obviously has persistence and a pretty good routing capability using Topics and Subscriptions. Is that the future answer?

Not quite – the service bus is passive and with a very thin administration layer. to handle the configuration of the routing and certainly to handle the end-to-end scenarios with retries etc one will have to build a solution on top of these two services. Not to mention technical log/monitoring and business activity monitoring if these were deemed important.

Which leads me to number 3 – error handling, monitoring and administration

Hold on – we’re building BizTalk all over again, aren’t we?

I truly don’t think that this is Microsoft’s intention. I equally don’t think that BizTalk Services is a bad direction, I just think they serve wholly different purposes. at least for the foreseeable future - 

BizTalk Server is a full blown integration-platform/Middleware / SOA / ESA / all that Jazz.     ‘nough said.

To me – BizTalk Services is emerging as a better way to connect an application with the outside world. But it’s probably still best viewed as a part of a specific solution rather than a generic messaging platform.

If you like – it’s a better way to do point to point integration.

Viewed this way it is the overall solution that takes responsibility for the end-to-end message delivery, including monitoring and error handling. BizTalk Services merely provides a ‘bridge’ (that term finally makes some sense!) between the application to the outside world, exposing easier ways(?) to deliver the mediation – enrichment/fransformation/adaptation.

Sure – there’s still some way to go for BizTalk Services, particularly around the tooling, and certainly there’s many more features that would be very nice to have, but when I think about it this way I am less annoyed that there’s no built in integration with the service bus or that there’s not much you can do by way of routing, etc.

If I’m deploying a web application to the cloud and need a light weight integration with a known third party, or indeed on-premises system, I might consider BizTalk Services for the job. if I don’t need, or can’t afford, a full blown integration platform, that is.

And of course – if BizTalk Services is an extension to a solution, providing mediation through the cloud, the solution itself could be BizTalk Server, either on-premises or in the cloud, too. At Solidsoft we see quite a few opportunities these days of using Windows Azure Service Bus in conjunction with BizTalk. BizTalk Services fits the very same pattern building on BizTalk’s persistence, configuration, error management, logging, etc. 

Final note – in many respects this is in par with my thinking of light-weight ESBs such as nServiceBus and the likes – they are very good frameworks/platforms (well – some are!) but they are light weight, and are typically suited as a component in a particular solution which will wrap them nicely.

Integrating BizTalk and BizTalk [Services]

Over the last few months I was constantly asked how does BizTalk Services (formerly known as Windows Azure Integration Services) compares to BizTalk and whether one can use one instead of the other…

Whilst, generally speaking, I like the fact that this had been renamed to BizTalk Services (even if it means that searching information online about it will be somewhat difficult, due to the raft of publications on BizTalk out there), I suspect that this will only add to the confusion.

In any case – my answer has been, and will be for the foreseeable future – BizTalk Services, for the most part, is a great complimentary component to BizTalk. Sure – in some cases it can be a component in a solution that does not involve BizTalk, but it is very far from doing everything that BizTalk does, and – as far as I know – there are no plans to make it so.

So – if the two are so complimentary – how does one integrate BizTalk and BizTalk services?

Going in one direction – BizTalk Services to BizTalk Server – is very easy – one can publish a web service on BizTalk and consume that from BizTalk Services, maybe leveraging the Relay Service, part of the Windows Azure Service Bus to traverse the firewall. In some case one might want to publish the message from BizTalk Services to a Queue or a Topic on the Windows Azure Service Bus and read that from BizTalk using the built in capability. There are plenty of options to choose from.

Going in the other direction, though, is less flexible. At the time of writing BizTalk Services only support 3 way to deliver messages to it – HTTP, FTP (ok – including FTP/S, so maybe 4) and SFTP. To my surprise there is no built in capability to reading messages from the Windows Azure Service Bus or the ability to publish a web service.

Considering all these potions, I suspected that the HTTP route is the best one for integrating with BizTalk Server, so I decided to give it a go. here’s how it went –

The first step was to use the MessageSender sample to confirm that my bridge is working correctly (baby steps are important!)

Next, I wanted to make sure BizTalk is working (and I still remember how to configure it!) so I created a simple, messaging only, scenario – I started with a simple File Receive Location to File Send Port.

Procrastination over it was time to get serious, so I went on to add a second send port, this time using the WCF-WebHttp adapter configured to my bridge’s address, which is easily found in it’s properties –

image

It is important to use HTTPs and not HTTP, I found (trying to be lazy). Using the latter, Iwill result in the following error –

<Error><Code>400</Code><Detail>Transport security is required to protect the security token.</Detail></Error>

BizTalk Services protects me from myself and prevents me from exchanging security tokens without ssl. good or bad – you decide…

 

At this point my port configuration looked like this –

image

– and I figured it’s time to give it a go. As I haven’t integrated ACS yet I don’t expect to get through to the bridge but it’s a useful experiment on its own right – firstly to see that indeed authentication is required (no real doubt there) but also to see the end-to-end behaviour

Sure enough dropping a message for the File receive adapter I quickly receive a suspended instance, with the following error –

image

Which, unsurprisingly, looks very much like what I get in a browser –

<Error>

<Code>401</Code>

<Detail>Manage claim is required for this operation..TrackingId:b44aa846-7105-4869-beb5-d7830eaeb2df_G0,TimeStamp:6/12/2013 9:02:58 PM</Detail>

</Error>

Good – so I’m failing consistently, that’s progress 🙂 now all I needed to do is add the required bits to obtain the ACS token and add it to the outgoing request…..easy 😦

To do this I needed a solution that will allow me to obtain the ACS’ token for this request and inject it as an HTTP header to the outgoing request.

At this point I was sure I’m going to need to implement a custom WCF behaviour that will get the ACS’ token for the request and manipulate the outgoing message’s http headers to include the token but then it hit me – BizTalk already knows how to do that, because it has built in support for Windows Azure Service bus integration, which uses exactly the same mechanism.

A quick look at the WCF-WebHttp adapter configuration revealed a little tick box and some properties behind it –

image image

Awesome – a big chunk of work I expected to have to do – create and configure a custom behaviour to call ACS and obtain a token and add it to every outgoing message – was all done for me!

At this point I had to try again, and it looked promising, alas I did get another error – this time about not being able to establish trust relationship for the SSL/TSL secure channel.

This makes sense, it is exactly the issue I had with deployment of my BizTalk Service project initially and of course it is because I’m using a self-signed certificate so I promptly added the certificate to the Trusted Root Certificate Authorities store on my BizTalk server and tried again.

Well – nearly – it looks like I did get to BizTalk services, as I’m getting back a tracking id, although I didn’t quite manage to see anything in the BizTalk Services tracking page. Unfortunately – the tracking id came with an error message –

The incoming request is not recognised as a namespace policy put request: Unsupported or missing Content-Type

Ah! –BizTalk to the rescue again – at this point I was happy with hardcoding everything, I just wanted to get this to work! 🙂 and it turns out the WCF-WebHttp Binding lets you set static Http headers you wish to add to the outgoing message –

image

That done and I’m ready for another attempt – message into BizTalk and……..nothing. No suspended instance….did it actually work? a quick check in the BizTalk Services Tracking reveals that it has! BizTalk had really made this easier than I expected and now I can integrate my on-premises BizTalk environment with BizTalk Services and through that…..the world! 🙂

Check what happens on failure on the BizTalk services side – do we still get an error on the BizTalk side? (so that we don’t lose the message)

 

Cross posted on the Solidsoft Blog

BizTalk Services – Xml Schema Editor

Here’s another short blurt on BizTalk Services – this time the Xml Schema Editor.

The team had clearly, and thankfully, ‘borrowed’ the Schema Editor that we all know and love from BizTalk Server.

The editor allows creating and modifying Xml Schemas without needing to worry about the exact syntax of the underlying XSD making the process much less painful and more productive. After all – selecting from menus and setting properties is something we’re all used to and typing lots of delicate xml isn’t

image

In addition to the editor itself, right clicking on an XSD file in the solution explorer reveals three options that exist on BizTalk as well-

image

Validate Schema – checks that the schema itself is valid
Validate Instance – checks whether a given xml file is valid according to the schema and
Generate Instance – generates an xml file based on the schema

In all my years of BizTalking I hardly ever used the first option. Not so much because I am an XSD guru (I was! :-)) but because the editor does a very good job saving myself from myself.

The other two options are very useful indeed though, whether during the development process itself or during debugging and troubleshooting.

Sadly – at this point in the preview they may give the misleading impression that they do not work (Richard – did you manage to fall into that trap as I did initially?) – BizTalk used to pop out the output windows whenever executing either operation, where information about the files used (with links!) and the outcome was registered. BizTalk services does write to the output window, but without popping it open the unsuspecting user  (that’s me!) is oblivious to the fact and thinks it did nothing!

It’s amazing what difference a small UI behaviour can make..

 

Note: as is the case with BizTalk the input file for validation and the location of the output file for instance generations are set as properties of the XSD file in the solution explorer:

image

Below are the screen shots showing BizTalk services does indeed report to the output window, I just wish it became visible at that point:

image

image

image

image

 

Cross posted on the Solidsoft Blog

A hidden gem in BizTalk Services?

A fellow Solidsofter– Ian McQueen had pointed out a real (hidden?) gem with BizTalk services I hadn’t heard about.

At the off-chance it isn’t just me, here it is –

Did you know that you get a BizTalk Standard license to use on-premises with BizTalk Services Premium edition?

We spotted this here

image

At this point I could not find any more details on this, but it is very interesting indeed as it opens the capability for more advanced integration to on-premises systems in conjunction with the ease and cost-effectiveness of BizTalk Services

FTP and BizTalk Services

Continuing with my exploration of BizTalk services I thought I’d start with a simple end-to-end scenario, although it is something that roughly maps to a real customer requirement we have right now –

I wanted a bridge to pick up a file from an FTP location and drop it onto a Service Bus Queue (from which BizTalk 2013 would pick it up, but that’s another story…)

 

I create a simple pass-through bridge configuration to begin with, with my topic details on the one side and the ftp details on the other –

image

For the FTP server I deployed a small IaaS instance on Azure and configured an FTP server on it, I then entered all the details for the topic and configured the FTP endpoint.

That done I deployed the solution only to see an error stating –

Failed to connect to the FTP server using specified configuration. Error message – ‘The remote certificate is invalid according to the validation procedure.’

It was nice to see such a detailed error an indeed I quickly realised that the Use SSL property on the FTP source is on by default, but I had not configured SSL for my FTP site so I promptly changed that to False and re-deployed.

Unfortunately, this one didn’t work either, and now the error wasn’t that useful – “The underlying connection was closed: An unexpected error occurred on a receive.” – but it was useful to be able to get more details on the BizTalk Services (silverlight based) portal, accessible through the Windows Azure Portal –

image

In the BizTalk Services Portal there’s a tracking tab, and in that I found the details of this particular issue –

image

Failed to connect to the FTP server using specified configuration. Error message – ‘The remote server returned an error: 227 Entering Passive Mode (100,84,86,13,192,39)..’

As far as I understand this is an issue with my FTP setup more than anything else – it is  fair to expect to need to use passive mode for FTP (BizTalk services will block incoming FTP connections required for active mode) and my Azure based FTP server configuration will struggle accepting incoming connection on semi-random ports, which I believe is what’s happening here (and if you want to read more, I found this very useful)

I’ve decided to ignore this self-introduced issue and use another public FTP server I have access to, but it was good to see the level of detailed errors one can get from the BizTalk portal with ease.

With this ‘proper’ FTP server and after a bit more fiddling with settings and re-deploying, it worked and I could see my ‘messages’ being picked up from my FTP location and placed on the Service Bus Queue

 

Cross posted on the Solidsoft blog

Deploying to BizTalk Services

The busy life at SolidSoft got in the way, but I finally got some hands-on time with the recently re-released, in preview, Windows Azure BizTalk Services.

It didn’t take long before I hit the first hurdle – it took a bit of time to get my BizTalk services account set-up, but lots have been written about this already, so I won’t repeat that, but beyond that – I created my first test project and wanted to deploy it, which didn’t work first time.

What was easily missed is that the deployment from Visual Studio to your BizTalk account uses SSL, established using the certificate you upload during the account creation. This certificate needs to be installed in your certificate store but if, like me and most others, you use a self-signed certificate, you must ensure you also install it in the “Trusted Root Certificate Authorities” store.

Without that the deployment from Visual Studio will fail with the error

The underlying connection was closed: Could not establish trust relationship for the SSL/TLS secure channel.

Cross posted on the Solidsoft blog

%d bloggers like this: