Hello world!

Welcome to WordPress.com. This is your first post. Edit or delete it and start blogging!

Posted in Uncategorized | 3 Comments

Come and meet me at Expert Days 2010!

On the week of the 22nd of November you are welcome to come and see me at the Israeli Expert days 2009 (http://www.expertdays.co.il/ ) in Ness College in Tel Aviv.

I will be giving two one-day workshops:

Everything you ever wanted to know about MSMQ 4.0

Windows Azure – Microsoft Cloud

The workshops will be given in Hebrew, but I would switch to English if any student will ask for it.

I can also give the same workshops on demand in your place.

See you there!

Yoel

Posted in Uncategorized | Leave a comment

PDC – Day 3 – Boku, .NET services and more on Azure queues

Day 3 keynotes were about Microsoft research, presented by Rick Rashid. Rick’s presentation was probably the most impressing presentation in PDC – even I did not know how large the Microsoft research team is (850 researchers – larger than most universities), and the breath of projects they are involved in – from software proofing, to weather control, health research (yes!) and surface programming. However to me as a father (and as a former LOGO instructor) the most impressing project presented is Boku – a programming environment for children. If you have children (and / or you are still, like me, a little bit of a child yourself) go see it.

After Boku, returning to the technical stuff was a bit hard…

I went to see Clemens Vasters‘ presentation on .NET services. Clemens showed how you can connect two applications through the cloud – even if they run on public Internet connections, corporate environments, NATS etc. To enable that, Microsoft is delivering a whole bunch of new bindings for WCF – the "relay" bindings – each corresponding to an existing "normal" binding (for example, BasicHttpRelayBinding corresponds to BasicHttpBinding. There is no MsmqRelayBinding yet – it may be a good exercise to develop one… According to Clemens, one of the reasons that Microsoft did not come up with such binding is the difficulty of keeping end-to-end transactional messages protocol through the cloud. However, I think it is a good idea anyway to avoid transactional messages.

Later in the day, I went to see Sriram Krishnan’s talk on best practices of programming on the Cloud. Sriram is an excellent presenter and I highly recommend that you go see his talk (should be available online on the PDC site in a few days) – especially if you are distributed applications architects  – not only those developing for the cloud. Sriram preaches for stateless, loosely coupled and simple solutions (did I tell you I hate transactions?). He strengthened my conclusion that cloud programming to the enterprise programmers is what VB.NET is to VB6 programmers – it looks the same, but it is actually quite different – and it will take time to adjust and think "the cloud way". You can see more in his site.

I learned a little more today about Azure queues. The interface to these queues is HTTP REST interface (which implies that duplications are part of life and there are no real transactions – but you know that already). Here are the APIs:

Queues:

Similar to get properties in MSMQ, except it also returns the message count (do I hear you clap?) and the other properties are free form name / value pairs.

Messages:

In the URI, you can also ask for the number of messages you want, and also specify a timeout. If you do not delete the message within the timeout, it will be returned to the queue.

Same as GET, but the message remains visible in the queue (you knew that already, didn’t you?)

"lightweight commit" You have to call that after GET – otherwise the message will be put back in the queue

Deletes all the messages in the queue (purge).

The Azure samples (provided with the SDK) wrap the REST methods with a .NET class, RestQueue, which itself is wrapped with a class called… Ahm… MessageQueue! Isn’t that fun? You already know that when programming with System.Messaging on a widows form you have to explicitly put the namespace for Message, now you also have MessageQueue in a different namespace. Should be fun.

That’s all folks… four queue methods, five message methods. Is that all we need? I guess that in most cases, the answer is surprisingly YES. I would love to hear your views.

Technorati Tags:
Posted in Uncategorized | Leave a comment

PDC – Day 2 – some more on Azure queues

Day 2’s keynotes where about Windows 7 – which apparently not going to include lots of bells and whistles (unless you are totally excited of the ability to re-order icons on the task bar), but hopefully will be faster and more stable, so people will migrate to it from Windows XP (and enjoy MSMQ 4.0 features, of course :-)).

I attended Brad Calder’s "Essential Azure storage services" and learned some more on Azure’s queues:

In general, queue is one of the three "native" storage object types in Azure (if you are not using SQL services) alongside with containers (containing blobs) and tables ("poor men’s database").

An Azure’s queue supports the following methods (exact names and parameters TBD):

  • Create
  • Clear
  • Delete
  • Get length (isn’t it nice to have this as a first-class method? :-))
  • Enqueue (Message)
  • Dequeue(timeout) –> returns a message. After the timeout, the message is returned to the queue unless explicitly deleted.
  • Delete(message)

A message can be up to 8K in size (that said, the real data will most likely be elsewhere – in a container, table, or database.

I kind of like the Enqueue / Dequeue / Delete paradigm. As devoted readers of this blog know, I always wanted some way to do "lightweight" transactional receive from a non transactional queue, and Azure provides just that (BTW, there is also a way to achieve that in MSMQ 4.0 using sub-queues – I may write a blog about that sometime). The queuing is far from being as feature-rich as MSMQ is, but I can definitely see scenarios when it will be the mechanism of choice. It worth following it up.

That is for today… Universal Studios visit was fun. Looking forward for tomorrow.

Technorati Tags:
Posted in Uncategorized | Leave a comment

PDC – Day 1 – the Azure day & yet another queuing

I guess that you all already read in the press or elsewhere about the new bomb Ray Ozzie had thrown to the keynotes hall in the PDC, declaring Microsoft’s new "operating system" – Microsoft Azure.

Azure is actually called "operating system" just because this is what Microsoft is used to call its products :-). In fact, this is a completely new beast – the move of Microsoft from selling the platform to owning & leasing it. At least at the first stage, Microsoft will not sell the new OS, but rather lease storage, bandwidth etc. on its own servers – what will put it as a competitor to companies like Rackspace, Godaddy, and, well, Google. In corridor talks I had, this was the most controversial aspect of the new move – Microsoft became such a great company because it had developed open systems (well, not open source :-)) and developed and echo-system – in contrast to the close approach of Apple computers. In the new move, Microsoft want to take the OS & tools alongside with the actual hosting & maintaining of the data – just as Google is doing with Google Apps. It is interesting to see if it will continue that way, or eventually sell the Azure as OS and bring along partners.

So far for business thoughts… On the technical side, Azure is a really exciting platform for easily hosting services, that brings "hosting" to a new level – from just storage to a complete set of enterprise-grade services.

In an "Azure Overview" talk I attended right after the keynotes, Munuvir Das describes that Azure to services is like OS to applications – it frees the developer for thinking of the ugly details of deployment and hosting. Indeed, the demonstration of a few clicks deployment of distributed application looks quite impressing. However, it is clear that Azure also requires the developer to learn a whole new set of tools and paradigms, just like VB programmers could not just "evolve" to VB.NET programmers…

Among other things, Azure presents yet another queuing (if you are not already confused enough between MSMQ and Message Broker). Azure queuing looks like the main paradigm Azure programmers will use, in order to allow the system to perform heavy background tasks and be a little more than ASP.NET++. One first observation from the code I saw (mainly in  Steve Marx "developing your first Azure application" presentation) is that Azure queuing had taken a "light transactional" approach – you can "GetMessage" – which give you the message exclusively – and when you are done, you should explicitly delete the message from the queue. If you fail to do so, the message will return to the queue after a timeout. There are not full transactions or distributed transactions. I kinda like this approach actually. I should devote some time to play with Azure queuing and come with some more solid conclusions…

This is all for now – I am almost late for 2nd day keynotes – see you soon!

Technorati Tags:
Posted in Computer and Internet | Leave a comment

PDC, Day 0 – pre-conference WCF session

Most people think PDC starts at Monday the 27th, however for me (and for ~1000 more people :-)) PDC started at Sunday with the pre-conference.

I went to hear Juval Lowy and Ron Jacobs speaking on WCF. Juval has a quite radical view on WCF – WCF is no less than the .NET killer and the new way to write every application in the world (well, maybe except high end graphics and device drivers). I must say his argument is quite convincing argument – with WCF you have logging, timeouts, performance counters & thread management out of the box, and this provides a step forward from .NET, exactly as .NET was a step forward from COM / C++.

Juval presented an interesting study – in the average enterprise solution, about 3% of the time is dedicated to solve the business solution, while 97% is dedicated to "plumbing" – messaging, connectivity, logging, security, etc. etc. This ration tends to worsen over time, as most of the maintenance time is also dedicated to plumbing problem.

I am not sure I totally agreed with the conclusion, that using WCF can make programmer 20 times more productive (I guess that even with WCF the time dedicated to plumbing does not go away so easily). However I definitely agree that doing "plumbing" right can be the difference between successful and failed project.

Ron presented REST services over WCF, which seems like a Microsoft attempt to make some peace with the web community and save some bandwidth in the way. Interestingly Juval presented WCF as something far larger than "web services platform" (as Microsoft’s "official" claim) while Ron showed the right way to work if you anyway want to use WCF for mere web services :-).

There were about 500 participants in the session (I did not know that so many people are willing to give up Sunday just to hear about WCF!). Interestingly, a vast majority of the audience was from overseas. I am sad to say that the ratio of males to females in the audience was worst than the ratio of plumbing to business code in Juval’s study (out of the 500, I think I spotted two or three women in the audience). Is infrastructure programming a male only profession? I find that hard to believe…

Takeaways:

  • Take your infrastructure (plumbing) code seriously
  • WCF is much more than just web services, but if you are building web services, look at WCF+REST
  • Look for building all your code using WCF (preferably with MSMQ binding, of course :-))

Overall a great session. I set for eight hours with no intermission except lunch and loved every minute.

Looking forward to the rest of the PDC….

Posted in Uncategorized | 2 Comments

Meet you in PDC 2008 + USA Visit + Is Send() thread safe?

It is hard to believe, but three years had passed since PDC 2005… The buzzwords then where WCF, WPF, LINQ & Vista. This time it is going to be cloud services, Oslo & Windows 7. Actually the "old" stuff that was presented in 2005 still looks a bit new for most developers (myself included  :-)). It would be interesting to see how fast all the new stuff will catch up.

I will come to the USA on the 22nd of October and leave at the 6th of November, and in-between I will be available for consulting (except, of course, the PDC itself – 27-30 of October). If you are interested in a one day (or more) of review of your MSMQ solution, or MSMQ training in your site – just drop me an email to yoel@msmq.biz . If you are coming to the PDC, I will be very happy to meet you and have a chat on MSMQ, distributed solution or any other subject – again, just drop me an email.

I wouldn’t like to end this post without giving you some technical value 🙂 . Well, recently two customers asked me why MessageQueue.Send is not thread safe. Indeed, MSDN documentation states (in "MessageQueue Class" documentation):

"Only the following methods are safe for multithreaded operations: BeginPeek, BeginReceive, EndPeek, EndReceive, GetAllMessages, Peek, and Receive."

Obviously, Send is not in the list… This looks a little weird, because MQSendMessage C++ API is definitely thread safe.

This question was raised in the MSMQ newsgroup about three years ago, and I have to admit I did not fully answer it… This time, I decided to do my own little research, and wrote the following program:

class Program
{
    static MessageQueue outQueue;
    static void Main(string[] args)
    {
        outQueue = new MessageQueue(@".\private$\mtQueue");

        for (int i = 0; i < 100; i++)
        {
            Thread thr = new Thread(new ThreadStart(MyThreadProc));
            thr.Start();
        }
    }

    static void MyThreadProc()
    {
        Message msg = new Message();
        for (int i = 0; i < 100; i++)
        {
            msg.Label = string.Format("{0} : {1}", Thread.CurrentThread.ManagedThreadId, i);
            outQueue.Send(msg);
        }
    }
}

(Note that .\private$\mtQueue creation code is not included – you can create it in advance using MMC).

As you can see, the program sends 10,000 messages using 100 thread. The program runs beautifully with no failure. Does it mean that MessageQueue.Send is always thread safe? Not necessarily…

Looking at MessageQueue.InternalSend code (using Lutz Roeder’s Reflector) reveals the following code:

private void SendInternal(object obj, MessageQueueTransaction internalTransaction,
                                        MessageQueueTransactionType transactionType)
{ <Some code deleted>
              Message cachedMessage = null;
              if (obj is Message) { cachedMessage = (Message) obj; }
              if (cachedMessage == null)
              {
                           cachedMessage = this.DefaultPropertiesToSend.CachedMessage;
                           cachedMessage.Formatter = this.Formatter;
                           cachedMessage.Body = obj;
              }
   <Code for sending cachedMessage using MQSendMessage>
}

Where is the problem, then? If you are using a Message object (as I did in my code) there is no problem – it is sent "as is" and as long as you do not use the same Message object in all your threads it would work.

The problem starts when you want to use the .NET "shortcut" for serializing and sending an arbitrary .NET object in MessageQueue.Send. In this case, the .NET code will use DefaultPropertiesToSend.CachedMessage – a member variable of MessageQueue, for doing the serialization. Since this member is the same in all threads, you will get an exception when using "Send" in multiple threads.

So, I changed MyThreadProc() in the original sample as follows:

    static void MyThreadProc()
    {
        for (int i = 0; i < 100; i++)
        {
             outQueue.Send("Hello, World!");
        }
    }

Bingo! I got an exception:

Unhandled Exception: System.InvalidCastException: Specified cast is not valid.
   at System.Messaging.Interop.MessagePropertyVariants.Lock()
   at System.Messaging.Message.Lock()
   at System.Messaging.MessageQueue.SendInternal(Object obj, MessageQueueTransaction internalTransaction, MessageQueueTransactionType transactionType)
   at System.Messaging.MessageQueue.Send(Object obj)
   at TestMultithreadSend.Program.MyThreadProc() in C:\Projects\TestMultithreadSend\TestMultithreadSend\Program.cs:line 32
   at System.Threading.ThreadHelper.ThreadStart_Context(Object state)
   at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, C
ontextCallback callback, Object state)
   at System.Threading.ThreadHelper.ThreadStart()

To sum up – Send is actually thread safe, as long as you always send a Message object and never use send a .NET object directly. Using the Message object, BTW, is always a good idea – since it lets you add label, timeouts, recoverable option and all this stuff that make your MSMQ solution a real enterprise solution.

Close to the end of my small research I found that this issue was actually addressed recently by Microsoft – you can find more about it here .

All the best – hope to see you in PDC,

Yoel

Posted in Uncategorized | 5 Comments

Expert Days 2008

Last Monday, I delivered an MSMQ training session in Expert Days 2008 (Warning- this link is in Hebrew :-)), organized by Expert4D. I had a small group of six devoted students from the Israeli Army, Orange telecom, and E&C Medical Intelligence (an interesting start-up developing solutions for Obstetrics), and another company I can’ remember right now (please respond to this point and remind me).

You can see the original syllabus of the training here.

How did it go? I started as planned, with a "getting started" session in which I showed how to transfer a simple "draw" program to bulleting board "disdraw" program. This is actually a variation of the first MSMQ demo I developed in 1995 for "Falcon" (later named MSMQ) technology preview. I really like this demo because it can clearly and visually show store-and-forward, addressing, multicast and more. You can see the source of the demo here.

To my surprise, I was actually able to develop the MSMQ part on stage and I did not need the backup solution I prepared. The gods of the demo were on my side this time! I wonder what are the odds of this to happen….

I continued with addressing, using the same program to demonstrate multiple readers from the same queue, multicast, MQF (multiple queue format) and HTTP addressing. Again I remembered why I like "disdraw" – when you put multiple readers on the same queue you see that each of them gets a part (dotted line…) of the complete draw, but when you use MQF to send to several queues and put a reader on each queue, each reader gets the full drawing.

image

The upper "disdraw" sends the drawing to "formatname:direct=os:.\private$\q1,direct=os:.\private$\q2,direct=os:.\private$\q3" (MQF). The readers in the middle reads the drawing from q1, the reader in the bottom left reads from q2, and in the bottom right – from q3. I hope you can see that the readers that read from q1 gets only part of the original drawing while the readers in the bottom get it all.

I tried to demonstrate multicast but the network I got seems to block it… Next time I should use virtual servers for that.

After lunch, I decided to use the second half of the day for "by request" sessions.

The people from E&C Medical asked to learn more on C++ APIs and I also got a request to learn how to use MSMQ from Javascript. I found a presentation delivered in 1998 by my boss – "Mr. MSMQ" Raphi Renous that covers both C++ and COM APIs. You can see it here . It looks like good old horses do not die that fast 🙂

Later, I used an excellent presentation that Shai Kariv delivered in DevCom 2002 (another proof that old horses do not die that fast) to demonstrate the MSMQ multicasting.

After talking about quality of service, recoverable messages and express messages (did you know that express messages are also saved to disk? I think I should devote a blog to that…), we talked about the MSMQ WMI provider (see my blog from April 2), and then we moved on to talk about new features – MSMQ 4.0 and WCF.

For WCF, I used two presentations I really like – one is the Teacher / Student presentation (WPF based "disdraw") by Pedram Rezaei . WPF makes it look much nicer than the original disdraw 🙂 and it also shows how you can easily switch between http and msmq bindings. The only downside of this presentation is that the teacher empties its queue as it starts, so you can’t really demo a student that starts before the teacher – a benefit of using MSMQ. I will fix this some day…

We ended the day by demonstrating sub-queues using Windows Vista’s Computer Management (actually we could overcome the message deletion in the student / teacher demo by "hiding" messages in sub-queue while the teacher canvas starts…), and then I showed the excellent presentation of Justin Wilcox on the MSMQ bindings in WCF .

I want to use this opportunity to thank Eyal Vardi and Sharon Ezra from Expert4D for organizing the Expert Days 2008 . Note that the Expert Days continue at the end of August – so you can still register for additional sessions (if you care in Israel and speak Hebrew).

Interested in attending similar sessions? Please let me know. I can also customize one for your company if needed.

Comments are welcome,

Yoel

Posted in Uncategorized | Leave a comment

MSMQ 5 wish list

It TechEd Eilat last week I had the pleasure of attending Ron Jacobs‘ presentation on REST web services in WCF. This is a subject that worth a blog of its own, as I feel that in supporting REST Microsoft actually admitted that not all bindings are alike in WCF, and individual protocols have their merits. There are obvious conclusions about MSMQ – I feel that NetMsmqBinding is hiding too much of the actual MSMQ features, and there is a need for MSMQ more"Native" protocol. Maybe MsmqIntegrationBinding can serve this purpose, with some changes…

Anyway I know Ron for several years as Microsoft’s "Mr. Architecture" – famous for his patterns and ArcCast. I was very glad to hear that he went out of his comfort zone and decided to join the WCF team as an evangelist.

In a corridor talk with Ron after his presentation, he asked me to compose a "wish list" for the MSMQ development team (which is part of the WCF team) towards the next version. I decided to make three lists – MSMQ core, System.Messaging and WCF integration. I tried to keep the lists short, and put them in priority order – as Ron rightfully warned me that it would be very difficult to get more than one or maximum two changes into the actual product.

You are, of course, welcome to comment and add your own wishes. I know most of you did not digest MSMQ 4.0 yet, but if we don’t act now, we won’t be able to make any impact  until MSMQ 6.0 ….

1. MSMQ core wishes

Transactional receive from non-transactional queues (or non transactional send to transactional queues)

Explanation: In MSMQ, "Transactional" messages are sent using a complex protocol that guarantees exactly-once and in-order delivery. According to my experience, most customers do not really need this protocol, and many are using it for the wrong reasons. Here is why:

      • Transactional messages are not "guaranteed delivery" because there is no such thing in a disconnected world. They just guarantee that messages are either delivered or end up in dead letter queue (or both, actually J). Most customers that use transactional messages, however, do not check the dead letter queue at all…
      • Most real world customers’ environment do not utilize MSMQ routing servers. Without routing servers, all the messages are sent using a direct TCP/IP session and out of order messages are very rare. In addition, MSMQ keeps a buffer of the last 65K messages received (a feature introduced in W2K) to avoid duplicate non-transactional messages.
      • The transactional order ack protocol may cause many problems in when the WAN is not highly reliable or introduce high latency. It may cause lots of unneeded re-sent, and sometimes cause the outgoing queue to get stuck for 6 hours (if lots of order acks were lost). In addition, it does not work well with of uni-directional firewall. Another source of problem here is the inseq files and the need to save messages in the logger – with performance cost and additional potential source of problems.

However, many customers really love transactional receive. Without transactional receive, all the open messages (which were read from a queue but not processed yet) would be lost in case of shutdown / power failure / cluster failover. This is especially important when the receiver is multithreaded and keeps lots of messages open at the same time.

Therefore, the best design of an MSMQ solution for most customers will be based on recoverable (non-transactional) messages, but transactional message receive.

If you can do only one change in MSMQ core for Windows 7, this is definitely my favorite.

Support more than 1000 messages in MSMQ snap-in

This one is actually my fault 🙂 . In MSMQ 2.0, ReceiveByLookupId was not supported yet, so there was no way to efficiently re-visit a message that was once peeked. Therefore, in order to be able to display the message body, the MSMQ snap-in kept a buffer of the first 256 bytes of each message’s body.

When the queue had a large number of messages, keeping that buffer pushed the snap-in memory consumption to the roof (at least in W2K times’ standards…). To avoid that, we decided to implement some kind of "receive as you go" algorithm for the snap-in, but in the meanwhile limit the number of messages peeked to 1000. There is even a comment there that I added "remove this limit after beta" – that is, the beta of Windows 2000…

After beta, of course, there was no time for change and the limitation remained.

In Windows XP / 2003 the ownership of the snap-in moved to someone else, and there was again no time for this change. In the Vista timeframe I was asked by Microsoft (already as a contractor) to improve the snap-in. This time I finally added support for ReceiveByLookupid, so there is no need to keep a buffer for each message. I then suggested to remove the 1000 messages limitation (for free…), but I was told "we need to spec it".

This 1000 messages limitation is really annoying, as many times queues contain lots of messages in times of problems – which is exactly when users want to understand what happened. This is one of the main reasons why people but 3rd party MSMQ browsers (sorry if this recommendation may cause problems to someone’s business).

Support messages larger than 4Mb

This issue was beaten to death throughout the MSMQ versions… It is a little harder to implement technically than the previous issues, so and there is a reasonable application level work around, so I will implement the two other features first. However there is no doubt that there is a significant segment of applications that need to send large messages – like pictures, videos, etc – and this segment  will only grow in the future.

Add performance counter / admin property for total bytes since MSMQ start

There are currently two performance counters – MSMQ Incoming Messages & MSMQ Outgoing Messages – that count the total number of messages since MSMQ service started. These counters are very useful in assessment of the load on an MSMQ server.

I would like to see additional counters for the number of incoming and outgoing bytes. Right now, I can get these only on the individual session level.

If all these values will be supported as admin properties in addition to performance counters, this will be even better…

2. System.Messaging wish list

With all the respect to WCF, System.Messaging is still the major programming model for MSMQ programmers, and is probably going to remain significant for the foreseeable future. The message-oriented programming model is different from the service-oriented model of WCF, and many real-world solutions would always prefer it.

The two main missing pieces in System.Messaging are MSMQ admin APIs –which were introduced in NT4 SP6 and are supported in the COM API since Windows XP – and MSMQ 4.0 features.

Apart from the general request (support those options!) I want to mention two changes – one of each category – that are easy to implement and will make the MSMQ programmers life much easier:

(MSMQ 4.0) – Support PROPID_M_ABORT_COUNT and PROPID_M_MOVE_COUNT

While most of the new MSMQ 4.0 features – like MQMoveMessage – can be easily supported by calling the MSMQ Win32 API from a System.Messaging application, there is no way to reach message properties that were not received by System.Messaging and the only way to get those properties is to receive the entire message using the Win32 API, or alternatively peek the message in System.Messaging and then re-receive it using Win32 APIs – which may be a performance issue.

Better yet – I would love to have a dynamic way to add "unsupported" properties to the message received by System.Messaging.

(Admin API) – Add MessageQueue.NumberOfMessages

The most popular post in my blog explains how to work around this System.Messaging limitation and get the number of messages in the queue by calling the admin API directly, so I guess it will make sense to add it to the main API (will my blog lose popularity after that?)

3. WCF wish list

MsmqBinding – It would definitely help to support various MSMQ features, like Label and AppSpecific properties, Peek & LookupId, etc, etc… However if I get only one wish, I would like support for formatname-based URL (like in MsmqIntegrationBinding) or alternatively support for Multicast. Multicast is an extremely useful feature for publishing (especially over LAN), and it should not be difficult to allow it in WCF

MsmqIntegrationBinding – I actually think that this binding is very useful not only for legacy / WCF integration, but also for everyone who wants to use the WCF model and still use the richness of MSMQ features. Therefore, I would like to see MSMQ 4.0 and admin features supported here as well – see my wishes on System.Messaging.

All the best,

Yoel

Posted in Uncategorized | 5 Comments

The MSMQ WMI Provider

One of the missing pieces that we always planned to add to MSMQ and never had time to is WMI provider.

In a way, the MSMQ performance counters already expose a way to query the MSMQ status through WMI. For example, the following query will find queues with more than 100 messages:

Dim Locator
Set Locator = CreateObject("WbemScripting.SWbemLocator")
Dim objs
Set Service = Locator.ConnectServer(".", "root\cimv2")
Set objs = Service.ExecQuery("Select * From Win32_PerfRawData_MSMQ_MSMQQueue where MessagesInQueue > 100")
For Each object In objs
    WScript.Echo "Name: " & object.Name
    WScript.Echo "…MessageCount: " & object.MessagesInQueue & ", BytesCount: " & object.BytesInQueue
Next

What is wrong with that code then? Nothing is wrong with code that works 🙂 . However, you should consider the following:

1. The list will contain both local and outgoing queues, and it may be a little tricky to distinguish between them (you can do that by analyzing the object’s name)

2. The list contains only the first 97 queues (local and outgoing queues) in your computer

3. Performance data may not be up to date – you may encounter a few seconds delay in data update

4. Some important data for monitoring, like queue state (for outgoing queue) or oldest message age, is missing

For these reasons, MSMQ has the local admin API, which let you query the same data but more accurately and reliably, without these limitations. However, the admin API do not have WMI interface, so it may be a problem calling them from standard monitoring tools.

To overcome those problems a present a simple and easy monitoring model for MSMQ, I recently developed an MSMQ WMI provider. This provider is already deployed with several customers and used by various monitoring tools.

The WMI provider exposes the following objects:

MSMQData – data on the MSMQ service in general.

MSMQOutgoingQueueData – Outgoing queues data

MSMQLocalActiveQueueData – Local active queues are queues which are open or have messages

For example, finding all local queues with more than 100 messages will look as follows:

Dim Locator ‘As New WbemScripting.SWbemLocator
Set Locator= CreateObject("WbemScripting.SWbemLocator")
Dim objs ‘ As ISWbemObjectSet
Set Service = Locator.ConnectServer(".", "root\cimv2")

Set objs = Service.ExecQuery("Select * from MSMQLocalActiveQueueData where MessageCount > 100")

For Each object in objs
    WScript.Echo "Name: " & object.Name
    WScript.Echo "…MessageCount: " & object.MessageCount & ", BytesCount: " & object.MessageSize
Next

The WMI provider lets me perform queries that performance counters cannot. For example, the following query finds outgoing queues which are "Waiting to connect" with oldest message older than 5 minutes (this usually indicate some communication problem, like unresolved address):

<Defining Locator and Service like in the previous scripts>

Set objs = Service.ExecQuery("Select * from MSMQOutgoingQueueData where State = ‘WAITING’ and OldestMessageAgeSeconds > 300")

For Each object in objs
    WScript.Echo "Name: " & object.Name & ", Oldest message: "  & object.OldestMessageAgeSeconds
Next

You can download and install the MSMQ WMI Provider here. Note that the installation package cannot be installed on Win2K – to get a Win2K compatible package please email meat yoel@msmq.biz .

You can find complete reference of the MSMQ WMI provider objects and properties here.

Enjoy – tell me about your experience and suggest future improvements.

Yoel

Posted in Uncategorized | 4 Comments