Pride in bad solutions

I solved a problem the other day.  It was a terrible solution. It works, but it's difficult to be proud of how I solved it.

The problem was with MSMQ, but let me describe the problem with a needlessly overwrought metaphor.

Every week for the last 5 years, you've been sending a bill to one of your clients.  You have an infinite supply of envelopes and when Friday comes around, you print out an invoice, seal it in an envelope, send it by courier to your client, and a couple of days later, you get paid.  With me so far?

One day, you decide to start using a different invoice management software package - it's much prettier and more stable than the old one.  You know, however, that the client doesn't want anything in that invoice to change.  You're not sure how they process it, but one time your printer smudged an invoice slightly and you didn't get paid.  So everything in that invoice has to be exact.  Luckly, the new software is capable of printing a pixel-perfect version of the old invoice layout.  So far so good.

You also think you should start using a different courier.  Your existing courier company (COM+) is fine, but frankly they're a bit behind the times.  The delivery drivers are about 80 and they're driving some vans that are generations old.  So you set up a deal with a new company called C-Sharp 3.5 to do the delivery for you.

Still with me?  I told you it was overwrought.

So anyway, you give your new system a try.  You print out the invoice, compare it to the old one (spot-on, 100% the same), put it in an envelope, and send it off using the new courier company.

You don't get paid.

You contact the customer but they don't say much.  Just that they didn't receive a valid invoice.  The new courier company swears they delivered it to the correct address and they provide proof.

You decide to try again next week and do some investigating.

The next week, you print out the invoice, compare it to the old ones (still the same), put it in an envelope, and organise for the new courier to pick it up.  This time though, you follow him.

He takes your letter, gets in his van, and drives to the client's address.  He gets out, puts your letter in the mailbox and drives off.  Nothing wrong so far, as far as you can tell, so you wait to see what happens.

Soon, a guy comes out of the house (which by the way looks exactly like a big black box), opens the mailbox, picks the letter up, and takes it inside.

You go home, satisfied that the letter made its journey this time.  It must have been a once off.  But you still don't get paid.

Again, you contact the client and all they'll tell you is that they didn't get a valid invoice.  You protest, telling them that you saw them pick it up but to no avail.

Over the next few weeks, you try everything you can think of to get this new system to work.  You try looking at the message again after it's put in the letterbox and you try sending an invoice created by your old program.  Every time, the seemingly perfect letter gets picked up my the man in the black box, and nothing happens.

No matter how hard you try to work out what's going on, nothing helps.  So, like any sensible person,  you give up and call the old courier company.  They turn up, pick up your invoice, and two days later, you get paid.  Despite the fact that they're apparently doing exactly the same as the new guys, their deliveries get you paid, and the other deliveries don't.  You resign yourself to using the old couriers until they or their vans all die.  It's just a matter of time...

Fun story, huh?

So in case you're a bit slow and didn't work out what it was all about, I replaced an app that put a message in a Windows message queue for another (black box) program to pick up.  The old app was VB6 using a COM MSMQ library, and the new one was C# using the native .Net MSMQ library in the 3.5 version of the .Net Framework.

No matter how hard I looked, I couldn't find ANYTHING different between the messages and where they got put.  I even dug up a copy of the black-box code and stepped it through a debugger.  When it got to the line saying Queue.Receive(), the message disappeared from the queue, and nothing came back.  There was no exception thrown!  I watched the message get picked up!  I did a binary comparison of the message contents from the old program and the new program!  No difference at all!  Inexplicable!

The fact that I was putting the message into the Windows Message Queue using a .Net library and picking it up using COM+ seemed to be all it took to break the thing.

This is the first time I can remember being absolutely, 100% stumped by what was going on.  I'd analysed what was going on as deeply as was practical and had come up with nothing.

So I did the logical thing and referenced the COM+ dll from my new app.  When I used this library to send the message, it all worked perfectly.

I really don't like this fix. Despite the fact that everything works perfectly now, it's still a failure in my eyes.  It's like a brand new Merc with duct tape holding the wheels on.

However, given that I'd spent way too long already on something that really shouldn't have been a problem, it was probably the right thing to do.  Cut my losses and take the easy way out.  I'm not proud.

Damian Brady

I'm an Australian developer, speaker, and author specialising in DevOps, MLOps, developer process, and software architecture. I love Azure DevOps, GitHub Actions, and reducing process waste.

--