Been a little quiet here lately

I've just become very aware that I haven't posted here since Friday's cynical link about BP being fined, but things have been busy here finalising and deploying the update hinted at in my Monitoring Custom Windows Services post.

It's not been an easy couple of weeks, but I think we're finally seeing the performance improvements that we were hoping for.

I know I mentioned posting some code samples about how to create your own performance counters, but I really haven't had time.

If you want to investigate using performance counters then probably the best thing to do is check out these MSDN articles:

• Category and Counter Management,
• Create Performance Counter Categories,
• How to Create Custom Performance Counters.
  

BP fined for environmental crimes and fraud

BP has been fined $373m by the US government for "environmental crimes and committing fraud".

"BP has committed serious environmental crimes in our two largest states, with terrible consequences for people and the environment," the Environment Protection Agency said.

Anyone else remember their advertising campaign that encouraged people to calculate their "carbon footprint"?

Seems BP should get their own house in order before telling the rest of us to.

Monitoring Custom Windows Services

A lot of my work at Esendex involves creating and maintaining Windows Services that form the guts of a much larger product that we collectively call the Esendex Messaging System. The work that these services do can differ wildly: everything from the relatively mundane to complex critical services that the business depends on.

Because of this, each service needs to be monitored in different ways. Sometimes its OK to trawl through log files to discover what the service is doing, but with other more widely used services this quickly becomes tedious.

We're currently in the process of rolling out an update to one of our critical Windows Services, and this update involves an architecture change that makes examining logs beyond tedious, it's actually unworkable for anything other than tracking historical events.

This is where we use custom performance counters. If you're a developer and don't know what these are, then read up on it, and quickly. Performance counters can be used to give you visibility on the near-realtime progress of your systems.

Here are some examples of what we monitor with performance counters:

• Execution time of certain key interface methods,
• Total counts of current throughput at various stages of a message's journey through the system,
  
• Rates of current throughput (how many messages per second, etc),
• Statuses of key services.
  

From these we can use the standard Windows Performance Monitor to watch what a service is doing. For a more bespoke monitoring solution you can also write your own applications that read your performance counters and display the results in a more friendly way.

If I get time I'll knock up some sample code demonstrating how to do this.

The Vulcan Flies Again

Something I've been waiting to see for a few years after hearing that the Vulcan to the Sky Trust were restoring one: yesterday it completed its first test flight.

Check out some of these photos of the flight.

Couple of things I miss from Visual C# Express

I know this great little application is free and so can't really complain, when you work with Visual Studio 2005 Professional five days a week the bits that are missing start to get to you:

1. No "delete all breakpoints" option.
2. Can't attach the debugger to another process, or set the project to load a different application

Doesn't sound like much, and I didn't think that was much, until you step through an XNA project, and then have to manually remove all your breakpoints. Or if you want to unit test something and you can't attach to the NUnit process, which means you can't step through your tests.

Glad it's got Intellisense though :)

The Stupidity of the Collective

I left work a little later than usual today, and so when I finally got onto the motorway the traffic was lighter than I normally have to drive through. Although I say "lighter" it was still busy, and I had to wait a few minutes before I could get into the outside lane to overtake a pair of lorries seemingly dancing together in the first two lanes.

The middle lane was clear after I passed, so I moved into that, and a little while later the first lane was clear, so I moved into that.

It was then that I realised the outside lane was pretty much bumper to bumper. The middle lane was clear, and the first lane that I was in was flowing smoothly at just under 70mph. The outside lane on the other hand was stumbling along at probably around 60mph, and the cars were so close that people had to keep braking, which was causing a traffic shockwave as far back as I could see in my mirror.

Why were these people in that lane? I have no idea really, but it must be for a similar reason to why cars try to overtake people when the next lane is going no faster than the one they're leaving.

Don't these people know that life in the fast lane doesn't always get them to their final destination any quicker? I mean, just look at Keith Richards..... ;)

What if Google Had to Design For Google?

To web developers the term SEO (search engine optimisation) can send chills down their backs. Usually it means adding things to the design of a web site in order to get it noticed by search engines--particularly Google.

Saw this linked from Slashdot: What if Google Had to Design For Google in which it shows what Google would look like if their own developers had to get their site noticed by themselves.

The outcome looks eerily like a Facebook or MySpace clone. Now you know why those sites can look so cluttered ;)

Log Visualiser using a particle system

I carried on my attempts to make logging output look a bit interesting, and what I think I wanted was a stream of pixels flowing down from a server name to give a waterfall effect. Each pixel streaming down would represent one log event, and the colour of the pixel would be determined by the log level of that event.

I set out to implement this, but then thought that someone else must have come up with a particle system for XNA already. And they have, check out this particle sample on the XNA Creators Club.

This sample shows how you can make explosions and smoke effects in XNA, and the end result is surprisingly realistic. I took this example and derived my own class from the abstract ParticleSystem class. It took some experimentation with the parameters I used, but now I've got a stream of pixels flowing in the way that I planned.

I've also got a mock application that just generates log entries for the XNA application, and I've generated 10,000 logging events in a very quick sequence. As the XNA application doesn't store the events for very long there doesn't appear to be much slow down, but it would be interesting to load test it against a number of servers generating a high volume of logs. At the moment I think there would be a slow down but I'm pretty sure I can refactor the event handling so that it will be more scalable.

There's some cleaning up to do, mainly with the positioning of new server entries and externalising a number of parameters to a config file, but I think I'm mostly there.

Visualising Logs - Progress Report

Just a short one as it's getting late now. I've continued a little with my XNA log viewer mentioned previously.

I've set up a mock application that generates logs on command, and I've set up log4net in the same way you would if you wanted to look at the logs in Chainsaw. So this fires off UDP packets containing the entry in XML. I've then got a class that listens asynchronously to a socket. When it receives data it deserialises it to an object, and then raises an event.

My XNA "game" has one of these socket listening classes, and provides an event handler to receive the logs.

On screen right now I'm just scrolling the messages. To do this I have a GameComponent that contains a rough implementation of a generic circular queue. The contents of this queue are drawn to the screen, and as it's always wrapping around, it gives the effect of scrolling without having to move the position of the string.

Next I need to figure out how I want the logs to be visualised.

How to prosecute a lying politician.

Did anyone else watch The Ministry of Truth on BBC2 just now? I saw a summary of it on last night's Newsnight, and was compelled to watch the full version today.

If you didn't get to see it, the crux of the documentary was to find out what you can do if an MP (including the Prime Minister) openly lies to the electorate.

The shocking answer, according to the makers of this new documentary, is: you can't.

If a politician lies to the electorate, there is no formal course of legal redress outside a general election. In a bid to close this gap in the constitution, an MP has drafted The Misrepresentation of the People Act: a bill that would force politicians to tell the truth - or face criminal prosecution.

Plaid Cymru MP Adam Price is now spearheading a national campaign to change British law so that any politician who deliberately misleads the electorate can be prosecuted.

Now, I don't normally get involved in politics, but watching the MPs in this documentary all agree that honesty was important, and then call the interviewer naive for wanting to put something into an actual law has put my back up quite a bit.

However, it was refreshing to see that no one backed down, and there were responses for every one of the MPs' objections. For instance:

The bill is not needed as Parliament regulates itself:
All of the regulatory bodies within Parliament report to the Prime Minister, who then decides if he wants to act on it. The programme likened this to being judge and jury at your own trial.

If an MP openly misleads Parliament their career is over:
Admittedly most MPs found to have lied do resign, but it doesn't prevent them from holding office again. Look at Peter Mandelson and David Blunkett. These MPs resigned only to find themselves elected officials once more.

Any MP in the House can ask "question after question after question" at Prime Minister's Question Time (Lord Falconer gave this little gem):
An MP can only ask a maximum of two questions per sitting, and the PM doesn't have to actually answer it.

To say this bill is needed is to suggest that all politicians are dishonest:
Not at all, claims the documentary. Just because we have laws against murder, robbery, rape, and theft doesn't mean that we would all do it if it was legal. (Although, an admittedly unscientific poll on the Ministry of Truth website asking "Do you think politicians lie to us" is currently sat at 96% for "Yes, on a regular basis")

You can't have a law against lying (particularly liked this one, that seemed to be spouted by most of the MPs):
Well, we do already. Trades descriptions laws, advertising laws, company laws... they all prevent individuals from deceiving their customers, or board members. Why should the government who, as one person on the documentary said, "are the board of directors for Great Britain PLC" be any different?

An Early Day Motion about the bill is set to be put in front of ministers next week. The cynical side of me thinks that a lot of smarmy spin doctors will shoot it down before anyone has had the chance to discuss it properly, but I truly hope that doesn't happen.

I wish Adam Price the best of luck next week, and sincerely hope he takes it all the way to an actual law.

Do you use goto in C#?

Looking through the code for the SmtpClient.Send method in Reflector and found this snippet:

switch (this.DeliveryMethod)
{

case SmtpDeliveryMethod.SpecifiedPickupDirectory:

if (this.EnableSsl)
{

throw new SmtpException(SR.GetString(
"SmtpPickupDirectoryDoesnotSupportSsl"));

}
break;

case SmtpDeliveryMethod.PickupDirectoryFromIis:

if (this.EnableSsl)
{

throw new SmtpException(SR.GetString(
"SmtpPickupDirectoryDoesnotSupportSsl"));

}
goto Label_021C;

default:

goto Label_022B;

}
MailWriter fileMailWriter = this.GetFileMailWriter(
this.PickupDirectoryLocation);
goto Label_025D;

Label_021C:

fileMailWriter = this.GetFileMailWriter(
IisPickupDirectory.GetPickupDirectory());
goto Label_025D;

Label_022B:

this.GetConnection();
fileMailWriter = this.transport.SendMail((message.Sender != null) ?
message.Sender : message.From, recipients,
message.BuildDeliveryStatusNotificationString(),
out exception);

Label_025D:

this.message = message;
message.Send(fileMailWriter, this.DeliveryMethod != SmtpDeliveryMethod.Network);
fileMailWriter.Close();
this.transport.ReleaseConnection();
if ((this.DeliveryMethod == SmtpDeliveryMethod.Network) && (exception != null))
{

throw exception;

}

I've always been taught to avoid goto like the plague, and it's something I've followed for years. I've never even considered it to be an option when writing code.

Thinking about it though, I did read an article a few years ago (which I can't find now) which said you could credibly use gotos only if you used it to jump down code, and not up: a rule that is stuck to here.

The thing is, I know this code can be written without using goto. But also, it might not be as readable as it is now.

I'm conflicted. But I'm sticking to the "no goto" rule until someone can come up with an example that really can't be coded without using a goto.

Sending SMS to a specific port using the Esendex API

I've written an Esendex Developers Blog entry describing how to send SMS to a specific port. It's quite technical, but we've had a few support cases raised about it in the last couple of weeks, so thought I'd write about it.

It's quite long as well, but I don't think I could make it be any shorter and still be meaningful.

Making logging output "interesting"

This slashdot article caught my interest last night: Logfiles Made Interesting with glTail. Basically a guy has written a Ruby on Rails program that parses Apache access logs and gives you a visual representation of them using OpenGL graphics.

The outcome of it looks pretty interesting, and it got me thinking about what else you can do. I'm wondering how we could visualise some log4net logging, maybe using the XNA framework to output the graphics.

So last night I coded up a simple UDP Socket listener, and made a mock application that logs out an XML event to a UDPAppender. I found it should be pretty simple to get the log events to an XNA application, and now I'm thinking how the logs that the Esendex system generates can be displayed in an interesting yet useful way.

I'll keep you posted if I do anything else.

Walkers Keywords from Google Analytics

Waiting through my lunch for an Orange conference call to continue I opened Google Analytics and checked up on who was getting to this blog. There's an unusually high proportion of people seemingly searching for Walkers crisps.

Some of the search keywords are:

• the purpose of walkers crisps
• internal customers at walkers crisps
• what are walkers crisps called in walmart orlando
• how do you make a packet of walkers crisps
• how much fat is there in a pack of walkers
• walkers crisp purpose of the business
• walkers crisps labels
• what is the main purpose of the packaging of walkers crisps

Also a couple of visits from what looks like an internal Walkers host name.

Seems to be a bit of a Walkers hub at the moment ;)

Batching Bulk SMS Submissions using the Esendex API

I've written a summary of how to submit batched messages using the Esendex APIs, which will help anyone having problems submitting large amounts of messages at once.

It's not code specific, but should point you in the right direction if you find that making many single submissions is giving you network errors.

Read this Esendex Developers Blog entry: Batching Bulk Submissions using the API for more information.

Documenting assumptions with Debug.Assert

Another one from Greg Beech today: Document your assumptions with Debug.Assert instead of comments.

I've never used Debug.Assert, but after reading this post I'm trying it out today. The post makes some excellent points:

When writing methods people frequently make assumptions, such as that the internal state of the class is correct, or that the arguments passed into a private method are valid (because they should have been checked by the non-private method calling it)

[...]

[If these assumptions are wrong] the program will carry on regardless until it fails at some point in the future. And because you didn't trap the problem at the point where it became clear that something was already wrong (i.e. at the comment documenting the assumption) then it's much harder to work out where the problem really occurred because the failure itself could occur further down the line in a largely unrelated method.

[...]

So every time you find yourself making an assumption "I know this will be there", "this should be in that state", "the argument should not be null as the public method will have validated it", document it by using Debug.Assert rather than a comment. Then, if your assumption is incorrect, your application will come to a grinding halt at the point where the problem became apparent.

So I'm giving it a go. It can only help during development, and won't affect anything in production code as the Debug class will only compile in debug builds.

There are a few warnings in Greg's post about when not to use it, so make sure you read those before giving it a go.

Esendex Website Updated

The new style web site went live yesterday afternoon, with surprisingly (but reassuringly) few problems. It's early days yet, and I'm sure we'll get a few calls from people who can't find things in the new format, but it looks like the deployment has been successful.

I was in two minds about posting about this actually, because I've not done that much work on it. I've paired with Neil and Kevin on the odd task, but I think of the developers here I've probably done the least on it.

The new website contains various blogs (like for support, development, and the company as a whole), so please check those out. I've already posted to the development blog, summarising the inbound multipart message update I blogged about here recently.

Right, onto the next story.

Product Knowledge Vs Coding Ability

I stumbled across Greg Beech's blog a few days ago while browsing the MSDN Forums and my attention was drawn to this post: Is product knowledge more important than coding ability?

The post didn't turn out to be what I thought it was, but it was interesting reading none the less. I thought it would be a discussion about whether a developer's knowledge of their product was more important than their individual coding ability, but the "product knowledge" that the title refers to is knowledge of third party software your company uses.

Greg gives a good example where his team solves a difficult problem quickly through their knowledge of the products they are using, rather than with any dramatic improvement in coding skill.

But isn't knowledge of these products just one measure of your ability as a developer? Without knowing how to use a framework (and I mean "framework" in the broadest possible terms) a developer isn't going to be able to do much quickly.

For example, if I wish to send an email from a program I don't open a socket to a mail server and throw SMTP commands across the wire. I use something like the System.Net.Mail classes which contains everything I need.

I know I could implement my own SmtpClient. I have the coding ability to write the code that will do it, but it would be a waste of my time to do so.

Coding ability and product knowledge are just two of the many weapons in a developer's arsenal that need to be used at different times to solve different problems.

I'd say the ability to judge when to use each is the more important skill though.

Sumo TV Trailerdrome

Flicking around Sky Digital channels a few weeks ago I stumbled across Sumo TV's Trailerdrome on channel 144. There's no website for the programme itself, the only thing I've found is the Sky.com TV listings page that tells you when it's on:

Featuring the best of obscure film trailers, public information films and newsreels from the last 70 years, when life was more innocent. Or was it?

And it is obscure, but I found it disturbingly compulsive viewing. The only problem is that they seem to show the same trailers on different days.

Among the trailers and public service films I've seen:

• a warning to young girls about "getting yourself in trouble", which contained absolutely no mention of pregnancy or sex, but ended with the girl being put into care for no other reason than being a single teenage mother
• an animated educational film about how capitalism works, and why it makes America great
• coverage of an "Economy Drive" from the 50s/60s, in which teenage drivers aim to follow a long street course using the least amount of fuel. I think the winner managed around 50 miles per gallon per ton.
• a documentary on how to have dinner with your family. Apparently no one person should dominate the conversation, and the girls of the family must help their mother prepare the meal.
• a short film about divorce, that only mentioned the word once, covered none of the legal side of actual proceedings, and which only really showed the beginning of a couple's relationship in which the woman said she could change the man to suit her own needs.
• far too many more to list here.
  

One of the great things though is that there seems to be no real adverts for the duration of the programme, which runs for about two hours. There are actually adverts, but they are old ones from the 50s and 60s that clearly aren't on sale now. There's a crazy one for Heinz Pickles that features an old lady beating up the waiting staff in a restaurant because the pickle she was served wasn't good enough.

So if you find yourself channel surfing and you happen across something in black and white that usually has a very condescending voice over on top then you've probably stumbled across Trailerdrome. Give it a go, it's a laugh.