Log Visualiser using a particle system

I carried on my attempts to make logging output look a bit interesting, and what I think I wanted was a stream of pixels flowing down from a server name to give a waterfall effect. Each pixel streaming down would represent one log event, and the colour of the pixel would be determined by the log level of that event.

I set out to implement this, but then thought that someone else must have come up with a particle system for XNA already. And they have, check out this particle sample on the XNA Creators Club.

This sample shows how you can make explosions and smoke effects in XNA, and the end result is surprisingly realistic. I took this example and derived my own class from the abstract ParticleSystem class. It took some experimentation with the parameters I used, but now I've got a stream of pixels flowing in the way that I planned.

I've also got a mock application that just generates log entries for the XNA application, and I've generated 10,000 logging events in a very quick sequence. As the XNA application doesn't store the events for very long there doesn't appear to be much slow down, but it would be interesting to load test it against a number of servers generating a high volume of logs. At the moment I think there would be a slow down but I'm pretty sure I can refactor the event handling so that it will be more scalable.

There's some cleaning up to do, mainly with the positioning of new server entries and externalising a number of parameters to a config file, but I think I'm mostly there.

Visualising Logs - Progress Report

Just a short one as it's getting late now. I've continued a little with my XNA log viewer mentioned previously.

I've set up a mock application that generates logs on command, and I've set up log4net in the same way you would if you wanted to look at the logs in Chainsaw. So this fires off UDP packets containing the entry in XML. I've then got a class that listens asynchronously to a socket. When it receives data it deserialises it to an object, and then raises an event.

My XNA "game" has one of these socket listening classes, and provides an event handler to receive the logs.

On screen right now I'm just scrolling the messages. To do this I have a GameComponent that contains a rough implementation of a generic circular queue. The contents of this queue are drawn to the screen, and as it's always wrapping around, it gives the effect of scrolling without having to move the position of the string.

Next I need to figure out how I want the logs to be visualised.

Making logging output "interesting"

This slashdot article caught my interest last night: Logfiles Made Interesting with glTail. Basically a guy has written a Ruby on Rails program that parses Apache access logs and gives you a visual representation of them using OpenGL graphics.

The outcome of it looks pretty interesting, and it got me thinking about what else you can do. I'm wondering how we could visualise some log4net logging, maybe using the XNA framework to output the graphics.

So last night I coded up a simple UDP Socket listener, and made a mock application that logs out an XML event to a UDPAppender. I found it should be pretty simple to get the log events to an XNA application, and now I'm thinking how the logs that the Esendex system generates can be displayed in an interesting yet useful way.

I'll keep you posted if I do anything else.

Enable Internal System.Net Logging

Having trouble with certificates while requesting information using HttpWebRequest, manifesting itself with the error "Could not create SSL/TLS secure channel". As far as we could tell the certificate should have been ok.

To diagnose the problem I found this great blog post to enable the internal logging from the System.Net classes by Durgaprasad Gorti over at the MSDN Blogs.

Just needs a new section in the app.Config file to log out what the System.Net classes are doing. One of the samples in the blog post worked straight off for me:

<?xml version="1.0" encoding="UTF-8" ?>
<configuration>
    <system.diagnostics>
        <trace autoflush="true" />
            <sources>
                <source name="System.Net" maxdatasize="1024">
                    <listeners>
                        <add name="MyTraceFile"/>
                    </listeners>
                </source>
              <source name="System.Net.Sockets" maxdatasize="1024">
                    <listeners>
                        <add name="MyTraceFile"/>
                    </listeners>
                </source>  
           </sources>

            <sharedListeners>
                <add
                  name="MyTraceFile"
                  type="System.Diagnostics.TextWriterTraceListener"
                  initializeData="System.Net.trace.log"
                />
            </sharedListeners>
            <switches>
                <add name="System.Net" value="Verbose" />
              <add name="System.Net.Sockets" value="Verbose" />
            </switches>
    </system.diagnostics>
</configuration>

It creates a file called System.Net.trace.log in the directory where the EXE is running. The logging is very comprehensive and will point you in the right direction for fixing your problems.

Is Log4cpp new, or has it been abandoned?

Any C++ programmers use Log4cpp? For C# we use Log4net for logging, and when I found that PC SMS used Log4cpp as it's logging framework I just assumed it would be as easy to configure as Log4net.

But Log4net has a very active user community, and the mailing list gets regular visits from the developers Ron and Nicko. No so with Log4cpp.

All I needed was a really simple basic setup to log to a file. Could I find anything? Well, would you find this useful, which says "The configuration file format is undocumented and may change without notice". Ok...

Luckily I found the log4cpp.properties file buried within the tests directory of the log4cpp download, which goes some way to giving the example I needed.

So, if anyone else is searching for the simplest Log4cpp configuration to log to a file, just use this:

log4j.rootCategory=DEBUG, FILE

appender.FILE=org.apache.log4j.FileAppender
appender.FILE.fileName=c:\filename.log
appender.FILE.layout=org.apache.log4j.BasicLayout

And no, that's not a typo, and I haven't cut and paste from the wrong file: Log4cpp uses Log4j stuff in its configuration file.

Durgaprasad Gorti's WebLog : Using System.Net Tracing

If you've ever used something like Ethereal to trace network traffic you'll know that it can be confusing, especially if you're sending lots of packets to different destinations.

I've been enlightened by a new .Net 2 feature that causes the System.Net classes to log what it's doing, and you can do it without making any code changes.

Check out this blog for details: Durgaprasad Gorti's WebLog : Using System.Net Tracing

Haven't given it a go yet, but it looks like something we can make use of to log what the framework is doing.

EventType clr20r3 From Windows Service

In the final throws of testing part of our system on the new .Net 2.0 code, we deployed a few of our Windows services to a production server in preparation for going live with it. Luckily for us that we did because we found that one of the services appeared to stop occasionally.

The service is responsible for notifying our customer's applications of events on their accounts as part of our Push SMS API service, which for many customers is a very important part of the services we offer. So we needed to get this problem fixed before going live, but we had a number of problems with finding out why the service was stopping.

We use log4net for logging within all of our applications, but these log files did not show any errors being thrown from the service.

However, the Event Viewer did trap an error. It was in the category of a .Net Runtime 2.0 Error, being logged in the Application section of the Event Viewer. The error message looked like this:

EventType clr20r3, P1 t1f44vcumbpfz21uvtobhdijm4egyfae, P2 1.0.2643.18660, P3 460a41c8, P4 mscorlib, P5 2.0.0.0, P6 4333ab80, P7 4052, P8 27, P9 qvnotafqzyql5inknkn5msslsene3ln1, P10 NIL.

I think you'll agree its not that much help. Most of the errors were the same, but some had different garbled text in the P9 section.

The code that was running hadn't changed since the 1.1 build, and the service never used to stop on any errors previously. We have error handling that will log any error and allow the service to carry on in the event of any failures. Or so we thought anyway.

I was maybe 5 minutes away from raising a support call with Microsoft, when I found this article that explained how I could make the error message more useful: Tech Notes :: DotNet Runtime Error UPDATE: Original link seems to have died. See below.

The error is an example of an unhandled exception being thrown out of the application. You can make some sense of the error by implementing a handler for the UnhandledException event on the current AppDomain that your application is running in. This event will pass in the exception object that has been thrown, so you can log it to the Event Viewer in a more meaningful fashion.

From this we found the error and have now corrected it so the service doesn't keep stopping.

When the service started it would perform each task it was required to do in a method that we had queued in the ThreadPool. If this method managed to throw out an exception, then of course that current Thread would exit. Now, we shouldn't have been throwing exceptions out of this method anyway, but the 1.1 Framework didn't seem to mind about it.

In the 2.0 Framework it seems that if a method that is in the ThreadPool throws, then that Exception takes down the application, in this case the Windows service. Personally I feel that this is a better way of doing it, as the 1.1 Framework was masking an error in our code. The error message could have been a bit more meaningful in the first place though.

UPDATE (3 Apr 2009): The original link that was the inspiration behind solving this no longer exists.  However if you are interested please check out Unhandled Exception Processing In The CLR from the MSDN Magazine for more information.