Saturday, September 14, 2013

Event Source Analyzer for Potential Run-time Errors

You might write the best EventSource with all the required methods for logging. But we must remember that your event source would generate manifest and the manifest is used at run-time to write ETW events. This might result in unexpected runtime errors. One wonders how those issues can be found out earlier in the development cycle. Semantic Logging Application Block provides EventSourceAnalyzer for just this purpose. It is used to verify the run-time checks for an EventSource instance.

The type provides Inspection methods. In case of a possible failure during verification, it throws EventSourceAnalyzerException. I think the best usage of this type is to write unit tests to verify such checks for the defined EventSource.

Checks Performed
Now what checks are performed by EventSourceAnalyzer. It performs a 3-point check on EventSource. The tests are as follows:
  1. Is there any error when the given EventSource is enabled by an EventListener?
  2. Can event schema be requested from the EventSource?
  3. Can all Event based methods be invoked in the specified EventSource class?
If the answer of all the above questions is Yes, the unit test is passed!

Let's Start Testing Our EventSource
Here is an method from an EventSource. The compiler generates manifest for ETW events based on the method parameters of Event method. So at run-time calling WriteEvent with a different parameter list than the generated manifest would cause failure. The same is the case with the following EventSource method.

If we use EventSourceAnalyzer for an EventSource containing this method, this should result in the an exception providing the details about the possible issue at runtime.

Generating an ETW event with a non-existent EventId would also result in failure at runtime. Let's see the following method where the Event method is decorated with an EventId which would be used to generate the manifest for the associated event.

Now at runtime it could generate an event with an Id in the manifest just because of an incorrect argument to WriteEvent method. We can determine that earlier in the cycle. The analyzer would result in the following exception in this case.

There is a typo here with the exception message. I have logged a connect item for this. If you really dislike this then you can vote for this.

There are only a handful of types which could be used for ETW events. Someone has specified a unofficial list here as follows: [ Boolean, SByte, Byte, Int16, UInt16, Int32, UInt32, Int64, UInt64, Single, Double, String, Guid]. Since WriteEvent accepts an object array as method argument, the compiler wouldn't have any issue if we pass an instance of an unsupported type. This would only cause failure at runtime. Look at the following method which is using DateTime type.

The Analyzer wouldn't like this and would result an error. Here is the generated exception for the above case:

The analyzer even checks for the order of the parameters in the WriteEvent and the EventSource method. They should be in the same order. In the following we are passing arguments to WriteEvent in a different order than EventSource method.

EventSourceAnalyzer can be a little flexible about this. We can use the following property so that the order is not checked.

Because of the way EventSourceAnalyzer inspects the EventSource, we might need to set ExcludeEventListenerEmulation as well. So that EventListener is not used, otherwise, it would mimic the runtime and would result in failure. The order is important at run-time so we should avoid setting this property to true generally.

The analyzer can also be used to verify that not only the parameters count and order should match, but the types of EventSource method and WriteEvent should also be matching. Here is the exact case in the following:

This would not be checked by EventSourceAnalyzer by default. We can set the EventSourceAnalyzer as follows so that this is also checked.

Since the methods in the EventSource are used to define Event Types for the resulting Event Provider. There should be no non-event methods in the EventSource class. But we do need helper methods sometime. All such helper methods should be clearly marked as NonEventAttribute. Yes, even if they are private.

Writing Unit Tests
As you might have noticed, EventSourceAnalyzer is provided by Semantic Logging Application Block [SLAB] in Enterprise Library 6.0. On the other hand EventSource is provided by .net framework 4.5. SLAB extends the EventSource by providing the amazing ways we can utilize the event data but it is not a requirement to generate ETW event data. We can even have an out-proc event listener which could utilize the generated data and direct it to any sink. The source project doesn't need any reference of the application block.

But we still want to test the logic, we can just install the package to our unit test project. In the unit test we can verify all these checks pretty early. These unit tests can be added to our continuous integration build. Let's install the application block nuget package to our unit test project.

We can use xUnit for our unit tests. As we have discussed before, we can add its support in IDE by installing a Resharper extension. Here we are installing a release version of the package.

Let's add the following unit test to inspect EventSource for runtime checks.

In case a test fails, it shows it as follows:


No comments: