Sunday, June 23, 2013

Nullable<T> is not a Reference Type

You probably might know it already. But I am sure for most of the developers out there it comes as a surprise. It kind of makes sense because they have seen null being assigned to reference types only. So if Nullable<T> can be assigned a null, is there a special preferential treatment going on here. In this post we are going to discuss how this is handled by the framework and how we can use it to write better and more efficient codes. null represents the absence of a value, so assigning null to a Nullable value type just means that it has no value. In C# official text, Nullable type is defined as a value type being an extension of all other value types with a null value. For every non-nullable value type there is a corresponding nullable value type denoting the same set of values plus the value null. The underlying type T of a nullable type cannot be a nullable type, otherwise, all other value types are supported. This is a beginner - intermediate level post.

In one of our previous discussions we pondered on the idea of Passing by Value and reference. We discussed that here Value and Reference have nothing to do with value and reference types. In both cases, passing happens by value until we use the ref keyword specifically which causes passing by reference. This would cause a re-assignment to be reflected in the caller code for the actual variable passed by reference. There is no issue passing a value type by reference. This is similar to the idea that says reference types are always allocated on a heap and value types, on a stack. What about the members of a reference type which are themselves value types?

Assigning T to a Nullable<T> type
There is no difference between directly assigning a variable of underlying type or newing up the Nullable<T> directly. In the following code, we are declaring and initializing an Int32. We are then assigning it to a nullable using the short notation and longer one. In Microsoft's text, Wrapping is defined as an operation of creating a non-null instance of a value type with a given value. On the other hand, Un-Wrapping is defined as accessing the Value property of the nullable type instance.

Let's see how the code created by these two options. As you can see below compiler would generate the same IL code in both situations.



Assigning Nullable<T> to T
Nullable<T> cannot be directly assigned to its non-nullable counterpart. Static type checking shows the error at compile time.



An explicit cast to underlying non-nullable type should get you out of the compile time issue. But in the case when the variable has a null value assigned, it results in InvalidOperationException as follows:



As we discussed above, we have to be careful when we use the nullable types in the expressions. Let's see the following example where a Nullable<T> has been added to a non-Nullable. We have used different order in both expressions. Let's see how it affects our result. Here we are checking if the Nullable has been passed a null value, we are defaulting to zero in this case. Null-coelscing operator (??) can be used on a Nullable<T>.

Let's see how these methods are compiled. We can compare the IL generated by both methods. Here Calc seems to generate more IL than Calc2.



Now let's try to use these two methods which should be performing similar by passing the same arguments and compare the results. It seems that they print different messages to the console. But why? Shouldn't they be performing similarly with Add being a commutative operation.



Actually this is just a matter of operator precedence. Null-coalescing operator's ?? precedence has a lower priority than + . So the addition is happening first [0 + 2] and then the expression becomes as [ num2?? 2]. Now since num2 is not null, 3 is assigned to the result. Using a parenthesis in the expression should result in the same result. This is a very common issue which results in surprisingly wrong results. You can read more about C# operators and their precedence on msdn. Further information about null-coelscing operator can be found here.



It is a lot easier if we use GetValueOrDefault method provided by Nullable<T>. We can also specify a fixed default value instead of the default value of wrapped type using another overload of the same method.


There is a lot of confusion on msdn about conversion between the Nullable and underlying type. Some statements are just wrong e.g. this statement: When the null value is converted to a non-nullable value type, it is implicitly converted to the zero value of the value type. This is certainly not the case, this results in InvalidOperationException. Additionally, this statement is also wrong: T? has a predefined narrowing conversion to T . We need an explicit cast here. We can give the writer benefit of doubt by thinking that it is explicit cast that she is talking about. But the example given is using an implicit cast, which is certainly not the case.

Boxing / Unboxing Nullable Types
Now that we have established that Nullable<T> are value types then, being value types, they need to go through the same boxing / unboxing when there is an assignment involved with a reference type. As a matter of fact, boxing a nullable type is actually boxing the underlying type. Following is the IL code generated for a code calling Console.Writeline with a Nullable<T> variable. Compiler decides to use the overload of the method with object type parameter. Please notice the special instruction for boxing in the IL here.



Similar IL instructions are generated when we make the same overload of Console.Writeline for an Int32 using explicit casting. Don't try this @ home ever.



Assigning an object to a Nullable type would result in unboxing. Since this is a narrowing conversion we need to use an explicit cast here.





C# language specification is clear in stating what is actually going on. Boxing a value of a nullable-type proceeds as follows: If the source value is null (HasValue property is false), the result is a null reference of the target type. Otherwise, the result is a reference to a boxed T produced by unwrapping and boxing the source value.

Nullable<T> and Null Reference Exception
What if we call Nullable<T>.HasValue? Since this hold a null value. Shouldn't this result in an error.



Basically assigning null to object type and Nullable<T> are not same. Let's look at the following code and generated IL. We can see that they don't result in similar IL. Assigning null to the former doesn't even actually assign null to the variable. How can the compiler do that as we have already established that Nullable<t> is not a reference type.



default Keyword & Nullable<T>
default keyword is generally used when we are writing algorithms based on Generics. This keyword is used to get the default value of a value or reference type used as type argument to a type or method based on Generics. For numeric types, it would be the default numeric value. Let's see what value would be used when we use this keyword with a Nullable<T> type.


The above code would initialize _taxRate with null, which is the default value of Nullable<T>. In the above code we can just use [ default(T) ] if we need to use the default value of the wrapped type. We can also use GetValueOrDefault() method as discussed above.

Operator Lifting
Operating lifting is the mechanism used when nullable types are used as operands. They can be applied with the same operators as the underlying types. If any operand is null, then the whole expression is evaluated to null. This is called null-propagation. Resharper is a great help here for obvious cases:



This excludes concat operation. Here we are using string.Format with a nullable holding null. Let's see how it is printed on the console:



In order to evaluate the expression, the candidate operators are lifted to be used from the wrapped type. Hence the term Lifting. The operands are converted to their non-nullable counterparts. After the evaluation of the expression, the result is converted back to nullable type. So Nullable<T> can use all the operators of the underlying type T, plus additional operators can also be applied including null-coelscing operator [??] as we discussed above.

Thursday, June 20, 2013

Referenced in 10 Laps Around Silverlight 5

Just realized that Michael Crump referenced one of my blog post in his famous ebook 10 Laps around Silverlight 5. The book was released in 2011. The ebook can be downloaded free from here:

http://www.silverlightshow.net/ebooks/10laps_silverlight5.aspx

Enjoy!

Partial Applications using Currying & Function Compositions

C# introduced Func and Action delegates in .net framework 3.5. They are part of C# 2 language specification. It also introduced expression trees and lambda expression in the same framework version. It's amazing how technologies evolve and new technologies are built on top of the existing ones. They were built on top of Generics introduce in .net framework 2.0. In this post we are going to discuss how we can use these constructs to introduce currying, partial applications and functional composition.

Previously we have discussed how we can use dynamic call site to avoid compile time check for arithmetic operations in a generic type. You can find the discussion here [http://www.shujaat.net/2012/05/c-generics-arithmetic-operators.html]. We will be assuming that calculator has been provided to us as a third party library and we will be deriving more meaningful operations (functions) from the existing methods using functional approaches. Referential transparency is the essence of functional programming providing side-effect free operations. We discussed about purely functional approach here [http://www.shujaat.net/2012/07/design-guidelines-purely-functional.html]

Currying
Currying is a transformation technique in functional programming space. It is used to convert a function with multiple parameters to a set of functions with single parameter each. It can be used to create refined algorithms where partial list of arguments can be provided to create more sophisticated operations. The nomenclature is in the honor of Haskell Curry, the creator of Haskell functional programming language. It must be remembered that the end-result of a curried version of a function should exactly be the same as its non-curried counterpart.

C# 4.0 provided optional parameters. These are the parameters with default values. If the caller doesn't want to pass arguments to those parameters then the default value is used instead. They are always at the end of the parameter list. This is opposite to the partial application case as the fixed arguments here. They would be the first parameters to a function. You might want to use these two concepts together as well. We can introduce the support for currying a function using extension methods. We can introduce these extension methods for the required overloads of Func and Action delegates. The following is an example of currying where we are converting a method with two parameters to a set of methods with single parameters.

The above code uses the Closure and Captured variables. Being a captured variable, operand1 is available inside the inner lambda expression. The whole feature is available due to closure where the life time of the context of lambda expression declaration is upgraded to the life time of expression itself allowing the use of captured variables. Actually we have discussed about closures in the context of async code execution using Dispatcher in WPF applications.

Partial application is about fixing the arguments to a function. An application has historically been referred as a set of functions. So the remaining method with some fixed arguments is referred as partial application. So we first obtain the curried version of a function and then provide fixed arguments. Wikipedia has a quick reference about the concept.

Now let's see how we can use currying and partial applications to create more meaningful operations. In the example below we are using the methods provided by the above Calculator class to derive calculatePlusPlus, calculateAdditiveInverse and calculateMultiplicativeInverse functions. Here we are fixing first arguments to the curried versions of the delegates based appropriate to the operation e.g. for calculatePlusPlus we can fix the first argument to 1.

The output of the above code would be as follows:



We are certainly not restricted to use only Func delegates. We can also use Action delegates if needed. Lets' create an extension method which curries an Action delegate with two parameters to a set of Action delegates. Don't get confused with the combination of Func and Action delegates being used here. You can just be concerned about the last Action, which refers to the actual operation. The rest of the design is just to curry it.

Here we are introducing a lambda statement to print a message specified number of times. We are assigning the statement to an Action delegate. After currying the delegate, we are creating a partial application printThreeTimes by fixing the first argument to 3. This would cause the message to be printed three times.

I can understand you might be thinking if this is possible to UnCurry a curried function or not. Actually we can do that too. Below is the extension method accepting a curried Func delegate and returning an Un-Curried version of the delegate.

Here we are UnCurrying a curried version of divide operation from Calculator. We are then calling it using the specified arguments as follows:

Functional Composition to construct data / processing chains
Functional composition is the method where multiple functions are used in a serial fashion in such a way that the output of one method serves as the input of other method.This can be used to build processing chains. I am deliberately not calling it a pipeline because a pipeline should have producer / consumer based intermediate queues. We have discussed how we can implement processing pipelines during the discussion of TPL Dataflow. You can find the discussion here: [http://www.shujaat.net/2013/05/processing-pipelines-with-tpl-dataflow.html].

The processing chain created through functional composition can also be specified using extension methods. The following defines an extension method on a Func delegate with two parameters. After getting the result from the first, it just passes the results to the second Func delegate. Obviously the next element in the chain should have only one parameter. After the second element completes, it just returns the result to the caller.

In the following code we are providing support of calculating percentage using the available Divide and Multiply methods from Calculator. As we know percentage of a part in a total is computed as [ percentage = (part / total) * 100 ]. So we need to obtain a curried version of Multiply & fix the first argument to 100 and use it after dividing part by the total.

It is also possible that the processing chain is not returning any data back to the caller. In this case, all the elements in the chain do pass the required data to the next element in the chain. The last element just uses the data for its purpose and nothing returns back to the caller. We can implement this using chain of Func delegates with an Action delegate at the end.

Let's see how we can use it to print the percentage of a part. Here we are using the function composition created for calculating the percentage. We are adding an element to construct a message. At the end we are using a processing element to print the message to the console. For the last element, we are using the curried version of printMessage with first argument fixed as 1.

The output of the above code would be as follows:


Saturday, June 8, 2013

Enterprise Scheduling using Quartz.net

Quartz.net is an open source enterprise class scheduler. It is available with Apache License V2.0. It is a port of Quartz from Java community. It can be used as a separate windows service. It can also be hosted inside your application providing scheduling of jobs easier. For a UI application, you can use different schedulers to poll the servers. But the main usage should be for server applications running schedules and batches. This post should serve an introduction of common features of Quartz.net in order to get a feel of the API including the main features and their usage.

Using Quartz.net
Quartz can be used as Hosted in your application. We can also use it as a separate process including hosted in a Windows Service. For our example we can use it as a scheduler hosted in our simple console application. Quartz is available as a Nuget package. You can use it when you want to host the scheduler in your application. You can also download the libraries package from Sourceforge.net.



Main Scheduling API Objects
If we take a moment to think about a scheduling API, there should be a main Scheduler which should be used to build and execute schedules. Each schedule should be a job executed based on a certain pre-configured trigger mechanism. So there are three types of objects. They are Scheduler, Job and Triggers. Quartz also follow the same conceptual model.



Quartz uses Triggers for "When to do" and Jobs for "What to do" needs". We also need a scheduler to register these jobs and triggers. The triggers would notify the scheduler when they are activated, which results in execution of jobs by the scheduler. There might be more than one jobs executed in a schedule. Similarly, we might need to execute the same job as a result of activation of more than one triggers.



Quartz Jobs
As we discussed above, a job is simply an operation that we need to execute when a trigger is saturated. Quartz Job's interface is just IJob interface which just defines a contract for Execute operation. The jobs might be interrupt-able.



Following is a simple job which just prints the firing time on the console. It implements IJob interface. The firing time can be obtained from IJobExecutionContext passed as argument to Execute method.

Quartz has also provided implementation of some of the most frequent cross-cutting jobs. The details are as follows:



Quartz Triggers
The scheduled job must be triggered based on some triggering event. All Quartz trigger interfaces are based on ITrigger.



The base trigger interface is inherited by a number of child interfaces providing more meaningful triggering details.



These interfaces are implemented by concrete classes in System.Impl.Triggers namespace. Here AbstractTrigger is the base class of these triggers. The available triggers are ConTriggerImpl, SimpleTriggerImpl, DailyTimeIntervalTriggerImpl and CalendarIntervalTriggerImpl.



Creating Schedules
As we discussed above, Triggers and Jobs can be used to create schedules. They can be registered with a Scheduler. Here Scheduler works as a Mediator which executes a Job based on saturation of a trigger's event. Job are specified using JobDetailImpl. This is a momento which would be used for persistence of jobs and their respective schedules. Here we have used an instance of MessageJob, defined above, and used it with CalendarIntervalTriggerImpl to create a schedule using Scheduler. As per Quartz documentation, every time a scheduler needs to execute the job, it creates a new instance of IJob. It then garbage collects the object later on.

The above code would result in the following output:



Calendars for Trigger's Exclusion List
Quartz provides a comprehensive list of calendars. These calendars allows us to provide exclusion list. All of them implement ICalendar through inheriting BaseCalendar. A calendar is associated with a trigger to provide exclusion list the the trigger inclusion list. A Calendar can be based on another ICalendar through CalendarBase property, which allows us to specify more sophisticated exclusion lists. It is easy to check if a given time is part of exclusion list through IsTimeInluded method of ICalendar. We can also get the next scheduled time after given time. In this case it just returns the next scheduled time if it is not in the exclusion list.



All the required calendars must be registered with the scheduler. A Trigger specifies the name of the calendar registered by the scheduler to provide the exclusion list. In the below code, we are just adding the calendar details to the trigger to include the exclusion details. The we are registering the actual calendar object with the key name with the scheduler. Here a DailyCalendar is use with start and ending time specified in 24-hour format.


Passing Data for Job Execution
As we discussed above, the scheduler uses JobFactory to create an instance of the IJob implementation. So we cannot directly instantiate the members of the job. Quartz provides JobDataMap for just that purpose. We can add key / value pairs to the JobDataMap associated with the JobDetail or Trigger. The JobDataMap values are assigned to the members of job after instantiation if key is same as one of the properties of Job. In the following example we are adding the JobDataMap to a JobDetail and Trigger. It must be remembered that multiple job details might use the same job with different JobDataMap, which can be used in other schedules.

Now we can modify the MessageJob to use the values passed by the scheduler. JobExecutionContext can be used to get the JobDataMap from Trigger and JobDetail. There is also a convenience JobDataMap which consolidates the values from both JobDataMap(s) where trigger ones override the JobDetails values. You can also notice additional attributes with the Job Definition. PersistJobDataAfterExecution would keep the new values added to the JobDataMap for JobDetails after job execution. DisallowConcurrentExecution avoids multiple execution of the job with the same JobDetails.


The above code would result in the following output. Please notice how JobDataMap values are injected into the properties.



Trigger Set for One Job
Quartz Scheduler allows the same JobDetails to be scheduled with multiple trigger. In the following code we are adding a SimpleTriggerImp and CronTriggerImpl. We are creating a JobDetails for MessageJob and then creating a schedule. Here CronTriggerImpl allows us to schedule based on the Cron expression as in Unix's Cron scheduler.


Here ISet<T> is the interface type provided by Quartz. The implementations including TreeSet<T>, HashSet<T> and ReadOnlySetISet<T>.



Quartz Listeners
Quartz provides listeners to allow execution of actions based on certain events on job, schedule and trigger. There are three types of interfaces IScheduleListener, ITriggerListener and IJobListener for listening of events on schedule, triggers and jobs. There are abstract implementation of these interfaces including TriggerListenerSupport, ScheduleListenerSupport and JobListenerSupport. These abstract implementations are mostly for providing convenience to implementers of the listener interfaces with protected constructors and virtual members. This makes it easier as we just need to override the required members.



There are also BroadCast implementations of these interfaces including BroadCastJobListener, BroadCastSchedulerListener and BroadcastTriggerListener. This is based on Composite pattern where multiple listeners can be added to these broadcast listener's instance. We just need to discuss an instance of this broadcast listener with scheduler instead of requiring the individual listeners' registrations.



In the following example, we are using JobListnerSupport to create a custom Job listener. Here we just want to print the job details to console after they are executed by overriding JobExecuted method.


All listeners must be added to scheduler's ListenerManager. It includes, job, trigger and scheduler listeners.


The above code would result in the following output:



Quartz Builder API
In order to make developer's lives easier, Quartz provides several builder interfaces to create schedules, triggers and jobs. In the following code, we are using TriggerBuilder, JobBuilder and ScheduleBuilderto create a schedule. These are fluent interfaces. Look at how easy it has become to create a trigger, assign job details to it, specify schedule using a schedule builder, provide exclusions using calendar. Since a job has already been assigned to the trigger, we don't need to specify it again for creating a schedule. We need to add the job details to the scheduler separately.


In the above example, we used SimpleScheduleBuilder. There are also additional scheduler builders available to create the scheduler specific to your needs.



Quartz Exceptions
It is OK to create a job with no associated trigger assigned. The job details would just be dormant in the scheduler. But Scheduler throws SchedulerException if we add a trigger which would never get fired.



Quartz also provides other exception types. The hierarchy is as follows:



Further Readings: