Monday, December 8, 2008

C# enums and aliases - weird behaviour

Jon has been playing with enum values, and fond some apparently inconsistent behaviour from the compiler.

Notes from this example:

  • If you’re going to set an enum value, set all of them.

  • If the value aligns to a table or values used in the database, always set all of the values in the enum.

Also, notice how TestA becomes an alias of TestB, yet TestE becomes an alias of TestD, see how this appears to be reversed.

I was quite surprised with this but it seems it has something to do with TestB being set prior to TestE being set.

using System;
namespace TestConsoleApp
   class Program
      enum TestType
         TestA, // TestA is an alias of TestB
         TestB = 0,
         TestE = 2, // TestE is an alias of TestD
         TestG = 6,
      static void Main(string[] args)
         TestType dc1 = TestType.TestB;
         if (dc1 == TestType.TestA)
            // since TestA is just an alias to TestB this works.
            Console.WriteLine("No compile warning is bad.");
         int a = (int)TestType.TestA; // alias of TestB
         int b = (int)TestType.TestB;
         int c = (int)TestType.TestC;
         int d = (int)TestType.TestD;
         int e = (int)TestType.TestE; // alias of TestD
         int f = (int)TestType.TestF;
         int g = (int)TestType.TestG;
         int h = (int)TestType.TestH;
         Console.WriteLine("{0} - {1}"TestType.TestA, a); // Prints 'TestB – 0'
         Console.WriteLine("{0} - {1}"TestType.TestB, b); // Prints 'TestB – 0'
         Console.WriteLine("{0} - {1}"TestType.TestC, c); // Prints 'TestC – 1'
         Console.WriteLine("{0} - {1}"TestType.TestD, d); // Prints 'TestD – 2'
         Console.WriteLine("{0} - {1}"TestType.TestE, e); // Prints 'TestD – 2'
         Console.WriteLine("{0} - {1}"TestType.TestF, f); // Prints 'TestF – 3'
         Console.WriteLine("{0} - {1}"TestType.TestG, g); // Prints 'TestG – 6'
         Console.WriteLine("{0} - {1}"TestType.TestH, h); // Prints 'TestH – 7'
         TestType type = (TestType)Enum.Parse(typeof(TestType), "TestA");
         Console.WriteLine("{0} - {1}", type, (int)type); // Prints 'TestB – 0'


Friday, October 24, 2008

SQL Server - Date Time accuracy

This does not work as expected:

SET @day1159pm = DATEADD(millisecond,-1,DATEADD(day,1,@dayMidnight))

If @dayMidnight was 1/1/01 then @day1159pm will equal 2/1/01. It seems that it doesn’t keep DATETIME with enough precision to subtract 1 millisecond. Use this instead:

SET @day1159pm = DATEADD(millisecond,-1,DATEADD(day,1,@dayMidnight))


The reason for this is provided in the Transact-SQL reference:

Date and time data from January 1, 1753 through December 31, 9999, to an accuracy of one three-hundredth of a second (equivalent to 3.33 milliseconds or 0.00333 seconds). Values are rounded to increments of .000, .003, or .007 seconds, as shown in the table.


Rounded example

01/01/98 23:59:59.999

1998-01-02 00:00:00.000

01/01/98 23:59:59.995,
01/01/98 23:59:59.996,
01/01/98 23:59:59.997, or
01/01/98 23:59:59.998

1998-01-01 23:59:59.997

01/01/98 23:59:59.992,
01/01/98 23:59:59.993,
01/01/98 23:59:59.994

1998-01-01 23:59:59.993

01/01/98 23:59:59.990 or
01/01/98 23:59:59.991

1998-01-01 23:59:59.990

Wednesday, August 6, 2008

Memory leak in FAXCOMEXLib

Yuan has been working on a fax / e-mail / SMS etc server we use internally and also give to our clients, and has come across some strange behavior in Microsoft's COM wrapper to the fax console.

Recently I have been working on our communications toolset – which manages our SMTP, FTP, SMS and Fax sending..

The application has 2 major components: An ASP.NET application to view, search and manage messages and message batches being sent; And a windows service which periodically queries the database to send any queued messages.

Each of the message types is run in its own thread with SMTP, FTP and SMS working for years with no problem.

However, a few weeks after deploying the new Fax component, we began to receive “Out of memory” log traces.

After a little investigation, I narrowed down the problem to a FAXCOMEXLib memory leak. Read below fo the details:

To queue a fax, we use FAXCOMEXLib - which is a Microsoft COM wrapper to the windows fax console. Once the server sends a fax to the console, the fax thread polls to check whether the fax job terminated successfully. The code we use to poll the fax queue is shown below:



            iterator = faxServer.Folders.OutgoingArchive.GetMessages(FAX_QUEUE_DEFAULT_PREFETCH) ' This routine exhibits the memory leak


            For i As Integer = 1 To filesCount


               If Not iterator.Message Is Nothing Then

                  Dim FaxRecipient As Recipient = GetRecipientForFax(iterator.Message.Id)


                  '     Recipient will be null if the fax job was added to the fax server

                  '     by anything other than this service.


                  If Not FaxRecipient Is Nothing AndAlso FaxRecipient.Status <> FaxRecipient.Statuses.sent.ToString() Then

                     ' Acknowledge fax has been sent



                     Dim LogMessage As String = String.Format("CR_ID: {0} Fax sent successfully", FaxRecipient.ID)

                     Log.write(MaxSoft.Common.Log.LevelEnum.INFO, Me, LOG_AREA, LogMessage)

                  End If


               End If






            If Not iterator Is Nothing Then


            End If

            iterator = Nothing

            faxServer = Nothing

         End Try

As you can see in the code above, we iterate through the files in the OutgoingArchive folder of the fax console and match the id with the id stored in our message queue. Unfortunately, every time we call the faxServer.Folders.OutgoingArchive.GetMessages the Microsoft COM dll leaks memory. With a little more research, we determined that if the ArchiveFolder was empty, there was no memory leak, and that the size of the memory leak was directly proportional to the number of files in the FAXCOMEXLib folder.

Once tracked down, we tried a number of options to dispose the object correctly including using Marshal.REleaseComObject, but with no luck. It appears that the problem is not in the disposal of the FAXCOMEXLib object, but in one of the underlying private objects or code used by FAXCOMEXLib. Since these objects are not exposed via COM, we cannot code around the memory leak.

After briefly considering restarting the windows service periodically I went and had a cup of coffee, and the solution came to me. Don't use FAXCOMEXLib.

Since I know that FAXCOMEXLib persists the completed faxes as a physical file in a known location, and that the filenames are the same as the id, we can get to the completed faxes by going through the physical filesystem rather then through the FAXCOMEXLib queue.

Here is the refactored code with no leaking:


            Dim unsentFaxRecipientTrackings As New RecipientDeliveryTrackingList

            unsentFaxRecipientTrackings.loadFromSQL("SELECT tr.* FROM tblCommRecipientDeliveryTracking tr JOIN tblCommRecipient cr ON tr.CR_Id = cr.CR_Id WHERE cr.CR_Status = 'sending'"Nothing, CommandType.Text, -1)

            If unsentFaxRecipientTrackings.Count > 0 Then

               Dim archiveFolder As String = faxServer.Folders.OutgoingArchive.ArchiveFolder

               Dim allFiles As New Hashtable

               ' Load files into hashtable

               For Each fileName As String In System.IO.Directory.GetFiles(archiveFolder)

                  ' The file name format is LoginUserID$MessageID.tif

                  If fileName.Split("$").Length > 1 Then

                     fileName = fileName.Split("$"c)(1).Replace(".tif""")

                     allFiles.Add(fileName, fileName)

                  End If


               For Each FaxTracking As RecipientDeliveryTracking In unsentFaxRecipientTrackings

                  If allFiles.ContainsKey(FaxTracking.FaxDocId) Then

                     Dim recipient As New recipient

                     recipient.loadFromSQL("SELECT * FROM tblCommRecipient WHERE CR_Id = " & FaxTracking.CR_Id.ToString(), Nothing, CommandType.Text, -1)

                     ' Might be a chance that someone remove the record from database manually

                     If Not recipient.isNew Then



                        Dim LogMessage As String = String.Format("CR_ID: {0} Fax sent successfully", FaxTracking.CR_Id)

                        Log.write(MaxSoft.Common.Log.LevelEnum.INFO, Me, LOG_AREA, LogMessage)

                     End If

                  End If


            End If




         End Try

So, rather than ask FAXCOMEXLib to return a list of messages, I go to the file system and get them myself. Happily FAXCOMEXLib aids with this approach, and will happily provide the physical path to Folders.OutgoingArchive.ArchiveFolder.

Since failed messages do not go into the OutgoingArchive.ArchiveFolder, we do not need any additional fax metadata.

Result: After running for 1 week, no memory leak found.

Tools: .NET Memory Profiler

Friday, July 4, 2008

QUT adds up the benefits of SmartaPay

QUT announced today that Queensland based company SmartaPay had completed installation of the SmartaPay solution. For the first time students will be able to pay ALL their University and Student fees through a new custom designed payment solution built by SmartaPay, specifically for QUT

QUT and SmartaPay have spent the past 18 months designing and building the payment solution to ensure that the functionality of SmartaPay’s solution will provide all QUT businesses with
• Shopping Cart
• Online Catalogue
• Pay Now and Pay Later
• Invoice Generation and Payment
• Reconciliation

“We made a decision two years ago to replace our previous student payment option with a more sophisticated and integrated payment solution" said Terry Leighton, Director of Corporate Finance at Queensland University of Technology. "This is just another step in ensuring our University stays at the forefront of Technology advancements as we provide the optimum solutions for our current students and our future students.”

QUT, ‘the University for the Real World’, is one of Australia’s largest universities servicing in excess of 35,000 students across four campuses. QUT aims to strengthen its distinctive national and international reputation by combining academic strength and practical engagement with the world of the professions, industry, government, and the broader community.
Tony Irvine, Chief Operating Officer of SmartaPay, “We’re excited to be chosen by QUT as a partner in building the University’s new Payment Solution. This solution will be a market leader in the education arena and an achievement that both QUT and SmartaPay will be justly proud of.”

SmartaPay is a payment facilitator for Universities and Private Schools Australia wide. SmartaPay’s solution will cater for all types of payers and payment channels, from online internet payments to paying fees over the counter or by cheque, and deliver a single interface to QUT.

“SmartaPay’s business is built on a foundation of providing unique and fully integrated end to end billing and payment solutions” explains Dorian Borin, Chief Information Officer of SmartaPay. “We have a proven track record in providing customized solutions to all our SmartaPay clients”.

For more information please contact SmartaPay on 07 5575 7422 or email

Tuesday, June 17, 2008

Simple dynamic compiling to mock a Console Application

Toby has written a follow up to ‘Spawning a console application and tracking its Standard Output’ found here.

This article deals with unit testing an application that relies on interaction with a console application.

The problem is making the console application behave in a specific manner for the unit tests. For instance, simulating network issues, file access issues, strange multiple file issues and so on can be a right pain in the neck. Throw in other factors such as the speed of the unit tests with a large console application running, trying to automate the setting up of an entire staged environment and invoking a complex, production only and sometimes cranky console application makes this a challenge I'm keen to avoid.

A simpler approach is to mock the external console application, so that the mocked application returns exactly what you need to test the behaviour of your own application.

We have a few options to solve this, along with the negatives of the approach:
  • Making a simple batch file to accept input, and spit out the correct text on demand
    - Limited interaction potential, validation etc
    - No access to .NET goodness
  • Writing a bunch of .NET console apps which are compiled as part of the solution
    - Too many projects to maintain
    - Hard coded and not flexible for different environments
  • Dynamically generate console apps on the fly with environmental changes as needed
    - Sounds like a solution to me !!!

Writing the mocked console application

The following test code sample shows messages being written to Standard Output and files being created. You will need a separate mock application for each test that you need to perform.

namespace TestApplication
   class Program
      static void Main(string[] args)
         Console.WriteLine(@"Connected to made.");
         // put download files here
         StreamWriter sw = new StreamWriter(@"C:\DownloadFiles\File1");
         Console.WriteLine("Downloading file File1");
         sw.Write("Text Content of File 1");

         sw = new StreamWriter(@"C:\DownloadFiles\File2");
         Console.WriteLine("Downloading file File2");
         sw.Write("Text Content of File 2");

         Console.WriteLine(@"Closed Connection.");

Dynamically generating mock console applications on the fly

The source code for each test application is included in my solution as a string resource called TestScript. The source code is then compiled on the fly to an executable file. The code to compile the resource is:

Microsoft.CSharp.CSharpCodeProvider cscp = new Microsoft.CSharp.CSharpCodeProvider();
System.CodeDom.Compiler.CompilerParameters param = new System.CodeDom.Compiler.CompilerParameters();
param.GenerateExecutable = true;
param.OutputAssembly = @"c:\App.EXE";

foreach (Assembly assembly in AppDomain.CurrentDomain.GetAssemblies())
System.CodeDom.Compiler.CompilerResults results = cscp.CompileAssemblyFromSource(param, TestScript.Test1);

Because this works from text, there is also the opportunity to use string.Replace to manage paths specific to the testing environment.

Setting param.GenerateExecutable to true causes an executable to be generated instead of a dll. Put the file name to generate to in param.OutputAssembly.

The above code assumes that the assemblies required to be referenced are the same as the currently executing application. This is achieved via the for loop. This can be altered by adding specific assemblies to param.ReferencedAssemblies in addition to, or instead of this loop.

If you are in an experimenting mood, any compiler directives available via the compiler options in Visual Studio are available to you. Have a play with System.CodeDom.Compiler.CompilerParameters.CompilerOptions;

Monday, June 2, 2008

XP SP3 changes the Remote Desktop Connection command

After installing XP SP 3 you will need to use this to connect to the console session on a server:

mstsc /admin

Instead of

mstsc /console

Wednesday, May 28, 2008

Creating a service using a batch file

Sometimes we need a quick and dirty method of installing a windows service.

The official method via the DOS prompt is uses the SC command. This command is one of the most unusual command line tools I have seen for a while ... the syntax for the parameters is "xxx= yy" - the space after the = and the lack of space before the = sign is mandatory. Adding quotes around pathnames requires escaping a quote after the initial quote. SC CREATE also appears not to set the ERRORLEVEL parameter for the batch file.

This all makes SC a highly infuriating tool.

Because of this, some users prefer hacking the registry directly while others create WIX install packages (the preferred method, but a but hard for a quick and dirty test).

The code sample below shows a batch file that can easily be modified to add a service with a dependancy on the TCPIP service that will automatically start up when Windows cranks up. There's another one below that to delete the service, but unfortunately, you seem to have to reboot to really get rid of the service.

It uses a nifty method to get the path of the batch file.

It really needs to be rewritten to use Windows PowerShell.

@echo off

REM Installs A service
REM -----------------------------------------
REM Created by MaxSoft Group KB
REM (c) 2008

REM -----------------------------------------
REM Set up the specific variables
REM Assumes that the service is from the same directory as the batch file unless specified as the first parameter

SET PROGRAM_NAME=BroadcastEventService.exe
SET DISPLAY_NAME=StrataPay Broadcast Events

REM -----------------------------------------

PROMPT $f$f$f$f


REM Get the directory of the install without destroying the current directory with the new PUSHD command
pushd %0\..
rem cd /d %0\..

REM Now install the service
REM SC does not seem to set the errorlevel, resulting in commands
REM being executed regardless of the success of the SC CREATE.

@echo on
SC CREATE %SERVICE_NAME% binpath= "\"%INSTALL_DIR%\%PROGRAM_NAME%\"" displayname= "%DISPLAY_NAME%" depend= Tcpip start= auto



@echo off
echo ))))-----------------------------------------------
echo ))))An error occurred.
echo ))))-----------------------------------------------


The command to delete a service is shown below (Less generic but just as easy):

@echo off
REM Delete the service

NET STOP SP_BroadcastEvents
SC delete SP_BroadcastEvents

echo You will have to restart the machine to action this command

SQL Error handling

SQL Server is a great tool, but when you are handling large volumes of queries that run automatically, you need a good framework wrapped around each query to ensure that if something out of the ordinary happens, you not only know about it, but it does not adversely imact your data.

Below is the an excerpt from our SQL Server wiki

Critical Errors can cause unexpected results within a transaction. Actions taken before the critical error will be rolled back and the transaction closed. Remaining actions will be performed without a transaction wrappper.

To overcome this we have developed a template for scripts that provide a reasonable expectation of all or nothing output for scripted updates.

This template is recommended for changes to existing schema particularly if data is present in the table being modified.

The template consists of a generic header and footer for the transaction handling. Each command is followed by a generic code block to ensure a transaction remains in effect.

FROM tempdb..sysobjects
WHERE id = Object_id('tempdb..#tmpErrors'))
DROP TABLE #tmperrors


CREATE TABLE #tmperrors (
error INT)


SET xact_abort on


SET TRANSACTION isolation LEVEL serializable






IF @@ERROR <> 0


INSERT INTO #tmperrors




IF @@ERROR <> 0


INSERT INTO #tmperrors



FROM #tmperrors)


PRINT 'The database update succeeded'

PRINT 'The database update failed'


DROP TABLE #tmperrors





Tuesday, April 22, 2008


Dorian: Never really looked at XHTML before. All I know was you had to close your tags. I learned a little more today.

All tags have to be lower case e.g. <img .. instead of <IMG.

I had to convert some HTML 4.01 markup to XHTML 1.0. Here is what I did.

I used to check my code. It will take a URL, a file upload or you can cut and paste your source in. I just cut and pasted my source as the site wasn’t public and it was derived markup from ASP.Net 3.5.

At first seeing 50 errors was daunting but it wasn’t hard to trim down to perfection.

The bulk of the issues it found were unclosed <br> tags which should be <br/> and unclosed <img …> tags. Every upper case <IMG> tag generated lots of errors because it also complained about each attribute because upper case IMG is an unknown tag. Making the <IMG> tags lower case stripped off heaps of errors.

The other big error count creator is no quotes on attributes. E.g. <table cellspacing=0> instead of <table cellspacing=”0”>

The one that took me a little more time to absorb was the markup like this:









The inner divs are invalid. You can’t have a div in a span in XHTML. The probably was the inner divs needed to be full width blocks. The span was automatically generated by a standard ASP.Net control. The way to solve it was this:



<span style=”display:block;”>


<span style=”display:block;”>




Of course I used CSS rather than doing a ugly style attribute hack.

So that was my 10 mins self learning intro into HTML to XHTML conversion. Interestingly while it seems popular Thinking that XHTML is the better way to go for future proofing HTML 5 which is still in draft has support for both nice XML tagging and old style HTML using different doc types. They probably don’t want to break the world.

One thing I’m yet to solve is that this is invalid.

<a href=”http://somewhere” disabled=”disabled”>something</a>

There appears to be no way to disable a link from being clicked without javascript. ASP.Net generates this markup. The only work around it to render the disabled links as labels.

ASP.NET tips

From Dorian: Here is a summary of the tips I’ve come up with so that you can build websites that can be deployed with precompilation.

TIP: Don’t use duplicate class names

While developing ASP.Net will keep all your pages and controls in separate dlls. It groups some controls together into one dll if they are in the same folder but at other times is keeps them separate. I’ve not determine exactly how it chooses. With this partitioning you would not very lucky run into a problem with two classes in your web site with the same name. If you attempt to merge the site into one dll (using aspnet_merge) however it would fail. I recommend naming your classes based on the directory structure using an _ instead of a slash. That will keep class names unique.

TIP: Don’t assume .ascx or .master files exist

Whilst in a development environment the .ascx and .master files exist under the website. After precompilation they do not. Never assume that they exist. The reason for doing this would be dynamic user control loading. Dynamic user control loading should be done by attempting to load the class and caching its presence. You will need to catch an exception to detect failure. I’ve spent a couple of hours looking for a work around and there is none that I can see.

TIP: Don’t reference generated classes

The following shows examples of good and bad. Whilst the bad code will work while you develop you will not be able to deploy with precompiliation.

User Control Reference

Dim singleProductDisplay As ASP.bvmodules_controls_singleproductdisplay_ascx = DirectCast(row.FindControl("SingleProductDisplay"), ASP.bvmodules_controls_singleproductdisplay_ascx)


Dim singleProductDisplay As BVModules_Controls_SingleProductDisplay = DirectCast(row.FindControl("SingleProductDisplay"), BVModules_Controls_SingleProductDisplay)

Master Page Reference

If TypeOf Me.Master Is ASP.bvadmin_bvadminproduct_master Then
DirectCast(Me.Master, ASP.bvadmin_bvadminproduct_master).ShowProductDisplayPanel = False


If TypeOf Me.Master Is BVAdmin_BVAdminProduct Then
DirectCast(Me.Master, BVAdmin_BVAdminProduct).ShowProductDisplayPanel = False

ASP.Net Precompilation

ASP.Net precompilation has a variety of options. You can chose to leave your aspx and ascx files in place and precompile the rest. This lets you change the HTML of those controls after deployment without recompilation. This is the same as VS2003 did it. You can chose to precompile everything. And you can go one step further to merge all the dlls into one big dll. For both VS2005 and VS2008 you can install a Web Deployment Project type into VS that will make it easier to do all this.

Thursday, April 10, 2008

Validating web service content against an XSD

Rodney has been working with a SOA implementation at one of our clients. As a result, we have changed how we deal with web services, to provide additional clarity and robustness to the WS-I standards that Microsoft utilises.

Recently we decided to split our web service definitions into 3 distinct files:

  1. An XSD to define the structure of the data (the parameters and returned data).
  2. A WSDL to define the operations that the web service will make available.
  3. A second WSDL to define the bindings (such as SOAP) used to communicate with the web service.

We create a C# interface from these three files using wsdl.exe. We create a web service in our web site, add this interface into our library and tweak the code to make it implicitly implement the interface.

This is quite easy and works very effectively. There is however one overlooked problem with this approach. Although the interface and associated classes were generated from an XSD and WSDL, the XSD is not strongly enforced from the web service. For example, the XSD may define a string maximum length:

   <xsd:simpleType name="String20">
      <xsd:restriction base="xsd:string">
         <xsd:maxLength value="20"/>

The problem is that the web service created in this manner will allow a client to pass a string of length > 20. Not the desired outcome, since the XSD should be doing the work for us.

The reason this happens is that the web service doesn’t reference the XSD, nor does it define any attributes on a property to restrict the length. When you look at the schema automatically generated from the web service (http://myserver/mywebservice.asmx?schema=schema1), it uses the class structure to define the schema and therefore does not have the restrictions.
Validating all the values using code can be quite cumbersome and might not necessarily match the validation of the XSD. If the XSD is changed, then the code will also need to be altered to reflect the changes made to the XSD. Again, not a desired outcome.

Another approach is to validate the input data against the XSD from within the body of the web method. This can cause some problems since every individual parameter needs to be serialized to XML and compared to the XSD. Again, if the XSD changes and a new parameter is added, then the programmer has to remember to add the new parameter and ensure that it is validated against the schema. If this is a really large web service, on a high load site, this can be a significant overhead.

Enter SoapExtensions.

SoapExtensions allow you to perform validation on the incoming XML string before it is de-serialised into CLR objects.

Once Only Setup

SoapExtensions require a once only setup to create custom attribute classes stored in your reusable library.

A number of classes derived from the SoapAttribute abstract class need to be created. Some will be used to decorate the web service class to define the schemas while others are required to decorate the web methods to force validation. Coded correctly, these allow you to pass in a specific XSD or a directory full of XSD files to validate against at runtime.

In addition you will need to create a ValidationExtension class to perform the actual validation using the .NET XML library on the incoming XML message stream and XSD collection.
The full detail of the implementation can be found at

Web Service Specific Setup

Armed with these new classes, validation on the incoming XML stream is now achieved via the following simple steps:

1. Setup the web.config to use the new ValidationExtension class

        <add type="MaxSoft.Web.Services.Validation.ValidationExtension, MaxSoftDllpriority="1" group="High" />

2. Decorate your class to point to the XSD file(s)

If you follow the msdn article above you will be able to point to a specific XSD, or a directory of XSD schema files. Both are shown below.


3. Decorate your web method to trigger the actual validation

[Validation(CheckAssertions = false)]

4. Optionally you can add more complex validation to compliment the XSD

If CheckAssertions is set to true, you can provide additional validation logic which cannot be expressed in the XSD schema. An example is shown below:

[Assert("(//t:length > //t:width)", "Length must be greater than width")]

Although the capability is there, we don't use this feature. In fact, we have set the default value of CheckAssertions to false.

Not only does CheckAssertions add overhead to the validation, but it stores important business logic in an attribute of the web method rather than in the entity library, obviously poor practice. This additional business logic should be in a library for reuse, testability and for a raft of other reasons.


This code in Microsoft's was written in 1.1 and you need to make some significant changes for it to run against newer versions of .Net as many of the classes are now marked as obsolete.