Yuan has been working on a fax / e-mail / SMS etc server we use internally and also give to our clients, and has come across some strange behavior in Microsoft's COM wrapper to the fax console.
Recently I have been working on our communications toolset – which manages our SMTP, FTP, SMS and Fax sending..
The application has 2 major components: An ASP.NET application to view, search and manage messages and message batches being sent; And a windows service which periodically queries the database to send any queued messages.
Each of the message types is run in its own thread with SMTP, FTP and SMS working for years with no problem.
However, a few weeks after deploying the new Fax component, we began to receive “Out of memory” log traces.
After a little investigation, I narrowed down the problem to a FAXCOMEXLib memory leak. Read below fo the details:
To queue a fax, we use FAXCOMEXLib - which is a Microsoft COM wrapper to the windows fax console. Once the server sends a fax to the console, the fax thread polls to check whether the fax job terminated successfully. The code we use to poll the fax queue is shown below:
Try
faxServer.Folders.OutgoingArchive.Refresh()
iterator = faxServer.Folders.OutgoingArchive.GetMessages(FAX_QUEUE_DEFAULT_PREFETCH) ' This routine exhibits the memory leak
iterator.MoveFirst()
For i As Integer = 1 To filesCount
iterator.MoveNext()
If Not iterator.Message Is Nothing Then
Dim FaxRecipient As Recipient = GetRecipientForFax(iterator.Message.Id)
'
' Recipient will be null if the fax job was added to the fax server
' by anything other than this service.
'
If Not FaxRecipient Is Nothing AndAlso FaxRecipient.Status <> FaxRecipient.Statuses.sent.ToString() Then
' Acknowledge fax has been sent
FaxRecipient.ActMarkAsSent()
FaxRecipient.save()
Dim LogMessage As String = String.Format("CR_ID: {0} Fax sent successfully", FaxRecipient.ID)
Log.write(MaxSoft.Common.Log.LevelEnum.INFO, Me, LOG_AREA, LogMessage)
End If
Marshal.ReleaseComObject(iterator.Message)
End If
Next
Finally
faxServer.Disconnect()
Marshal.ReleaseComObject(faxServer.Folders.OutgoingArchive)
Marshal.ReleaseComObject(faxServer)
If Not iterator Is Nothing Then
Marshal.ReleaseComObject(iterator)
End If
iterator = Nothing
faxServer = Nothing
End Try
As you can see in the code above, we iterate through the files in the OutgoingArchive folder of the fax console and match the id with the id stored in our message queue. Unfortunately, every time we call the faxServer.Folders.OutgoingArchive.GetMessages the Microsoft COM dll leaks memory. With a little more research, we determined that if the ArchiveFolder was empty, there was no memory leak, and that the size of the memory leak was directly proportional to the number of files in the FAXCOMEXLib folder.
Once tracked down, we tried a number of options to dispose the object correctly including using Marshal.REleaseComObject, but with no luck. It appears that the problem is not in the disposal of the FAXCOMEXLib object, but in one of the underlying private objects or code used by FAXCOMEXLib. Since these objects are not exposed via COM, we cannot code around the memory leak.
After briefly considering restarting the windows service periodically I went and had a cup of coffee, and the solution came to me. Don't use FAXCOMEXLib.
Since I know that FAXCOMEXLib persists the completed faxes as a physical file in a known location, and that the filenames are the same as the id, we can get to the completed faxes by going through the physical filesystem rather then through the FAXCOMEXLib queue. Here is the refactored code with no leaking:
Try
Dim unsentFaxRecipientTrackings As New RecipientDeliveryTrackingList
unsentFaxRecipientTrackings.loadFromSQL("SELECT tr.* FROM tblCommRecipientDeliveryTracking tr JOIN tblCommRecipient cr ON tr.CR_Id = cr.CR_Id WHERE cr.CR_Status = 'sending'", Nothing, CommandType.Text, -1)
If unsentFaxRecipientTrackings.Count > 0 Then
Dim archiveFolder As String = faxServer.Folders.OutgoingArchive.ArchiveFolder
Dim allFiles As New Hashtable
' Load files into hashtable
For Each fileName As String In System.IO.Directory.GetFiles(archiveFolder)
' The file name format is LoginUserID$MessageID.tif
If fileName.Split("$").Length > 1 Then
fileName = fileName.Split("$"c)(1).Replace(".tif", "")
allFiles.Add(fileName, fileName)
End If
Next
For Each FaxTracking As RecipientDeliveryTracking In unsentFaxRecipientTrackings
If allFiles.ContainsKey(FaxTracking.FaxDocId) Then
Dim recipient As New recipient
recipient.loadFromSQL("SELECT * FROM tblCommRecipient WHERE CR_Id = " & FaxTracking.CR_Id.ToString(), Nothing, CommandType.Text, -1)
' Might be a chance that someone remove the record from database manually
If Not recipient.isNew Then
recipient.ActMarkAsSent()
recipient.save()
Dim LogMessage As String = String.Format("CR_ID: {0} Fax sent successfully", FaxTracking.CR_Id)
Log.write(MaxSoft.Common.Log.LevelEnum.INFO, Me, LOG_AREA, LogMessage)
End If
End If
Next
End If
Finally
faxServer.Disconnect()
Marshal.ReleaseComObject(faxServer)
End Try
So, rather than ask FAXCOMEXLib to return a list of messages, I go to the file system and get them myself. Happily FAXCOMEXLib aids with this approach, and will happily provide the physical path to Folders.OutgoingArchive.ArchiveFolder.
Since failed messages do not go into the OutgoingArchive.ArchiveFolder, we do not need any additional fax metadata.
Result: After running for 1 week, no memory leak found.
Tools: .NET Memory Profiler
