Avoiding File Locks and Cannot Access File Exceptions 

While working through some issues recently with some legacy code writen by someone else I was faced with a horrible case of a shared object that was writing to the file.  This was a "custom" logging implementation and opened a file for append, inserted the line, and then closed the file.  Well in times of heavy load the system would encounter errors such as "Cannot access ___ because it is being used by another process".  So in effect the file was either still open, or the lock was not yet released.  This post goes through a bit of the detail on how I resolved the issue.

The Issue

The main manifestation of this issue was with a series of very agressive log writes, 100+ calls in a row.  So for testing I created an application that would write out 2000+ entries one after another to simulate heavy load.  The previous developers had included a "retry" process to get around the issue.  If the file couldn't be opened, it pause 10 milliseconds then re-tried.  For a 5000 record set the app would average 1800 retries.  This process was clunky at best, and involved exception handling at every attempt.

My Fix

I decided to look at a locking scenario, and came up with the following minimized code.

   1:  Imports System.IO
   2:  Imports System.Threading
   3:   
   4:  PublicClass FileWriteTest
   5:  Private _filePath AsString = "C:\Testing\myfile.txt"
   6:  Private _fileMutex AsNew Mutex()
   7:   
   8:   
   9:  PublicSub WriteLine(ByVal sMsg AsString)
  10:  'Wait until free
  11:          _fileMutex.WaitOne()
  12:  Dim ofs As FileStream
  13:  Try
  14:                 ofs = New FileStream(_filePath, FileMode.Append, _
  15:                   FileAccess.Write, FileShare.None)
  16:  Dim oFile AsNew StreamWriter(ofs)
  17:   
  18:  Dim sLine AsString = sMsg & ControlChars.NewLine
  19:   
  20:                  oFile.Close()
  21:                  ofs.Close()
  22:  Catch e As Exception
  23:  Throw
  24:  EndTry
  25:  Finally
  26:  'Release the mutex to allow others to call
  27:              _fileMutex.ReleaseMutex()
  28:  EndTry
  29:  EndSub
  30:  EndClass

 Now, this fix was a very simple I created a Mutex inside of the class that actually did the file operations, and then used WaitOne() to ensure that I could be the only one accessing the code, then after all file operations were complete, including the closing of the file, I release the mutex.  This removed the need for a continual "retry" method as was previously used and improved performance by over 35%.

Now, the Mutex is a much more severe and costly operation than a SyncLock, but due to the nature of the code that I am working with, I couldn't get a SyncLock to work the way it needed to, thus the Mutex solution.

Conclusion

I hope that this example has helped someone out as I know when I was looking for solutions, I found it VERY hard to find any solution other than a "keep trying" approach, which to me just wasn't an option!  Feel free to share comments below, and if you have any specific implementation questions please feel free to post to the forum!

Posted by Mitchel on Thursday, January 22, 2009
 

Comments

Some comments...

1) if an exception is thrown, the file will not be closed and will remain opened until the garbage collector kicks in. Next attempts to write will fail, since the share mode is set to none. When GC comes in, it will close the file, then next attempts will succeed. It would be good to close the file in the finally block, or enclose the code in a "using" statement (C#, I hate VB...)

2) The sync approach should work fine. I'd suggest locking on the type (lock(typeof(MyWriter)), so that code executes in sync if there are many instances of the class. You could also make your Mutex static (Shared in VB I think) so that it is the same for all instances of the class.

3) Disk IO is really expensive when considering performance. It would be better to implement some sort of caching, then periodically flush the cache to the file. To the least, execute this on a separate thread.

By Etienne Richard on Monday, January 26, 2009 at 1:37 PM

All very great comments.

The lack of finally statements and closing of the file was from an accidential omission when posting the code.

As for the use of Sync, that would most likely work, the biggest issue is that in our implementation the log writing object is shared and only has one instance per app, and Sync was not working.

I also agree 100% that a caching/flushing method would be much better, but again, our specific requirements require that logs must go to disk right away, without any caching.

By mitchel.sellers@gmail.com on Monday, January 26, 2009 at 2:01 PM

Very cool! I have been intending to read this entry for a while now, as I have had similar issues in the past. Thanks for the example!

By Will Strohl on Thursday, February 12, 2009 at 10:42 PM

Name (required)

Email (required)

Website

CAPTCHA image
Enter the code shown above:

Content provided in this blog is provided "AS-IS" and the information should be used at your own discretion.  The thoughts and opinions expressed are the personal thoughts of Mitchel Sellers and do not reflect the opinions of his employer.

Friend of RedGate

www.datasprings.com - DotNetNuke ModulesICG

Click here for advertising information.

Content in this blog is copyright protected.  Re-publishing on other websites is allowed as long as proper credit and backlink to the article is provided.  Any other re-publishing or distribution of this content is prohibited without written permission from Mitchel Sellers.