Performance Matters: Choose Your Library Wisely

Recently, someone forwarded me a link to an article describing itself as an independent benchmark, designed to compare the performance of Java, C#, and VB.NET. The conclusion was that Java was faster than VB but slower than C#. This caused a lot of consternation among folks who weren't expecting to see a performance difference between VB and C# at all. Sure enough, it turned out that the difference was that the code in the comparison apps was not, in fact, the same. The C# code looked like this:

/**
 * Write max lines to a file, then read
 * max lines back in from file.
 */
static long io(int ioMax)
{
   long elapsedMilliseconds;
   startTime = DateTime.Now;

   String fileName = "C:\\TestCSharp.txt";
   String textLine = 
    "abcdefghijklmnopqrstuvwxyz1234567890
     abcdefghijklmnopqrstuvwxyz1234567890abcdefgh";
   int i = 0;
   String myLine = "";
   try
   {
     StreamWriter streamWriter = 
            new StreamWriter(fileName);
      while (i++ < ioMax)
      {
         streamWriter.WriteLine(textLine);
      }
      streamWriter.Close();

      i = 0;
      StreamReader streamReader = 
                 new StreamReader(fileName);
      while (i++ < ioMax)
      {
         myLine = streamReader.ReadLine();
      }
   }
   catch (IOException e)
   {
     System.Console.Write(e.Message);
   }

   stopTime = DateTime.Now;
   elapsedTime = stopTime.Subtract(startTime);
   elapsedMilliseconds = (long)elapsedTime.TotalMilliseconds;

   Console.WriteLine("IO elapsed time: " + elapsedMilliseconds + 
                     " ms with max of " + ioMax);
   return elapsedMilliseconds;
}

This code writes a long line of text to a file over and over, and then reads it. It's using a StreamReader and a StreamWriter, from the System::IO namespace. Now, from the same benchmark, here's the VB code:

Function IO(ByVal ioMax As Integer) As Long
   Dim milliseconds As Long
   startTime = Now
   Dim fileName As String = "C:\TestVB.txt"
   Dim i As Integer = 0
   Dim myString As String = _
       "abcdefghijklmnopqrstuvwxyz1234567890 _
        abcefghijklmnopqrstuvwxyz1234567890abcdefgh"
   Dim readLine As String
   FileOpen(1, fileName, Microsoft.VisualBasic.OpenMode.Output)
   Do While (i < ioMax)
      PrintLine(1, myString)
      i += 1
   Loop
   FileClose(1)
   FileOpen(2, fileName, Microsoft.VisualBasic.OpenMode.Input)
   i = 0
   Do While (i < ioMax)
      readLine = LineInput(2)
      i += 1
   Loop
   FileClose(2)

   stopTime = Now
   elapsedTime = stopTime.Subtract(startTime)
   milliseconds = CLng(elapsedTime.TotalMilliseconds)
   Console.WriteLine("I/O elapsed time: " & milliseconds & _
                     " ms with a max of " & ioMax)
   Console.WriteLine(" i: " & i)
   Console.WriteLine(" readLine: " & readLine)
   Return milliseconds
End Function

This code does the same thing: Writes text to a file and then reads it back, but it's using FileOpen and the like. These are "compatibility" functions for Visual Basic programmers: They're in the Microsoft::VisualBasic namespace and you have to add a reference to Microsoft.VisualBasic.dll to use them. VB.NET apps get the reference automatically, which makes life simpler for VB 6 programmers who are moving to VB.NET. There's something else you should know about these backward-compatible functions: They're slow. Much slower than the StreamReader and StreamWriter combination.

To prove it, I decided to convert both these code snippets into Managed C++, so there's no language issue in the comparison. I wrote a console application to keep everything simple, and put the timing and exception-handling code in the main(), then called separate functions for the actual I/O loops. The main reads like this:

int _tmain()
{
    DateTime startTime, stopTime;
    TimeSpan elapsedTime;
    long elapsedMilliseconds;

    startTime = DateTime::Now;
    try
    {
        IOStreamWriter::main();
    }
    catch (IOException* e)
    {
        Console::WriteLine(e->Message);
    } 
    stopTime = DateTime::Now;
    elapsedTime = stopTime.Subtract(startTime);
    elapsedMilliseconds = (long)elapsedTime.TotalMilliseconds;

    Console::WriteLine("StreamWriter elapsed time: {0} ms with
                        max of {1}" ,
                        __box(elapsedMilliseconds), __box(ioMax));

    startTime = DateTime::Now;
    try
    {
        IOFileOpen::main();
    }
    catch (IOException* e)
    {
        Console::WriteLine(e->Message);
    }
    stopTime = DateTime::Now;
    elapsedTime = stopTime.Subtract(startTime);
    elapsedMilliseconds = (long)elapsedTime.TotalMilliseconds;
    Console::WriteLine("FileOpen elapsed time: {0} ms with max
                        of {1}" , 
        __box(elapsedMilliseconds), __box(ioMax));
}

(To be honest, it's a little more complicated than this, because on each run of main(), I want to run only one of these blocks to be sure they don't interfere with each other, but I'm leaving that out here to keep it simple.)

The C# block reads like this in C++:

int i = 0;

String* fileName = "C:\\TestStreamWriter.txt";
String* textLine = 
        "abcdefghijklmnopqrstuvwxyz1234567890
         abcdefghijklmnopqrstuvwxyz1234567890abcdefgh";
String* Line = "";

StreamWriter* streamWriter = new StreamWriter(fileName);
while (i++ < ioMax)
{
    streamWriter->WriteLine(textLine);
}
streamWriter->Close();

i = 0;
StreamReader* streamReader = new StreamReader(fileName);
while (i++ < ioMax)
{
    Line = streamReader->ReadLine();
}