While driving home from work today, I was thinking of how to write an algorithm in C# 2.0 that takes a string representing a multi-line text file, and return the lines, one at a time.

I know Mono's System.IO.StringReader class can do this, but I was thinking there must be a more efficient way of doing it. Here's what I came up with:

 public class LineReader
 {
     private string _source;
 
     public LineReader(string src)
     {
         _source = src;
     }
 
     public IEnumerator GetEnumerator()
     {
         bool finished=false;
         int nextChar=0;
         int readTo;
         string nextLine;
         while(!finished) {
             readTo = _source.IndexOf ('\n', nextChar);
             if (readTo==-1){
                 readTo=_source.Length-1;
                 finished=true;
                 if (readTo<nextChar){
                     yield break;
                 }
             }
             nextLine = _source.Substring(nextChar,readTo-nextChar);
             nextChar = readTo+1;
             yield return nextLine;
         }
     }
 }
 

Timing tests show that with the following test harness running on my 1.33GHz iBook G4 running Mono 1.2.5, parsing a 1000 line text file 1000 times takes 2.7 seconds on average using this algorithm and 3.4 seconds on average using StringReader.

 public static void Main(string[] args)
 {
     StreamReader sr = new StreamReader("sample.text");
     string test = sr.ReadToEnd();
     System.DateTime start = DateTime.Now;
     for (int i=0;i<1000;i++){
         LineReader lineReader = new LineReader(test);
         foreach (string s in lineReader){
         }
     }
     System.DateTime finish = DateTime.Now;
     double seconds = (((double)finish.Ticks) - ((double)start.Ticks)) / 10000000;
     Console.WriteLine("{0} seconds",seconds);
 }
 

Yes, StringReader handles both LF and CRLF line endings, and has a PushBack method to allow jumping back to the previous line, but for pure speed of parsing standard text with LF line endings, this algorithm seems to be around 20% faster than StringReader, and 30% faster than using Split to turn large string into an array of one string per line.

Update on 14th November 2007:
I ran the same tests on Windows XP with Microsoft .NET 2.0, on a dual core 3GHz PC, and produced this graph which compares various ways of splitting the large string into one string per line: