Character replacement in strings in VB.NET

Question

How fast can I replace characters in a string?

So the background on this question is this: We have a couple of applications that communicate with each other and with clients' applications through sockets. These socket messages contain non-printable characters (e.g. chr(0)) which need to get replaced with a predetermined string (e.g "{Nul}"}, because the socket messages are kept in a log file. On a side note, not every log message will need to have characters replaced.

Now I started off on this little adventure reading from this MSDN link which I found from a different post from this site.

The current method we used...at the beginning of the day...was using StringBuilder to check for all the possible replacements such as...

    Public Function ReplaceSB(ByVal p_Message As String) As String
      Dim sb As New System.Text.StringBuilder(p_Message)

      sb.Replace(Chr(0), "{NUL}")
      sb.Replace(Chr(1), "{SOH}")

      Return sb.ToString
    End Function

Now as the blog post points out leaving StringBuilder out and using string.replace does yield faster results. (Actually, using StringBuilder was the slowest method of doing this all day long.)

    p_Message = p_Message.Replace(Chr(0), "{NUL}")
    p_Message = p_Message.Replace(Chr(1), "{SOH}")

Knowing that not every message would need to go through this process I thought it would save time to not have to process those messages that could be left out. So using regular expressions I first searched the string and then determined if it needed to be processed or not. This was about the same as using the string.replace, basically a wash from saving the time of not processing all the strings, but losing time from checking them all with regular expressions.

Then it was suggested to try using some arrays that matched up their indexes with the old and the new and use that to process the messages. So it would be something like this...

Private chrArray() As Char = {Chr(0), Chr(1)}
Private strArray() As String = {"{NUL}", "{SOH}"}

Public Function TestReplace(ByVal p_Message As String) As String
    Dim i As Integer

    For i = 0 To ((chrArray.Length) - 1)
        If p_Message.Contains(chrArray(i).ToString) Then
            p_Message = p_Message.Replace(chrArray(i), strArray(i))
        End If
    Next

    Return p_Message
End Function

This so far has been the fastest way I have found to process these messages. I have tried various other ways of going about this as well like converting the incoming string into a character array and comparing along with also trying to loop through the string rather than the chrArray.

So my question to all is: Can I make this faster yet? What am I missing?

Answer 1

You might be able to squeeze out a little more speed by reducing some lookups. Take for example this:

    If p_Message.Contains(chrArray(i).ToString) Then

The .Contains method is O(n). In the worst case, you're going to traverse all the chars in the entire string without finding anything, so you expect to traverse at least one time for each character in your array, so its O(nm) where n is the length of your string and m is the number of chars you're replacing.

You might get a little better performance doing the following (my VB-fu is rusty, has not been tested ;) ):

Private Function WriteToCharList(s as String, dest as List(Of Char))
    for each c as Char in s
        dest.Add(c)
    Next
End Function

Public Function TestReplace(ByVal p_Message As String) As String
    Dim chars as new List(Of Char)(p_Message.Length)

    For each c as Char in p_Message
        Select Case c
            Case Chr(0): WriteToCharList("{NUL}", chars)
            Case Chr(1): WriteToCharList("{SOH}", chars)
            Case Else: chars.Add(c);
        End Select
    Next

    Return New String(chars)
End Function

This will traverse chars in p_Message at most twice (once for traversing, once when the string constructor copies the char array), making this function O(n).

Answer 2

StringBuilder offers the fastest Replace() function in .NET.

Answer 3

This should also be faster:

    Private Shared strList As New Dictionary(Of Char, String)

    Shared Sub New()
        strList.Add(Chr(0), "{NUL}")
        strList.Add(Chr(1), "{SOH}")
    End Sub

    Public Function TestReplace(ByVal p_Message As String) As String
        For Each c As Char In strList.Keys
            If p_Message.IndexOf(c) <> -1 Then
                p_Message = p_Message.Replace(c, strList(c))
            End If
        Next

        Return p_Message
    End Function

Answer 4

Have a look at this example. It has some benchmark statistics comparing the two methods.

Answer 5

A couple of general notes here:

1. You might be able to improve on the search function by using a plain .IndexOf() or .Contains() search, since you're only looking for single characters.
2. You might be able to improve your total throughput by returning the StringBuilder object from your function directly and providing overloads for other functions that accept stringbuilders as input or calling .ToString() somewhere later on in the process (note: You can also call .ToString() on objects that are already strings)
3. You should definitely be able to further improve performance/throughput by using a StringReader/TextReader further up the chain and continue to treat everything as streams that keep getting passed up the chain.

At very least you can modify your final method this way:

Public Function TestReplace(ByVal p_Message As String) As String
    Static chrArray() As Char = {ChrW(0), ChrW(1)}
    Static strArray() As String = {"{NUL}", "{SOH}"}

    Dim rdr As New StringReader(p_Message)
    Dim result As New StringWriter()

    Dim i As Integer
    While (i = rdr.Read()) <> -1
        Dim c As Char = ChrW(i)
        Dim index As Integer = Array.IndexOf(chrArray, c)
        If index >= 0 Then result.Write(strArray(index)) Else result.Write(c)
    End While

    Return result.ToString()
End Function

Note that your benchmarks will greatly depend on the kind of strings you're throwing at it, so make sure you're using the most representative sample (and it should be a good sized sample) possible.