Can't get the proper Index to return

Question

Alright, so firstly I want to thank everyone for helping me so much in the last couple weeks, here's another one!!!

I have a file and I'm using Regex to find how many times the term "TamedName" comes up. That's the easy part :)

Originally, I was setting it up like this

            StreamReader ff = new StreamReader(fileName);
        String D = ff.ReadToEnd();
        Regex rx = new Regex("TamedName");
        foreach (Match Dino in rx.Matches(D))
        {
            if (richTextBox2.Text == "")
                richTextBox2.Text += string.Format("{0} - {1:X} - {2}", Dino.Value, Dino.Index, ReadString(fileName, (uint)Dino.Index));
            else
                richTextBox2.Text += string.Format("\n{0} - {1:X} - {2}", Dino.Value, Dino.Index, ReadString(fileName, (uint)Dino.Index));
        }

and it was returning completely incorrect index points, as pictured here

I'm fairly confident I know why it's doing this, probably because converting everything from a binary file to string, obviously not all the characters are going to translate, so that throws off the actual index count, so trying to relate that back doesn't work at all... The problem, I have NO clue how to use Regex with a binary file and have it translate properly :(

I'm using Regex vs a simple search function because the difference between each occurrence of "TamedName" is WAY too vast to code into a function.

Really hope you guys can help me with this one :( I'm running out of ideas!!

Answer 1

The problem is that you are reading in a binary file and the streamreader does some interpretation when it reads it into a Unicode string. It needed to be dealt with as bytes.

My code is below.(Just as an FYI, you will need to enable unsafe compilation to compile the code - this was to allow a fast search of the binary array)

Just for proper attribution, I borrowed the byte version of IndexOf from this SO answer by Dylan Nicholson

namespace ArkIndex
{
    class Program
    {
        static void Main(string[] args)
        {
            string fileName = "TheIsland.ark";
            string searchString = "TamedName";
            byte[] bytes = LoadBytesFromFile(fileName);
            byte[] searchBytes = System.Text.ASCIIEncoding.Default.GetBytes(searchString);

            List<long> allNeedles = FindAllBytes(bytes, searchBytes);    
        }

        static byte[] LoadBytesFromFile(string fileName)
        {
            FileStream fs = new FileStream(fileName, FileMode.Open);
            //BinaryReader br = new BinaryReader(fs);
            //StreamReader ff = new StreamReader(fileName);

            MemoryStream ms = new MemoryStream();
            fs.CopyTo(ms);
            fs.Close();
            return ms.ToArray();   
        }

        public static List<long> FindAllBytes(byte[] haystack, byte[] needle)
        {
            long currentOffset = 0;
            long offsetStep = needle.Length;
            long index = 0;
            List<long> allNeedleOffsets = new List<long>();
            while((index = IndexOf(haystack,needle,currentOffset)) != -1L)
            {
                allNeedleOffsets.Add(index);
                currentOffset = index + offsetStep;
            }
            return allNeedleOffsets;
        }

        public static unsafe long IndexOf(byte[] haystack, byte[] needle, long startOffset = 0)
        {
            fixed (byte* h = haystack) fixed (byte* n = needle)
            {
                for (byte* hNext = h + startOffset, hEnd = h + haystack.LongLength + 1 - needle.LongLength, nEnd = n + needle.LongLength; hNext < hEnd; hNext++)
                    for (byte* hInc = hNext, nInc = n; *nInc == *hInc; hInc++)
                        if (++nInc == nEnd)
                            return hNext - h;
                return -1;
            }
        }    
    }
}