Why do I have unwanted extra bytes at the beginnin

Question

Question:

I've made a homepage for my brother, accessible here:
http://www.daniel-steiger.ch

It uses Microsoft ASP.NET MVC3 on Linux with mono 3, over fastcgi with nginx (plus my own DNS server).

Now I know this is an unusual constellation, but so far, it all works fine.
However, I ran into a minor very subtle bug.

When in the gallery one clicks on a thumbnail image, I wanted to display the full-sized image via the FullImage method of the gallery controller in a new tab.
For example, this direct url:
http://www.daniel-steiger.ch/gallery/FullImage/001.jpg


In internet Explorer, I got the image as text instead.
In all other browsers displayed an "invalid image" message.
I solved the problem by calling the image via it's direct file URL, which works fine:
http://www.daniel-steiger.ch/Content/images/gallery/001.jpg?LastWriteTimeUTC=1358694795000

Subsequently, I reported the bug to the mono mailing list
http://mono.1490590.n4.nabble.com/Bug-in-mono-3-0-1-MVC3-File-FileResult-td4658382.html

Now I got as a response, that this is all my fault, because I set the wrong image mime type, which is/was true.
However, I found it strange that if that was the case, the same code works fine on Windows, plus smart browsers like Chrome usually detect a wrong mime setting and use the correct one.

So I changed the mime-type from "image/jpg" to "image/jpeg", and redeployed the project to the server.
I also checked with the file utility if the image is actually a jpeg image, and it is.

What is strange is that it still doesn't show the image.
On internet explorer, I now get "not available".
On all other browsers, I get: the image cannot be displayed because it contains errors.

I did now wget the image from the URL where the image contains errors.
wget http://www.daniel-steiger.ch/gallery/fullimage/001.jpg

Then I ran a binary compare between the invalid and the original file:

cmp -l 001.jpg 001a.jpg   | awk '{printf "%08X %02X %02X\n", $1, strtonum(0$2), strtonum(0$3)}' >> comparison.txt

And this is the comparison result:

What springs into my eye, is that the image that internet explorer says he cannot find is actually 1.7 MB in size, and contains the extra bytes:

31 39 36 62 36 38 0D 0A 

at the beginning...
Anybody has an idea what's going wrong here/ where these bytes could come from (apart from the fact that it's most likely from a bug in mono/fastcgi) ?

This is the new controller code btw:

namespace Homepage.Controllers
{


    public class GalleryController : Controller
    {


        protected static string GetImageDirectory()
        {
            string bd = AppDomain.CurrentDomain.BaseDirectory;
            string strImageDirectory = System.IO.Path.Combine(bd,
            "Content");
            strImageDirectory =
            System.IO.Path.Combine(strImageDirectory, "images");
            strImageDirectory =
            System.IO.Path.Combine(strImageDirectory, "gallery");

            return strImageDirectory;
        } // End Function GetImageDirectory


        protected static string strImageDirectory = GetImageDirectory();


        public FileResult FullImage(string id)
        {
            string strFileName =
            System.IO.Path.Combine(strImageDirectory, id);

            //return new FilePathResult("CorrectFullPathAndFileName", "CorrectMime");
            //return File(strFileName, "image/jpg"); // Old
            return File(strFileName, "image/jpeg"); // New
        } // End Action FullImage



        public FileResult Thumb(string id)
        {
            //return Redirect(id);

            string strFileName =
            System.IO.Path.Combine(strImageDirectory, id);

            System.IO.Stream ms =
            Tools.Imaging.GetThumbnailStream(strFileName,
            System.Drawing.Imaging.ImageFormat.Png);
            return File(ms, "image/png");
            /*
            using (System.IO.Stream ms =
            Tools.Imaging.GetThumbnailStream(strFileName,
            System.Drawing.Imaging.ImageFormat.Png))
            {
                return File(ms, "image/png");
            }*/
        } // End Action Thumb


    } // End Class GalleryController : Controller


} // End Namespace Homepage.Controllers

Edit:
it gets stranger:
There is also an extra sequence with

0d 0a 0d 0a 30 0d 0a 0d  0a 

at the end...

I just did the hexdump, and found the original file has filesize (hold your breath):

00196b68

Here the hexdumps (8 MB each):

Canonical format (hexdump -C 001.jpg > 001.txt):
Original file: http://www.daniel-steiger.ch/001.txt
Botched file: http://www.daniel-steiger.ch/001a.txt

Pure dump (hexdump 001.jpg > 001_1.txt):
Original file: http://www.daniel-steiger.ch/001_1.txt
Botched file: http://www.daniel-steiger.ch/001a_1.txt

Hmmm, the hex dump for the botched files is 5 MB, the one for the original is 8.2 MB...

I use the latest stable nginx:

sudo -s
nginx=stable # use nginx=development for latest development version
add-apt-repository ppa:nginx/$nginx
apt-get update 
apt-get install nginx

The full code for Tools.Imaging (renamed Tools to MvcTools in the meantime, to match the assembly namespace)

using System;
using System.Text;


namespace MvcTools
{


    public class Imaging
    {

        //public static System.Drawing.Size m_sMaxThumbNailDimensions = new System.Drawing.Size(200, 200);
        public static System.Drawing.Size m_sMaxThumbNailDimensions = new System.Drawing.Size(300, 300);

        public static System.IO.Stream GetImageAsStream(
                                                 string strOrgFileName
                                                , System.Drawing.Imaging.ImageFormat ifOutputFormat
                                           )
        {
            return GetImageAsStream(strOrgFileName, ifOutputFormat, 1024);
        } // End Function GetImage


        public static System.IO.Stream GetImageAsStream(
                                                 string strOrgFileName
                                                ,System.Drawing.Imaging.ImageFormat ifOutputFormat
                                                ,int rez
                                           )
        {
            System.IO.MemoryStream ms = null;

            if (!System.IO.File.Exists(strOrgFileName))
                throw new System.IO.FileNotFoundException(strOrgFileName);

            try
            {

                using (System.Drawing.Image imgSourceImage = System.Drawing.Image.FromFile(strOrgFileName))
                {

                    ms = new System.IO.MemoryStream();
                    imgSourceImage.Save(ms, ifOutputFormat);
                    ms.Position = 0;
                } // End Using imgSourceImage

            } // End Try
            catch (Exception ex)
            {
                System.Windows.Forms.MessageBox.Show(ex.GetType().ToString());
                System.Windows.Forms.MessageBox.Show(ex.Message);
                //Response.Write(ex.Message);
            } // End Catch

            return ms;
        } // End Function GetImageAsStream


        public static System.Drawing.Size GetThumbnailSize(string strOrgFileName)
        {
            System.Drawing.Size sThumbNailSizeToUse = new System.Drawing.Size();

            try
            {

                using (System.Drawing.Image imgSourceImage = System.Drawing.Image.FromFile(strOrgFileName))
                {
                    decimal decPixToSubstract = 0;
                    decimal decPercentage;

                    if (m_sMaxThumbNailDimensions.Width < imgSourceImage.Size.Width || m_sMaxThumbNailDimensions.Height < imgSourceImage.Size.Height)
                    {
                        if (imgSourceImage.Size.Width > imgSourceImage.Size.Height)
                        {
                            decPercentage = (((decimal)imgSourceImage.Size.Width - (decimal)m_sMaxThumbNailDimensions.Width) / (decimal)imgSourceImage.Size.Width);
                            decPixToSubstract = decPercentage * imgSourceImage.Size.Height;
                            sThumbNailSizeToUse.Width = m_sMaxThumbNailDimensions.Width;
                            sThumbNailSizeToUse.Height = imgSourceImage.Size.Height - (int)decPixToSubstract;
                        } // End if (imgSourceImage.Size.Width > imgSourceImage.Size.Height)
                        else
                        {
                            decPercentage = (((decimal)imgSourceImage.Size.Height - (decimal)m_sMaxThumbNailDimensions.Height) / (decimal)imgSourceImage.Size.Height);
                            decPixToSubstract = decPercentage * (decimal)imgSourceImage.Size.Width;
                            sThumbNailSizeToUse.Height = m_sMaxThumbNailDimensions.Height;
                            sThumbNailSizeToUse.Width = imgSourceImage.Size.Width - (int)decPixToSubstract;
                        } // End else of if (imgSourceImage.Size.Width > imgSourceImage.Size.Height)

                    } // End if (m_sMaxThumbNailDimensions.Width < imgSourceImage.Size.Width || m_sMaxThumbNailDimensions.Height < imgSourceImage.Size.Height)
                    else
                    {
                        sThumbNailSizeToUse.Width = imgSourceImage.Size.Width;
                        sThumbNailSizeToUse.Height = imgSourceImage.Size.Height;
                    } // End else of if (m_sMaxThumbNailDimensions.Width < imgSourceImage.Size.Width || m_sMaxThumbNailDimensions.Height < imgSourceImage.Size.Height)

                } // End Using imgSourceImage

            } // End Try
            catch (Exception ex)
            {
                System.Windows.Forms.MessageBox.Show(ex.Message);
                //Response.Write(ex.Message);
            } // End Catch

            return sThumbNailSizeToUse;
        } // End Sub GetThumbnailSize(string strOrgFileName)


        // http://stackoverflow.com/questions/7319842/mvc3-razor-thumbnail-resize-image-ideas
        // http://stackoverflow.com/questions/1528525/alternatives-to-system-drawing-for-use-with-asp-net/1528908#1528908
        public static void GenerateThumbnailFile(
                                        string strPhysicalPath,
                                        string strOrgFileName, string strThumbnailFileName,
                                        System.Drawing.Imaging.ImageFormat ifOutputFormat, int rez
                                     )
        {

            try
            {

                using (System.Drawing.Image imgSourceImage = System.Drawing.Image.FromFile(strOrgFileName))
                {
                    //System.Drawing.Image oImg = System.Drawing.Image.FromStream(fil.InputStream);

                    decimal decPixToSubstract = 0;
                    decimal decPercentage;

                    //default
                    System.Drawing.Size sThumbNailSizeToUse = new System.Drawing.Size();
                    if (m_sMaxThumbNailDimensions.Width < imgSourceImage.Size.Width || m_sMaxThumbNailDimensions.Height < imgSourceImage.Size.Height)
                    {
                        if (imgSourceImage.Size.Width > imgSourceImage.Size.Height)
                        {
                            decPercentage = (((decimal)imgSourceImage.Size.Width - (decimal)m_sMaxThumbNailDimensions.Width) / (decimal)imgSourceImage.Size.Width);
                            decPixToSubstract = decPercentage * imgSourceImage.Size.Height;
                            sThumbNailSizeToUse.Width = m_sMaxThumbNailDimensions.Width;
                            sThumbNailSizeToUse.Height = imgSourceImage.Size.Height - (int)decPixToSubstract;
                        } // End if (imgSourceImage.Size.Width > imgSourceImage.Size.Height)
                        else
                        {
                            decPercentage = (((decimal)imgSourceImage.Size.Height - (decimal)m_sMaxThumbNailDimensions.Height) / (decimal)imgSourceImage.Size.Height);
                            decPixToSubstract = decPercentage * (decimal)imgSourceImage.Size.Width;
                            sThumbNailSizeToUse.Height = m_sMaxThumbNailDimensions.Height;
                            sThumbNailSizeToUse.Width = imgSourceImage.Size.Width - (int)decPixToSubstract;
                        } // End else of if (imgSourceImage.Size.Width > imgSourceImage.Size.Height)

                    } // End if (m_sMaxThumbNailDimensions.Width < imgSourceImage.Size.Width || m_sMaxThumbNailDimensions.Height < imgSourceImage.Size.Height)
                    else
                    {
                        sThumbNailSizeToUse.Width = imgSourceImage.Size.Width;
                        sThumbNailSizeToUse.Height = imgSourceImage.Size.Height;
                    } // End else of if (m_sMaxThumbNailDimensions.Width < imgSourceImage.Size.Width || m_sMaxThumbNailDimensions.Height < imgSourceImage.Size.Height)

                    using (System.Drawing.Bitmap bmpThumbnail = new System.Drawing.Bitmap(sThumbNailSizeToUse.Width, sThumbNailSizeToUse.Height))
                    {
                        bmpThumbnail.SetResolution(rez, rez);
                        using (System.Drawing.Image imgThumbNail = bmpThumbnail)
                        {

                            using (System.Drawing.Graphics gGraphicsContext = System.Drawing.Graphics.FromImage(imgThumbNail))
                            {
                                gGraphicsContext.CompositingQuality = System.Drawing.Drawing2D.CompositingQuality.HighQuality;
                                gGraphicsContext.SmoothingMode = System.Drawing.Drawing2D.SmoothingMode.HighQuality;
                                gGraphicsContext.InterpolationMode = System.Drawing.Drawing2D.InterpolationMode.HighQualityBicubic;

                                System.Drawing.Rectangle rThumbnailDimension = new System.Drawing.Rectangle(0, 0, sThumbNailSizeToUse.Width, sThumbNailSizeToUse.Height);
                                gGraphicsContext.DrawImage(imgSourceImage, rThumbnailDimension);
                            } // End Using gGraphicsContext

                            imgThumbNail.Save(System.IO.Path.Combine(strPhysicalPath, strThumbnailFileName), ifOutputFormat);
                        } // End Using imgThumbNail

                    } // End Using bmpThumbnail

                } // End Using imgSourceImage

            } // End Try
            catch (Exception ex)
            {
                System.Windows.Forms.MessageBox.Show(ex.Message);
                //Response.Write(ex.Message);
            } // End Catch

        } // End Function GenerateThumbNail


        public static System.IO.Stream GetThumbnailStream(
                                                               string strOrgFileName
                                                             , System.Drawing.Imaging.ImageFormat ifOutputFormat
                                                         )
        {
            return GetThumbnailStream(strOrgFileName, ifOutputFormat, 1024);
        } // End Function GetThumbnailStream


        public static System.IO.Stream GetThumbnailStream(
                                      string strOrgFileName
                                     ,System.Drawing.Imaging.ImageFormat ifOutputFormat
                                     ,int rez
                                  )
        {

            System.IO.MemoryStream ms = null;

            try
            {

                using (System.Drawing.Image imgSourceImage = System.Drawing.Image.FromFile(strOrgFileName))
                {
                    decimal decPixToSubstract = 0;
                    decimal decPercentage;

                    System.Drawing.Size sThumbNailSizeToUse = new System.Drawing.Size();
                    if (m_sMaxThumbNailDimensions.Width < imgSourceImage.Size.Width || m_sMaxThumbNailDimensions.Height < imgSourceImage.Size.Height)
                    {
                        if (imgSourceImage.Size.Width > imgSourceImage.Size.Height)
                        {
                            decPercentage = (((decimal)imgSourceImage.Size.Width - (decimal)m_sMaxThumbNailDimensions.Width) / (decimal)imgSourceImage.Size.Width);
                            decPixToSubstract = decPercentage * imgSourceImage.Size.Height;
                            sThumbNailSizeToUse.Width = m_sMaxThumbNailDimensions.Width;
                            sThumbNailSizeToUse.Height = imgSourceImage.Size.Height - (int)decPixToSubstract;
                        } // End if (imgSourceImage.Size.Width > imgSourceImage.Size.Height)
                        else
                        {
                            decPercentage = (((decimal)imgSourceImage.Size.Height - (decimal)m_sMaxThumbNailDimensions.Height) / (decimal)imgSourceImage.Size.Height);
                            decPixToSubstract = decPercentage * (decimal)imgSourceImage.Size.Width;
                            sThumbNailSizeToUse.Height = m_sMaxThumbNailDimensions.Height;
                            sThumbNailSizeToUse.Width = imgSourceImage.Size.Width - (int)decPixToSubstract;
                        } // End else of if (imgSourceImage.Size.Width > imgSourceImage.Size.Height)

                    } // End if (m_sMaxThumbNailDimensions.Width < imgSourceImage.Size.Width || m_sMaxThumbNailDimensions.Height < imgSourceImage.Size.Height)
                    else
                    {
                        sThumbNailSizeToUse.Width = imgSourceImage.Size.Width;
                        sThumbNailSizeToUse.Height = imgSourceImage.Size.Height;
                    } // End else of if (m_sMaxThumbNailDimensions.Width < imgSourceImage.Size.Width || m_sMaxThumbNailDimensions.Height < imgSourceImage.Size.Height)


                    using (System.Drawing.Bitmap bmpThumbnail = new System.Drawing.Bitmap(sThumbNailSizeToUse.Width, sThumbNailSizeToUse.Height))
                    {
                        bmpThumbnail.SetResolution(rez, rez);
                        using (System.Drawing.Image imgThumbNail = bmpThumbnail)
                        {

                            using (System.Drawing.Graphics gGraphicsContext = System.Drawing.Graphics.FromImage(imgThumbNail))
                            {
                                gGraphicsContext.CompositingQuality = System.Drawing.Drawing2D.CompositingQuality.HighQuality;
                                gGraphicsContext.SmoothingMode = System.Drawing.Drawing2D.SmoothingMode.HighQuality;
                                gGraphicsContext.InterpolationMode = System.Drawing.Drawing2D.InterpolationMode.HighQualityBicubic;

                                System.Drawing.Rectangle rThumbnailDimension = new System.Drawing.Rectangle(0, 0, sThumbNailSizeToUse.Width, sThumbNailSizeToUse.Height);
                                gGraphicsContext.DrawImage(imgSourceImage, rThumbnailDimension);

                                ms = new System.IO.MemoryStream();
                                imgThumbNail.Save(ms, ifOutputFormat);
                                ms.Position = 0;
                            } // End Using gGraphicsContext

                        } // End Using imgThumbNail

                    } // End Using bmpThumbnail


                    /*
                    byte[] buffer = null;
                    using (System.IO.MemoryStream ms = new System.IO.MemoryStream())
                    {
                        imgThumbNail.Save(ms, ifOutputFormat);
                        buffer = ms.ToArray();
                    }
                    */

                    // Exerts from Page_Load method 
                    //Response.ContentType = "image/" + extension;
                    //Response.OutputStream.Write(pBuffer, 0, pBuffer.Length);
                    //Response.End();

                    //imgThumbNail.Save(System.IO.Path.Combine(strPhysicalPath, strThumbnailFileName), ifOutputFormat);
                } // End Using imgSourceImage

            } // End Try
            catch (Exception ex)
            {
                //Console.WriteLine(ex.Message);
                System.Windows.Forms.MessageBox.Show(ex.Message);
                //Response.Write(ex.Message);
            } // End Catch

            //System.Windows.Forms.MessageBox.Show("image/" + ifOutputFormat.ToString().ToLower());
            return ms;
        } // End Function GenerateThumbNail


    } // End Class Imaging


} // End Namespace Tools

Answer 1

31 39 36 62 36 38 0D 0A

is 196b68/r/n in ascii.

• does this color some how related to your code?
• does the string 196b68/r/n say something related to your platforms?

Answer 2

I don't recognize that Tools.Imaging.GetThumbnailStream() function, and neither does Google. Assuming that's your own code or a 3rd party library, I'd look there for the source of the strange bytestream.

Answer 3

This seems to be a bug involving Chuncked transfer encoding in

Class: System.Web.HttpResponse (or one of its dependencies)
Method: TransmitFile(string filename)

Edit:
There is this code in the constructor:

if (worker_request != null)
      use_chunked = (worker_request.GetHttpVersion () == "HTTP/1.1");

Patched it to check for CGI (if CGI, the server handles the file transfer, so there may be no chuncked encoding given back from the FastCGI server as per RFC 3875.

        internal HttpResponse (HttpWorkerRequest worker_request, HttpContext context) : this ()
        {
            WorkerRequest = worker_request;
            this.context = context;

#if !TARGET_J2EE
            if (worker_request != null)
            {

                if(worker_request.GetHttpVersion () == "HTTP/1.1")
                {
                    string GatewayIface = context.Request.ServerVariables["GATEWAY_INTERFACE"];
                    use_chunked = (GatewayIface == null || !GatewayIface.StartsWith("CGI"));
                }
                else
                    use_chunked = false;

            }
#endif
            writer = new HttpWriter (this);
        }

Added patch to https://bugzilla.xamarin.com/show_bug.cgi?id=10001
Fixed in mono 3.2.3