WebRequest POST with both file and parameters

Question

I'm trying to upload a file and a send along a few parameters to my site using .NET / C#. Having read a few tutorials that do either a few parameters or a file, I've tried, unsuccessfully, to combine them. Here is how I try doing it:

WebRequest req = WebRequest.Create(baseURL + "upload");
req.Credentials = new NetworkCredential(username, password);
String boundary = "B0unD-Ary";
req.ContentType = "multipart/form-data; boundary=" + boundary;
req.Method = "POST";
((HttpWebRequest)req).UserAgent = "UploadTester v0.1";

string postData = "--" + boundary + "\nContent-Disposition: form-data\n";
postData += "myId=123&someFk=456";
postData += "\n--" + boundary + "\nContent-Disposition: form-data; name=\"file\" filename=\"upload.pdf\" Content-Type: application/pdf\n\n";
byte[] byteArray = Encoding.UTF8.GetBytes(postData);

byte[] filedata = null;
using (BinaryReader reader = new BinaryReader(File.OpenRead("myfile.pdf")))
    filedata = reader.ReadBytes((int)reader.BaseStream.Length);

req.ContentLength = byteArray.Length + filedata.Length;
req.GetRequestStream().Write(byteArray, 0, byteArray.Length);
req.GetRequestStream().Write(filedata, 0, filedata.Length);

WebResponse response = req.GetResponse();
Stream data = response.GetResponseStream();
StreamReader sReader = new StreamReader(data);
String sResponse = sReader.ReadToEnd();
response.Close();

When I execute it, I get a 500 exception, saying "Header section has more than 10240 bnytes (maybe it is not properly terminated)" and Wireshark informs me that the request sent was a malformed package, where the MIME multipart was malformed.

There are probably several issues here, so please let me know all the problems you can spot

Update: to separate MIME from C#/.NET, I've spawned a thread here: https://stackoverflow.com/questions/1880002/error-in-mime-packet-for-http-post

Update 2: So the backend indeed has issues with the content-length, saying that the amount of bytes available for reading is smaller than the stated content-length. BUT! If I reduce the content-length in req.ContentLength accordingly, I don't have a buffer size large enough for sending the data. Any suggestions?

Update 3: Actually, it looks like the header has a too large size compared to how much data it contains

Answer 1

I guess the immediate problem is a mismatch between the declared and the actual length.

Where did you miscalculated I am not sure but if I remember correctly the length should include every byte of the response (including headers) except for the first line.

To be sure I would build a simple html page to generate the post you want and examined the post with fiddler (or firebug)

Answer 2

The problem is that you're missing a '\n'. The following line:

string postData = "--" + boundary + "\nContent-Disposition: form-data\n";

should be:

string postData = "--" + boundary + "\nContent-Disposition: form-data\n\n";

And this line:

postData += "\n--" + boundary + "\nContent-Disposition: form-data; name=\"file\" filename=\"upload.pdf\" Content-Type: application/pdf\n\n"

is missing a '\n' before 'Content-Type'. It should be:

postData += "\n--" + boundary + "\nContent-Disposition: form-data; name=\"file\" filename=\"upload.pdf\"\nContent-Type: application/pdf\n\n"

Answer 3

Try to change

string postData = "--" + boundary + "\nContent-Disposition: form-data\n";

to

string postData = "\n\n--" + boundary + "\nContent-Disposition: form-data\n\n";

And better also replace all \n to \r\n. At least first two newlines worked for me solving "Header section has more than 10240 bytes (maybe it is not properly terminated)" problem.

And this code doesn't look correct:

postData += "myId=123&someFk=456";

My objective-C code for iPhone app which I am using and which works now looks like this:

NSMutableData *reqData = [NSMutableData data];
NSURL *encUrl;
encUrl = [NSURL URLWithString:[NSString url];
NSMutableURLRequest *request = [NSMutableURLRequest requestWithURL:encUrl];
request.HTTPMethod = @"POST";
request.timeoutInterval = kServerConnectionTimeout;
    NSString *stringBoundary = [NSString stringWithString:@"Multipart-Boundary"]; // TODO - randomize this
    NSString *contentType = [NSString stringWithFormat:@"multipart/form-data; boundary=\"%@\"",stringBoundary];
    [request addValue:contentType forHTTPHeaderField: @"Content-Type"];

[reqData appendData:[[NSString stringWithFormat:@"\r\n\r\n--%@\r\n",stringBoundary] dataUsingEncoding:NSUTF8StringEncoding]];
for (id key in [self.parameters allKeys])
{
    if (![key isEqualToString:@"image"])
    {
        NSString *val = [self.parameters objectForKey:key];
        if (val != nil)
        {
            [reqData appendData:[[NSString stringWithFormat:@"Content-Disposition: form-data; name=\"%@\"\r\n",key] dataUsingEncoding:NSUTF8StringEncoding]];
            [reqData appendData:[[NSString stringWithFormat:@"Content-Type: text/plain\r\n"] dataUsingEncoding:NSUTF8StringEncoding]];
            [reqData appendData:[[NSString stringWithFormat:@"Content-Transfer-Encoding: 8bit\r\n\r\n"] dataUsingEncoding:NSUTF8StringEncoding]];
            [reqData appendData:[[NSString stringWithFormat:@"%@",val] dataUsingEncoding:NSUTF8StringEncoding]];
            [reqData appendData:[[NSString stringWithFormat:@"\r\n--%@\r\n",stringBoundary] dataUsingEncoding:NSUTF8StringEncoding]];
        }
    }
}

// send binary part last
if ([self.parameters objectForKey:@"image"])
{
    NSString *key  = @"image";
    NSData *imageData = [self.parameters objectForKey:key];
    if (imageData != nil)
    {
        [reqData appendData:[[NSString stringWithFormat:@"Content-Disposition: form-data; name=\"image\";filename=\"avatar.png\"\r\n"] dataUsingEncoding:NSUTF8StringEncoding]];
        [reqData appendData:[[NSString stringWithFormat:@"Content-Type: image/png\r\n"] dataUsingEncoding:NSUTF8StringEncoding]];

        [reqData appendData:[[NSString stringWithFormat:@"Content-Transfer-Encoding: binary\r\n\r\n"] dataUsingEncoding:NSUTF8StringEncoding]];
        [reqData appendData:imageData];
        [reqData appendData:[[NSString stringWithFormat:@"\r\n--%@--\r\n",stringBoundary] dataUsingEncoding:NSUTF8StringEncoding]];
    }
}
NSString * dataLength = [NSString stringWithFormat:@"%d", [reqData length]];
[request addValue:dataLength forHTTPHeaderField:@"Content-Length"];

request.HTTPBody = reqData;
NSLog(@"postBody=%@", [[NSString alloc] initWithData:reqData encoding:NSASCIIStringEncoding]);

Hope this helps somebody.