Java 9 HttpClient send a multipart/form-data reque

Question

Below is a form:

<form action="/example/html5/demo_form.asp" method="post" 
enctype=”multipart/form-data”>
   <input type="file" name="img" />
   <input type="text" name=username" value="foo"/>
   <input type="submit" />
</form>

when will submit this form, the request will look like this:

POST /example/html5/demo_form.asp HTTP/1.1
Host: 10.143.47.59:9093
Connection: keep-alive
Content-Length: 326
Accept: application/json, text/javascript, */*; q=0.01
Origin: http://10.143.47.59:9093
X-Requested-With: XMLHttpRequest
User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/60.0.3112.90 Safari/537.36
Content-Type: multipart/form-data; boundary=----WebKitFormBoundaryEDKBhMZFowP9Leno
Accept-Encoding: gzip, deflate
Accept-Language: en-US,en;q=0.8,zh-CN;q=0.6,zh;q=0.4

Request Payload
------WebKitFormBoundaryEDKBhMZFowP9Leno
Content-Disposition: form-data; name="username"

foo
------WebKitFormBoundaryEDKBhMZFowP9Leno
Content-Disposition: form-data; name="img"; filename="out.txt"
Content-Type: text/plain


------WebKitFormBoundaryEDKBhMZFowP9Leno--

please pay attention to the "Request Payload", you can see the two params in the form, the username and the img(form-data; name="img"; filename="out.txt"), and the finename is the real file name(or path) in your filesystem, you will receive the file by name(not filename) in your backend(such as spring controller).
if we use Apache Httpclient to simulate the request, we will write such code:

MultipartEntity mutiEntity = newMultipartEntity();
File file = new File("/path/to/your/file");
mutiEntity.addPart("username",new StringBody("foo", Charset.forName("utf-8")));
mutiEntity.addPart("img", newFileBody(file)); //img is name, file is path

But in java 9, We could write such code:

HttpClient client = HttpClient.newHttpClient();
HttpRequest request = HttpRequest.
        newBuilder(new URI("http:///example/html5/demo_form.asp"))
       .method("post",HttpRequest.BodyProcessor.fromString("foo"))
       .method("post", HttpRequest.BodyProcessor.fromFile(Paths.get("/path/to/your/file")))
       .build();
HttpResponse response = client.send(request, HttpResponse.BodyHandler.asString());
System.out.println(response.body());

Now you see, how could I set the "name" of the param?

Answer 1

A direction in which you can attain making a multiform-data call could be as follows:

BodyProcessor can be used with their default implementations or else a custom implementation can also be used. Few of the ways to use them are :

1. Read the processor via a string as :

   HttpRequest.BodyProcessor dataProcessor = HttpRequest.BodyProcessor.fromString("{\"username\":\"foo\"}")
   

2. Creating a processor from a file using its path

   Path path = Paths.get("/path/to/your/file"); // in your case path to 'img'
   HttpRequest.BodyProcessor fileProcessor = HttpRequest.BodyProcessor.fromFile(path);
   

OR

3. You can convert the file input to a byte array using the apache.commons.lang(or a custom method you can come up with) to add a small util like :

   org.apache.commons.fileupload.FileItem file;
   
   org.apache.http.HttpEntity multipartEntity = org.apache.http.entity.mime.MultipartEntityBuilder.create()
          .addPart("username",new StringBody("foo", Charset.forName("utf-8")))
          .addPart("img", newFileBody(file))
          .build();
   multipartEntity.writeTo(byteArrayOutputStream);
   byte[] bytes = byteArrayOutputStream.toByteArray();
   

   and then the byte[] can be used with BodyProcessor as:

   HttpRequest.BodyProcessor byteProcessor = HttpRequest.BodyProcessor.fromByteArray();
   

---

Further, you can create the request as :

HttpRequest request = HttpRequest.newBuilder()
            .uri(new URI("http:///example/html5/demo_form.asp"))
            .headers("Content-Type","multipart/form-data","boundary","boundaryValue") // appropriate boundary values
            .POST(dataProcessor)
            .POST(fileProcessor)
            .POST(byteProcessor) //self-sufficient
            .build();

---

The response for the same can be handled as a file and with a new HttpClient using

HttpResponse.BodyHandler bodyHandler = HttpResponse.BodyHandler.asFile(Paths.get("/path"));

HttpClient client = HttpClient.newBuilder().build();

as:

HttpResponse response = client.send(request, bodyHandler);
System.out.println(response.body());

Answer 2

I struggled with this problem for a while, even after seeing and reading this page. But, using the answers on this page to point me in the right direction, reading more about multipart forms and boundaries, and tinkering around, I was able to create a working solution.

The gist of the solution is to use Apache's MultipartEntityBuilder to create the entity and its boundaries (HttpExceptionBuilder is a homegrown class):

import java.io.BufferedInputStream;
import java.io.File;
import java.io.FileInputStream;
import java.io.FileNotFoundException;
import java.io.IOException;
import java.io.InputStream;
import java.util.Optional;
import java.util.function.Supplier;

import org.apache.commons.lang3.Validate;
import org.apache.http.HttpEntity;
import org.apache.http.entity.BufferedHttpEntity;
import org.apache.http.entity.ContentType;
import org.apache.http.entity.mime.MultipartEntityBuilder;

/**
 * Class containing static helper methods pertaining to HTTP interactions.
 */
public class HttpUtils {
    public static final String MULTIPART_FORM_DATA_BOUNDARY = "ThisIsMyBoundaryThereAreManyLikeItButThisOneIsMine";

    /**
     * Creates an {@link HttpEntity} from a {@link File}, loading it into a {@link BufferedHttpEntity}.
     *
     * @param file     the {@link File} from which to create an {@link HttpEntity}
     * @param partName an {@link Optional} denoting the name of the form data; defaults to {@code data}
     * @return an {@link HttpEntity} containing the contents of the provided {@code file}
     * @throws NullPointerException  if {@code file} or {@code partName} is null
     * @throws IllegalStateException if {@code file} does not exist
     * @throws HttpException         if file cannot be found or {@link FileInputStream} cannot be created
     */
    public static HttpEntity getFileAsBufferedMultipartEntity(final File file, final Optional<String> partName) {
        Validate.notNull(file, "file cannot be null");
        Validate.validState(file.exists(), "file must exist");
        Validate.notNull(partName, "partName cannot be null");

        final HttpEntity entity;
        final BufferedHttpEntity bufferedHttpEntity;

        try (final FileInputStream fis = new FileInputStream(file);
                final BufferedInputStream bis = new BufferedInputStream(fis)) {
            entity = MultipartEntityBuilder.create().setBoundary(MULTIPART_FORM_DATA_BOUNDARY)
                    .addBinaryBody(partName.orElse("data"), bis, ContentType.APPLICATION_OCTET_STREAM, file.getName())
                    .setContentType(ContentType.MULTIPART_FORM_DATA).build();

            try {
                bufferedHttpEntity = new BufferedHttpEntity(entity);
            } catch (final IOException e) {
                throw HttpExceptionBuilder.create().withMessage("Unable to create BufferedHttpEntity").withThrowable(e)
                        .build();
            }
        } catch (final FileNotFoundException e) {
            throw HttpExceptionBuilder.create()
                    .withMessage("File does not exist or is not readable: %s", file.getAbsolutePath()).withThrowable(e)
                    .build();
        } catch (final IOException e) {
            throw HttpExceptionBuilder.create()
                    .withMessage("Unable to create multipart entity from file: %s", file.getAbsolutePath())
                    .withThrowable(e).build();
        }

        return bufferedHttpEntity;
    }

    /**
     * Returns a {@link Supplier} of {@link InputStream} containing the content of the provided {@link HttpEntity}. This
     * method closes the {@code InputStream}.
     *
     * @param entity the {@link HttpEntity} from which to get an {@link InputStream}
     * @return an {@link InputStream} containing the {@link HttpEntity#getContent() content}
     * @throws NullPointerException if {@code entity} is null
     * @throws HttpException        if something goes wrong
     */
    public static Supplier<? extends InputStream> getInputStreamFromHttpEntity(final HttpEntity entity) {
        Validate.notNull(entity, "entity cannot be null");

        return () -> {
            try (final InputStream is = entity.getContent()) {
                return is;
            } catch (final UnsupportedOperationException | IOException e) {
                throw HttpExceptionBuilder.create().withMessage("Unable to get InputStream from HttpEntity")
                        .withThrowable(e).build();
            }
        };
    }
}

And then a method that uses these helper methods:

private String doUpload(final File uploadFile, final String filePostUrl) {
    assert uploadFile != null : "uploadFile cannot be null";
    assert uploadFile.exists() : "uploadFile must exist";
    assert StringUtils.notBlank(filePostUrl, "filePostUrl cannot be blank");

    final URI uri = URI.create(filePostUrl);
    final HttpEntity entity = HttpUtils.getFileAsBufferedMultipartEntity(uploadFile, Optional.of("partName"));
    final String response;

    try {
        final Builder requestBuilder = HttpRequest.newBuilder(uri)
                .POST(BodyPublisher.fromInputStream(HttpUtils.getInputStreamFromHttpEntity(entity)))
                .header("Content-Type", "multipart/form-data; boundary=" + HttpUtils.MULTIPART_FORM_DATA_BOUNDARY);

        response = this.httpClient.send(requestBuilder.build(), BodyHandler.asString());
    } catch (InterruptedException | ExecutionException e) {
        throw HttpExceptionBuilder.create().withMessage("Unable to get InputStream from HttpEntity")
                    .withThrowable(e).build();
    }

    LOGGER.info("Http Response: {}", response);
    return response;
}

Answer 3

It is possible to use multipart/form-data or any other content type - but you have to encode the body in the correct format yourself. The client itself does not do any encoding based on the content type.

That means your best option is to use another HTTP client the like Apache HttpComponents client or only use the encoder of another library like in the example of @nullpointer's answer.

---

If you do encode the body yourself, note that you can't call methods like POST more than once. POST simply sets the BodyProcessor and calling it again will just override any previously set processors. You have to implement one processor that produces the whole body in the correct format.

For multipart/form-data that means:

1. Set the boundary header to an appropriate value

2. Encode each parameter so that it looks like in your example. Basically something like this for text input:

   boundary + "\nContent-Disposition: form-data; name=\" + name + "\"\n\n" + value + "\n"
   

   Here, the name refers to the name attribute in the HTML form. For the file input in the question, this would img and the value would be the encoded file content.