Sending HTTP Request GET/POST to form with Java?

Question

So I have this piece of code, and I got it to work, and now it basically allows me to send http post and get requests to almost any external website I want UNLESS the elements don't contain a name attribute. Here's an example:

This is the Java code:

    public static String sendPostRequest(String url) {

    StringBuffer sb = null;

    try {

        String data = URLEncoder.encode("user", "UTF-8") + "="
                + URLEncoder.encode("myUserName", "UTF-8") + "&"
                + URLEncoder.encode("submit", "UTF-8") + "="
                + URLEncoder.encode("Submit", "UTF-8");


        URL requestUrl = new URL(url);
        HttpURLConnection conn = (HttpURLConnection) requestUrl
                .openConnection();
        conn.setDoOutput(true);
        conn.setRequestMethod("GET");

        OutputStreamWriter osw = new OutputStreamWriter(
                conn.getOutputStream());
        osw.write(data);
        osw.flush();

        BufferedReader br = new BufferedReader(new InputStreamReader(
                conn.getInputStream()));

        String in = "";
        sb = new StringBuffer();

        while ((in = br.readLine()) != null) {
            sb.append(in + "\n");
        }

        osw.close();
        br.close();
    } catch (UnsupportedEncodingException e) {
        // TODO Auto-generated catch block
        e.printStackTrace();
    } catch (MalformedURLException e) {
        // TODO Auto-generated catch block
        e.printStackTrace();
    } catch (IOException e) {
        // TODO Auto-generated catch block
        e.printStackTrace();
    }
    return sb.toString();
}

This is the form I'm trying to send a request to (it's a form on the w3schools site, this being the site http://www.w3schools.com/html/html_forms.asp):

<form name="input0" target="_blank" action="html_form_action.asp" method="get">

Username: 

<input type="text" name="user" size="20" />

<input type="submit" value="Submit" />

</form>

Now because the Submit button doesn't have a name attribute, I can't send a proper HTTP Get/Post request to it (I know it's a get method in this case). What do I replace the String data with (what proper keys/values) so that it actually sends a request to this form?

Answer 1

I have a ClientHttpRequest class that does all multiparts, files etc, with optional progress tracing and cancelation. It's been around for about 10 years. Pretty simple to use. Now there's a Scala version too. https://github.com/vpatryshev/ScalaKittens/blob/master/src/main/scala/scalakittens/ClientHttpRequest.scala

http://myjavatools.com/

Answer 2

//Making http get request

HttpClient httpClientDefault1 = new DefaultHttpClient();
HttpPost httpPost = new HttpPost("http://www.your.targer.url.com/page.html");

//setup headers (Server understand request throw by some browser)

httpPost.setHeader("Connection", "keep-alive");
httpPost.setHeader("User-Agent", "Mozilla/5.0 (Windows NT 6.1; rv:9.0.1) Gecko/20100101 Firefox/9.0.1");
httpPost.setHeader("Accept", " text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8");
httpPost.setHeader("Accept-Language", "en-us,en;q=0.5");
httpPost.setHeader("Host", "ec2-23-20-44-83.compute-1.amazonaws.com");

httpPost.setHeader("Referer",resultUrl+resultUrlAsp);

//Set parameters

ArrayList<NameValuePair> nameValuePair = new ArrayList<NameValuePair>();
nameValuePair.add(new BasicNameValuePair("key",""));
nameValuePair.add(new BasicNameValuePair("txtenroll","095020693015"));
httpPost.setEntity(new UrlEncodedFormEntity(nameValuePair));

//Send request

HttpResponse httpRespnse = httpClientDefault1.execute(httpPost);

//Get Response body

if(httpRespnse.getStatusLine().getStatusCode() != 200) {
    InputStream in =  httpRespnse.getEntity().getContent();
    byte b[] = new byte[1024] ;
    StringBuilder html = new StringBuilder("");
    while(in.read(b) != -1) {
        html.append((new String(b)).toString());
        b = new byte[1024];
    }
    System.out.println(html);
}

also you can get headers, http parameters, cookies, manage the session through the java code... :) :)

Answer 3

I am using HttpClient for generating http request

HttpClient is open source apache project. you can get widely code. HttpClient version 4.1 is good set of Http api

HttpClient Learning Artical

Answer 4

You don't add the submit part to your data at all. This is just for the browser to know that "Submit" button triggers the action. Notice how the URL of the newly opened site looks: http://www.w3schools.com/html/html_form_action.asp?user=myUserName - no submit part here. So your data code should look like this:

String data = URLEncoder.encode("user", "UTF-8") + "="
            + URLEncoder.encode("myUserName", "UTF-8"); // end here