Tuesday, April 23, 2013

Simulate user browser behaviour in java - forms

There are two major approaches:

  • use a browser library. There you define all actions you want to do.
  • parse the browser response manually and fill the forms by on low-level.

Browser library

  • very powerful;
  • manages well when the output of web server is changed; 


  • heavy;
  • slow

Free Java libraries:

  • prowser - no info, please make a comment
  • htmlunit - no info, please make a comment

Manual submitting

  • very light for simple tasks; 
  • no need of additional library and studying its API; 
  • faster;


  • catching form fields in not reliable. If someone wants to lay it, it easily could do it.
  • when the output of the parsed page changes, it could break parsing;
  • forget about compatibility with javascript this way;

I choose manual submitting because my tasks were quite simple.

A example code:

        HttpURLConnection connection = null;
        try {
            OutputStreamWriter wr = null;
            BufferedReader rd  = null;

            URL serverUrl = new URL( serverAddress );

            log.info( "trying to connect to " + serverUrl.toURI() );
            connection = (HttpURLConnection)serverUrl.openConnection();
            connection.setRequestMethod( "POST" );
            connection.setDoOutput( true );
            connection.setRequestProperty("Content-Type", "application/x-www-form-urlencoded"); // needed for sumbitting forms data
            connection.setInstanceFollowRedirects( followRedirects );

        StringBuilder pageSubmitBody = new StringBuilder();
        pageSubmitBody.append("auth_token=").append( authToken );
        pageSubmitBody.append("&utf8=").append( ala_bala );



            wr = new OutputStreamWriter(connection.getOutputStream());
            wr.write( pageSubmitBody.toString() );


No comments:

Post a Comment