Doing HTTP request in Scala

Question

I am trying to issue a simple POST request to a webservice which returns some XML in Scala.

It seems that Dispatch is the standard library used for this task, but I cannot find documentation for it. The main site, which I link above, explains at length what is a promise and how to do asynchronous programming, but does not actually document the API. There is a periodic table - which looks a bit scary - but it only seems useful to people who already know what to do and only need a reminder for the cryptic syntax.

It also seems that Scalaz has some facility for HTTP, but I cannot find any documentation for it either.

Answer 1

Another option is Typesafe's play-ws, which is the Play Framework WS library broken out as a standalone lib:

http://blog.devalias.net/post/89810672067/play-framework-seperated-ws-library

I wouldn't necessarily offer this as the best option, but worth mentioning.

Answer 2

You could use spray-client. The documentation is lacking (it took me some digging to find out how to make GET requests with query parameters) but it's a great option if you are already using spray. And the documentation is better than dispatch.

We're using it at AI2 over dispatch because the operators are less symbolic and we're already using spray/actors.

import spray.client.pipelining._

val url = "http://youruri.com/yo"
val pipeline: HttpRequest => Future[HttpResponse] = sendReceive

// Post with header and parameters
val responseFuture1: Future[String] = pipeline(Post(Uri(url) withParams ("param" -> paramValue), yourPostData) map (_.entity.asString)

// Post with header
val responseFuture2: Future[String] = pipeline(Post(url, yourPostData)) map (_.entity.asString)

Answer 3

I'm using dispatch: http://dispatch.databinder.net/Dispatch.html

They've just released a new version (0.9.0) with a complete new api that I really like. And it is async.

Example from project page:

import dispatch._
val svc = url("http://api.hostip.info/country.php")
val country = Http(svc OK as.String)

for (c <- country)
  println(c)

edit: This might help you https://github.com/dispatch/reboot/blob/master/core/src/main/scala/requests.scala

Answer 4

I had to do the same to test one end point (in Integration test). So following is the code to fetch response from GET request in Scala. I am making use of scala.io.Source to read from endpoint and ObjectMapper for json to object conversion.

private def buildStockMasterUrl(url:String, stockStatus:Option[String]) = {
      stockStatus match  {
        case Some(stockStatus) => s"$url?stockStatus=${stockStatus}"
        case _ => url
    }
  }

    private def fetchBooksMasterData(stockStatus:Option[String]):  util.ArrayList[BooksMasterData] = {
    val url: String = buildBooksMasterUrl("http://localhost:8090/books/rest/catalogue/booksMasterData",stockStatus)
    val booksMasterJson : String = scala.io.Source.fromURL(url).mkString
    val mapper = new ObjectMapper()
    apper.readValue(booksMasterJson,classOf[util.ArrayList[BooksMasterData]])
}

case class BooksMasterData(id:String,description: String,category: String)

And here is my test method for the same

test("validate booksMasterData resource") {
    val booksMasterData = fetchBooksMasterData(Option(null))
    booksMasterData.size should be (740)
  }

Answer 5

Why not use Apache HttpComponents ? Here's the application FAQ, which covers a wide range of scenarios.

Answer 6

I use the following: https://github.com/scalaj/scalaj-http.

Here's a simple GET request:

import scalaj.http.Http

Http("http://foo.com/search").param("q", "monkeys").asString

and an example of a POST:

val result = Http("http://example.com/url").postData("""{"id":"12","json":"data"}""")
  .header("Content-Type", "application/json")
  .header("Charset", "UTF-8")
  .option(HttpOptions.readTimeout(10000)).asString

Scalaj HTTP is available through SBT:

libraryDependencies += "org.scalaj" % "scalaj-http_2.11" % "2.3.0"