AkkaHTTP: Getting content out of an HttpEntity

2020-05-06 17:28发布

问题:

I'm working with AkkaHTTP in Scala, trying to interact with an API, but I'm having some trouble getting data with certain API calls. Basically, I have a curl command which works in my terminal for any valid API call to this certain API, but in my current code I only get the actual data for calls whose responses are small enough.

My first attempt was using sys.process._ and the !! method on a string with "curl (url) -X GET -H \"Authorization: Bearer (auth token)\" (which is the command that works in my terminal), but that failed with an error in scala due to issues parsing the header; so after making no headway on that and realizing I wanted to use AkkaHTTP anyway, I abandoned that approach and came to the code below, whose format I borrowed from another stackoverflow post.

implicit val system = ActorSystem()
implicit val materializer = ActorMaterializer()
implicit val executionContext = system.dispatcher

val responseFuture: Future[HttpResponse] = Http().singleRequest(HttpRequest(
  uri = "(url)").withHeaders(
  RawHeader("Authorization", "Bearer (auth token)")
))


responseFuture
  .onComplete {
    case Success(res) => println(res)
    case Failure(_)   => "failure"
  }

Now, I've tried this on two different API calls; the first, which in my terminal returns a JSON string with 327 characters, returns a blob, where res.entity is HttpEntity.Strict(application/json,(the same JSON blob as in the terminal)). However, while the second call returns in my terminal a JSON blob with 203,413 characters (much bigger than the first one), sending that request in scala returns a blob where res.entity is just HttpEntity.Chunked(application/json). I would presume the payload is split into multiple packets ("Chunked" being a fairly applicable name for that scenario), but I can't seem to figure out how to get the payload information from here, and I'd appreciate some help.

回答1:

If it's acceptable to load the entire entity into memory, you can call toStrict on the ResponseEntity:

import scala.concurrent.Future
import scala.concurrent.duration._

val entityFut: Future[String] =
  responseFuture.flatMap(_.entity.toStrict(5 seconds).map(_.data.utf8String))

entityFut.onComplete {
  case Success(body) =>
    println(body)
  case Failure(_) =>
    println("failure")
}

You could also use a built-in unmarshaller:

import akka.http.scaladsl.unmarshalling.Unmarshal

val entityFut: Future[String] =
  responseFuture.flatMap(resp => Unmarshal(resp.entity).to[String])