Testing Pipelined HTTP Requests with WebMock

By: Tim Blair

Tags:

  • ruby
  • http
  • piplining
  • testing
  • webmock
  • net-http-pipeline

We’ve just released a small gem called webmock-net-http-pipeline (how’s that for a mouthful?) that helps us with testing Ruby code that makes calls to external services via pipelined HTTP requests, by enabling us to mock those requests via WebMock. Read on for more details…

What is HTTP Pipelining?

HTTP Pipelining is a feature of HTTP 1.1. It basically provides the ability to send multiple idempotent requests (principally GET and HEAD) along a single HTTP connection, without waiting for each response individually. The image below is a representation of how pipelining multiple requests can be of benefit when trying to minimise wait time between the start of the first request, and the end of the last response:

Pipelining allows us to minimise the time spent waiting for responses from the external service by providing the ability to send all requests first, then receive all the responses in one go. The HTTP 1.1 specification states that compliant servers must return responses in the same order that the requests are received in (and TCP guarantees the ordering on the wire), so it’s simple to batch requests without having to untangle a mess of response objects when they come back.

Why do we use it?

We use HTTP pipelining for efficiency when making a number of calls to a single internal HTTP service. There are a few articles around on the web implying that there’s not really much benefit to be had from pipelining requests over fast connections. However, that’s not been our experience.

When someone using our mobile application performs a search, the request is effectively split in to two parts. Firstly, a sub-request gets made to one internal HTTP service to perform the search and return a list of member IDs; the application then requests the details of those 20 members from a separate HTTP service, so they can be displayed to the end-user. The following benchmark script mimics the requests to retrieve the member details using both serial and pipeline methods.

require "net/http/pipeline"
require "benchmark"

http = Net::HTTP.start("member-service.wld", 80)
http.pipelining = true

requests = (1..20).map { Net::HTTP::Get.new("/members/123456789") }
repeat = 100

Benchmark.bm do |bm|
  bm.report("series:  ") do
    repeat.times do
      requests.each { |r| http.request(r) }
    end
  end

  bm.report("pipeline:") do
    repeat.times do
      http.pipeline requests.dup
    end
  end
end
               user     system      total        real
series:    1.170000   0.110000   1.280000 (  4.419424)
pipeline:  1.010000   0.110000   1.120000 (  2.371025)

The numbers speak for themselves: a 46% speed increase by using pipelined requests. Running this a number of times to avoid any issues with occasional network latency, garbage collection and so on, this number comes out as just over 43%, still making pipelined requests nearly twice as fast as individual requests.

Our library of choice for performing pipelined requests is net-http-pipeline by Eric Hodel. It sits on top of Net::HTTP (part of the Ruby standard library) and gives us just the right amount of power without over-complicating things.

So what’s the problem with testing?

We use the excellent WebMock library for both unit and integration tests. It’s a great tool for mocking out HTTP requests so you don’t have to rely on external services for your regular test runs. The problem we had was that, due to the way net-http-pipeline makes use of the underlying socket connections within Net::HTTP, it bypasses WebMock’s mocking behaviour.

WebMock’s Net::HTTP support works by effectively overriding the Net::HTTP#request method, through which all other request methods get routed. In the vast majority of cases, net-http-pipeline uses the “internal use only” method Net::HTTPGenericRequest#exec directly, bypassing WebMock’s behaviour.

Introducing webmock-net-http-pipeline

I was butting up against exactly the problem above a couple of days ago: trying to refactor some tests which seemed to be testing the mocking ability of the test library, rather than testing the behaviour of the application itself. I started digging into both WebMock and -pipeline to try and understand why they didn’t just “work” together; after that digging, I came to the conclusion that directly modifying either library to work with the other was a yak-shave too far, so started looking for a more pragmatic approach.

Because the unit tests I was writing were relatively simple (pipeline x GET requests and do something with the responses), and during the tests I had no reason to care about over-the-wire performance, I went for the simplest approach I could: don’t pipeline the requests. Now I don’t mean that my app didn’t pipeline the requests: I made the library not pipeline them. If we’re mocking requests, who cares if the “requests” are made in parallel or one by one?

After getting that far, I shared what I had with some of the team here, at which point Mat pointed me to a very similar, but cleaner approach he’d already used in one of our other apps; it’s that approach which is now at the centre of this very small, but useful gem.

How do I use it?

With great ease. Simply install the webmock-net-http-pipeline gem (either directly or via your project’s Gemfile), require webmock/net/http/pipeline after you’ve already required webmock, and you’re good to go. Mock your pipelined requests to your heart’s content.

require "webmock"
require "webmock/net/http/pipeline"

include WebMock::API

host = "www.example.com"
stub_request(:any, host)

http      = Net::HTTP.start(host, 80).tap { |http| http.pipelining = true }
requests  = (1..3).map { Net::HTTP::Get.new("/") }
responses = http.pipeline(requests)

p responses   #=> [#<Net::HTTPOK 200  readbody=true>, ...]

Please note that, as strange as it may sound, webmock-net-http-pipeline does not actually require net-http-pipeline at all: it simply mimics the behaviour of that library within the WebMock framework.

If you’ve got any suggestions for tidying the library up, or any comments about the approach, then please either comment here, raise an issue on GitHub, or send me a tweet.


About the Author

Tim Blair

Tim Blair is the systems architect for Venntro, and holds the coveted position of Committer #1 to our WhiteLabelDating platform. He can often be found chasing pieces of plastic around a field for the GB Ultimate team, or charging around the countryside by foot, bike or boat on an adventure race.