We’ve just released a small gem called webmock-net-http-pipeline (how’s that for a mouthful?) that helps us with testing Ruby code that makes calls to external services via pipelined HTTP requests, by enabling us to mock those requests via WebMock. Read on for more details…
What is HTTP Pipelining?
HTTP Pipelining is a feature of HTTP 1.1. It basically provides
the ability to send multiple idempotent requests (principally
along a single HTTP connection, without waiting for each response individually.
The image below is a representation of how pipelining multiple requests can be
of benefit when trying to minimise wait time between the start of the first
request, and the end of the last response:
Pipelining allows us to minimise the time spent waiting for responses from the external service by providing the ability to send all requests first, then receive all the responses in one go. The HTTP 1.1 specification states that compliant servers must return responses in the same order that the requests are received in (and TCP guarantees the ordering on the wire), so it’s simple to batch requests without having to untangle a mess of response objects when they come back.
Why do we use it?
We use HTTP pipelining for efficiency when making a number of calls to a single internal HTTP service. There are a few articles around on the web implying that there’s not really much benefit to be had from pipelining requests over fast connections. However, that’s not been our experience.
When someone using our mobile application performs a search, the request is effectively split in to two parts. Firstly, a sub-request gets made to one internal HTTP service to perform the search and return a list of member IDs; the application then requests the details of those 20 members from a separate HTTP service, so they can be displayed to the end-user. The following benchmark script mimics the requests to retrieve the member details using both serial and pipeline methods.
The numbers speak for themselves: a 46% speed increase by using pipelined requests. Running this a number of times to avoid any issues with occasional network latency, garbage collection and so on, this number comes out as just over 43%, still making pipelined requests nearly twice as fast as individual requests.
Our library of choice for performing pipelined requests is net-http-pipeline by Eric Hodel. It sits on top of Net::HTTP (part of the Ruby standard library) and gives us just the right amount of power without over-complicating things.
So what’s the problem with testing?
We use the excellent WebMock library for both unit and integration tests. It’s a great tool for mocking out HTTP requests so you don’t have to rely on external services for your regular test runs. The problem we had was that, due to the way net-http-pipeline makes use of the underlying socket connections within Net::HTTP, it bypasses WebMock’s mocking behaviour.
WebMock’s Net::HTTP support works by effectively overriding the
Net::HTTP#request method, through which all other request
methods get routed. In the vast majority of cases, net-http-pipeline uses the
“internal use only” method
bypassing WebMock’s behaviour.
I was butting up against exactly the problem above a couple of days ago: trying to refactor some tests which seemed to be testing the mocking ability of the test library, rather than testing the behaviour of the application itself. I started digging into both WebMock and -pipeline to try and understand why they didn’t just “work” together; after that digging, I came to the conclusion that directly modifying either library to work with the other was a yak-shave too far, so started looking for a more pragmatic approach.
Because the unit tests I was writing were relatively simple (pipeline x
requests and do something with the responses), and during the tests I had no
reason to care about over-the-wire performance, I went for the simplest
approach I could: don’t pipeline the requests. Now I don’t mean that my app
didn’t pipeline the requests: I made the library not pipeline them. If we’re
mocking requests, who cares if the “requests” are made in parallel or one by
After getting that far, I shared what I had with some of the team here, at which point Mat pointed me to a very similar, but cleaner approach he’d already used in one of our other apps; it’s that approach which is now at the centre of this very small, but useful gem.
How do I use it?
With great ease. Simply install the
webmock-net-http-pipeline gem (either
directly or via your project’s
after you’ve already required
webmock, and you’re good to go. Mock your
pipelined requests to your heart’s content.
Please note that, as strange as it may sound, webmock-net-http-pipeline does not actually require net-http-pipeline at all: it simply mimics the behaviour of that library within the WebMock framework.
If you’ve got any suggestions for tidying the library up, or any comments about the approach, then please either comment here, raise an issue on GitHub, or send me a tweet.