Persistent

Persistent connections for Net::HTTP

Bundler::Persistent::Net::HTTP::Persistent maintains persistent connections across all the servers you wish to talk to. For each host:port you communicate with a single persistent connection is created.

Multiple Bundler::Persistent::Net::HTTP::Persistent objects will share the same set of connections.

For each thread you start a new connection will be created. A Bundler::Persistent::Net::HTTP::Persistent connection will not be shared across threads.

You can shut down the HTTP connections when done by calling #shutdown. You should name your Bundler::Persistent::Net::HTTP::Persistent object if you intend to call this method.

Example:

require 'bundler/vendor/net-http-persistent/lib/net/http/persistent'

uri = URI 'http://example.com/awesome/web/service'

http = Bundler::Persistent::Net::HTTP::Persistent.new 'my_app_name'

# perform a GET
response = http.request uri

# or

get = Net::HTTP::Get.new uri.request_uri
response = http.request get

# create a POST
post_uri = uri + 'create'
post = Net::HTTP::Post.new post_uri.path
post.set_form_data 'some' => 'cool data'

# perform the POST, the URI is always required
response http.request post_uri, post

Note that for GET, HEAD and other requests that do not have a body you want to use URI#request_uri not URI#path. The request_uri contains the query params which are sent in the body for other requests.

SSL

SSL connections are automatically created depending upon the scheme of the URI. SSL connections are automatically verified against the default certificate store for your computer. You can override this by changing verify_mode or by specifying an alternate cert_store.

Here are the SSL settings, see the individual methods for documentation:

#certificate	This client’s certificate
#ca_file	The certificate-authority
#cert_store	An SSL certificate store
#private_key	The client’s SSL private key
#reuse_ssl_sessions	Reuse a previously opened SSL session for a new connection
#ssl_version	Which specific SSL version to use
#verify_callback	For server certificate verification
#verify_mode	How connections should be verified

Proxies

A proxy can be set through #proxy= or at initialization time by providing a second argument to ::new. The proxy may be the URI of the proxy server or :ENV which will consult environment variables.

See #proxy= and #proxy_from_env for details.

Headers

Headers may be specified for use in every request. #headers are appended to any headers on the request. #override_headers replace existing headers on the request.

The difference between the two can be seen in setting the User-Agent. Using http.headers[‘User-Agent’] = ‘MyUserAgent’ will send “Ruby, MyUserAgent” while http.override_headers[‘User-Agent’] = ‘MyUserAgent’ will send “MyUserAgent”.

Tuning

Segregation

By providing an application name to ::new you can separate your connections from the connections of other applications.

Idle Timeout

If a connection hasn’t been used for this number of seconds it will automatically be reset upon the next use to avoid attempting to send to a closed connection. The default value is 5 seconds. nil means no timeout. Set through #idle_timeout.

Reducing this value may help avoid the “too many connection resets” error when sending non-idempotent requests while increasing this value will cause fewer round-trips.

Read Timeout

The amount of time allowed between reading two chunks from the socket. Set through #read_timeout

Max Requests

The number of requests that should be made before opening a new connection. Typically many keep-alive capable servers tune this to 100 or less, so the 101st request will fail with ECONNRESET. If unset (default), this value has no effect, if set, connections will be reset on the request after max_requests.

Open Timeout

The amount of time to wait for a connection to be opened. Set through #open_timeout.

Socket Options

Socket options may be set on newly-created connections. See #socket_options for details.

Non-Idempotent Requests

By default non-idempotent requests will not be retried per RFC 2616. By setting retry_change_requests to true requests will automatically be retried once.

Only do this when you know that retrying a POST or other non-idempotent request is safe for your application and will not create duplicate resources.

The recommended way to handle non-idempotent requests is the following:

require 'bundler/vendor/net-http-persistent/lib/net/http/persistent'

uri = URI 'http://example.com/awesome/web/service'
post_uri = uri + 'create'

http = Bundler::Persistent::Net::HTTP::Persistent.new 'my_app_name'

post = Net::HTTP::Post.new post_uri.path
# ... fill in POST request

begin
  response = http.request post_uri, post
rescue Bundler::Persistent::Net::HTTP::Persistent::Error

  # POST failed, make a new request to verify the server did not process
  # the request
  exists_uri = uri + '...'
  response = http.get exists_uri

  # Retry if it failed
  retry if response.code == '404'
end

The method of determining if the resource was created or not is unique to the particular service you are using. Of course, you will want to add protection from infinite looping.

Connection Termination

If you are done using the Bundler::Persistent::Net::HTTP::Persistent instance you may shut down all the connections in the current thread with #shutdown. This is not recommended for normal use, it should only be used when it will be several minutes before you make another HTTP request.

If you are using multiple threads, call #shutdown in each thread when the thread is done making requests. If you don’t call shutdown, that’s OK. Ruby will automatically garbage collect and shutdown your HTTP connections when the thread terminates.

http.socket_options << [Socket::SOL_SOCKET, Socket::SO_KEEPALIVE, 1]

SSL_connect [...] read finished A: unexpected message (OpenSSL::SSL::SSLError)

lib/bundler/vendor/net-http-persistent/lib/net/http/persistent.rb