You are looking at historical revision 22244 of this page. It may differ significantly from its current revision.

http-client

Description

Http-client is a highlevel HTTP client library.

Author

Peter Bex

Requirements

Requires the intarweb and openssl extensions.

Documentation

This egg is still under development; the API might change a bit in future versions.

Main request procedures

[procedure] (call-with-response request writer reader)

This is the core http-client procedure. It is only necessary to use this when you want the most control over the request/response cycle. request is the request object that contains information about the request to perform. reader is a procedure that receives the response object and should read the request body, writer is a procedure that receives the request object and should write the request body.

The writer should be prepared to be called several times; if the response is a redirect or some other status that indicates the server wants the client to perform a new request, the writer should be ready to write a request body for this new request. In case digest authentication with message integrity checking is used, writer is always invoked at least twice, once to determine the message digest of the response and once to actually write the response.

Returns three values: The result of the call to reader, the request-uri of the last request and the response object. The request-uri is useful because this is to be used as the base uri of the document. This can differ from the initial request in the presence of redirects.

[procedure] (call-with-input-request uri-or-request writer reader)

This procedure is a convenience wrapper around call-with-response.

It is much less strict - uri-or-request can be an intarweb request object, but also an uri-common object or even a string with the URI in it, in which case a request object will be automatically constructed around the URI, using the GET method.

writer can be either #f (in which case nothing is written), a string containing the raw data to send, an alist (in which case the data is written out as using www-form-urlencoding, useful for POST requests), or a procedure that accepts a port and writes the response data to it. If you supply a procedure, do not forget to set the content-length header! In the other cases, the length is calculated and the header automatically set for you. If you supplied an alist, the content-type header is automatically set to application/x-www-form-urlencoded.

reader is a procedure which accepts a port and reads out the data.

Returns three values: The result of the call to reader, the request-uri of the last request and the response object. If the response code is not in the 200 class, it will throw an exception of type (exn http client-error), (exn http server-error) or (exn http unexpected-server-response), depending on the response code. This includes 404 not found (which is a client-error).

[procedure] (with-input-from-request uri-or-request write-thunk read-thunk)

Same as call-with-input-request, except this accepts thunks (lambdas of no arguments) which will be executed with the current input (or output) port to the request or response port, respectively.

(use http-client)

(with-input-from-request "http://wiki.call-cc.org/" #f read-string)
 => ;; [the chicken wiki page HTML contents]


(use http-client uri-common intarweb)

;; Perform a POST of the key "test" with value "value" to an echo service
(with-input-from-request
  (make-request method: 'POST
                uri: (uri-reference "http://localhost/echo-service"))
  '((test . "value")) read-string)
 => "You posted: test=value"

Request handling parameters

[parameter] (max-retry-attempts [number])

When a request fails because of an I/O or network problem (or simply because the remote end closed a persistent connection while we were doing something else), the library will try to establish a new connection and perform the request again. This parameter controls how many times this is allowed to be done. If #f, it will never give up.

Defaults to 1.

[parameter] (retry-request? [predicate])

This procedure is invoked when a retry should take place, to determine if it should take place at all. It should be a procedure accepting a request object and returning #f or a true value. If the value is true, the new request will be sent. Otherwise, the error that caused the retry attempt will be re-raised.

Defaults to idempotent?, from intarweb. This is because non-idempotent requests cannot be safely retried when it is unknown whether the previous request reached the server or not.

[parameter] (max-redirect-depth [number])

The maximum number of allowed redirects, or #f if there is no limit. Currently there's no automatic redirect loop detection algorithm implemented.

Defaults to 5.

[parameter] (client-software [software-spec])

This is the names, versions and comments of the software packages that the client is using, for use in the user-agent header which is automatically added to each request.

Defaults to (("Chicken Scheme HTTP-client" VERSION #f)), where VERSION is the version of this egg.

Connection management

[procedure] (close-connection! uri)

Close the connection to the server associated with the URI.

[procedure] (close-all-connections!)

Close all connections to all servers.

http-client's cookie management is supposed to be as automatic and DWIMmy as possible. This means it will write any cookie as instructed by a server and all stored cookies are automatically sent back to the server upon a new request.

However, in some cases you may want to take control of how cookies are stored.

The API described here should be considered unstable and it may change dramatically when someone comes up with a better way to handle cookies.

[procedure] (get-cookies-for-uri uri)

Fetch a list of all cookies which ought to be sent to the given URI. Cookies are vectors of two elements: a name/value pair and an alist of attributes. In other words, these are the exact same values you can put in a cookie header.

[procedure] (store-cookie! cookie-info set-cookie)

Store a cookie in the cookiejar corresponding to the Set-Cookie header given by set-cookie. This overwrites any cookie that is equal to this cookie, as defined by RFC 2965, section 3.3.3. Practically, this means that when the cookie's name, domain and path are equal to an existant one, it will be overwritten by the new one. These attributes are taken from the cookie-info alist and expected to be there.

Generally, attributes should be taken from set-cookie, but if missing they ought to be taken from the request URI that responded with the set-cookie.

[procedure] (delete-cookie! cookie-name cookie-info)

Removes any cookie from the cookiejar that is equal to the given cookie (again, in the sense of RFC 2965, section 3.3.3). The cookie-name must match and the path and domain values for the cookie-info alist must match.

Authentication support

When a 401 Unauthorized response is received, in most interactive clients, the user is normally asked to authenticate. To support this type of interaction, http-client offers the following parameter:

[parameter] (determine-username/password [HANDLER])

The procedure in this parameter is called whenever the remote host requests authentication via a 401 Unauthorized response.

The HANDLER is a procedure of two arguments; the URI for the resource currently being requested and the realm (a string) which wants credentials. The procedure should return two string values: the username and the password to use for authentication.

The default value is a procedure which extracts the username and password components from the URI.

For proxy authentication support, see determine-proxy-username/password in the next section.

Proxy support

http-client has support for sending requests through proxy servers.

[parameter] (determine-proxy [HANDLER])

Whenever a request is sent, the library invokes the procedure stored in this parameter to determine through what proxy to send the request, if any.

The HANDLER procedure receives one argument, the URI about to be requested, and returns either an URI-common absolute URI object representing the proxy or #f if no proxy should be used.

The URI's path and query, if present, are ignored; only the scheme and authority (host, port, username, password) are used.

The default value of this parameter is determine-proxy-from-environment.

If you just want to disable proxy support, you can do:

(determine-proxy (constantly #f))   ; From unit data-structures
[procedure] (determine-proxy-from-environment URI)

This procedure implements the common behaviour of HTTP software under UNIX:

Some UNIX software expects plain hostnames or hostname port combinations separated by colons, but (currently) this library expects full URIs, like most modern UNIX programs.

[parameter] (determine-proxy-username/password [HANDLER])

The procedure in this parameter is called whenever the proxy requests authentication via a 407 Proxy Authentication Required response. This basically works the same as authentication against an origin server.

The HANDLER is a procedure of two arguments; the URI for the proxy currently being used and the realm (a string) which wants credentials. The procedure should return two string values: the username and the password to use for authentication.

The default value is a procedure which extracts the username and password components from the proxy's URI.

Changelog

License

 Copyright (c) 2008-2010, Peter Bex
 Parts copyright (c) 2000-2004, Felix L. Winkelmann
 All rights reserved.
 
 Redistribution and use in source and binary forms, with or without
 modification, are permitted provided that the following conditions are
 met:
 
 Redistributions of source code must retain the above copyright
 notice, this list of conditions and the following disclaimer.
 
 Redistributions in binary form must reproduce the above copyright
 notice, this list of conditions and the following disclaimer in the
 documentation and/or other materials provided with the distribution.
 
 Neither the name of the author nor the names of its contributors may
 be used to endorse or promote products derived from this software
 without specific prior written permission.
 
 THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
 "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
 LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS
 FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE
 COPYRIGHT HOLDERS OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT,
 INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES
 (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR
 SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION)
 HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT,
 STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
 ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED
 OF THE POSSIBILITY OF SUCH DAMAGE.