You are looking at historical revision 4953 of this page. It may differ significantly from its current revision.
http
- http
- Description
- Author
- Requirements
- Documentation
- http-client
- http-server
- Usage
- General operation
- http:make-server
- http:content-parser
- http:write-response-header
- http:write-error-response
- http:request-method-handler
- http:add-resource
- http:remove-resource
- http:find-resource
- http:fallback-handler
- http:error-response-handler
- http:current-request-count
- http:request-count-limit
- http:startup-hook
- http:error-hook
- http:log-hook
- http:url-transformation
- http:listen-procedure
- http:accept-procedure
- http:get-addresses-procedure
- http:hard-close-procedure
- http-utils
- Examples
- Changelog
- License
Description
An easy to use HTTP client and server package. The server is fully multithreaded and supports persistent connections.
Author
Requirements
Requires the regex-case egg at compile-time.
http-client requires the url egg at runtime.
Both http-client and http-server require the http-utils module at runtime.
Documentation
What follows is a description of the http extension, which is separated into three sub-extensions: http-client (HTTP client functionality), http-server (serving HTTP requests) and http-utils (utility functions common to client and server code).
If you want to run a web-server, you should also take a look at the more featureful spiffy extension.
http-client
Usage
(require-extension http-client)
http:send-request
[procedure] (http:send-request REQUEST [INPUT-PORT OUTPUT-PORT])
Sends an HTTP request represented by REQUEST, which may either be a HTTP request object, or a string that specifies an URL. The request is sent to it's destination and four values are returned: a string containing the first line of the response received from the server, an a-list that maps HTTP headers to values (strings) and the input- and output-port of the connection. Note that the connection is still open, and the returned ports can be passed in subsequent invocations of http:send-request to achieve a persistent connection. The ports are closed automatically, if no longer referenced.
If an URL is given instead of a request record, a request is sent of the form
GET url HTTP/1.0 Connection: close
http:GET
[procedure] (http:GET REQUEST)
Sends a GET request represented by REQUEST, which may be a string (an URL) or a HTTP request object, and returns a string containing the body of the servers response. The REQUEST keeps a Cookie header that is set by Set-Cookie headers in a response, unsafely (hack), regardless of the storing policy. Be aware of it when reusing REQUEST.
http:POST
[procedure] (http:POST REQUEST [ARGUMENTS] [DELIM1] [DELIM2] [ENCODER])
Sends a POST request represented by REQUEST, which may be a string (an URL) or a HTTP request object, and returns a string containing the body of the servers response. ARGUMENTS should be either a list of parameters passed in the body of the POST request and should contain strings of the form "name=value", or an alist of "name" and "value".
If REQUEST is a string or a HTTP request object whose Content-Type attribute is application/x-www-form-urlencoded, the HTTP body is created with the form "name1=value1&name2=value2...". Otherwise, the HTTP body is concatenated using DELIM1, DELIM2, and ENCODER. The REQUEST keeps a Cookie header, as with http:GET
http:close-all-connections!
[procedure] (http:close-all-connections!)
Close all persistent connections kept in the current thread.
http:read-body
[procedure] (http:read-body ARGUMENTS PORT)
Read the HTTP Body from the input port PORT, according to the Content-Length header or the Transfer-Encoding header.
http:add-proxy!
[procedure] (http:add-proxy! PROXY-HOST PROXY-PORT [SERV-PATTERN] [HOST-PATTERN] [PORT-PATTERN] [PATH-PATTERN])
Add a http proxy. PATTERNs are either a string, a regex, or #t. The first added proxy has the least priority.
Note: CONNECT method for SSL is not supported (yet).
http:remove-all-proxies!
[procedure] (http:remove-all-proxies!)
Remove all http proxies.
http-server
Usage
(require-extension http-server)
General operation
The server maps URLs to resource-handlers, which are procedures that process incoming client-requests. A resource-handler is responsible for generating a response by writing output to the value of (current-output-port).
The data contained in a client-request is parsed by a so-called content-parser, which is a procedure that reads the request-body from the port given by (current-input-port). Content-types are encoded as symbols.
A parser for application/x-www-form-urlencoded is predefined, other content-parsers have to be defined by application-code using the procedure http:content-parser, or the default parser will be invoked (which reads the content as a plain string).
The content-parser for text returns the request-body as a string. The content-parser for urlencoded data returns the request-body as an a-list that maps variables to strings.
http:make-server
[procedure] (http:make-server PORTNUMBER #!key SERVERNAME LOCATION PROTOCOL BACKLOG RESTRICTHOST INIT)
Creates and returns a server-procedure. SERVERNAME defaults to something silly, LOCATION to "localhost", PROTOCOL to #f (ignored), BACKLOG to 40 and RESTRICTHOST to #f. INIT should be a procedure of no arguments that will be called after the networking initialisation has taken place (specifically, after the invocation of tcp-listen).
To run the server-loop, invoke the returned procedure, which takes an optional boolean argument (passing #t will generate debugging output). PORTNUMBER, BACKLOG and RESTRICTHOST are directly passed on to tcp-listen (see Unit tcp for more information about those parameters).
http:content-parser
[procedure] (http:content-parser CONTENTTYPE [PROC])
Returns or sets the parser-procedure (a procedure of three arguments: the size of the content (may be #f), the a-list of request headers and an input port) for CONTENTTYPE, which should be a symbol.
The content-parser procedure PROC should return two values: the parsed and the unparsed (raw) request body.
http:write-response-header
[procedure] (http:write-response-header [CODE MSG [ALIST [PORT [PROTOCOL]]]])
Writes a HTTP response header to PORT, containing the server-name and -location. CODE and MSG default to 200 and OK, respectively. The optional ALIST may contain pairs with header-names and -values. If given, PROTOCOL specifes the HTTP protocol to use for the reply, which should be a symbol (either HTTP/1.0 or HTTP/1.1) and defaults to the protocol given in http:make-server.
http:write-error-response
[procedure] (http:write-error-response CODE MESSAGE [PORT])
Writes a HTTP error-response to PORT, which defaults to the value of (current-output-port). Uses the result returned by the value of (http:error-response-handler).
http:request-method-handler
[procedure] (http:request-method-handler METHOD [PROC])
Reads ot sets the handler procedure PROC for the request-method METHOD (which should be a symbol). PROC should accept one argument, a request-object. During execution of the handler, the current input- and output ports are bound to ports connected to the client.
Method-handlers for GET and POST requests are predefined.
http:add-resource
[procedure] (http:add-resource URL HANDLER)
Defines a new resource for URL (which may be a string, a symbol or a list of strings/symbols). HANDLER should be a procedure of two arguments: a request structure, and an a-list mapping urlencoded arguments to values.
During execution of the handler the current input- and an output-ports are bound to ports communicating with the client.
http:remove-resource
[procedure] (http:remove-resource URL)
Removes the resource defined under URL (string, symbol or list).
http:find-resource
[procedure] (http:find-resource URL)
Returns the handler-procedure for the resource defined under URL or #f if no such resource is registered.
http:fallback-handler
[parameter] http:fallback-handler
Contains a procedure that is is invoked on requests for resources that could not be found. This procedure is then called with the original request object. The default handler generates a 404 response.
http:error-response-handler
[parameter] http:error-response-handler
Contains a procedure that will be called when an HTTP error-response should be generated. The procedure is called with the error-code and message and should return a string containing HTML that will be sent in the body of the error-response.
http:current-request-count
[procedure] (http:current-request-count)
Returns the number of requests handled since startup.
http:request-count-limit
[parameter] http:request-count-limit
Maximum number of concurrently handled requests.
http:startup-hook
[parameter] http:startup-hook
A procedure that will be called on server startup, before accepting first request. The default procedure does nothing.
http:error-hook
[parameter] http:error-hook
A procedure that will be called when a thread triggers an unhandled exception. The exception is passed as an argument to the hook procedure.
http:log-hook
[parameter] http:log-hook
A procedure that will be called upon completion of a HTTP request. The procedure will be called with two arguments: the request object and the IP address of the client. The default value of this parameter does nothing.
http:url-transformation
[parameter] http:url-transformation
A procedure that allows arbitrary transformations of the URL part of each incoming request. The procedure is called with a single argument (the URL string) and should return the same URL, or a transformed one. The default transformation returns the original url unchanged.
http:listen-procedure
[parameter] http:listen-procedure
Holds a procedure that will be used to create a socket-listener - defaults to tcp-listen.
http:accept-procedure
[parameter] http:accept-procedure
Holds a procedure that will be used to accept a socket-connection - defaults to tcp-accept.
http:get-addresses-procedure
[parameter] http:get-addresses-procedure
Holds a procedure that will be used to obtain peer addresses - defaults to tcp-addresses.
http:hard-close-procedure
[parameter] http:hard-close-procedure
Holds a procedure that will be used to close a connection prematurely - defaults to tcp-abandon-port.
http-utils
Usage
(require-extension http-utils)
http:decode-url
[procedure] (http:decode-url URL)
Canonicalizes the string passed in URL and returns two values: the location-path and an alist containing argument <-> value pairs.
http:make-request
[procedure] (http:make-request METHOD URL [ATTRIBUTES [BODY [PROTOCOL [IP]]]])
Returns a freshly created HTTP request object (see below for the meaning of the arguments).
http:request?
[procedure] (http:request? X)
Returns #t if X is a request object, or #f otherwise.
http-request accessor methods
[procedure] (http:request-url REQUEST) -> URL [procedure] (http:request-protocol REQUEST) -> PROTOCOL [procedure] (http:request-attributes REQUEST) -> ATTRIBUTES [procedure] (http:request-body REQUEST) -> X [procedure] (http:request-method REQUEST) -> METHOD [procedure] (http:request-ip REQUEST) -> STRING [procedure] (http:request-sslctx REQUEST) -> <ssl-client-context> [procedure] (http:request-url-set! REQUEST URL) [procedure] (http:request-protocol-set! REQUEST PROTOCOL) [procedure] (http:request-attributes-set! REQUEST ATTRIBUTES) [procedure] (http:request-body-set! REQUEST X) [procedure] (http:request-method-set! REQUEST METHOD) [procedure] (http:request-ip-set! REQUEST STRING) [procedure] (http:request-sslctx-set! REQUEST <ssl-client-context>)
Accessor procedures for the components of a HTTP request structure. URL is a string, METHOD and PROTOCOL are symbols, BODY is either #f or a string and ATTRIBUTES is an a-list where each pair contains an attribute string and a value string.
http:read-line-limit
[parameter] http:read-line-limit
The maximum length of a header-line.
http:read-request-attributes
[procedure] (http:read-request-attributes PORT)
Reads MIME type headers from PORT until end of file is reached or a line is not a valid MIME header and returns an a-list where each pair holds the header (converted to lowercase) and the value (both strings).
http:canonicalize-string
[procedure] (http:canonicalize-string STRING)
Canonicalizes STRING by substituting %XX and + sequences.
Examples
Server example
A simple "Hello, world" server:
(require-extension http-server) (http:add-resource '("/" "/index.html") (lambda (r a) (http:write-response-header) (let ([msg "<h1>Hello, world!</h1>"]) (print "Content-type: text/html\r\nContent-length: " (add1 (string-length msg)) "\r\n\r\n" msg) ) ) ) ((http:make-server 4242) #t)
To try it out, simply load the code into the interpreter and point your browser to localhost:4242.
Client example
(require-extension http-client) (define-values (h a i o) (http:send-request "localhost:4242/")) (pretty-print (read-lines i)) (close-input-port i) (close-output-port o)
Loading this into the interpreter will print
("<h1>Hello, world!</h1>")
(provided the hello-world server is running)
Changelog
- 1.54 Fix passing of POST arguments to handlers defined with http:add-resource
- 1.53 Generation of debugging info from get/post-handler (ticket #253) [Mario Goulart]
- 1.52 Added requirement of url egg to meta info [thanks to Mario Goulart]
- 1.51 Client fix for handling closed keep-alive connections [Daishi Kato]
- 1.50 Client API for http proxies [Daishi Kato]
- 1.49 Client API flushes output before reading to be able to work with buffered ports
- 1.48 Bugfix in http-client.scm [Daishi Kato]
- 1.47 Removed multipart support, added unparsed request body field, content-parsers return additional raw body
- 1.46 Bugfix in url-parser
- 1.45 Cookie hack for http:GET and http:POST [Daishi Kato]
- 1.44 http:POST now handles www-form-urlencoded content-type, which is the default [Daishi Kato]
- 1.43 Client support for chunked transfer encoding [Daishi Kato]
- 1.42 Client support for persistent connections [Daishi Kato]
- 1.41 Client support for https (requires openssl egg) [Daishi Kato]
- 1.40 Client requests did finalize ports in the wrong situation [Thanks to Tim Reid]
- 1.39 URL canonicaliation fix #2 [Thanks to Zbigniew Szadkowski]
- 1.38 URL-canonicalization fix by Peter Busser
- 1.37 Added hidden slot to request-structure
- 1.36 Fixed bug in http:write-error-response [Thanks to Peter Bex]
- 1.35 Any error-response forces closing of client connection [Suggested by Diashi Kato]
- 1.34 Added -lws2_32 to build-command for http-client on Windows [Thanks to Daishi Kato]
- 1.33 Added parameterized socket operations to server
- 1.32 Adapted to SRFI-69 compatible hash-tables
- 1.31 The ports returned by http:send-request, http:GET and http:POST are finalized [Thanks to Reed Sheridan]
- 1.30 Added ip field to request object
- 1.29 Added http:url-transformation, debug output includes response headers
- 1.28 Replaced use of (end-of-file) with #!eof
- 1.27 Added http:request-limit; fixed http:POST (which didn't handle the second argument always correctly)
- 1.26 Special-cased regex strings to work around pregexp bugs [Thanks to Peter Bex]
- 1.25 Retries on failed tcp-accept, error-message output is limited
- 1.24 Fixed bug in server example [Thanks to Graham Fawcett]
- 1.23 Added note about SIGPIPE [Thanks Graham Fawcett]
- 1.22 Fixed regex bug in http-utils.scm [Thanks to Patrick Brannan]
- 1.21 Added http:log-hook
- 1.20 Content-parser got a third argument (attribute a-list); support for multipart/form-data content type; default handler for unknown content types
- 1.19 Added documentation for http:read-request-attributes
- 1.18 Added http:GET and http:POST
- 1.17 URL parsing of requests and header-generation is somewhat more robust [Thanks to Peter Bex]
- 1.16 http:send-request assumed old meaning of request body
- 1.15 If the content-type is not known, a request-body has the empty list (()) as the body [Thanks to Peter Bex]
- 1.14 Urlencoded arguments without argument are handled better and escapes in the argument name are processed correctly
- 1.13 URL parsing regex fix #394493849 [Thanks to Lars Rustemeier]
- 1.12 Added http:current-request-count
- 1.11 http:make-server takes keyword arguments now and accepts two additional arguments
- 1.10 http:write-response-header accepts yet another optional argument (protocol)
- 1.9 Fixed bug that required uint32_t which wasn't necessarily everywhere available [Thanks to Mikel Evins]
- 1.8 Adapted to new setup scheme
- 1.7 URL parsing in (http-client) again. Will it ever stop?
- 1.6 URL parsing in (http-client) works now with PCRE
- 1.5 Fixed bug in regular expression for URL parsing and another one in the regexp for HTTP headers
- 1.4 The URL parsing in (http-client) didn't handle XXX.XXX.XXX.XXX-style URLs [Thanks to Lars Rustemeier]
- 1.3 http:write-response-header doesn't override headers given in the alist.
- 1.2 http:write-response-header accepts more optional arguments. Added default content-parser for application/x-www-form-urlencoded bodies. http:canonicalize-string.
- 1.1 The content-type from a request may be followed by optional parameters. Fixed bug in canonicalize-string. The argument to the fallback-handler is now a request object.
- 1.0 Initial release
License
Copyright (c) 2003, Felix L. Winkelmann All rights reserved. Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met: Redistributions of source code must retain the above copyright notice, this list of conditions and the following disclaimer. Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the documentation and/or other materials provided with the distribution. Neither the name of the author nor the names of its contributors may be used to endorse or promote products derived from this software without specific prior written permission. THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDERS OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.