You are looking at historical revision 4952 of this page. It may differ significantly from its current revision.
http
- http
- Description
- Author
- Requirements
- Documentation
- http-client
- http-server
- Usage
- General operation
- http:make-server
- http:content-parser
- http:write-response-header
- http:write-error-response
- http:request-method-handler
- http:add-resource
- http:remove-resource
- http:find-resource
- http:fallback-handler
- http:error-response-handler
- http:current-request-count
- http:request-count-limit
- http:startup-hook
- http:error-hook
- http:log-hook
- http:url-transformation
- http:listen-procedure
- http:accept-procedure
- http:get-addresses-procedure
- http:hard-close-procedure
- http-utils
- Examples
- Changelog
Description
An easy to use HTTP client and server package. The server is fully multithreaded and supports persistent connections.
Author
Requirements
Requires the regex-case egg at compile-time.
http-client requires the url egg at runtime.
Both http-client and http-server require the http-utils module at runtime.
Documentation
What follows is a description of the http extension, which is separated into three sub-extensions: http-client (HTTP client functionality), http-server (serving HTTP requests) and http-utils (utility functions common to client and server code).
If you want to run a web-server, you should also take a look at the more featureful spiffy extension.
http-client
Usage
(require-extension http-client)
http:send-request
[procedure] (http:send-request REQUEST [INPUT-PORT OUTPUT-PORT])
Sends an HTTP request represented by REQUEST, which may either be a HTTP request object, or a string that specifies an URL. The request is sent to it's destination and four values are returned: a string containing the first line of the response received from the server, an a-list that maps HTTP headers to values (strings) and the input- and output-port of the connection. Note that the connection is still open, and the returned ports can be passed in subsequent invocations of http:send-request to achieve a persistent connection. The ports are closed automatically, if no longer referenced.
If an URL is given instead of a request record, a request is sent of the form
GET url HTTP/1.0 Connection: close
http:GET
[procedure] (http:GET REQUEST)
Sends a GET request represented by REQUEST, which may be a string (an URL) or a HTTP request object, and returns a string containing the body of the servers response. The REQUEST keeps a Cookie header that is set by Set-Cookie headers in a response, unsafely (hack), regardless of the storing policy. Be aware of it when reusing REQUEST.
http:POST
[procedure] (http:POST REQUEST [ARGUMENTS] [DELIM1] [DELIM2] [ENCODER])
Sends a POST request represented by REQUEST, which may be a string (an URL) or a HTTP request object, and returns a string containing the body of the servers response. ARGUMENTS should be either a list of parameters passed in the body of the POST request and should contain strings of the form "name=value", or an alist of "name" and "value".
If REQUEST is a string or a HTTP request object whose Content-Type attribute is application/x-www-form-urlencoded, the HTTP body is created with the form "name1=value1&name2=value2...". Otherwise, the HTTP body is concatenated using DELIM1, DELIM2, and ENCODER. The REQUEST keeps a Cookie header, as with http:GET
http:close-all-connections!
[procedure] (http:close-all-connections!)
Close all persistent connections kept in the current thread.
http:read-body
[procedure] (http:read-body ARGUMENTS PORT)
Read the HTTP Body from the input port PORT, according to the Content-Length header or the Transfer-Encoding header.
http:add-proxy!
[procedure] (http:add-proxy! PROXY-HOST PROXY-PORT [SERV-PATTERN] [HOST-PATTERN] [PORT-PATTERN] [PATH-PATTERN])
Add a http proxy. PATTERNs are either a string, a regex, or #t. The first added proxy has the least priority.
Note: CONNECT method for SSL is not supported (yet).
http:remove-all-proxies!
[procedure] (http:remove-all-proxies!)
Remove all http proxies.
http-server
Usage
(require-extension http-server)
General operation
The server maps URLs to resource-handlers, which are procedures that process incoming client-requests. A resource-handler is responsible for generating a response by writing output to the value of (current-output-port).
The data contained in a client-request is parsed by a so-called content-parser, which is a procedure that reads the request-body from the port given by (current-input-port). Content-types are encoded as symbols.
A parser for application/x-www-form-urlencoded is predefined, other content-parsers have to be defined by application-code using the procedure http:content-parser, or the default parser will be invoked (which reads the content as a plain string).
The content-parser for text returns the request-body as a string. The content-parser for urlencoded data returns the request-body as an a-list that maps variables to strings.
http:make-server
[procedure] (http:make-server PORTNUMBER #!key SERVERNAME LOCATION PROTOCOL BACKLOG RESTRICTHOST INIT)
Creates and returns a server-procedure. SERVERNAME defaults to something silly, LOCATION to "localhost", PROTOCOL to #f (ignored), BACKLOG to 40 and RESTRICTHOST to #f. INIT should be a procedure of no arguments that will be called after the networking initialisation has taken place (specifically, after the invocation of tcp-listen).
To run the server-loop, invoke the returned procedure, which takes an optional boolean argument (passing #t will generate debugging output). PORTNUMBER, BACKLOG and RESTRICTHOST are directly passed on to tcp-listen (see Unit tcp for more information about those parameters).
http:content-parser
[procedure] (http:content-parser CONTENTTYPE [PROC])
Returns or sets the parser-procedure (a procedure of three arguments: the size of the content (may be #f), the a-list of request headers and an input port) for CONTENTTYPE, which should be a symbol.
The content-parser procedure PROC should return two values: the parsed and the unparsed (raw) request body.
http:write-response-header
[procedure] (http:write-response-header [CODE MSG [ALIST [PORT [PROTOCOL]]]])
Writes a HTTP response header to PORT, containing the server-name and -location. CODE and MSG default to 200 and OK, respectively. The optional ALIST may contain pairs with header-names and -values. If given, PROTOCOL specifes the HTTP protocol to use for the reply, which should be a symbol (either HTTP/1.0 or HTTP/1.1) and defaults to the protocol given in http:make-server.
http:write-error-response
[procedure] (http:write-error-response CODE MESSAGE [PORT])
Writes a HTTP error-response to PORT, which defaults to the value of (current-output-port). Uses the result returned by the value of (http:error-response-handler).
http:request-method-handler
[procedure] (http:request-method-handler METHOD [PROC])
Reads ot sets the handler procedure PROC for the request-method METHOD (which should be a symbol). PROC should accept one argument, a request-object. During execution of the handler, the current input- and output ports are bound to ports connected to the client.
Method-handlers for GET and POST requests are predefined.
http:add-resource
[procedure] (http:add-resource URL HANDLER)
Defines a new resource for URL (which may be a string, a symbol or a list of strings/symbols). HANDLER should be a procedure of two arguments: a request structure, and an a-list mapping urlencoded arguments to values.
During execution of the handler the current input- and an output-ports are bound to ports communicating with the client.
http:remove-resource
[procedure] (http:remove-resource URL)
Removes the resource defined under URL (string, symbol or list).
http:find-resource
[procedure] (http:find-resource URL)
Returns the handler-procedure for the resource defined under URL or #f if no such resource is registered.
http:fallback-handler
[parameter] http:fallback-handler
Contains a procedure that is is invoked on requests for resources that could not be found. This procedure is then called with the original request object. The default handler generates a 404 response.
http:error-response-handler
[parameter] http:error-response-handler
Contains a procedure that will be called when an HTTP error-response should be generated. The procedure is called with the error-code and message and should return a string containing HTML that will be sent in the body of the error-response.
http:current-request-count
[procedure] (http:current-request-count)
Returns the number of requests handled since startup.
http:request-count-limit
[parameter] http:request-count-limit
Maximum number of concurrently handled requests.
http:startup-hook
[parameter] http:startup-hook
A procedure that will be called on server startup, before accepting first request. The default procedure does nothing.
http:error-hook
[parameter] http:error-hook
A procedure that will be called when a thread triggers an unhandled exception. The exception is passed as an argument to the hook procedure.
http:log-hook
[parameter] http:log-hook
A procedure that will be called upon completion of a HTTP request. The procedure will be called with two arguments: the request object and the IP address of the client. The default value of this parameter does nothing.
http:url-transformation
[parameter] http:url-transformation
A procedure that allows arbitrary transformations of the URL part of each incoming request. The procedure is called with a single argument (the URL string) and should return the same URL, or a transformed one. The default transformation returns the original url unchanged.
http:listen-procedure
[parameter] http:listen-procedure
Holds a procedure that will be used to create a socket-listener - defaults to tcp-listen.
http:accept-procedure
[parameter] http:accept-procedure
Holds a procedure that will be used to accept a socket-connection - defaults to tcp-accept.
http:get-addresses-procedure
[parameter] http:get-addresses-procedure
Holds a procedure that will be used to obtain peer addresses - defaults to tcp-addresses.
http:hard-close-procedure
[parameter] http:hard-close-procedure
Holds a procedure that will be used to close a connection prematurely - defaults to tcp-abandon-port.
http-utils
Usage
(require-extension http-utils)
http:decode-url
[procedure] (http:decode-url URL)
Canonicalizes the string passed in URL and returns two values: the location-path and an alist containing argument <-> value pairs.
http:make-request
[procedure] (http:make-request METHOD URL [ATTRIBUTES [BODY [PROTOCOL [IP]]]])
Returns a freshly created HTTP request object (see below for the meaning of the arguments).
http:request?
[procedure] (http:request? X)
Returns #t if X is a request object, or #f otherwise.
http-request accessor methods
[procedure] (http:request-url REQUEST) -> URL [procedure] (http:request-protocol REQUEST) -> PROTOCOL [procedure] (http:request-attributes REQUEST) -> ATTRIBUTES [procedure] (http:request-body REQUEST) -> X [procedure] (http:request-method REQUEST) -> METHOD [procedure] (http:request-ip REQUEST) -> STRING [procedure] (http:request-sslctx REQUEST) -> <ssl-client-context> [procedure] (http:request-url-set! REQUEST URL) [procedure] (http:request-protocol-set! REQUEST PROTOCOL) [procedure] (http:request-attributes-set! REQUEST ATTRIBUTES) [procedure] (http:request-body-set! REQUEST X) [procedure] (http:request-method-set! REQUEST METHOD) [procedure] (http:request-ip-set! REQUEST STRING) [procedure] (http:request-sslctx-set! REQUEST <ssl-client-context>)
Accessor procedures for the components of a HTTP request structure. URL is a string, METHOD and PROTOCOL are symbols, BODY is either #f or a string and ATTRIBUTES is an a-list where each pair contains an attribute string and a value string.
http:read-line-limit
[parameter] http:read-line-limit
The maximum length of a header-line.
http:read-request-attributes
[procedure] (http:read-request-attributes PORT)
Reads MIME type headers from PORT until end of file is reached or a line is not a valid MIME header and returns an a-list where each pair holds the header (converted to lowercase) and the value (both strings).
http:canonicalize-string
[procedure] (http:canonicalize-string STRING)
Canonicalizes STRING by substituting %XX and + sequences.
Examples
Server example
A simple "Hello, world" server:
(require-extension http-server) (http:add-resource '("/" "/index.html") (lambda (r a) (http:write-response-header) (let ([msg "<h1>Hello, world!</h1>"]) (print "Content-type: text/html\r\nContent-length: " (add1 (string-length msg)) "\r\n\r\n" msg) ) ) ) ((http:make-server 4242) #t)
To try it out, simply load the code into the interpreter and point your browser to localhost:4242.
Client example
(require-extension http-client) (define-values (h a i o) (http:send-request "localhost:4242/")) (pretty-print (read-lines i)) (close-input-port i) (close-output-port o)
Loading this into the interpreter will print
("<h1>Hello, world!</h1>")
(provided the hello-world server is running)
Changelog
- 1.54 Fix passing of POST arguments to handlers defined with http:add-resource
- 1.53 Generation of debugging info from get/post-handler (ticket #253) [Mario Goulart]
- 1.52 Added requirement of url egg to meta info [thanks to Mario Goulart]
- 1.51 Client fix for handling closed keep-alive connections [Daishi Kato]
- 1.50 Client API for http proxies [Daishi Kato]
- 1.49 Client API flushes output before reading to be able to work with buffered ports
- 1.48 Bugfix in http-client.scm [Daishi Kato]
- 1.47 Removed multipart support, added unparsed request body field, content-parsers return additional raw body
- 1.46 Bugfix in url-parser
- 1.45 Cookie hack for http:GET and http:POST [Daishi Kato]
- 1.44 http:POST now handles www-form-urlencoded content-type, which is the default [Daishi Kato]
- 1.43 Client support for chunked transfer encoding [Daishi Kato]
- 1.42 Client support for persistent connections [Daishi Kato]
- 1.41 Client support for https (requires openssl egg) [Daishi Kato]
- 1.40 Client requests did finalize ports in the wrong situation [Thanks to Tim Reid]
- 1.39 URL canonicaliation fix #2 [Thanks to Zbigniew Szadkowski]
- 1.38 URL-canonicalization fix by Peter Busser
- 1.37 Added hidden slot to request-structure
- 1.36 Fixed bug in http:write-error-response [Thanks to Peter Bex]
- 1.35 Any error-response forces closing of client connection [Suggested by Diashi Kato]
- 1.34 Added -lws2_32 to build-command for http-client on Windows [Thanks to Daishi Kato]
- 1.33 Added parameterized socket operations to server
- 1.32 Adapted to SRFI-69 compatible hash-tables
- 1.31 The ports returned by http:send-request, http:GET and http:POST are finalized [Thanks to Reed Sheridan]
- 1.30 Added ip field to request object
- 1.29 Added http:url-transformation, debug output includes response headers
- 1.28 Replaced use of (end-of-file) with #!eof
- 1.27 Added http:request-limit; fixed http:POST (which didn't handle the second argument always correctly)
- 1.26 Special-cased regex strings to work around pregexp bugs [Thanks to Peter Bex]
- 1.25 Retries on failed tcp-accept, error-message output is limited
- 1.24 Fixed bug in server example [Thanks to Graham Fawcett]
- 1.23 Added note about SIGPIPE [Thanks Graham Fawcett]
- 1.22 Fixed regex bug in http-utils.scm [Thanks to Patrick Brannan]
- 1.21 Added http:log-hook
- 1.20 Content-parser got a third argument (attribute a-list); support for multipart/form-data content type; default handler for unknown content types
- 1.19 Added documentation for http:read-request-attributes
- 1.18 Added http:GET and http:POST
- 1.17 URL parsing of requests and header-generation is somewhat more robust [Thanks to Peter Bex]
- 1.16 http:send-request assumed old meaning of request body
- 1.15 If the content-type is not known, a request-body has the empty list (()) as the body [Thanks to Peter Bex]
- 1.14 Urlencoded arguments without argument are handled better and escapes in the argument name are processed correctly
- 1.13 URL parsing regex fix #394493849 [Thanks to Lars Rustemeier]
- 1.12 Added http:current-request-count
- 1.11 http:make-server takes keyword arguments now and accepts two additional arguments
- 1.10 http:write-response-header accepts yet another optional argument (protocol)
- 1.9 Fixed bug that required uint32_t which wasn't necessarily everywhere available [Thanks to Mikel Evins]
- 1.8 Adapted to new setup scheme
- 1.7 URL parsing in (http-client) again. Will it ever stop?
- 1.6 URL parsing in (http-client) works now with PCRE
- 1.5 Fixed bug in regular expression for URL parsing and another one in the regexp for HTTP headers
- 1.4 The URL parsing in (http-client) didn't handle XXX.XXX.XXX.XXX-style URLs [Thanks to Lars Rustemeier]
- 1.3 http:write-response-header doesn't override headers given in the alist.
- 1.2 http:write-response-header accepts more optional arguments. Added default content-parser for application/x-www-form-urlencoded bodies. http:canonicalize-string.
- 1.1 The content-type from a request may be followed by optional parameters. Fixed bug in canonicalize-string. The argument to the fallback-handler is now a request object.
- 1.0 Initial release