You are looking at historical revision 6805 of this page. It may differ significantly from its current revision.

http

Description

An easy to use HTTP client and server package. The server is fully multithreaded and supports persistent connections.

Author

Felix Winkelmann

Requirements

Requires the regex-case egg at compile-time.

http-client requires the url egg at runtime.

Both http-client and http-server require the http-utils module at runtime.

Documentation

What follows is a description of the http extension, which is separated into three sub-extensions: http-client (HTTP client functionality), http-server (serving HTTP requests) and http-utils (utility functions common to client and server code).

If you want to run a web-server, you should also take a look at the more featureful spiffy extension.

http-client

Usage

(require-extension http-client)

http:send-request
[procedure] (http:send-request REQUEST [INPUT-PORT OUTPUT-PORT])

Sends an HTTP request represented by REQUEST, which may either be a HTTP request object, or a string that specifies an URL. The request is sent to it's destination and four values are returned: a string containing the first line of the response received from the server, an a-list that maps HTTP headers to values (strings) and the input- and output-port of the connection. Note that the connection is still open, and the returned ports can be passed in subsequent invocations of http:send-request to achieve a persistent connection. The ports are closed automatically, if no longer referenced.

If an URL is given instead of a request record, a request is sent of the form

GET url HTTP/1.0
Connection: close
http:GET
[procedure] (http:GET REQUEST)

Sends a GET request represented by REQUEST, which may be a string (an URL) or a HTTP request object, and returns a string containing the body of the servers response. The REQUEST keeps a Cookie header that is set by Set-Cookie headers in a response, unsafely (hack), regardless of the storing policy. Be aware of it when reusing REQUEST.

http:POST
[procedure] (http:POST REQUEST [ARGUMENTS] [DELIM1] [DELIM2] [ENCODER])

Sends a POST request represented by REQUEST, which may be a string (an URL) or a HTTP request object, and returns a string containing the body of the servers response. ARGUMENTS should be either a list of parameters passed in the body of the POST request and should contain strings of the form "name=value", or an alist of "name" and "value".

If REQUEST is a string or a HTTP request object whose Content-Type attribute is application/x-www-form-urlencoded, the HTTP body is created with the form "name1=value1&name2=value2...". Otherwise, the HTTP body is concatenated using DELIM1, DELIM2, and ENCODER. The REQUEST keeps a Cookie header, as with http:GET

http:close-all-connections!
[procedure] (http:close-all-connections!)

Close all persistent connections kept in the current thread.

http:read-body
[procedure] (http:read-body ARGUMENTS PORT)

Read the HTTP Body from the input port PORT, according to the Content-Length header or the Transfer-Encoding header.

http:add-proxy!
[procedure] (http:add-proxy! PROXY-HOST PROXY-PORT [SERV-PATTERN] [HOST-PATTERN] [PORT-PATTERN] [PATH-PATTERN])

Add a http proxy. PATTERNs are either a string, a regex, or #t. The first added proxy has the least priority.

Note: CONNECT method for SSL is not supported (yet).

http:remove-all-proxies!
[procedure] (http:remove-all-proxies!)

Remove all http proxies.

http-server

Usage

(require-extension http-server)

General operation

The server maps URLs to resource-handlers, which are procedures that process incoming client-requests. A resource-handler is responsible for generating a response by writing output to the value of (current-output-port).

The data contained in a client-request is parsed by a so-called content-parser, which is a procedure that reads the request-body from the port given by (current-input-port). Content-types are encoded as symbols.

A parser for application/x-www-form-urlencoded is predefined, other content-parsers have to be defined by application-code using the procedure http:content-parser, or the default parser will be invoked (which reads the content as a plain string).

The content-parser for text returns the request-body as a string. The content-parser for urlencoded data returns the request-body as an a-list that maps variables to strings.

http:make-server
[procedure] (http:make-server PORTNUMBER #!key NAME PROTOCOL BACKLOG ACCEPT INIT)

Creates and returns a server-procedure. NAME defaults to something silly, PROTOCOL to #f (ignored), BACKLOG to 40 and ACCEPT to #f. INIT should be a procedure of no arguments that will be called after the networking initialisation has taken place (specifically, after the invocation of tcp-listen).

To run the server-loop, invoke the returned procedure, which takes an optional boolean argument (passing #t will generate debugging output). PORTNUMBER, BACKLOG, and ACCEPT are directly passed on to tcp-listen (see Unit tcp for more information about those parameters).

http:content-parser
[procedure] (http:content-parser CONTENTTYPE [PROC])

Returns or sets the parser-procedure (a procedure of three arguments: the size of the content (may be #f), the a-list of request headers and an input port) for CONTENTTYPE, which should be a symbol.

The content-parser procedure PROC should return two values: the parsed and the unparsed (raw) request body.

http:write-response-header
[procedure] (http:write-response-header REQ [CODE MSG [ALIST [PORT [PROTOCOL]]]])

Writes a HTTP response header for request REQ to PORT, containing the server-name. CODE and MSG default to 200 and OK, respectively. The optional ALIST may contain pairs with header-names and -values. If given, PROTOCOL specifes the HTTP protocol to use for the reply, which should be a symbol (either HTTP/1.0 or HTTP/1.1) and defaults to the protocol given in http:make-server.

http:write-error-response
[procedure] (http:write-error-response CODE MESSAGE [PORT])

Writes a HTTP error-response to PORT, which defaults to the value of (current-output-port). Uses the result returned by the value of (http:error-response-handler).

http:request-method-handler
[procedure] (http:request-method-handler METHOD [PROC])

Reads ot sets the handler procedure PROC for the request-method METHOD (which should be a symbol). PROC should accept one argument, a request-object. During execution of the handler, the current input- and output ports are bound to ports connected to the client.

Method-handlers for GET and POST requests are predefined.

http:add-resource
[procedure] (http:add-resource URL HANDLER)

Defines a new resource for URL (which may be a string, a symbol or a list of strings/symbols). HANDLER should be a procedure of two arguments: a request structure, and an a-list mapping urlencoded arguments to values.

During execution of the handler the current input- and an output-ports are bound to ports communicating with the client.

http:remove-resource
[procedure] (http:remove-resource URL)

Removes the resource defined under URL (string, symbol or list).

http:find-resource
[procedure] (http:find-resource URL)

Returns the handler-procedure for the resource defined under URL or #f if no such resource is registered.

http:fallback-handler
[parameter] http:fallback-handler

Contains a procedure that is is invoked on requests for resources that could not be found. This procedure is then called with the original request object. The default handler generates a 404 response.

http:error-response-handler
[parameter] http:error-response-handler

Contains a procedure that will be called when an HTTP error-response should be generated. The procedure is called with the error-code and message and should return a string containing HTML that will be sent in the body of the error-response.

http:current-request-count
[procedure] (http:current-request-count)

Returns the number of requests handled since startup.

http:request-count-limit
[parameter] http:request-count-limit

Maximum number of concurrently handled requests.

http:startup-hook
[parameter] http:startup-hook

A procedure that will be called on server startup, before accepting first request. The default procedure does nothing.

http:error-hook
[parameter] http:error-hook

A procedure that will be called when a thread triggers an unhandled exception. The exception is passed as an argument to the hook procedure.

http:log-hook
[parameter] http:log-hook

A procedure that will be called upon completion of a HTTP request. The procedure will be called with two arguments: the request object and the IP address of the client. The default value of this parameter does nothing.

http:url-transformation
[parameter] http:url-transformation

A procedure that allows arbitrary transformations of the URL part of each incoming request. The procedure is called with a single argument (the URL string) and should return the same URL, or a transformed one. The default transformation returns the original url unchanged.

http:listen-procedure
[parameter] http:listen-procedure

Holds a procedure that will be used to create a socket-listener - defaults to tcp-listen.

http:accept-procedure
[parameter] http:accept-procedure

Holds a procedure that will be used to accept a socket-connection - defaults to tcp-accept.

http:get-addresses-procedure
[parameter] http:get-addresses-procedure

Holds a procedure that will be used to obtain peer addresses - defaults to tcp-addresses.

http:hard-close-procedure
[parameter] http:hard-close-procedure

Holds a procedure that will be used to close a connection prematurely - defaults to tcp-abandon-port.

http-utils

Usage

(require-extension http-utils)

http:decode-url
[procedure] (http:decode-url URL)

Canonicalizes the string passed in URL and returns two values: the location-path and an alist containing argument <-> value pairs.

http:make-request
[procedure] (http:make-request METHOD URL [ATTRIBUTES [BODY [PROTOCOL [IP]]]])

Returns a freshly created HTTP request object (see below for the meaning of the arguments).

http:request?
[procedure] (http:request? X)

Returns #t if X is a request object, or #f otherwise.

http-request accessor methods
[procedure] (http:request-url REQUEST) -> URL
[procedure] (http:request-protocol REQUEST) -> PROTOCOL
[procedure] (http:request-attributes REQUEST) -> ATTRIBUTES
[procedure] (http:request-body REQUEST) -> X
[procedure] (http:request-method REQUEST) -> METHOD
[procedure] (http:request-ip REQUEST) -> STRING
[procedure] (http:request-sslctx REQUEST) -> <ssl-client-context>
[procedure] (http:request-url-set! REQUEST URL)
[procedure] (http:request-protocol-set! REQUEST PROTOCOL)
[procedure] (http:request-attributes-set! REQUEST ATTRIBUTES)
[procedure] (http:request-body-set! REQUEST X)
[procedure] (http:request-method-set! REQUEST METHOD)
[procedure] (http:request-ip-set! REQUEST STRING)
[procedure] (http:request-sslctx-set! REQUEST <ssl-client-context>)

Accessor procedures for the components of a HTTP request structure. URL is a string, METHOD and PROTOCOL are symbols, BODY is either #f or a string and ATTRIBUTES is an a-list where each pair contains an attribute string and a value string.

http:read-line-limit
[parameter] http:read-line-limit

The maximum length of a header-line.

http:read-request-attributes
[procedure] (http:read-request-attributes PORT)

Reads MIME type headers from PORT until end of file is reached or a line is not a valid MIME header and returns an a-list where each pair holds the header (converted to lowercase) and the value (both strings).

http:canonicalize-string
[procedure] (http:canonicalize-string STRING)

Canonicalizes STRING by substituting %XX and + sequences.

Examples

Server example

A simple "Hello, world" server:

(require-extension http-server)

(http:add-resource '("/" "/index.html")
  (lambda (r a)
    (http:write-response-header)
    (let ([msg "&lt;h1>Hello, world!&lt;/h1>"])
      (print "Content-type: text/html\r\nContent-length: " (add1 (string-length msg)) "\r\n\r\n" msg) ) ) )

((http:make-server 4242) #t)

To try it out, simply load the code into the interpreter and point your browser to localhost:4242.

Client example

(require-extension http-client)

(define-values (h a i o) (http:send-request "localhost:4242/"))

(pretty-print (read-lines i))
(close-input-port i)
(close-output-port o)

Loading this into the interpreter will print

("<h1>Hello, world!</h1>")

(provided the hello-world server is running)

Changelog

License

Copyright (c) 2003, Felix L. Winkelmann
All rights reserved.

Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following
conditions are met:

  Redistributions of source code must retain the above copyright notice, this list of conditions and the following
    disclaimer. 
  Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following
    disclaimer in the documentation and/or other materials provided with the distribution. 
  Neither the name of the author nor the names of its contributors may be used to endorse or promote
    products derived from this software without specific prior written permission. 

THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY EXPRESS
OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY
AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDERS OR
CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR
SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR
OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
POSSIBILITY OF SUCH DAMAGE.