spiffy-request-vars

Introduction

spiffy-request-vars provides easy access to variables from HTTP requests.

Thanks to Moritz Heidkamp for the implementation of the content body reader and for several terminology suggestions (including the egg name). Also thanks to Peter Bex for the helpful discussions and lots of implementation suggestions.

Repository

https://github.com/mario-goulart/spiffy-request-vars

Procedures parameters and macros

Procedure

request-vars

[procedure] (request-vars #!key (source 'request-method) max-content-length)

request-vars returns a procedure which can be used to access variables from the HTTP request. The returned procedure takes the name of the variable (either a symbol or a string) as argument. You can (optionally) also pass a converter procedure to be used as a type converter for the variable value (see Converter procedures) or a default value.

request-vars accepts some keyword arguments:

source: 'query-string tells request-vars to parse the query string only (for GET variables). 'request-body tells request-vars to parse the request body only (e.g., for POST variables). 'both tells request-vars to parse both the request body and the query string. 'request-method tells request-vars to parse only the source that matches the request method (e.g., for a GET request, only the query string will be read; for a POST request, only the request body will be read). The default value for source is 'request-method (since version 0.18 -- previous versions used 'both). Notice that when 'both is used, variables from the request body have precedence over the ones from the query string. Warning: using 'both as source for request-vars may be a security issue (see the release notes for version 0.18 for more details).
max-content-length: the maximum content length (in characters) to be read from the request body. Default is #f (no limit).

Converter procedures

The following procedures are intended to be used by the procedure returned by request-vars. The variables/values parameter is an alist mapping variable names to their corresponding values, resulting from parsing the request.

as-string

[procedure] (as-string variable variables/values)

If the given variable is set, return its value as a string (that's the default behavior if no converter is specified).

as-symbol

[procedure] (as-symbol variable variables/values)

If the given variable is set, convert its value to a symbol using string->symbol.

as-number

[procedure] (as-number variable variables/values)

If the given variable is set, convert its value to a number using string->number.

as-boolean

[procedure] (as-boolean variable variables/values)

If the variable is set and its value is one of the values yield by the true-boolean-values parameter, return #t, otherwise return #f. It also returns #t if the variable is passed in the request but is not bound to any value.

as-list

[procedure] (as-list variable variables/values)

If the variable is set once, returns a list with a single element (the value for the given variable). If the variable is set multiple times, return a list with the multiple values for the variable. If the variable is not set, return #f.

as-alist

[procedure] (as-alist variable variables/values)

Returns an alist represented by the request variables/values for the given variable. The request representation for alists is variable.key=value. Example: foo.a=1 would result in '((a . "1")) for the foo variable.

Example:

;; considering a http://server:port/path?foo.a=0&foo.b=1 request

(let (($ (request-vars)))
  ($ 'foo as-alist))   ;; => ((a . "0") (b . "1"))

as-alist returns #f when the wanted variable is not sent in the request or it is sent not in the dot notation (e.g., foo=0).

as-hash-table

[procedure] (as-hash-table variable variables/values)

The same as as-alist, but returns a hash-table object instead of an alist.

as-vector

[procedure] (as-vector variable variables/values)

Returns a vectir represented by the request variables/values for the given variable. The request representation for vectors is variable.numeric-index=value. Example: foo.0=1 would result in #("1") for the foo variable.

Example:

;; considering a http://server:port/path?foo.0=a&foo.1=b

(let (($ (request-vars)))
  ($ 'foo as-vector))   ;; => #("a" "b")

as-vector returns #f when the wanted variable is not sent in the request or it is sent not in the dot notation (e.g., foo=0).

If the vector represented by the request is sparse, the missing items are unspecified values.

Combinator

nonempty

[procedure] (nonempty converter)

A combinator to be used with converters. Returns the converter value if the variable is set and its value is not null. Returns #f if its value is null.

It can be useful for handling values from form-submited data, when all form fields are submited, but some are null. If you are only interested in values that are not null, you can just check if the return value of nonempty is not #f (otherwise you'd have to check if the variable was actually in the request and if its value is not null).

Example:

(let ((var (or ($ 'var (nonempty as-string)) "not set")))
   var)

Parameters

true-boolean-values

[parameter] (true-boolean-values [list])

A list of values (strings) to be considered as #t for request variables when as-boolean is used as converter.

The default value is '("y" "yes" "1" "on" "true"). The values are compared using string-ci=?.

compound-variable-separator

[parameter] (compound-variable-separator [string])

A string representing the separator for request variable names bound to compound data types (vectors, alists, hash-tables). The default value is ".".

For example, if (compound-variable-separator) yields "." and the query string is ?foo.A=0&foo.B=1, if you bind it as an alist, you'll get ((A . 0) (B . 1)). If you want the same behavior, but for query string with variables like ?foo_A=0&foo_B=1, you can set compound-variable-separator to "_".

Example

  (let (($ (request-vars)))
    ($ 'var1)
    ($ 'var2 "") ;; if var12 is not set, return ""
    ($ 'var3 as-number)) ;; if var3 is not set, return #f; if it is
                         ;; set, convert its value to a number

Macros

with-request-vars

[syntax] (with-request-vars [getter] (var1 var2 ... varN) expr1 expr2 ... exprN)

Bind the given identifiers to the corresponding query string and request body variable values and evaluate the expressions. The optional getter argument (the return value of request-vars) may be used in situations when you already have the getter and don't want to reparse the query string and request body. With with-request-vars*, the given getter will be used and no reparsing will be performed. When the syntax is ambiguous (e.g., (with-request-vars (request-vars) (var1 var2) (body)), with-request-vars* can be used).

Examples

(with-request-vars (a b c)
   (conc "a = " a
         "b = " b
         "c = " c))

(let (($ (request-vars)))
  (with-request-vars $ (a b c)
     (conc "a = " a
           "b = " b
           "c = " c)))

A converter procedure can also be used to specify the type of the variable values:

(let (($ (request-vars)))
  (with-request-vars $ (a (b as-list) (c as-number))
     (conc "a = " a
           "b = " b
           "c = " c)))

with-request-vars*

[syntax] (with-request-vars* getter (var1 var2 ... varN) expr1 expr2 ... exprN)

The same as with-request-vars, but the getter is mandatory.

More examples

Considering

(define $ (request-vars))

here are some expected results for the given requests:

;; http://host:port/

($ 'foo)             => #f
($ 'foo 'bar)        => bar
($ 'foo as-list)     => #f
($ 'foo as-boolean)  => #f
($ 'foo as-number)   => #f


;; http://host:port/?foo=bar

($ 'foo)             => "bar"
($ 'foo 'bar)        => "bar"
($ 'foo as-list)     => ("bar")
($ 'foo as-boolean)  => #f
($ 'foo as-number)   => #f


;; http://host:port/?foo=bar&foo=baz

($ 'foo)             => "bar"
($ 'foo 'bar)        => "bar"
($ 'foo as-list)     => ("bar" "baz")
($ 'foo as-boolean)  => #f
($ 'foo as-number)   => #f


;; http://host:port/?foo=0

($ 'foo)             => "0"
($ 'foo 'bar)        => "0"
($ 'foo as-list)     => ("0")
($ 'foo as-boolean)  => #f
($ 'foo as-number)   => 0


;; http://host:port/?foo=yes

($ 'foo)             => "yes"
($ 'foo 'bar)        => "yes"
($ 'foo as-list)     => ("yes")
($ 'foo as-boolean)  => #t
($ 'foo as-number)   => #f


;; http://host:port/

(with-request-vars (foo (bar as-list) (baz 5))
  (list foo bar baz) => (#f #f 5)


;; http://host:port/?foo=10

(with-request-vars (foo (bar as-list) (baz 5))
  (list foo bar baz) => ("10" #f 5)


;; http://host:port/?foo=10&bar=1

(with-request-vars (foo (bar as-list) (baz 5))
  (list foo bar baz) => ("10" ("1") 5)


;; http://host:port/?foo=10&bar=1&bar=2

(with-request-vars (foo (bar as-list) (baz 5))
  (list foo bar baz) => ("10" ("1" "2") 5)


;; http://host:port/?foo=10&bar=1&bar=2&baz=-8

(with-request-vars (foo (bar as-list) (baz 5))
  (list foo bar baz) => ("10" ("1" "2") "-8")


;; http://host:port

(with-request-vars ((foo as-alist) (bar as-number) (baz as-vector) (bool as-boolean))
  (list foo bar baz bool)) => (#f #f #f #f)


;; http://host:port/?foo.A=0&foo.B=1&bar=0&baz.0=a&baz.1=b&bool=yes

(with-request-vars ((foo as-alist) (bar as-number) (baz as-vector) (bool as-boolean))
  (list foo bar baz bool)) => (((A . "0") (B . "1")) 0 #("a" "b") #t)


;; http://host:port/?foo=0&bar=a&baz=0&bool=3

(with-request-vars ((foo as-alist) (bar as-number) (baz as-vector) (bool as-boolean))
  (list foo bar baz bool)) => (#f #f #f #f)

Tips and tricks

If you want to specify both converters and default values, you can use the following trick:


;; Define a procedure to return the default value if the
;; variable is not set.
(define ((as-number/default default) var vars/vals)
  (or (as-number var vars/vals) default))


;; http://host:port/

(with-request-vars (foo (bar as-list) (baz (as-number/default 3)))
  (->string (list foo bar baz))) => (#f #f 3)


;; http://host:port/?baz=9

(with-request-vars (foo (bar as-list) (baz (as-number/default 3)))
  (->string (list foo bar baz))) => (#f #f 9)

License

Copyright (c) 2008-2018, Mario Domenech Goulart
All rights reserved.

Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions are
met:

Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.

Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.

Neither the name of the author nor the names of its contributors may
be used to endorse or promote products derived from this software
without specific prior written permission.

THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
"AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS
FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE
COPYRIGHT HOLDERS OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT,
INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES
(INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR
SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION)
HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT,
STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED
OF THE POSSIBILITY OF SUCH DAMAGE.

Requirements

Version history

version 0.19

Drop awful as dependency for tests (fix circular dependency)
CHICKEN 5 support

version 0.18

The default value for request-vars' source keyword parameter is now request-method.

Using 'both as the default value for request-vars' source keyword params is dangerous. Consider the following example (in awful):

(use awful)

(enable-sxml #t)

(define-page (main-page-path)
  (lambda ()
    (with-request-variables (user admin)
      (if user
          `(ul
            (li ,user)
            (li ,(or admin "--")))
          `(form (@ (method "post"))
                 (input (@ (type "text") (name "user")))
                 (input (@ (type "submit")))))))
  method: '(get post))

An attacker could maliciously make an user follow a link to the form with the query string set to ?admin=bar. If source is bound to 'both, request-vars will read from both the query string and the request body, leading to user=<what user filled> and admin=bar.

This change sets the default value for source to request-method, that is, request-vars will read the query string or the request body depending on the request method (never both).

With this change, in the example mentioned above, if an attacker makes an user follow a link to the form with the query string set to ?admin=bar, request-vars will read the request body only, since the form method is post. Thus, the handler will get user=<what user filled> and admin=#f.

Thanks to Peter Bex for the heads-up and discussions on this issue.

version 0.17

Add compound-variable-separator (suggested by Taylor Venable)

version 0.16

Test fixes

version 0.15

Allow calling getter returned by request-vars with no arguments to get an alist of all request vars (patch by Moritz Heidkamp -- #719)

version 0.14

Fixed multiple evaluation of request-vars in with-request-vars* (by Moritz)
Bug fix: source: 'both makes request-vars actually read from both the request body and query string (thanks to Moritz for reporting that).

version 0.13

Added as-astring and as-symbol converters and nonempty combinator

version 0.12

Bug fixes: request-vars returns #f when the content-type is not application/x-www-form-urlencoded (thanks to Peter Bex for reporting this problem). req-vars/vals always return a list.

version 0.11

Applied patch by Moritz Heidkamp which enhances as-boolean in two ways: It adds "true" to the default list of true-boolean-values and it considers the mere presence of the parameter as #t, i.e. a query string like "?foo" would bind foo to #t when it is cast as-boolean.

version 0.10

Bug fix for some corner cases

version 0.9

Added as-vector, as-alist and as-hash-table as converters.

version 0.8

Bug fix/improvement: don't bother reading the request body when the content-length is zero (fixes some awful/jquery ajax issues)

version 0.7

Bug fix. Interpret request body before query string.

version 0.6

support for specifying types to variables
support for receiving lists from the URI
Warning: compatibility with previous versions has been broken: now the procedure returned by request-vars accepts only two arguments, not three. So, if you are using converter procedures as argument to the procedure returned by request-vars, beware that your code will break.

version 0.5

with-request-vars accepts a getter as argument when the syntax is not ambiguous.

version 0.4

with-request-vars* resurrected. For the cases when the syntax of with-request-vars is ambiguous. Thanks to Moritz Heidkamp for catching this bug.

version 0.3

Removed with-request-vars*. with-request-vars accepts an optional getter argument

version 0.2

Added with-request-vars and with-request-vars*

version 0.1

Initial release