You are looking at historical revision 31630 of this page. It may differ significantly from its current revision.

websockets

Description

websockets is a fast, lightweight, and simple implementation of the websockets protocol. It currently only supports version 13 of the websocket protocol and no extensions.

websockets includes both a high level and low level API. The high level API provides both a blocking and concurrent interface. Note that contrary to most or even all other websocket implementations it does not provide an asynchronous interface but it does however provide a concurrent backed interface. See the section on the high level interface for more details. The low level API provides all of the primitives for working directly with websocket connections, messages, and fragments. It can be used for special circumstances, like when more fine grained control is desired for accepting or processing fragments, or for building a different high level API. The provided high level API is based on the exposed low level API.

All high level procedures are thread safe. See the low level interface for details on which procedures are and are not thread safe at that level.

All errors triggered by the library are of condition type websocket and all are continuable (not that they should be continued in all cases).

websockets passes all of the relevant Autobahn websockets tests. In the individual message sending and processing benchmark it performs better than most websocket or possibly all implementations in most cases especially with larger message sizes. It uses less memory than all the tested implementations for all of the test cases and benchmarks. It does not fare well, relatively, in the round-trip-time tests but I do not consider that a problem or something to optimize for since they are far removed from real world usage. An API tailored for that use case could be built on the low-level API but nonetheless it isn't currently provided in this library.

Author

Thomas Hintz with contributions from Seth Alves.

Please send an email to t@thintz.com or chicken-users@nongnu.org with questions, bug reports, or feature requests.

Repository

The git repository for the websockets source code is hosted by bitbucket: https://bitbucket.org/thomashintz/websockets.

Requirements

The following eggs are required:

Quick start example

Put these two files in the same folder.

index.html

<html>
  <body>
    <script type="text/javascript">
      var ws = new WebSocket("ws://localhost:8080/web-socket");
      ws.onmessage = function(evt) {
        alert(evt.data);
      };
      ws.onopen = function() {
        ws.send('Hello!');
      }
    </script>
  </body>
</html>

echo.scm

(import chicken scheme)
(use spiffy websockets)

(handle-not-found
 (lambda (path)
   (when (string= path "/web-socket")
         (with-websocket
          (lambda ()
            (send-message (string-append "you said: " (receive-message))))))))

(root-path ".")
(start-server port: 8080)

Then, in the same directory, run:

csi -s echo.scm

You should see an alert saying "you said: Hello!".

Parameters

current-websocket
[parameter] (current-websocket [websocket])

Only available with with-websocket or with-concurrent-websocket. (current-websocket) will be bound to a websocket object. Many procedures provide an optional argument for a websocket object and it is bound to current-websocket by default.

ping-interval
[parameter] (ping-interval [number])

How often to ping the client, in seconds, in the background. If 0 then automatic pinging will be disabled. This defaults to 15 seconds.

If this is set to a value greater than 0 then a thread will run in the background sending ping messages to the client at the specified interval. This is used to keep connections open that will otherwise be closed. Often connections without a transmission will be killed after awhile a short while. Receiving pongs in response to a ping will also reset the connection close timer.

pong responses to ping messages are not passed through to the user when using the high level API but they are used to update the timestamp that the connection timeout thread uses to decide if it should kill a connection.

close-timeout
[parameter] (close-timeout [number])

How long to wait, in seconds, for the client to respond to a connection close request before timing out. If 0 then the connection timeout will be disabled. The default value is 5 seconds.

connection-timeout
[parameter] (connection-timeout [number])

The length of time in seconds without a response (of any kind) from the client before the connection to the client will be cut off. If 0 then the connection will never be timed out by the websocket (something else can, of course, timeout causing the websocket connection to fail anyways). The default is 60 seconds.

accept-connection
[parameter] (accept-connection [procedure])

A one-argument (URL path of the current page) procedure which tells whether to accept the websocket connection. If #t accept, #f reject. IT IS HIGHLY RECOMMENDED that this parameter be set otherwise it opens up your application to potential security vulnerabilities. You will probably want to verify it is coming from a specific source. The default is to accept all connections.

access-denied
[parameter] (access-denied [procedure])

A procedure to be called when the origin header value is rejected by the accept-connection procedure.

The default is:

(lambda () (send-status 'forbidden "<h1>Access denied</h1>"))
drop-incoming-pings
[parameter] (drop-incoming-pings [boolean])

Clients should usually not initiate pings so the default is to drop all incoming pings without responding with a pong. This defaults to #t. Set this to #f to respond to incoming pings with a pong.

propagate-common-errors
[parameter] (propagate-common-errors [boolean])

A lot of errors that occur in the lifecycle of a websocket connection are more-or-less expected and it often is not interesting to receive and deal with these errors. The default is to correctly close the connection when an error occurs with the required opcode but the error is then not signaled to the user. The default is #f. Set to #t to receive all errors.

All websocket specific errors are covered by this. The only error you may receive if this is #f is of type websocket and unexpected-error if something goes terribly wrong with the implementation and usually means there is a bug in the implementation. This does not affect errors that are caused by user code, they will always be passed on but the websocket will be properly with an error code of 1011 (unexpected-error).

Note that this parameter is only relevant when using with-websocket or with-concurrent-websocket. If you don't use one of those then all errors will be propagated.

max-frame-size
[parameter] (max-frame-size [number])

The maximum allowed frame payload size. The default is 1MiB. If a frame exceeds this size then its payload will not be read and the connection will be dropped immediately. This signals a message-too-large error.

The maximum frame size supported by the current websockets implementation is 1GiB.

max-message-size
[parameter] (max-message-size [number])

The maximum allowed total message size. The default is 1MiB. If a frame will cause the max-message-size to be exceeded then its payload is not read and the connection is dropped immediately. This signals a message-too-large error.

The maximum message size supported by the current websockets implementation is 1GiB.

High level interface

As noted in the description, the high level interface is not a "traditional" asynchronous API as is often seen with other websocket libraries. Instead a blocking and synchronous interface is provided as well as an asynchronous backed interface that looks and behaves like a blocking interface. See the function definitions below for further details.

Note that if you don't use with-websocket or with-concurrent-websocket then you will need to manually handle opening and closing the websocket connection as well as all error and protocol violations. See the low level interface section for more details.

The main difference between with-websocket and with-concurrent-websocket is that messages are guaranteed to be delivered in order with with-websocket. This also means that no messages will be read into memory and processed unless a call is made to receive-message. This includes pong messages that may be sent in response to any pings sent in the background ping thread, if enabled. See the definition of ping-interval for more details on how background pings and pongs work.

A list of possible message types and their respective websocket codes:

message type (symbol) code
continuation 0
text 1
binary 2
connection-close 8
ping 9
pong 10

with-websocket

[procedure] (with-websocket [procedure])

with-websocket handles the process of setting up and closing off a websocket connection. procedure is a thunk to be executed after the websocket has been successfully setup. When you use with-websocket you can be assured that the connection will always be closed correctly even if errors occur in your application code and for all protocol violations.

with-websocket sends and receives all messages in a blocking fashion. Only one message will be sent or received at a time unless more threads are introduced in procedure. Even then the processing will block sending and receiving whereas it will not with with-concurrent-websocket.

Unless you expect a high volume of messages or messages of very large size on one websocket connection then this is the recommended procedure to use. It is easier to program when you know messages will arrive in order and it is more lightweight.

with-concurrent-websocket

[procedure] (with-concurrent-websocket [procedure])

This will, like with-websocket, handle setting up and closing a websocket connection. procedure is a thunk to be executed after the websocket has been successfully setup.

with-concurrent-websocket adds to with-websocket by running a thread in the background to read in messages as they arrive and spins off a new thread to process each message. Messages can arrive out-of-order, especially if there are a mix of very large and very small messages. with-concurrent-websocket can be very useful if very large messages are expected since they can take a long time to unmask and UTF8 validate.

Sending messages are still done in a blocking fashion but sending is relatively fast and will likely not be a problem. This could change in the future though so don't rely on it.

receive-message

[procedure] (receive-message #!optional (ws (current-websocket)))

Read in a message from the client. Returns two values. The first is the message and the second is the message type, either 'text or 'binary. Regardless of the message type the message itself will always be a string. Takes an optional websocket object that is bound to (current-websocket) by default. It will always block until a message is received but if used within with-concurrent-websocket it will not block other messages from being received or processed.

receive-message is "concurrent" aware so if it is used within with-websocket it will behave in a purely blocking fashion and when used within with-concurrent-websocket it will utilize the provided concurrency mechanism internally.

It is thread safe.

If a message is of type binary then converting it to something possibly more "binary" like, such as a u8vector, could be done with the following. It will incur one copy of the data though.

(blob->u8vector/shared (string->blob msg))

You could also use a string port to read the data in different fashions.

(with-input-from-string msg
  (lambda ()
    (read-byte)
    (read-u8vector))) ; etc

send-message

[procedure] (send-message data #!optional (message-type 'text) (ws (current-websocket)))

Send a message to the client. data is a string or u8vector to be sent to the client. message-type is one of the types listed under the message-types section and defaults to 'text. Usually this will be 'text or 'binary but any valid type is allowed. Note that if the data argument is a string it must be copied before being sent due to some CHICKEN internal limitations, strings will not. send-message also takes an optional websocket object that is bound to (current-websocket) by default.

It is thread safe.

Low level interface

An object of type websocket is used throughout. For purposes of this library it is an opaque object and none of its properties or methods are exposed.

upgrade-to-websocket

[procedure] (upgrade-to-websocket)

Transforms the current request into a websocket request. Returns a websocket object. When this procedure completes successfully the websocket has been setup with the client and is ready to start sending and receiving messages.

Can signal an error of condition type missing-upgrade-header.

close-websocket

[procedure] (close-websocket #!optional (ws (current-websocket)) #!key (close-reason 'normal) (data ""))

Closes off the websocket connection. ws can be a websocket object that defaults to (current-websocket). data can be a string or u8vector and will be sent in the close frame payload and defaults to an empty body. Note that according to the specification control frame bodies must be less than 126 octets. close-reason may be one of the following symbols and defaults to 'normal.

close-reason (symbol) code
normal 1000
going-away 1001
protocol-error 1002
unknown-data-type 1003
invalid-data 1007
violated-policy 1008
message-too-large 1009
extension-negotiation-failed 1010
unexpected-error 1011
unknown 3000-5000

close-websocket will wait up to (close-connection-timeout), if specified.

read-frame

[procedure] (read-frame total-size websocket)

Read in a frame. total-size is an integer used to track the total received message size in multi-frame messages. It will be checked against (max-message-size). Note that you can read in frames larger than the noted maximum message size listed under max-message-size but you may get get unexpected and potentially harmful results if a message too large is passed into the unmask or valid-utf8? method. websocket is of type websocket. Returns an object of type fragment. read-frame does read in the frame payload but does not unmask or UTF8 validate it.

A composite condition of types websocket, reserved-bits-not-supported, and protocol-error will be signaled if the frame has any reserved bits set.

A composite condition of types websocket and message-too-large will be signaled if the message or frame payload exceeds (max-message-size) or (max-frame-size) respectively.

A composite condition of types websocket and unhandled-opcode will be signaled if an unsupported opcode is specified in the frame. The condition property optype will contain the opcode.

This is not thread safe.

receive-fragments

[procedure] (receive-fragments #!optional (ws (current-websocket)))

A higher level procedure that reads in all message fragments and checks for protocol violations. Returns two values: The first is a list of fragment objects that make up the message and the second is the type of the message.

Can signal various composite conditions of types websocket and protocol-error with the protocol-error condition's msg property set to a description of the violation.

This is thread safe.

process-fragments

[procedure] (process-fragments fragments message-type #!optional (ws (current-websocket)))

Takes in a list made of fragments and unmasks and UTF8 validates the message. Returns two values. The first is a string containing the whole message and the second is the type of the message. It is possible for the return value to share memory with message fragment's payloads. message-type is the type for the list of fragments. Only messages of type 'text are UTF8 validated.

Can signal a composite condition of types websocket and invalid-data if the UTF8 validation fails.

This is thread safe assuming the same fragments are not being processed at the same time.

unmask

[procedure] (unmask fragment)

Takes in a fragment and returns an unmasked version of the frame payload (not the fragment). The unmasked value will share memory with the payload slot in the passed in fragment. unmask will not attempt to unmask a fragment's payload that is not masked.

valid-utf8?

[procedure] (valid-utf8? s)

Takes in a string and returns a boolean #t if valid and #f if not.

control-frame?

[procedure] (control-frame? [symbol])

Takes in a symbol representing the message type and returns #t if it is a control frame and #f if it isn't.

send-frame

[procedure] (send-frame websocket message-type data last-frame)

websocket is a websocket object. message-type is a symbol representing the optype of the message. data is a string or u8vector to be sent and last-frame is a boolean, #t if it is the last frame otherwise #f.

Fragments

Fragments are an opaque object representing a partially or full processed fragment of a message. Messages may be contained in one fragment or spread across multiple fragments. See the websocket specification for more information on fragments. Fragments cannot be constructed but are returned by methods related to reading and processing fragments and messages.

All of the following methods in the "Fragments" section take a fragment object as the only input argument.

fragment?

[procedure] (fragment? fragment)

Is fragment a fragment.

fragment-payload

[procedure] (fragment-payload fragment)

Returns the payload of the fragment. It may or may not be masked.

fragment-length

[procedure] (fragment-length fragment)

Returns the length of the fragment payload.

masked?

[procedure] (masked? fragment)

Returns whether the fragment payload is currently masked.

fragment-masking-key

[procedure] (fragment-masking-key fragment)

Returns a four element vector containing the masking key.

fragment-last?

[procedure] (fragment-last? fragment)

Returns #t if the fragment is the last fragment in a message otherwise #f.

fragment-optype

[procedure] (fragment-optype fragment)

Returns the optype (or message type) of the fragment as a symbol.

Exceptions

Possible exceptions and a brief description. See individual function definitions for details on when specific ones will be triggered.

type description
websocket A part of every websocket error as a composite condition. You can use this as a catchall for all websocket related errors.
protocol-error Anytime something happens that violates the websocket protocol. The msg error property will contain details on the protocol broken.
invalid-optype If an invalid message type (optype) is passed into any method that takes it as an argument.
reserved-bits-not-supported A frame had a reserved bit set.
message-too-large A message exceeds the set (max-frame-size) or (max-message-size) parameters.
unhandled-optype The implementation received a frame with an opcode it could not handle. The optype property contains the unhandleable optype (as produced by (opcode->optype)).
invalid-data Currently only signaled when a text payload contains invalid UTF8 codepoints. The msg property will hold a description of the "invalid data".
missing-header-upgrade The request does not contain an "upgrade" parameter or it is not set to "websocket".
connection-timeout The connection has timed out.

Examples

echo server for the Autobahn test suite

To run the tests make sure to have the test suites installed.

Autobahn test suite

echo-server.scm

(import chicken scheme)
(use spiffy websockets)

(ping-interval 0)
(drop-incoming-pings #f)

(handle-not-found
 (lambda (path)
   (with-websocket
     (lambda ()
       (let loop ()
         (receive (data type) (receive-message)
                  (unless (eq? type 'connection-close)
                          (send-message data type)
                          (loop))))))))

(start-server port: 8080)

ws-test.spec

{
    "servers": [
        {"agent": "AutobahnServer",
            "url": "ws://localhost:8080/web-socket",
            "options": {"version": 13}}
        ],
    "cases": ["1.*", "2.*", "3.*", "4.*", "5.*"],
    "exclude-cases": [],
    "exclude-agent-cases": {}
}

Run and compile it:

terminal

csc -O3 echo-server.scm
./echo-server

terminal 2

wstest -d -m fuzzingclient -s ws-test.spec

Contributors

Thanks to Seth Alves for developing the initial version. Thanks also to Andy Bennet and Peter Bex for helping solve implementation problems and providing advice as well as all the others on #chicken for their contributions.

Versions

0.0.2

First release.

0.0.1

Initial version.

License

Copyright (c) 2014, Thomas Hintz, Seth Alves
All rights reserved.

Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions
are met:

1. Redistributions of source code must retain the above copyright
   notice, this list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright
   notice, this list of conditions and the following disclaimer in the
   documentation and/or other materials provided with the distribution.
3. The name of the authors may not be used to endorse or promote products
   derived from this software without specific prior written permission.

THIS SOFTWARE IS PROVIDED BY THE AUTHORS ``AS IS'' AND ANY EXPRESS
OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
ARE DISCLAIMED.  IN NO EVENT SHALL THE AUTHORS BE LIABLE FOR ANY
DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE
GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER
IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR
OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN
IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.