## Generalized Arrays

This repository holds an implementation of an R7RS library implementing generalized arrays. As a reference, this repository is meant to build a module for CHICKEN Scheme.

### Usage

Below are the instructions for using and working with the generalized-arrays egg.

#### Installation

$ chicken-install -s generalized-arrays

#### Importing the Library

To import the complete library, use the following import specification:

(import generalized-arrays)

#### Module Structure

The library is broken apart into several smaller modules, however. These can be individually imported as follows:

(import (generalized-arrays intervals) (generalized-arrays storage) (generalized-arrays arrays))

Each sub-module containing the procedures and record definitions as specified in each independent module documentation below.

#### Dependencies

This egg aims to be fairly light on dependencies where possible, and tries its best to be R7RS compatible. To that end, it uses the following dependencies:

- r7rs
- srfi-48
- srfi-128
- srfi-133
- srfi-160
- check-errors
- transducers (GitLab)

Most of these dependencies are SRFIs. `check-errors` could probably be replaced with a thin shim for type-checking arguments, and is a very small dependency. `transducers` is another R7RS library that is maintained and necessary for working with arrays.

### Reporting Issues

Please report issues in the repository or on the CHICKEN bug tracker.

### Repository

The repository can be located on GitLab.

### Why Arrays?

Scheme provides a variety of contiguous data structures (e.g. vectors, SRFI-4 vectors) that can reference data over a singular index in an interval. Scheme does not, however, provide any means by default for working with data indexed over a multi-dimensional interval. In C, for example, we might declare a multi-dimensional array as follows:

// Declares an array `arr` that is N x N in size, holding `double`-typed values. double (*arr)[N] = malloc(sizeof(double[N][N])); for (size_t col = 0; col < N; ++col) { for (size_t row = 0; row < N; ++row) { if (row == col) { arr[col][row] = 1.0; } } }

This can be useful for a variety of representations:

- Images holding individual pixel data at each location
- Point clouds holding multiple points with multi-dimensional data (XYZ)
- A grid of "cells" holding more structured data
- Mathematical data such as matrices, tensors, etc. depending on the rank of the array

Usually arrays are most useful at reducing complex lookup mechanisms over multiple keys into a lookup mechanism that merely uses integers within a range. This can aid in making lookup faster, but more importantly provides a means to abstract how data is "represented" in a spatial sense, even if the actual underlying representation is different.

#### Scheme Arrays

Unlike C (mentioned above), Scheme is dynamically typed, which opens arrays up to be much more flexibly used than their counterpart in C. However we still want to keep open the possibility of having strong(ish)ly typed arrays as well — many mathematical applications perform some construction of an array into contiguous memory expecting some layout, and then pass along said array to a much more optimized library with a specific layout to handle the larger number crunching.

The generalized-arrays egg supports arrays holding nothing but Scheme types, but also supports arrays that can be more strongly typed (e.g. by using SRFI-4 containers for array storage). This is done by separating the following concepts:

- Intervals: the shape, size, stride, etc. of an array. This describes how to traverse an array and what the underlying "shape" or structure of the array is.
- Storage: the container or record type that is used to store the array's data. This is often a "flattened" vector or vector-like container of some kind.
- Arrays: the actual objects we want to work on and their associated high-level API.

Each of these concepts is described in brief in the following sections.

#### Intervals

Intervals are an abstraction provided by this library to express multi-dimensional intervals expressed through integer ranges. Intervals here follow the mathematical definition of an interval, specifically, an interval can be expressed as a starting index and an ending index. In math this might look like `[0 10]` to express the range of integers starting at zero and up to ten, inclusive.

This library provides multi-dimensional intervals as a first class concept. One should think about intervals as being the shape or the boundaries of an array. For example, if we use the prior example of an `N` by `N` matrix, we might say that our interval is something like `[(0, 0) (N - 1, N - 1)]`. If this same matrix were transformed or sliced to be `N` by `M`, then we might say that the interval is `[(0, 0) (N - 1, M - 1)]`.

Another point to note is that for the sake of API convenience this library always treats all intervals as half-open; that is, intervals are specified up-to-but-not-including their upper bound. So in the previous example where we used the inclusive "mathematical" representation of ranges, in this library you might reference a library as being `[(0, 0) (N, N))` with unbalanced parentheses.

The reasons for such will become more obvious later, but if you were to create an example program to express a range in this library it might look like:

(import (generalized-arrays intervals)) (defineN 10) (make-interval #(0 0) #(N N))

This interval is half-open, in that it does not include the index `#(N N)`. Indices in the generalized-arrays egg are expressed as Scheme vectors containing fixnums describing each dimension. An array can be any number of dimensions, but when making an interval the number of dimensions (the rank) must match.

**NOTE**: The generalized-arrays egg always assumes that all indices (i.e. the members of an interval) are represented as Scheme vectors holding non-negative fixnums.

For more information on intervals, see the API reference on intervals below.

#### Storage

The next core concept in this library is storage. `generalized-arrays` provides what are known as "storage classes" which comprise APIs that relate to how data in an array are stored. This part of the API generally does not require as much familiarity as default storage classes are provided:

`vector-storage-class``u8vector-storage-class``s8vector-storage-class``u16vector-storage-class``s16vector-storage-class``u32vector-storage-class``s32vector-storage-class``u64vector-storage-class``s64vector-storage-class``f32vector-storage-class``f64vector-storage-class``c64vector-storage-class``c128vector-storage-class`

Each of these "storage classes" are designed to be instances of a unique record type that holds all the procedures necessary for working with these storage classes. This pattern is sometimes known as the type-class pattern, and you may be familiar with it if you have ever used SRFI-128 (Comparators). If not, fret not, because in all likelihood you won't have to touch most of this API unless you are making your own storage class.

In any case, the storage class chosen defines how the array will store the data. Data is stored in continguous, one-dimensional storage (e.g. it is not a set of nested vectors, or vectors of f32 vectors, or lists of vectors, or whatever have you). Most usage of storage classes will pertain to getting the storage object (which is an object that a storage class can operate on), or specifying which storage class one wishes to use as the "backing storage" of an array.

For more information on storage and storage classes, see the API reference on intervals below.

#### Arrays

The arrays API may appear a bit sparse initially, but this library provides every procedure necessary in order to be able to perform any individual array operation. This includes but is not limited to:

- Array traversal
- Array slicing
- Copying
- Getting array metadata (shape, rank, stride, etc)
- Checking equality
- Comparing arrays
- Setting and getting values from the array
- Printing an array to the REPL
- etc.

Many procedures which you may be accustomed to in a data-structure library may appear missing; this is because this library is meant to be used exclusively with the `transducers` library. It is advised to become acquainted with transducers before trying to dig too deep into what is missing from the arrays API.

### Some History

This project started many years ago when I was in grad school, looking for something analagous to Numpy in Scheme. At the time, I made an attempt to implement the API that eventually became SRFIs 122, 179, and 231. To my credit, early versions of this library predated those SRFIs by quite some time; conversely, to my detriment my early implementation was pretty poor.

I did not like what eventually became SRFI 231 (supercedes all the other aforementioned SRFIs) for a handful of reasons:

- The API is largely "lazy" in that it does not immediately perform the work that a given map / etc. might invoke on all the data in an array the same way that a map might work on e.g. a list.
- While the library provides a storage-class-like interface, it does not always make it easy to pull an FFI-compatible structure out from the "array" (again, due to the "lazy" API).

The largest reason of which being the first: the laziness invoked by SRFI 231's API. It ends up muddying the distinction between intervals and arrays, and behaves very differently than what many Scheme libraries would leave one to expect. The laziness has a purpose in SRFI 231: it serves to avoid making lots of expensive intermediate copies of arrays in memory as operations stack up. The latter reason listed above is a downstream effect of the laziness in SRFI 231, which makes it sometimes awkward to use.

Avoiding expensive intermediate copies of large arrays is important; however, this is better solved by composing operations over a transducer than by letting that concern bleed through the entire API. What's more, intervals (as provided by this library, not SRFI 231) combined with transducers are useful unto themselves. The "basic" implementation of indices-as-vectors likewise leads to a pleasantly expressive (and fast) way to quickly loop over multiple dimensions.

In the end, I chose to depart from implementing as many procedures as SRFI 231, in particular avoiding:

- One-off whole-array transformations, such as
`array-map`,`array-any`,`array-every`, etc. - Conversions to other data structures such as vectors and lists.* A conversion
`array->nested-list`is provided purely for providing a pretty-printer for array objects - Inner and outer products

The first two points above are a consequence of making the arrays API transducer-aware. The latter is a conscious culling of functionality. Quite bluntly: no direct Scheme API will ever perform large-scale mathematics (such as matrix operations) efficiently, nor will it be easy to optimize. What is more important than providing a variety of poorly-optimized mathematical operations is to provide a data structure that enables ease of construction and manipulation, as well as easy access to ways in which the array can be conveniently passed to some foreign function interface (FFI) so as to allow a lower-level mathematical language (C, Fortran, Rust) to run much more tightly optimized mathematical code on the array.

It is under that belief that I say this: "If you're using generalized-arrays in the hope that you're going to implement BLAS/LAPACK, or Eigen3, or SuiteSparse, be warned that you'd be better served wrapping those libraries in your favourite Scheme's FFI and instead passing the underlying array to them." Full stop.

For that reason, generalized-arrays is not a great library for doing math. But if you need to store, manipulate, print, slice, or traverse an array that you plan to send to some other library that does math, you're in the right place.

### API Reference

The below API reference is best understood in conjunction with `transducers`. If you are unsure of any of the examples, it is strongly suggested that you start by understanding how transducers work first.

#### (generalized-arrays intervals)

*[record]*

`<interval>`

A record type that represents a multi-dimensional, half-open interval. It is described by two Scheme vectors, the `interval-start` and `interval-end`, which represent the lower and upper bounds of the interval, respectively.

*[constant]*

`vector-index-comparator`

A SRFI-128 comparator that is meant for comparing Scheme vector indices in an interval.

*[procedure]*

`(check-interval loc arg arg-name)`

Checks if the interval `arg` is an interval. If it is not, signals an `error` that describes that `arg` is not an interval in the function with the name `loc` (a symbol) and name `arg-name` (also a symbol).

(import (generalized-arrays intervals)) (check-interval 'my-procedure 3 'interval) ; => Error: (my-procedure) bad `interval` argument type - not an interval: 3

` `

*[procedure]*

`(make-interval start end)`

Constructs a new half-open interval from the `start` index up to but not including the `end` index. Both indices must be vectors of the same length (rank). If any dimension inside the start index is greater than or equal to its equivalent dimension in the end index, then the interval is empty.

Indices in this library cannot contain dimension values less than zero.

*[procedure]*

`(make-default-interval end)`

Constructs a default interval from the provided `end` index. The "start" index for this interval is an index with the same rank as the `end` index, but where all the dimensions are equal to zero.

*[procedure]*

`(interval? i)`

A predicate for evaluating if an object is an interval. True iff the interval is an `<interval>` record.

*[procedure]*

`(interval-start i)`

*[procedure]*

`(interval-end i)`

Procedures that grab the start and end indices of an interval, respectively.

*[procedure]*

`(interval-length i)`

Procedure that computes the total number of elements in the interval. This is equivalent to taking the lengths along each dimension and multiplying them.

(import (generalized-arrays intervals) (test)) (test "Length of [(0, 0) (2, 2)) is 4" 4 (make-default-interval (vector 2 2))) (test "Length of [(1, 1) (3, 4)) is 6" 6 (make-interval (vector 1 1) (vector 3 4)))

` `

*[procedure]*

`(interval-empty? i)`

Predicate which returns true iff the interval `i` has a length of zero.

*[procedure]*

`(interval-contains? interval idx)`

A predicate which returns true iff the index `idx` is contained within the `interval`.

(import (generalized-arrays intervals) (test)) (test-assert "Index #(1 1) is contained within [(0, 0) (3, 3))" (interval-contains? (make-default-interval (vector 3 3)) (vector 1 1)))

` `

*[procedure]*

`(interval-fold f sentinel xs)`

*[procedure]*

`(reverse-interval-fold f sentinel xs)`

Transducer-aware folding operations over intervals (forward and reverse).

(import (generalized-arrays intervals) (transducers) (test)) (test "Transducing over interval values" (list (vector 0 0) (vector 0 1) (vector 1 0) (vector 1 1)) (transduce interval-fold values (collect-list) (make-default-interval (vector 2 2)))) (test "Transducing over interval values in reverse" (list (vector 1 1) (vector 1 0) (vector 0 1) (vector 0 0)) (transduce reverse-interval-fold values (collect-list) (make-default-interval (vector 2 2))))

` `

*[procedure]*

`flatten-interval`

*[procedure]*

`reverse-flatten-interval`

Transducer that flattens transduced intervals either in forward or reverse order.

(import (generalized-arrays intervals) (transducers) (test)) (test "Flattening intervals constructed by a separate transducer procedure" (list (vector 0 0) (vector 0 1) (vector 1 0) (vector 1 1) (vector 5 5) (vector 5 6) (vector 6 5) (vector 6 6)) (transduce list-fold (compose (zip-list (list (vector 2 2) (vector 7 7))) (map (lambda(pair) (apply make-interval pair))) flatten-interval) (collect-list) (list (vector 0 0) (vector 5 5))))

` `

*[procedure]*

`(chain-interval interval)`

*[procedure]*

`(reverse-chain-interval interval)`

Transducers that chain the indices contained in the intervals to the transduction operation, in either forward or reverse order.

(import (generalized-arrays intervals) (transducers) (test)) (test "Chaining an interval to a list of symbols" (list 'a 'b 'c 'd (vector 0 0) (vector 0 1) (vector 1 0) (vector 1 1)) (transduce list-fold (chain-interval (make-default-interval (vector 2 2))) (collect-list) (list 'a 'b 'c 'd)))

` `

*[procedure]*

`(zip-interval interval)`

*[procedure]*

`(reverse-zip-interval interval)`

Transducers that zip the indices contained within the interval to another transduction, in forward or reverse order.

(import (generalized-arrays intervals) (transducers) (test)) (test "Zipping an interval onto elements of a list" (list (cons 'a (vector 0 0)) (cons 'b (vector 0 1)) (cons 'c (vector 1 0)) (cons 'd (vector 1 1))) (transduce list-fold (zip-interval (make-default-interval (vector 2 2))) (collect-list) (list 'a 'b 'c 'd 'e)))

` `

*[procedure]*

`(interleave-interval interval)`

*[procedure]*

`(reverse-interleave-interval interval)`

Transducers that interleave the indeices contained within the interval with another transduction, in forward or reverse order.

(import (generalized-arrays intervals) (transducers) (test)) (test "Interleaving an interval onto elements of a list" (list 'a (vector 0 0) 'b (vector 0 1) 'c (vector 1 0) 'd (vector 1 1)) (transduce list-fold (zip-interval (make-default-interval (vector 2 2))) (collect-list) (list 'a 'b 'c 'd 'e)))

#### (generalized-arrays storage)

Storage classes are a type-class like object that holds procedures for working with storage objects. Storage classes themselves do not hold data; however, they can be used to construct a storage object (e.g. a vector, an f64vector) that can hold data. These storage objects can be manipulated by the corresponding procedures for a given storage class.

*[record]*

`<storage-class>`

A unique record type that denotes an object which is a storage-class.

*[procedure]*

`(make-storage-class short-id constructor ref set length copy copy! transducible comparator default-element)`

Constructor for creating a new storage class. See the following accessors for more information about what each of the individual elements should be.

*[procedure]*

`(storage-class? sc)`

A type predicate which returns true iff the provided object is a storage-class.

*[procedure]*

`(storage-class-short-id sc)`

A short ID for the storage class `sc`. This identifier exists to be a unique short-code for a storage class' type or kind. For example, a vector-storage-class (which uses a Scheme vector as the underlying storage type) would have a short ID that is the symbol 'v. This is primarily used for reader and writer syntax.

*[procedure]*

`(storage-class-constructor sc)`

A procedure of the form `(make-storage size #!optional fill)` which constructs a new storage object of the implementing storage class `sc`.

*[procedure]*

`(storage-class-ref sc)`

A procedure of the form `(storage-ref storage index)` which gets an individual element at `index` from the `storage` object of the implementing storage class `sc`.

*[procedure]*

`(storage-class-set sc)`

A procedure of the form (storage-set! storage index value) which sets `value` to the element at `index` in `storage` object of the implementing storage class `sc`.

*[procedure]*

`(storage-class-length sc)`

A procedure of the form (storage-length storage) which gets the number of total elements in the `storage` object of the implementing storage class `sc`.

*[procedure]*

`(storage-class-copy sc)`

A procedure of the form (storage-copy storage) which creates a direct copy of the `storage` object of the implementing storage class `sc` and returns it.

*[procedure]*

`(storage-class-copy! sc)`

A procedure of the form (storage-copy storage) which creates a direct copy of the `storage` object of the implementing storage class `sc` and returns it.

*[procedure]*

`(storage-class-transducible sc)`

A transducible type-class over the storage object kind. See the documentation of transducible type-classes in the `transducers` library for more details.

*[procedure]*

`(storage-class-comparator sc)`

A SRFI-128 comparator for storage objects of the storage class `sc`.

*[procedure]*

`(storage-class-default-element sc)`

A value which represents the default element of the storage object. This is used when producing a default storage object from storage class `sc`, or filling individual elements with a default value when filtered.

*[procedure]*

`(make-storage-object storage-class n #!optional fill)`

Constructs a storage object of the provided `storage-class` with `n` elements defaulting to the provided `fill` value. If `fill` is not provided, the storage-class` default element will be used instead.

*[procedure]*

`(storage-object-ref storage-class storage-object i)`

Gets the `i`th element contained inside the `storage-object`.

*[procedure]*

`(storage-object-set! storage-class storage-object i value)`

Sets the `i`th element contained inside the `storage-object` to `value`.

*[procedure]*

`(storage-object-length storage-class storage-object)`

Gets the length of the `storage-object`.

*[procedure]*

`(storage-object-copy storage-class storage-object start #!optional end)`

Constructs a new storage-object of length `(- end start)` and copies the values from `storage-object` into this newly constructed object.

*[procedure]*

`(storage-object-copy! to at from #!optional start end)`

Copies the values in storage-object `from` into storage-object `to` starting at index `at`. The values copied are from `start` to `end` (length `(- end start)`) in the `from` object.

*[constant]*

`vector-storage-class`

*[constant]*

`u8vector-storage-class`

*[constant]*

`s8vector-storage-class`

*[constant]*

`u16vector-storage-class`

*[constant]*

`s16vector-storage-class`

*[constant]*

`u32vector-storage-class`

*[constant]*

`s32vector-storage-class`

*[constant]*

`u64vector-storage-class`

*[constant]*

`s64vector-storage-class`

*[constant]*

`f32vector-storage-class`

*[constant]*

`f64vector-storage-class`

*[constant]*

`c64vector-storage-class`

*[constant]*

`c128vector-storage-class`

Standard, preconfigured storage classes. Each storage class uses the specific "vector" Scheme type (either standard Scheme vectors or SRFI-4 / SRFI-160 vectors) as a backing storage, as specified by its name.

#### (generalized-arrays arrays)

*[record]*

`<array>`

A unique record type for representing multi-dimensional array data.

*[procedure]*

`(array? obj)`

A predicate procedure that returns `#t` iff the object is an array (of type `<array>`), otherwise `#f`.

*[procedure]*

`(array-view? array)`

Predicate which evaluates to `#t` iff the array is some kind of array view, otherwise `#f`. An array can be a slice, or any array that has had its axes re-arranged in some way (tranposition, swap-axes, squeeze, expand), while maintaining the same underlying storage object and interval rank.

*[procedure]*

`(make-array storage-class dimension #!optional fill)`

Constructs an array by allocating a storage object of the provided storage-class and dimension. If `fill` is provided, every value in the array is set to `fill`, otherwise the array is defaulted with the value as if calling `(storage-class-default-element storage-class)`.

(import generalized-arrays test) (test-assert "Array is filled with #f" (array=? #a2v((#f #f #f) (#f #f #f) (#f #f #f)) (make-array vector-storage-class #(3 3) #f)))

` `

*[procedure]*

`(make-array-from-storage storage-class dimension storage-object)`

Constructs an array of the provided storage-class and dimension with a provided storage-object. Unlike `make-array`, this does not allocate any storage object on its own. Additionally, it is an error if:

- The
`storage-object`is not a storage object of the provided`storage-class` - The length of the interval constructed by
`(make-default-interval dimension)`is not exactly equal to the length of the storage-object.

This procedure is helpful if you are working with array data via SRFI-4 and / or SRFI-160 storage kinds but are constructing or managing the storage across the FFI boundary.

(import generalized-arrays test) (definevec #f64(1.0 2.0 3.0 4.0 5.0 6.0 7.0 8.0 9.0)) (definearr (make-array-from-storage f64vector-storage-class #(3 3) vec)) (test "Storage object in array is equal to original f64vector" vec (array-storage-object arr))

` `

*[procedure]*

`(array-tabulate storage-class dimension proc)`

Constructs an array by allocating a storage-object of the provided `storage-class` with the provided `dimension`(s) as `make-array` does. Unlike `make-array` which fills the array with a single value, `array-tabulate` sets the value at each index by calling `(proc index)` for every index in the array.

(import generalized-arrays test) (test "Constructed array contains all of its indices." #a2v((#(0 0) #(0 1) #(0 2)) (#(1 0) #(1 1) #(1 2)) (#(2 0) #(2 1) #(2 2))) (array-tabulate vector-storage-class #(3 3) (lambda(idx) idx)))

` `

*[procedure]*

`(array-broadcast array value)`

Constructs a new array by copying the storage-class and shape of the input `array` and setting all the elements of the newly allocated array to the provided `value`.

(import generalized-arrays test) (definea (make-array vector-storage-class #(3 3) 0)) (defineb (array-broadcast a "abc")) (test "Shapes of arrays are the same." (array-shape a) (array-shape b)) (test "Storage class of arrays are the same." (array-storage-class a) (array-storage-class b))

` `

*[procedure]*

`(array-storage-class array)`

Returns the storage-class of the provided array.

*[procedure]*

`(array-interval array)`

Returns the "interval" of a given array. The current interval of a given array is the range of multi-dimensional indices that are valid for that array object. It is not always the case that the interval starts at the default (zero) multi-index. For example:

(import generalized-arrays) (definea (make-array vector-storage-class #(3 3) 0)) (test "Interval of array a" (make-interval #(0 0) #(3 3)) (array-interval a)) (defineb (array-slice a #(1 1) #(2 2))) (test "Interval of array b" (make-interval #(1 1) #(2 2)) (array-interval b))

In the above example, the array `b` is a slice of array `a`, from `#(1 1)` to `#(2 2)`, so while `b` shares the same storage-object as `a`, the interval of `b` is smaller than that of `a`.

`array-interval` is meant to be used as a low-level mechanism for being able to work with slices over a shared storage-object. In most cases, you'll want to use `array-shape` to get the semantic interval over the array, rather than `array-interval`.

*[procedure]*

`(array-shape array)`

Returns the shape of a given array. The shape of an array is the default interval up to the maximum length of each dimension of the array or view. Unlike `array-interval`, the shape of an array always starts at the default (zero) index, and ends at the maximum dimension of the array.

In almost all instances, `array-shape` is the way one should get the full interval of indices in a given array. While `array-interval` provides the actual interval used to index into the storage-object of the array, `array-shape` is more useful when using most array APIs (e.g. `array-ref`); of which, these APIs will expect an array that is in the bounds of the shape rather than the interval, which is meant to be a tool for developers who are operating on the raw, underlying array structure directly.

(import generalized-arrays) (definea (make-array vector-storage-class #(3 3) 0)) (test "Shape of array a" (make-interval #(0 0) #(3 3)) (array-shape a)) (defineb (array-slice a #(1 1) #(2 3))) (test "Interval of array b" (make-interval #(1 1) #(2 3)) (array-interval b)) (test "Shape of array b" (make-interval #(0 0) #(1 2)) (array-shape b))

` `

*[procedure]*

`(array-storage-object array)`

Gets the underlying storage-object of the array. This is primarily meant if one wants to get the underlying storage in order to pass it to some specialized procedure that can operate on the array contents directly. An example of this could be passing the underlying storage object to the FFI, such as to use BLAS or LAPACK optimized procedures over the underlying array storage.

For example, see the following example (copied from the BLAS egg):

(import generalized-arrays blas srfi-4 test) (defineorder RowMajor) (definetransa NoTrans) (definem 4) (definen 4) (definealpha 1) (definebeta 0) (definea (make-array-from-storage f64vector-storage-class (vector m n) (f64vector 1 2 3 4 1 1 1 1 3 4 5 6 5 6 7 8))) (definex (f64vector 1 2 1 1)) (definey (f64vector 0 0 0 0)) (dgemv! order transa m n alpha (array-storage-object a) x beta y) (test "Result of multiplying the array a by the vector x" y (f64vector 12 5 22 32))

Generally speaking, the above is a contrived example; However, this contrived example demonstrates how one might start to build a more complete (and optimized) mathematics library that can be utilized in conjunction with the provided arrays API.

*[procedure]*

`(array-rank array)`

Returns the rank (number of dimensions) of the provided array.

(import generalized-arrays test) (test "Rank of array is 2" 2 (make-array vector-storage-class #(2 2))) (test "Rank of array is 3" 3 (make-array vector-storage-class #(2 3 3)))

` `

*[procedure]*

`(array-stride)`

Gets the stride of each dimension in the array.

*[procedure]*

`(array-ref array idx)`

Gets the element at `idx` in the `array`.

(import generalized-arrays test) (definea (make-array-from-storage vector-storage-class (vector 3 3) (vector 1 2 3 4 5 6 7 8 9))) (test "Element at #(1 1) is equal to 5" 5 (array-ref a #(1 1)))

` `

*[procedure]*

`(array-set! array idx value)`

Sets the element in `array` at `idx` to the provided `value`. Analagous to something like `vector-set!`, but for arrays.

(import generalized-arrays test) (definea (make-array-from-storage vector-storage-class (vector 3 3) (vector 1 2 3 4 5 6 7 8 9))) (test "Element at #(1 1) is equal to 5" 5 (array-ref a #(1 1))) (array-set! a #(1 1) "abc") (test "Element at #(1 1) is now equal to 'abc'" "abc" (array-ref a #(1 1)))

` `

*[procedure]*

`(array-update! array idx proc)`

Updates the element in the `array` at `idx` as if applying `proc` to it.

(import generalized-arrays test) (definea (make-array-from-storage vector-storage-class (vector 3 3) (vector 1 2 3 4 5 6 7 8 9))) (test "Element at #(1 1) is equal to 5" 5 (array-ref a #(1 1))) (array-update! a #(1 1) (lambda(elem) (- elem 20))) (test "Element at #(1 1) is now equal to -15" -15 (array-ref a #(1 1)))

` `

*[procedure]*

`(array=? lhs rhs)`

A predicate function which evalutes if the dimensions of the left and right arrays (`lhs` and `rhs` respectively) are equal, and all of the elements at each index position in the left and right arrays are equal as if applying `equal?`.

This is mostly useful in writing tests, and is unlikely to be too useful outside of them.

*[procedure]*

`(array-fold f sentinel xs)`

*[procedure]*

`(reverse-array-fold f sentinel xs)`

Transducer-aware folding operations over arrays. Every element in the array is folded in lexicographic order, starting at the default (zero) index of the array and ending at the maximum dimension of the array.

(import generalized-arrays transducers test) (definea (make-array-from-storage vector-storage-class (vector 3 3) (vector 1 2 3 4 5 6 7 8 9))) (test "Sum of all array values is 45" 45 (transduce array-fold values (collect-sum 0) a)

`array-reverse-fold` works like `array-fold`, but folds over the array in reverse-lexicographic order, starting at the maximum dimension of the array and ending at the default (zero) index of the array.

*[procedure]*

`(array-interval-fold f sentinel xs)`

*[procedure]*

`(reverse-array-interval-fold f sentinel xs)`

Transducer-aware folding operations that serve a convenience to fold over all the indices in an array. This can be used as a shorthand form of calling `interval-fold` over the `(array-shape array)`.

*[procedure]*

`flatten-array`

*[procedure]*

`reverse-flatten-array`

Transducers that flatten arrays into their contained values in a transduction. `flatten-array` flattens the elements of the array into the transduction in lexicographic ordering. `reverse-flatten-array` is similar to `flatten-array`, but flattens the values in reverse-lexicographic ordering.

(import generalized-arrays transducers test) (definea (make-array-from-storage vector-storage-class (vector 3 3) (vector 1 2 3 4 5 6 7 8 9))) (test "Flattening a list of arrays" (vector 1 2 3 4 5 6 7 8 9 1 2 3 4 5 6 7 8 9) (transduce list-fold flatten-array (collect-vector) (list a a)))

` `

*[procedure]*

`(chain-array array)`

*[procedure]*

`(reverse-chain-array array)`

Transducer that chains the contents of the provided array onto the end of the transduction. `reverse-chain-array` works similar to `chain-array`, but chains the array onto the transduction in reverse-lexicographic ordering.

(import generalized-arrays transducers test) (definea (make-array-from-storage vector-storage-class (vector 3 3) (vector 'a 'b 'c 'd 'e 'f 'g 'h 'i))) (test "Chaining array to end of list-transduction" (list 1 2 3 'a 'b 'c 'd 'e 'f 'g 'h 'i) (transduce list-fold (chain-array a) (collect-list) (list 1 2 3)))

` `

*[procedure]*

`(zip-array array)`

*[procedure]*

`(reverse-zip-array array)`

Transducer that zips the contents of the provided array onto the transduction. `reverse-zip-array` works similar to `zip-array`, but zips the array onto the transduction in reverse-lexicographic ordering.

(import generalized-arrays transducers test) (definea (make-array-from-storage vector-storage-class (vector 3 3) (vector 'a 'b 'c 'd 'e 'f 'g 'h 'i))) (test "Zip array to transduction" (list (cons 1 'a) (cons 2 'b) (cons 3 'c)) (transduce list-fold (zip-array a) (collect-list) (list 1 2 3)))) (test "Reverse zip array to transduction" (list (cons 1 'i) (cons 2 'h) (cons 3 'g)) (transduce list-fold (reverse-zip-array a) (collect-list) (list 1 2 3)))

` `

*[procedure]*

`(interleave-array array)`

*[procedure]*

`(reverse-interleave-array array)`

Transducer that interleaves the contents of the provided array into the transduction. `reverse-interleave-array` works similar to `interleave-array`, but interleaves the array onto the transduction in reverse-lexicographic ordering.

(import generalized-arrays transducers test) (definea (make-array-from-storage vector-storage-class (vector 3 3) (vector 'a 'b 'c 'd 'e 'f 'g 'h 'i))) (test "Interleave array to transduction" (list 1 'a 2 'b 3 'c) (transduce list-fold (interleave-array a) (collect-list) (list 1 2 3)))) (test "Reverse zip array to transduction" (list 1 'i 2 'h 3 'g) (transduce list-fold (reverse-interleave-array a) (collect-list) (list 1 2 3)))

` `

*[procedure]*

`(collect-array storage-class dimension)`

A transducer-collector (reducer) used to collect values into an array of the provided `storage-class` and `dimension`. The number of elements collected must be exactly equal to the length of the default-interval generated from the provided `dimension`, or an error is signaled.

*[constant]*

`array-transducible`

*[constant]*

`reverse-array-transducible`

Transducible type-classes over the lexicographic and reverse-lexicographic transducer operations over arrays.

*[procedure]*

`(array-slice array start #!optional end)`

Slices an array starting from the `start` index to the (optional) `end` index. This procedure produces an array view (that satisfies `array-view?`) representing the sub-interval defined by `[start end)` in the `array`.

This procedure doesn't copy any elements of the array or modify the internal storage object shared with the original `array`, which makes it useful for designating smaller sub-sections of the array, for e.g. transduction or mutation.

(import generalized-arrays test) (definea (make-array-from-storage vector-storage-class (vector 3 3) (vector 'a 'b 'c 'd 'e 'f 'g 'h 'i))) ;; Slice the array ;; ;; By default, `end` is the end of the array `a` (defineb (array-slice a #(1 1))) (test-assert "Slice of array a" (array=? b (make-array-from-storage vector-storage-class (vector 2 2) (vector 'e 'f 'h 'i)))

` `

*[procedure]*

`(array-transpose array)`

Performs a monadic transposition of the major through minor axes of the array. This is equivalent to reversing the ordering of the major through minor axes. This procedure returns an array view (i.e. satisfies `array-view?`) without copying any of the array's internal data or modifying the underlying storage-object.

(import generalized-arrays test) (definea (make-array vector-storage-class (vector 1 2 3 4) 0)) (test "Array dimensions are (1 2 3 4)" (make-default-interval (vector 1 2 3 4)) (array-shape a)) (definea-t (array-transpose a)) ;; For a rank-2 array, this would be swapping an #(i j) with #(j i) (test "Transpose dimensions are (4 3 2 1)" (make-default-interval (vector 1 2 3 4)) (array-shape a-t))

Note that while the internal storage-object is not modified, fold operations still iterate through the array's elements (specifically `a-t` in the example above) in lexicographic order according to the array's shape / interval. This may or may not provide efficient access into the storage-object, since sequential elements in lexicographic order may not necessarily be contiguous in memory.

*[procedure]*

`(array-swap-axes array to from)`

Perform a triadic transposition of the `to` and `from` axes. This is equivalent to swapping the ordering of the two provided axes in the array's shape. This procedure returns an array view (i.e. satisfies `array-view?`) without copying any of the array's internal data or modifying the underlying storage-object.

Both the `to` and `from` axes must be fixnums between 0 and the less than the rank of the array.

(import generalized-arrays test) (definea (make-array vector-storage-class #(2 1 3 4) 0)) (test "Shape of array a" (make-default-interval (vector 2 1 3 4)) (array-shape a)) (defineb (array-swap-axes a 0 1)) (test "Shape of array b" (make-default-interval (vector 1 2 3 4)) (array-shape b)) (definec (array-swap-axes b 0 3)) (test "Shape of array c" (make-default-interval (vector 4 2 3 1)) (array-shape b)) (defined (array-swap-axes a 2 3)) (test "Shape of array d" (make-default-interval (vector 2 1 4 3)) (array-shape d))

` `

*[procedure]*

`(array-squeeze-axis array axis)`

Removes the `axis` from the array's shape, if and only if the length of that axis is 1. If the length of that axis is not 1, an error is signalled. This procedure returns an array view (i.e. satisfies `array-view?`) without copying any of the array's internal data or modifying the underlying storage-object.

(import generalized-arrays test) (definea (make-shape vector-storage-class (vector 2 1 2) 0)) (test "Squeezing array axis" (make-default-interval (vector 2 2)) (array-shape (array-squeeze-axis a 1))) (test-error "Squeezing an incorrect axis" (array-squeeze-axis a 0))

` `

*[procedure]*

`(array-squeeze array)`

Removes all axes in the array's shape whose length is 1. If every axis would be squeezed by this operation, returns the singular value in the array. This procedure returns an array view (i.e. satisfies `array-view?`) without copying any of the array's internal data or modifying the underlying storage-object.

(import generalized-arrays test) (definea (make-array-from-storage vector-storage-class (vector 3 1 3) (vector 'a 'b 'c 'd 'e 'f 'g 'h 'i))) (test "Squeeze array axes" (make-default-interval (vector 3 3)) (array-shape (array-squeeze a))) (test "Get single value from slice" 'e (array-squeeze (array-slice a #(1 0 1) #(2 1 2))))

` `

*[procedure]*

`(array-expand-axis array axis)`

Adds an axis of length 1 before `axis` to the internal shape of the array. This procedure returns an array view (i.e. satisfies `array-view?`) without copying any of the array's internal data or modifying the underlying storage-object.

This procedure can be though of as the opposite of `array-squeeze-axis`.

(import generalized-arrays test) (definea (make-array-from-storage vector-storage-class (vector 3 3) (vector 'a 'b 'c 'd 'e 'f 'g 'h 'i))) (test "Shape of expanded array" (make-default-interval (vector 1 3 3)) (array-shape (array-expand-axis a 0)))

` `

*[procedure]*

`(array-copy array #!optional start end)`

Makes a copy of the `array` by allocating a new storage object, with the values copied into the new storage-object in lexicographic order.

This procedure is often most useful when one wants to pass the storage object of an array to some lower-level API (e.g. BLAS). Array views (i.e. satisfies `array-view?`) share the underlying storage object with an array allocated previously, and so need to be coerced into a newly allocated copy of the storage object if one wants to use the storage object with e.g. BLAS.

(import generalized-arrays blas srfi-4 test) (defineorder ColMajor) (definetransa NoTrans) (definealpha 1) (definebeta 0) (definea (make-array-from-storage f64vector-storage-class (vector 5 5) ;; Padded zeros to demonstrate how slicing might be wrong (f64vector 0 0 0 0 0 0 1 1 3 5 0 2 1 4 6 0 3 1 5 7 0 4 1 6 8))) (definex (f64vector 1 2 1 1)) (definey (f64vector 0 0 0 0)) (defineb (array-transpose (array-slice a #(1 1)))) ;; WRONG! Do not do the following, since the storage object of `b` is the same ;; storage object of `a`. Instead, we need to first define a copy `c` that ;; copies the data so that the storage object has the correct layout. ;; ;; (dgemv! order transa 4 4 alpha (array-storage-object b) x beta y) (definec (array-copy b)) (dgemv! order transa 4 4 alpha (array-storage-object a) x beta y) (test "Result of multiplying the array c by the vector x" y (f64vector 12 5 22 32))

` `

*[procedure]*

`(array-copy! to at from #!optional start end)`

Copies the elements of array `from` into the array `to` starting at index `at`. If `start` and `end` are provided, these specify the array interval inside `from` to copy across. An error is signaled if the shapes of the array `to` and the interval specified by `start` and `end` are incongruent.

(import generalized-arrays test) (definea (make-array vector-storage-class #(3 3) 0)) (defineb (make-array vector-storage-class #(4 4) 1)) (array-copy! (array-slice a #(1 1) #(3 3)) (vector 0 0) b) (test-assert "Array a has had its lower corner copied from b" (array=? a (make-array-from-storage vector-storage-class #(3 3) (vector 0 0 0 0 1 1 0 1 1))))

` `

*[procedure]*

`(array-append axis arr brr)`

Appends the array `brr` to the array `arr` along the provided `axis`. An error is signaled if the arrays have different ranks, if the provided axis is less than zero or greater than the rank of the arrays, or if the axes outside of `axis` are not equal.

(import generalized-arrays test) (definea (make-array vector-storage-class #(2 2) 0)) (defineb (make-array vector-storage-class #(2 2) 1)) (definec (array-append 0 a b)) (test-assert "Appending arrays a and b produces 4x2 array" (array=? c (make-array-from-storage vector-storage-class #(4 2) (vector 0 0 0 0 1 1 1 1))))

` `

*[procedure]*

`(array-reshape array dimension)`

Reshapes the `array` to the default interval constructed by the provided `dimension`. This is done by performing a full copy of the array (as opposed to creating an array view). An error is signaled if the new shape does not have the same length as the existing shape.

(import generalized-arrays test) (definea (make-array-from-storage vector-storage-class #(4) (vector 1 2 3 4))) (test-assert "Reshaping array to a 2x2" (array=? (array-reshape a #(2 2)) (make-array-from-storage vector-storage-class #(2 2) (vector 1 2 3 4)))) (test-assert "Array-reshape never makes an array view" (not (array-view? (array-reshape a #(2 2)))))

` `

*[procedure]*

`(array-reclassify array storage-class)`

Reclassifies the storage of an array by copying the data to a newly constructed array and storage-object of the provided `storage-class`.

(import generalized-arrays srfi-4 test) (definea (make-array vector-storage-class #(3 3) 0)) ;; Reclassify into f32vector storage (defineb (array-reclassify a f32vector-storage-class)) (test-assert "Original array holds vector storage object" (vector? (array-storage-object b))) (test-assert "Reclassified array holds f32vector storage object" (f32vector? (array-storage-object b)))

` `

*[procedure]*

`(array->nested-list array)`

Converts an array's data into a nested-list representation. This API is mostly not intended for broad use and is only provided for writing data out in a structured manner.

*[procedure]*

`(array-read #!optional port)`

Reads an array (similar to `read`) from the provided `port`, or `(current-input-port)` if port is not specified. The array is read in using the optional reader syntax provided with this library:

;; #a for array ;; 2 for rank 2 ;; v for vector-storage-class (short-id of the storage-class) #a2v((1 2 3) (4 5 6) (7 8 9))

Note that if the storage class is unknown (unknown short-id) then the procedure will default to reading in the array with a `vector-storage-class`.

*[procedure]*

`(array-write #!optional port)`

Writes an array (similar to `write`) to the provided `port`, or `(current-output-port)` if port is not specified. The array is written out using the optional reader syntax provided with this library:

;; #a for array ;; 2 for rank 2 ;; v for vector-storage-class (short-id of the storage-class) #a2v((1 2 3) (4 5 6) (7 8 9))

### License

The code in this library is distributed according to the 3-clause BSD license. See LICENSE for more details.