HTTP/1.0 and HTTP/1.1 message codecs, plus a server, a client and test tools on top of them for fibers. A jaic
extension, not official Jai: other Jai compilers have no such module, and it may change between jaic releases. See
docs/extensions/http-codec.md, http-server.md and http-client.md.
The codec has no IO of its own. A Parser reads requests or responses from byte buffers the caller fills, in pieces of
any size, and reports the head, the body in pieces and the end of each message with strict RFC 9112 framing; the writers
build heads and chunked bodies. Nothing in it touches a socket, so it runs on every target, wasm included. The server
(App, serve) and the client (Client, get, send) need fibers and Extensions/Io, and are not built for wasm.
Failures are values: the parser's .ERROR event and Parser.error, an Http_Error returned by a handler, a
Client_Error returned by a client call. Memory the parser or a response owns is released with parser_free and
close; everything else is borrowed or in the temporary allocator, as each procedure says.
The spelling of method on the wire, such as "GET". .OTHER has no spelling of its own and gives ""; use
Head.method_text for the token a request actually carried. The string is a constant. The client refuses to send .OTHER
(.INVALID_METHOD).
True when a comma-separated header value (Connection, Upgrade, Accept-Encoding...) lists
token, compared without regard to case. Parameters after ; are ignored.
Prepares p to read requests or responses. Any state the parser had is released first, so it is safe to call on a parser
that was used. limits caps line, head and body sizes (the defaults are in Limits); a message over a limit is refused with
the error kind that maps to the matching status. For a response parser, set p.response_to to the request's method before
the response head arrives.
Releases the parser's buffers. The parser must not be used afterwards, except to start again with parser_init. Strings taken
from its head, trailers or events are invalid after this.
Between messages: gives back the head and trailer storage that grew past limit bytes (each of
the six arrays on its own), so a connection that once saw a huge head does not keep it. The
previous message's head and trailers are gone afterwards, like when the next message begins.
Does nothing while a message is under way.
Reads from data and returns the number of bytes used and one event. Call again with the
rest until NEED_MORE (the input is then used up); END and HEAD can come with no bytes used, so
loop on the event, not on the byte count. BODY data is a slice of data.
The connection has closed. Ends a body that runs until close; reports CLOSED when it fell
between messages and an UNEXPECTED_EOF error when it fell inside one.
Appends a request head, framing included, and returns how the body must be sent: with a
non-negative body_length, Content-Length is added (except for a GET, HEAD, DELETE, CONNECT or
TRACE without a body) and the body is sent as is (.LENGTH, or .NONE for length 0); with -1 the
length is not known, Transfer-Encoding: chunked is added and the body goes through
write_chunk and write_last_chunk (.CHUNKED). ok is false, and out is left as it was,
when a header is not valid or sets the framing itself.
Appends a response head and returns how the body must be sent (see write_request_head).
body_length is the length of the body the response is for; for a HEAD response (set
request_method) it only fills in Content-Length, and nothing may be sent after the head. 1xx,
204 and CONNECT 2xx responses get no framing headers. A length of -1 means unknown: chunked on
HTTP/1.1, and for HTTP/1.0 the body ends when the connection closes (Connection: close is
added and the caller must close).
Appends data as one chunk. An empty data writes nothing: a chunk of size 0 would end the
body (use write_last_chunk).
Appends data to out as one chunk of a chunked body: the size in hex, CRLF, the bytes, CRLF. Nothing is written for empty
data. out is a growing array the caller owns (it uses the context allocator); end the body with write_last_chunk.
The zero-size chunk with optional trailers, which ends a chunked body. False, with nothing
written, when a trailer is not valid or is a field a trailer may not be (framing, Host).
Appends the zero-size chunk that ends a chunked body, with no trailers. Returns true.
Decodes %XX escapes (and + as a space when plus_is_space, which forms and queries use but
paths do not). The text itself is returned, with no allocation, when there is nothing to
decode; otherwise the result comes from context.allocator. A % not followed by two hex
digits stays as it is.
Splits a=1&b=two&flag into decoded pairs (flag has an empty value). The pairs and the
strings that needed decoding come from context.allocator; the server calls this with
the temporary allocator.
An Http_Error with status 400, for when the request is malformed or its content is invalid.
format and args make the detail text as tprint does, in the temporary allocator.
An Http_Error with status 401, for when the caller has not authenticated. Add WWW-Authenticate with with_header.
format and args make the detail text as tprint does, in the temporary allocator.
An Http_Error with status 403, for when the caller is known but not allowed.
format and args make the detail text as tprint does, in the temporary allocator.
An Http_Error with status 404, for when there is nothing at the requested resource.
format and args make the detail text as tprint does, in the temporary allocator.
error := not_found("No user %", 7);
assert(error.status == 404);
assert(error.title == "Not Found");
assert(error.detail == "No user 7");
An Http_Error with status 409, for when the request clashes with the current state, such as a duplicate.
format and args make the detail text as tprint does, in the temporary allocator.
An Http_Error with status 410, for when the resource existed and has been removed for good.
format and args make the detail text as tprint does, in the temporary allocator.
An Http_Error with status 422, for when the request is well-formed but its content cannot be processed.
format and args make the detail text as tprint does, in the temporary allocator.
An Http_Error with status 429, for when the caller is over a rate limit; add Retry-After with with_header.
format and args make the detail text as tprint does, in the temporary allocator.
An Http_Error with status 500, for when something failed that is not the caller's fault.
format and args make the detail text as tprint does, in the temporary allocator.
An Http_Error with status 503, for when the service cannot answer now; add Retry-After with with_header.
format and args make the detail text as tprint does, in the temporary allocator.
The default format, RFC 9457: application/problem+json with type, title, status,
detail, instance (the request path) and, for bad input, location and field.
Copies the next bytes of the body, at most buffer.count, into buffer and returns how many.
Zero means the body is over (an empty buffer also reads zero). Waits for bytes when none have arrived,
within config.body_timeout. A body over the route's limit (max_body, else
config.max_body_size) is a 413, and a client that stops sending a 408; the error stays the
same on later calls.
The rest of the body. The bytes live in the connection's own buffer and stay valid until the
response has been written; copy them to keep them longer. Read as read: over the limit is a
413, and a declared length over the limit is refused without reading anything.
The rest of the body decoded as JSON into a T, with app.json_options. Content-Type: application/json (or +json) is required (415); a body that is not JSON is a 400 and one of
the wrong shape a 422, both naming the JSON path.
(path: string, content_type := "", status := 200) -> Response
A response with the contents of the file at path, read in pieces as the client takes them
(Content-Length is the file's size when the response is sent). An empty content_type is
chosen from the file's extension (application/octet-stream when unknown). A file that cannot
be opened is a 404, or a 500 for any other failure; a file that shrinks while it is sent cuts
the connection.
Asks the servers started by serve to stop: they stop accepting and finish the requests in
progress. Safe to call from any thread; serve returns once they are done.
Routes under prefix that share middleware, which runs after the app's own. The prefix is part of the group's type, so
a handler's Path(...) is checked against the whole route. The group belongs to the app and is released with it: do not
free it. Groups nest with group(parent, "/more"), and the middleware add up.
get(*app, "/users/{id}", handler). max_body (all the verbs take it) is the most request body
bytes the route reads: 0 uses config.max_body_size, -1 allows any size (for a handler that
streams its *Body); a body over it is a 413 from Json, Form, read and read_all. The
template and the handler are compile-time arguments:
the template is parsed and every extractor of the handler is checked against it while the
program compiles, and the code that fills them in is generated for this route. A route that
does not fit is an error that points at the call.
HEAD and OPTIONS are answered for every route by default (HEAD by the GET handler without the
body, OPTIONS with an Allow list); register them only to change that.
Adds a HEAD route. Register one only to change the default, which answers HEAD from the GET handler without sending the
body. The template, handler and max_body work as for get.
Adds an OPTIONS route. Register one only to change the default, which answers with an Allow list of the methods the path
has. The template, handler and max_body work as for get.
A Link over an accepted or connected TCP stream. peer is the stream's remote address, or empty if it cannot be read.
The link does not copy the stream: close the link (not the stream as well) when done.
Starts listening on the running scheduler (call it inside Io.io_run, from a fiber) and
returns at once; connections are served by fibers of this scheduler. Fails if the address
cannot be bound, the routes conflict or config.tls is not usable. stop and wait end it.
Starts the graceful shutdown: no new connections, idle ones close, requests in progress finish
(up to config.shutdown_timeout, then the connections are cut). Returns at once; wait
returns when it is done.
Blocks the calling fiber until the server has stopped and its connections are gone. The
Server itself stays valid (its fibers may still be winding down) and costs a few hundred
bytes; nothing needs it afterwards.
Runs the app until it is stopped: SIGINT or SIGTERM (when config.handle_signals), or
shutdown(*app) from another thread. One scheduler per config.threads (default: the
processor count), each with its own listener on the same port. Returns once the servers have
stopped; the error says why it could not start (the port is taken, two routes conflict, the TLS
configuration is unusable) and is .NONE for a clean stop.
A header sent with every request of client that does not set it: set_header(*api, "Authorization", "Bearer abc"). Setting a name again replaces the value.
Sends request and returns once the status line and headers have arrived. The response is
whatever the server answered (a 404 is a response, not an error); the body is read from it with
read_text, read_bytes, read_json or read_some. Errors are only about not getting a
response: the URL, the connection, a timeout, a reply that is not HTTP, too many redirects.
The whole body as text, in memory from the context allocator (release it with free). A body
longer than the request's max_response_size is a .TOO_LARGE error and is not read.
The Io.Reader read procedure over a response body: it reads through read_some, stops at the request's
max_response_size, and keeps the client error it ran into in the Body_Reader (it can report only an Io_Error).
Reads the next part of the body into buffer and returns how many bytes it wrote: at least one
while there is more, then 0 once the body has ended (and on every call after that). It waits
for data when none has arrived. For a body that should not be held in memory:
A non-2xx response as an error, for code that treats those as failures: .NONE for a 2xx. The
error carries status, title and detail (from an RFC 9457 problem body, the kind the
server sends, when there is one) and the body text, so this reads the body. Release the error
with free_error.
The counts of the running scheduler's pools, for tests and diagnostics. Call it on a scheduler, inside Io.io_run or a
test client call. A program that closes or reads every response to the end shows no open responses.
A client for app. The app must stay alive and unchanged in its routes while the client is
used; its state pointer is yours to inspect between requests. The client belongs to the test
run (run_tests frees it afterwards); free_test_client releases one sooner.
Releases a client made by test_client before the test ends. run_tests frees the clients of a test by itself, so call this
only for a client that should not live until then. The client and its responses must not be used afterwards.
The response has the header, with exactly this value (any case in the name). For
Content-Type a value without parameters matches the media type: "application/json" accepts
application/json; charset=utf-8.
(response: Client_Response, $T: Type, status := 0, loc := #caller_location) -> T
The body decoded as JSON into a T. The status must be status, or 2xx when it is left out,
and the Content-Type JSON. On failure the value is empty. The strings in it are allocated
with the context allocator; a test may leave them.
(response: Client_Response, status: int, detail := "", loc := #caller_location) -> Problem
The response is an RFC 9457 problem document with this status and, when detail is given, a
detail that contains it. Returns the decoded document (empty when the check failed).
Runs the tests one after another on the calling thread, prints a line per test (and the
details of what failed), then a summary. Exits with status 1 when any expectation failed.
Clients made by test_client are released after each test.
Collects the failure text that would be printed instead of printing it, and does not count the
failures against the run: for tests of the test tools themselves. end_failure_capture
returns the text and how many failures there were.
Ends a begin_failure_capture and returns the failure text collected since, in the temporary allocator, and how many
expectations failed. Failures that were captured are not counted against the run.
The request methods the codec knows. A token that is not one of them is .OTHER and its spelling stays in
Head.method_text. parse_method turns text into a Method and method_name turns it back.
The HTTP versions the codec speaks: HTTP_1_0 and HTTP_1_1. Anything else in a start line is refused as
.VERSION_NOT_SUPPORTED (or .BAD_VERSION when it is not shaped like a version).
Well-known header names. Parsed headers carry the id so code can compare an enum instead of
strings; HEADER_NAMES holds the canonical spelling the writers use.
One header field line. name keeps the spelling on the wire; id says which well-known
header it is (.UNKNOWN for others). value has no leading or trailing blanks. Strings of a
parsed head point into the parser's buffer.
Why the parser refused a message, one value per rule it enforces. The comments give the status a server answers with
(error_status returns the same number). The kind is in Parser.error after an .ERROR event, and a failed parser stays failed.
parser: Parser;
parser_init(*parser, .REQUEST);
deferparser_free(*parser);
// An HTTP/1.1 request without a Host header.
text := "GET / HTTP/1.1\r\n\r\n";
_, event := parse(*parser, .{ text.count, text.data });
assert(event.kind == .ERROR);
assert(parser.error.kind == .MISSING_HOST);
assert(error_status(parser.error.kind) == 400);
What parse found. Loop on the event, not on the number of bytes used: HEAD and END can come with none. The comments
say where the data of each kind lives.
One result of parse: its kind and, for .BODY, data, a slice of the input that was handed to parse (not a copy), so it
is valid only as long as that input is. For the other kinds data is empty.
A header line of the head or the trailers, as offsets into the buffer that holds it (the buffer
moves while it grows; strings are made once the head is complete).
An incremental HTTP/1.x parser for one direction of a connection. Start it with parser_init, feed it with parse in
pieces of any size, and release it with parser_free. The result does not depend on how the bytes were split.
A parsed request, as handlers and middleware see it. The strings (target, header names and
values) stay valid until the response has been written: copy what has to outlive the request.
The body is not here: take a *Body parameter to read it (see Body), or Json(T), Form(T).
What a handler returns when it wants full control. The headers and body must stay valid until
the response is written: use string literals or the temporary allocator (tprint, ,, temp),
which the server resets after each response. The builders below do.
What a handler returns to answer with a status instead of a value. A handler's .{} means
no error. The server sends it as an RFC 9457 application/problem+json document:
find_user :: (id: int) -> string, Http_Error {
return"", not_found("No user with id %", id);
}
_, error := find_user(7);
// Sent as 404 {"type":"about:blank","title":"Not Found","status":404,"detail":"No user with id 7"}assert(error.kind == .STATUS);
assert(error.status == 404);
assert(error.detail == "No user with id 7");
Where a stream's bytes come from. Called once, after the head has been sent, on the
connection's fiber; it may wait (Io.sleep, channels) between pieces. Not called for a HEAD
request.
One {name} of the route, parsed as a T: integers, floats, bools, strings and enums (by
member name). A value that does not parse is a 400 that names the parameter.
The query string, decoded into a T with the rules and notes of JSON (default, rename,
alias...): a field without default is required, repeated keys fill arrays. The T lives in
temporary storage for the request.
The whole request body as JSON, read before the handler runs (read_json does the same from a
*Body). Content-Type: application/json is required (415 otherwise), a body that is not
valid JSON is a 400 and one of the wrong shape a 422; both name the JSON path. A body over the
route's max_body is a 413. The T lives in temporary storage for the request: copy the
strings you keep.
One request header that may be missing, parsed as a T when it is there. present says which; when it is false value is
the zero value. A header that is present but does not parse as T is still a 400. Strings point into the request and
last as long as it does.
The shared state set with app.state = *my_state. Take it as *State(My_State); the fields
of My_State are reachable directly. The pointer is the state itself, not a copy: handlers on
several threads see the same memory, so guard what they change.
What an App does when nothing else is said. Every number is a limit or a timeout in
milliseconds unless it says otherwise; 0 turns the ones that allow it off.
Called for every error response (a handler's Http_Error, a failed extractor, 404, 405, 413...)
to make the response. Set app.on_error to use your own format instead of problem+json.
A web application: its routes, middleware, settings and shared state. A zeroed App is a working one. Add routes with
get, post and the other verbs, then serve it with serve or start, or drive it in memory with test_client. Routes
are added before serving and not changed while the app serves; the handlers run on the scheduler threads. The app owns
the router, groups and setup errors it builds: release them with free_app after the servers have stopped. Strings you
store in its fields (config.address, ...) are yours and are not copied.
hello :: () -> string {
return"hello";
}
main :: () {
app: App;
deferfree_app(*app);
get(*app, "/hello", hello);
api := test_client(*app); // the app, driven in memory: no socket
response, error := get(api, "/hello");
deferclose(*response);
assert(error.kind == .NONE && response.status == 200);
text, read_error := read_text(*response);
defer free(text);
assert(read_error.kind == .NONE && text == "hello");
}
A prefix and middleware shared by the routes registered through it. The prefix is part of the
type, so a handler's Path("org", ...) is checked against the whole route, prefix included.
The connection as the server loop and the client exchange see it: the reader and writer halves and the address of the
peer. Both halves share one socket or pipe end, so close on the link (or on either half) ends the connection once.
A transport of your own only has to produce a Link.
Why a server did not start (Serve_Error.kind), with .NONE for success. The comments say what each kind points at. Routes are
checked when the server starts, not when they are added, so a conflict shows up here.
What went wrong when the client did not get, or could not read, an answer (Client_Error.kind), with .NONE for success. A
response with any HTTP status, a 404 included, is not an error; error_for_status turns a non-2xx status into .STATUS.
// A URL the client cannot request is refused before anything is sent.
api: Client;
response, error := get(*api, "ftp://example.com/file");
assert(error.kind == .INVALID_URL);
assert(response.status == 0);
What the client returns when something went wrong. A handler's Http_Error is about answering
with a status; this is about not getting, or not reading, an answer. error_text makes one
line of it.
Pool settings, for the whole program. Idle connections are closed when they have waited
idle_timeout milliseconds or when more than max_idle_per_host pile up for one host.
The body of a response as an Io.Reader, for read_json: each read is a read_some that stops
at the request's max_response_size and keeps the client error it ran into, which the reader
can only report as an Io_Error.
What the running scheduler's pools hold right now, for tests and diagnostics (nothing here is
needed to use the module). Call it on a scheduler: inside Io.io_run or a test client call.
One test for run_tests: a name to print and a procedure to run. The procedure makes its checks with the expect_ calls;
a failed one is printed and counted, and the test goes on.
check_math :: () {
expect_equal(1 + 1, 2, "addition");
}
main :: () {
run_tests(.[.{ name = "math", run = check_math }]);
}