Unicode module

utf8_iter

procedure · 1 overloadSource

Loops over the code points of a string: for: utf8_iter text { ... }. The body sees it, the code point, and it_index, the number of characters before it (not the byte offset). for <: goes from the last character to the first, and it_index is -1 there. Iterating by pointer is not allowed.

utf8_iter :: (s: *string, body: Code, flags: For_Flags)

The iteration consumes text: the string passed in is advanced as it goes. It is strict: on invalid UTF-8 (an overlong form, a surrogate, a bad or stray continuation byte or a character cut short) the loop logs invalid UTF-8 as an error through the context logger and stops, so the rest of the string is left in text and the body does not see it. Going backwards, the last character and any stray bytes after it are checked first.

Parameters

s: *string
body: Code
flags: For_Flags
ExampleOpen in playground ↗
text := "a\xc3\xa9\xe2\x82\xac";
codes: [..] u32;
defer array_free(codes);
for: utf8_iter text array_add(*codes, it);
assert(codes.count == 3 && codes[0] == #char "a" && codes[1] == 0xe9 && codes[2] == 0x20ac);

See also