tokenizer -- split a 0-terminated stralloc object into a list of strings
#include <afblib/tokenizer.h>
int tokenizer(stralloc* input, strlist* tokens);
The function tokenizer accepts a nullbyte-terminated stralloc object and splits it into whitespace-separated tokens. All characters for which isspace (see the ctype manpage) returns true are treated as whitespace. Sequences of whitespaces are considered as one field separator. All whitespaces within input are replaced by nullbytes.
tokens is initially emptied, and then subsequently filled with all tokens found. Each of the tokens consists of at least one non-whitespace character. Hence, the list of tokens will be empty if input is empty (beside the terminating nullbyte) or is filled with whitespaces only.
In case of failures, tokenizer returns 0. Otherwise the return value is 1.
Andreas F. Borchert