NAME

tokenizer -- split a 0-terminated stralloc object into a list of strings


SYNOPSIS

   #include <afblib/tokenizer.h>
   int tokenizer(stralloc* input, strlist* tokens);


DESCRIPTION

The function tokenizer accepts a nullbyte-terminated stralloc object and splits it into whitespace-separated tokens. All characters for which isspace (see the ctype manpage) returns true are treated as whitespace. Sequences of whitespaces are considered as one field separator. All whitespaces within input are replaced by nullbytes.

tokens is initially emptied, and then subsequently filled with all tokens found. Each of the tokens consists of at least one non-whitespace character. Hence, the list of tokens will be empty if input is empty (beside the terminating nullbyte) or is filled with whitespaces only.


DIAGNOSTICS

In case of failures, tokenizer returns 0. Otherwise the return value is 1.


AUTHOR

Andreas F. Borchert