This shows you the differences between two versions of the page.

Link to this comparison view

reference:tokenize [2018/08/23 14:43]
reference:tokenize [2018/08/23 14:43] (current)
Line 1: Line 1:
 +=====  TOKENIZE =====
 +Note that this reference documentation is identical to the help that is displayed in MATLAB when you type "help tokenize"​.
 +  <a href=/​reference/​tokenize><​font color=green>​TOKENIZE</​font></​a>​ cuts a string into pieces, returning the pieces in a cell array
 +  Use as
 +    t = tokenize(str)
 +    t = tokenize(str,​ sep)
 +    t = tokenize(str,​ sep, rep)
 +  where
 +    str = the string that you want to cut into pieces
 +    sep = the separator at which to cut (default is whitespace)
 +    rep = whether to treat repeating separator characters as one (default is false)
 +  With the optional boolean flag "​rep"​ you can specify whether repeated
 +  separator characters should be squeezed together (e.g. multiple
 +  spaces between two words). The default is rep=1, i.e. repeated
 +  separators are treated as one.