Class Tokenizer
- java.lang.Object
-
- org.sonatype.maven.polyglot.atom.parsing.Tokenizer
-
public class Tokenizer extends Object
Taken from the Loop programming language compiler pipeline.- Author:
- dhanji@gmail.com (Dhanji R. Prasanna)
-
-
Method Summary
All Methods Static Methods Instance Methods Concrete Methods Modifier and Type Method Description static String
detokenize(List<Token> tokens)
List<Token>
tokenize()
-