diff --git a/app/preproc/PdePreprocessor.java b/app/preproc/PdePreprocessor.java index c598ae309..94b2455a0 100644 --- a/app/preproc/PdePreprocessor.java +++ b/app/preproc/PdePreprocessor.java @@ -39,6 +39,98 @@ import antlr.collections.impl.*; import com.oroinc.text.regex.*; +/** + * Class that orchestrates preprocessing p5 syntax into straight Java. + *

+ * The PDE Preprocessor is based on the Java Grammar that comes with + * ANTLR 2.7.2. Moving it forward to a new version of the grammar + * shouldn't be too difficult. + *

+ * Here's some info about the various files in this directory: + *

+ * java.g: this is the ANTLR grammar for Java 1.3/1.4 from the + * ANTLR distribution. It is in the public domain. The only change to + * this file from the original this file is the uncommenting of the + * clauses required to support assert(). + *

+ * java.tree.g: this describes the Abstract Syntax Tree (AST) + * generated by java.g. It is only here as a reference for coders hacking + * on the preprocessor, it is not built or used at all. Note that pde.g + * overrides some of the java.g rules so that in PDE ASTs, there are a + * few minor differences. Also in the public domain. + *

+ * pde.g: this is the grammar and lexer for the PDE language + * itself. It subclasses the java.g grammar and lexer. There are a couple + * of overrides to java.g that I hope to convince the ANTLR folks to fold + * back into their grammar, but most of this file is highly specific to + * PDE itself. + * PdeEmitter.java: this class traverses the AST generated by + * the PDE Recognizer, and emits it as Java code, doing any necessary + * transformations along the way. It is based on JavaEmitter.java, + * available from antlr.org, written by Andy Tripp , + * who has given permission for it to be distributed under the GPL. + *

+ * ExtendedCommonASTWithHiddenTokens.java: this adds a necessary + * initialize() method, as well as a number of methods to allow for XML + * serialization of the parse tree in a such a way that the hidden tokens + * are visible. Much of the code is taken from the original + * CommonASTWithHiddenTokens class. I hope to convince the ANTLR folks + * to fold these changes back into that class so that this file will be + * unnecessary. + *

+ * TokenStreamCopyingHiddenTokenFilter.java: this class provides + * TokenStreamHiddenTokenFilters with the concept of tokens which can be + * copied so that they are seen by both the hidden token stream as well + * as the parser itself. This is useful when one wants to use an + * existing parser (like the Java parser included with ANTLR) that throws + * away some tokens to create a parse tree which can be used to spit out + * a copy of the code with only minor modifications. Partially derived + * from ANTLR code. I hope to convince the ANTLR folks to fold this + * functionality back into ANTLR proper as well. + *

+ * whitespace_test.pde: a torture test to ensure that the + * preprocessor is correctly preserving whitespace, comments, and other + * hidden tokens correctly. See the comments in the code for details about + * how to run the test. + *

+ * All other files in this directory are generated at build time by ANTLR + * itself. The ANTLR manual goes into a fair amount of detail about the + * what each type of file is for. + *

+ * Current Preprocessor Subsitutions: + *

+ * Other preprocessor functionality + * + */ public class PdePreprocessor { static final int JDK11 = 0;