Skip to content

Commit 1ca9ed6

Browse files
committed
Declare other tokens used later in the reference grammar
There were some tokens used in the grammar but not declared. Antlr doesn't really seem to care and happily uses them, but they appear in RustLexer.tokens in a potentially-unexpected order.
1 parent cdb6e1e commit 1ca9ed6

File tree

1 file changed

+3
-3
lines changed

1 file changed

+3
-3
lines changed

src/grammar/RustLexer.g4

+3-3
Original file line numberDiff line numberDiff line change
@@ -12,10 +12,10 @@ tokens {
1212
MINUS, STAR, SLASH, PERCENT, CARET, AND, OR, SHL, SHR, BINOP,
1313
BINOPEQ, AT, DOT, DOTDOT, DOTDOTDOT, COMMA, SEMI, COLON,
1414
MOD_SEP, RARROW, FAT_ARROW, LPAREN, RPAREN, LBRACKET, RBRACKET,
15-
LBRACE, RBRACE, POUND, DOLLAR, UNDERSCORE, LIT_CHAR,
15+
LBRACE, RBRACE, POUND, DOLLAR, UNDERSCORE, LIT_CHAR, LIT_BYTE,
1616
LIT_INTEGER, LIT_FLOAT, LIT_STR, LIT_STR_RAW, LIT_BINARY,
17-
LIT_BINARY_RAW, IDENT, LIFETIME, WHITESPACE, DOC_COMMENT,
18-
COMMENT, SHEBANG
17+
LIT_BINARY_RAW, QUESTION, IDENT, LIFETIME, WHITESPACE, DOC_COMMENT,
18+
COMMENT, SHEBANG, UTF8_BOM
1919
}
2020

2121
import xidstart , xidcontinue;

0 commit comments

Comments
 (0)