lexer | elegant armor-plated JavaScript lexer | Theme library

I'm stuck on a problem with writing a parser in Haskell that I hope someone can help out with!

It is a bit more complicated than my usual parser because there are two layers of parsing. First a language definition is parsed into an AST, then that AST is transformed into another parser that parses the actual language.

I have made pretty good progress so far but I'm stuck on implementing recursion in the language definition. As the language definition is transformed from AST into a parser in a recursive function, I can't work out how it can call itself if it doesn't exist yet.

I'm finding it a bit hard to explain my problem, so maybe an example will help.

The language definition might define that a language consists of three keywords in sequence and then optional recursion in brackets.

I have a scannerless parser grammar utilizing the CharsAsTokens faux lexer which generates a usable Java Parser class for ANTLR4 versions through 4.6. But when updating to ANTLR 4.7.2 through 4.9.3-SNAPSHOT, the tool generates code producing dozens of compilation errors from the same grammar file, as detailed below.

My question here is simply: Are scannerless parser grammars no longer supported, or must their character-based terminals be specified differently in 4.7 and beyond?

Update:

Unfortunately, I cannot post my complete grammar here as it is derived from FOUO security marking guidance, access to which is retricted by the U.S. government (I am a DoD/IC contractor).

The incompatible upgrade issue however is entirely reproducible with the CSQL.g4 scannerless parser grammar example referred to by Ter in Section 5.6 of The Definitive ANTLR 4 Reference.

As does my grammar, the CSQL example uses CharsAsTokens.java for its tokenizer, and CharVocab.tokens as its token vocabulary.

Note that every token name is specified by its ASCII character-literal equivalent, as in:

I'm attempting build a query that will check an entity Project and check two columns either project_owner or project_contributor where project contributor is a Many to One relation.

The query I attempted to make was this (in the project entity repository):

I'm parsing C++-style scoped names, e.g., A::B. I want to parse such a name as a single token for a type name if it was previously declared as a type; otherwise, I want to parse it as three tokens A, ::, and B.

Given:

I have a scannerless security markings conversion grammar that generates code correctly and runs fine using antlr-4.4-complete.jar. But when I upgrade to antlr4-4.6-complete.jar or newer, code generation fails with "error(50): <.g4 file path>::: syntax error: mismatched character ':' expecting '{'" and other errors.

What changed in ANTLR v4.6 (or possibly v4.5 as I haven't tried that version) that would cause its lexer to err on grammars recognized by v4.4?

Sorry I can't provide a grammar snippit here, but access to the code is restricted.

I am trying to parse a file in order to reformat it. For this, I need to be able to distinguish between full line comments and end of line comments. I have been able to get lex to recognize full line comments properly, but am having issues with end of line comments.

For example: "a = 0; //This; works; fine" but "a = 0; //This, does; not;".

What confuses me the most is that re is able to recognise both comments without issue and yet lex can not.

Here is the relevant code (FL=full line, EL=end of line):

I've created my class in Dart this way, but I'm getting the Non-nullable instance field 'text' must be initialized. Try adding an initializer expression, or add a field initializer in this constructor, or mark it 'late'. I would like to know if there's a way to do it in a 'Python' style where this kind of class creation is possible, thank you in advance.

I am trying to add syntax highlighting to the text widget in tkinter i am using the code from another stack overflow question Pygments syntax highlighter in python tkinter text widget i binded the function for syntax hyghlighting to the root but the problem is that all the other default binds like CTRL A stops working. They work fine if i bind it to the text widget but the last entered letter doesnt get highlighted. Heres the code(i am new to programming so there might be many silly mistakes)

Problem: