Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[go target] Build errors with PartiQL parser #4632

Open
h3n4l opened this issue May 27, 2024 · 8 comments
Open

[go target] Build errors with PartiQL parser #4632

h3n4l opened this issue May 27, 2024 · 8 comments

Comments

@h3n4l
Copy link
Contributor

h3n4l commented May 27, 2024

The ANTLR4 version I used is 4.13.1.
Use the PartiQL grammar in PartiQL site, https://partiql.org/syntax/antlr.html. And generate go target with command antlr -Dlanguage=Go -package parser -long-messages -listener *.g4, the target has build errors, for examples:

cannot use ctx (variable of type *ExprOrBaseContext) as antlr.RuleNode value in argument to v.VisitChildren: *ExprOrBaseContext does not implement antlr.RuleNode (wrong type for method GetParent)
		have GetParent() IExprAndContext
		want GetParent() antlr.Tree
cannot use ctx (variable of type *ExprAndBaseContext) as antlr.RuleNode value in argument to v.VisitChildren: *ExprAndBaseContext does not implement antlr.RuleNode (wrong type for method GetParent)
		have GetParent() IExprNotContext
		want GetParent() antlr.Tree
cannot use ctx (variable of type *ExprNotBaseContext) as antlr.RuleNode value in argument to v.VisitChildren: *ExprNotBaseContext does not implement antlr.RuleNode (wrong type for method GetParent)
		have GetParent() IExprPredicateContext
		want GetParent() antlr.Tree
cannot use ctx (variable of type *PredicateBaseContext) as antlr.RuleNode value in argument to v.VisitChildren: *PredicateBaseContext does not implement antlr.RuleNode (wrong type for method GetParent)
		have GetParent() IMathOp00Context
		want GetParent() antlr.Tree
cannot use ctx (variable of type *MathOp00Context) as antlr.RuleNode value in argument to v.VisitChildren: *MathOp00Context does not implement antlr.RuleNode (wrong type for method GetParent)
		have GetParent() IMathOp01Context
		want GetParent() antlr.Tree
cannot use ctx (variable of type *MathOp01Context) as antlr.RuleNode value in argument to v.VisitChildren: *MathOp01Context does not implement antlr.RuleNode (wrong type for method GetParent)
		have GetParent() IMathOp02Context
		want GetParent() antlr.Tree
cannot use ctx (variable of type *MathOp02Context) as antlr.RuleNode value in argument to v.VisitChildren: *MathOp02Context does not implement antlr.RuleNode (wrong type for method GetParent)
		have GetParent() IValueExprContext
		want GetParent() antlr.Tree
cannot use ctx (variable of type *ValueExprContext) as antlr.RuleNode value in argument to v.VisitChildren: *ValueExprContext does not implement antlr.RuleNode (wrong type for method GetParent)
		have GetParent() IExprPrimaryContext
		want GetParent() antlr.Tree
cannot use s (variable of type *ExprOrBaseContext) as antlr.RuleContext value in return statement: *ExprOrBaseContext does not implement antlr.RuleContext (wrong type for method GetParent)
		have GetParent() IExprAndContext
		want GetParent() antlr.Tree
cannot use s (variable of type *ExprOrBaseContext) as antlr.RuleNode value in argument to t.VisitChildren: *ExprOrBaseContext does not implement antlr.RuleNode (wrong type for method GetParent)
		have GetParent() IExprAndContext
		want GetParent() antlr.Tree
cannot use NewExprOrBaseContext(p, localctx) (value of type *ExprOrBaseContext) as IExprOrContext value in assignment: *ExprOrBaseContext does not implement IExprOrContext (wrong type for method GetParent)
		have GetParent() IExprAndContext
		want GetParent() antlr.Tree
impossible type assertion: localctx.(*ExprOrBaseContext)
	*ExprOrBaseContext does not implement IExprOrContext (wrong type for method GetParent)
		have GetParent() IExprAndContext
		want GetParent() antlr.Tree
cannot use s (variable of type *ExprAndBaseContext) as antlr.RuleContext value in return statement: *ExprAndBaseContext does not implement antlr.RuleContext (wrong type for method GetParent)
		have GetParent() IExprNotContext
		want GetParent() antlr.Tree
cannot use s (variable of type *ExprAndBaseContext) as antlr.RuleNode value in argument to t.VisitChildren: *ExprAndBaseContext does not implement antlr.RuleNode (wrong type for method GetParent)
		have GetParent() IExprNotContext
		want GetParent() antlr.Tree
cannot use NewExprAndBaseContext(p, localctx) (value of type *ExprAndBaseContext) as IExprAndContext value in assignment: *ExprAndBaseContext does not implement IExprAndContext (wrong type for method GetParent)
		have GetParent() IExprNotContext
		want GetParent() antlr.Tree
impossible type assertion: localctx.(*ExprAndBaseContext)
	*ExprAndBaseContext does not implement IExprAndContext (wrong type for method GetParent)
		have GetParent() IExprNotContext
		want GetParent() antlr.Tree
cannot use s (variable of type *ExprNotBaseContext) as antlr.RuleContext value in return statement: *ExprNotBaseContext does not implement antlr.RuleContext (wrong type for method GetParent)
		have GetParent() IExprPredicateContext
		want GetParent() antlr.Tree
cannot use s (variable of type *ExprNotBaseContext) as antlr.RuleNode value in argument to t.VisitChildren: *ExprNotBaseContext does not implement antlr.RuleNode (wrong type for method GetParent)
		have GetParent() IExprPredicateContext
		want GetParent() antlr.Tree
cannot use NewExprNotBaseContext(p, localctx) (value of type *ExprNotBaseContext) as IExprNotContext value in assignment: *ExprNotBaseContext does not implement IExprNotContext (wrong type for method GetParent)
		have GetParent() IExprPredicateContext
		want GetParent() antlr.Tree
impossible type assertion: localctx.(*ExprNotBaseContext)
	*ExprNotBaseContext does not implement IExprNotContext (wrong type for method GetParent)
		have GetParent() IExprPredicateContext
		want GetParent() antlr.Tree
cannot use s (variable of type *PredicateBaseContext) as antlr.RuleContext value in return statement: *PredicateBaseContext does not implement antlr.RuleContext (wrong type for method GetParent)
		have GetParent() IMathOp00Context
		want GetParent() antlr.Tree
cannot use s (variable of type *PredicateBaseContext) as antlr.RuleNode value in argument to t.VisitChildren: *PredicateBaseContext does not implement antlr.RuleNode (wrong type for method GetParent)
		have GetParent() IMathOp00Context
		want GetParent() antlr.Tree
cannot use NewPredicateBaseContext(p, localctx) (value of type *PredicateBaseContext) as IExprPredicateContext value in assignment: *PredicateBaseContext does not implement IExprPredicateContext (wrong type for method GetParent)
		have GetParent() IMathOp00Context
		want GetParent() antlr.Tree
impossible type assertion: localctx.(*PredicateBaseContext)
	*PredicateBaseContext does not implement IExprPredicateContext (wrong type for method GetParent)
		have GetParent() IMathOp00Context
		want GetParent() antlr.Tree
duplicate method GetParent
duplicate method SetParent
duplicate method GetParent (see details)
duplicate method SetParent (see details)
cannot use s (variable of type *MathOp00Context) as antlr.RuleContext value in return statement: *MathOp00Context does not implement antlr.RuleContext (wrong type for method GetParent)
		have GetParent() IMathOp01Context
		want GetParent() antlr.Tree
cannot use s (variable of type *MathOp00Context) as antlr.Tree value in argument to antlr.TreesStringTree: *MathOp00Context does not implement antlr.Tree (wrong type for method GetParent)
		have GetParent() IMathOp01Context
		want GetParent() antlr.Tree
cannot use s (variable of type *MathOp00Context) as antlr.RuleNode value in argument to t.VisitChildren: *MathOp00Context does not implement antlr.RuleNode (wrong type for method GetParent)
		have GetParent() IMathOp01Context
		want GetParent() antlr.Tree
cannot use _prevctx (variable of type IMathOp00Context) as antlr.ParserRuleContext value in variable declaration: IMathOp00Context does not implement antlr.ParserRuleContext (wrong type for method GetParent)
		have GetParent() IMathOp01Context
		want GetParent() antlr.Tree
cannot use localctx (variable of type IMathOp00Context) as antlr.ParserRuleContext value in argument to p.EnterRecursionRule: IMathOp00Context does not implement antlr.ParserRuleContext (wrong type for method GetParent)
		have GetParent() IMathOp01Context
		want GetParent() antlr.Tree
cannot use localctx (variable of type IMathOp00Context) as antlr.ParserRuleContext value in argument to p.EnterOuterAlt: IMathOp00Context does not implement antlr.ParserRuleContext (wrong type for method GetParent)
		have GetParent() IMathOp01Context
		want GetParent() antlr.Tree
cannot use localctx (variable of type IMathOp00Context) as antlr.ParserRuleContext value in argument to p.PushNewRecursionContext: IMathOp00Context does not implement antlr.ParserRuleContext (wrong type for method GetParent)
		have GetParent() IMathOp01Context
		want GetParent() antlr.Tree
duplicate method GetParent
duplicate method SetParent
duplicate method GetParent (see details)
duplicate method SetParent (see details)
cannot use s (variable of type *MathOp01Context) as antlr.RuleContext value in return statement: *MathOp01Context does not implement antlr.RuleContext (wrong type for method GetParent)
		have GetParent() IMathOp02Context
		want GetParent() antlr.Tree
cannot use s (variable of type *MathOp01Context) as antlr.Tree value in argument to antlr.TreesStringTree: *MathOp01Context does not implement antlr.Tree (wrong type for method GetParent)
		have GetParent() IMathOp02Context
		want GetParent() antlr.Tree
cannot use s (variable of type *MathOp01Context) as antlr.RuleNode value in argument to t.VisitChildren: *MathOp01Context does not implement antlr.RuleNode (wrong type for method GetParent)
		have GetParent() IMathOp02Context
		want GetParent() antlr.Tree
cannot use _prevctx (variable of type IMathOp01Context) as antlr.ParserRuleContext value in variable declaration: IMathOp01Context does not implement antlr.ParserRuleContext (wrong type for method GetParent)
		have GetParent() IMathOp02Context
		want GetParent() antlr.Tree
cannot use localctx (variable of type IMathOp01Context) as antlr.ParserRuleContext value in argument to p.EnterRecursionRule: IMathOp01Context does not implement antlr.ParserRuleContext (wrong type for method GetParent)
		have GetParent() IMathOp02Context
		want GetParent() antlr.Tree
cannot use localctx (variable of type IMathOp01Context) as antlr.ParserRuleContext value in argument to p.EnterOuterAlt: IMathOp01Context does not implement antlr.ParserRuleContext (wrong type for method GetParent)
		have GetParent() IMathOp02Context
		want GetParent() antlr.Tree
cannot use localctx (variable of type IMathOp01Context) as antlr.ParserRuleContext value in argument to p.PushNewRecursionContext: IMathOp01Context does not implement antlr.ParserRuleContext (wrong type for method GetParent)
		have GetParent() IMathOp02Context
		want GetParent() antlr.Tree
duplicate method GetParent
duplicate method SetParent
duplicate method GetParent (see details)
duplicate method SetParent (see details)
cannot use s (variable of type *MathOp02Context) as antlr.RuleContext value in return statement: *MathOp02Context does not implement antlr.RuleContext (wrong type for method GetParent)
		have GetParent() IValueExprContext
		want GetParent() antlr.Tree
cannot use s (variable of type *MathOp02Context) as antlr.Tree value in argument to antlr.TreesStringTree: *MathOp02Context does not implement antlr.Tree (wrong type for method GetParent)
		have GetParent() IValueExprContext
		want GetParent() antlr.Tree
cannot use s (variable of type *MathOp02Context) as antlr.RuleNode value in argument to t.VisitChildren: *MathOp02Context does not implement antlr.RuleNode (wrong type for method GetParent)
		have GetParent() IValueExprContext
		want GetParent() antlr.Tree
cannot use _prevctx (variable of type IMathOp02Context) as antlr.ParserRuleContext value in variable declaration: IMathOp02Context does not implement antlr.ParserRuleContext (wrong type for method GetParent)
		have GetParent() IValueExprContext
		want GetParent() antlr.Tree
cannot use localctx (variable of type IMathOp02Context) as antlr.ParserRuleContext value in argument to p.EnterRecursionRule: IMathOp02Context does not implement antlr.ParserRuleContext (wrong type for method GetParent)
		have GetParent() IValueExprContext
		want GetParent() antlr.Tree
cannot use localctx (variable of type IMathOp02Context) as antlr.ParserRuleContext value in argument to p.EnterOuterAlt: IMathOp02Context does not implement antlr.ParserRuleContext (wrong type for method GetParent)
		have GetParent() IValueExprContext
		want GetParent() antlr.Tree
cannot use localctx (variable of type IMathOp02Context) as antlr.ParserRuleContext value in argument to p.PushNewRecursionContext: IMathOp02Context does not implement antlr.ParserRuleContext (wrong type for method GetParent)
		have GetParent() IValueExprContext
		want GetParent() antlr.Tree
duplicate method GetParent
duplicate method SetParent
duplicate method GetParent (see details)
duplicate method SetParent (see details)
cannot use s (variable of type *ValueExprContext) as antlr.RuleContext value in return statement: *ValueExprContext does not implement antlr.RuleContext (wrong type for method GetParent)
		have GetParent() IExprPrimaryContext
		want GetParent() antlr.Tree
cannot use s (variable of type *ValueExprContext) as antlr.Tree value in argument to antlr.TreesStringTree: *ValueExprContext does not implement antlr.Tree (wrong type for method GetParent)
		have GetParent() IExprPrimaryContext
		want GetParent() antlr.Tree
cannot use s (variable of type *ValueExprContext) as antlr.RuleNode value in argument to t.VisitChildren: *ValueExprContext does not implement antlr.RuleNode (wrong type for method GetParent)
		have GetParent() IExprPrimaryContext
		want GetParent() antlr.Tree
cannot use localctx (variable of type IValueExprContext) as antlr.ParserRuleContext value in argument to p.EnterRule: IValueExprContext does not implement antlr.ParserRuleContext (wrong type for method GetParent)
		have GetParent() IExprPrimaryContext
		want GetParent() antlr.Tree
cannot use localctx (variable of type IValueExprContext) as antlr.ParserRuleContext value in argument to p.EnterOuterAlt: IValueExprContext does not implement antlr.ParserRuleContext (wrong type for method GetParent)
		have GetParent() IExprPrimaryContext
		want GetParent() antlr.Tree
cannot use localctx (variable of type IValueExprContext) as antlr.ParserRuleContext value in argument to p.EnterOuterAlt: IValueExprContext does not implement antlr.ParserRuleContext (wrong type for method GetParent)
		have GetParent() IExprPrimaryContext
		want GetParent() antlr.Tree
impossible type assertion: localctx.(*MathOp00Context)
	*MathOp00Context does not implement antlr.RuleContext (wrong type for method GetParent)
		have GetParent() IMathOp01Context
		want GetParent() antlr.Tree
cannot use t (variable of type *MathOp00Context) as antlr.RuleContext value in argument to p.MathOp00_Sempred: *MathOp00Context does not implement antlr.RuleContext (wrong type for method GetParent)
		have GetParent() IMathOp01Context
		want GetParent() antlr.Tree
impossible type assertion: localctx.(*MathOp01Context)
	*MathOp01Context does not implement antlr.RuleContext (wrong type for method GetParent)
		have GetParent() IMathOp02Context
		want GetParent() antlr.Tree
cannot use t (variable of type *MathOp01Context) as antlr.RuleContext value in argument to p.MathOp01_Sempred: *MathOp01Context does not implement antlr.RuleContext (wrong type for method GetParent)
		have GetParent() IMathOp02Context
		want GetParent() antlr.Tree
impossible type assertion: localctx.(*MathOp02Context)
	*MathOp02Context does not implement antlr.RuleContext (wrong type for method GetParent)
		have GetParent() IValueExprContext
		want GetParent() antlr.Tree
cannot use t (variable of type *MathOp02Context) as antlr.RuleContext value in argument to p.MathOp02_Sempred: *MathOp02Context does not implement antlr.RuleContext (wrong type for method GetParent)
		have GetParent() IValueExprContext
		want GetParent() antlr.Tree
unreachable code
...
@jimidle
Copy link
Collaborator

jimidle commented May 27, 2024

I am reworking the Go visitor code as there are some other issues. Though it generally works, there are some issues that need manual intervention, which is not ideal. It may take a week or so to get this ready for release. though.

Looks like you are using the Listener here though, which does work, so I woudl need to see your code to work out what is wrong here.

Please don't use images for text errors as they are hard to read.

@kaby76
Copy link
Contributor

kaby76 commented May 27, 2024

I have several comments on this grammar. There are several problems.

  • grammar PartiQL; is a declaration for a combined grammar. The grammar works, but I've seen similar situations where it doesn't. I would recommend changing the declaration to parser grammar PartiQLParser;.
  • options { caseInsentive=true; } has no meaning in a combined or parser grammar. Remove.
  • statement is a start rule (i.e., a rule that terminates with EOF), but it is also used in root. This could be a problem if the statement is followed by any terminal or non-terminal symbol in some rule, e.g., root. (What token can occur after the end of file?) The Antlr tool will not warn if you try.
  • There are two unused parser rules, tableName and tableconstrantName. The rules should be deleted.
$ bash /c/Users/Kenne/Documents/GitHub/g4-scripts/find-unused-parser-symbols.sh  *.g4
Arguments were provided.
Finding unused parser symbols in grammars...
PartiQLParser.g4:L77: tableName : symbolPrimitive;
                      ^
PartiQLParser.g4:L78: tableConstraintName : symbolPrimitive;
                      ^
05/27-12:38:16 ~/issues/g4-4632
$ grep tableName *.g4
PartiQLParser.g4:tableName : symbolPrimitive;
05/27-12:38:30 ~/issues/g4-4632
$ grep tablecontraintName *.g4
05/27-12:38:43 ~/issues/g4-4632
  • There are many lexer symbols declared that are not used at all in the parser grammar, not explicitly (via name, e.g., ABSOLUTE is not on the right-hand side of a parser rule), implicitly (via a string literal, e.g.,'absolute' in the right-hand side of a parser rule), or with a dot in the right-hand side of a parser rule.
$ bash /c/Users/Kenne/Documents/GitHub/g4-scripts/find-unused-lexer-symbols.sh  *.g4
Arguments were provided.
Finding unused lexer symbols in grammars...
1st pass: fold string literals in parser rules to make explicit lexer rule used.
2nd pass: find unreferenced lexer rule symbols.
PartiQLLexer.g4:ABSOLUTE
PartiQLLexer.g4:ACTION
PartiQLLexer.g4:ADD
PartiQLLexer.g4:ALLOCATE
PartiQLLexer.g4:ALTER
PartiQLLexer.g4:ARE
PartiQLLexer.g4:ASSERTION
PartiQLLexer.g4:AUTHORIZATION
PartiQLLexer.g4:BEGIN
PartiQLLexer.g4:BIT
PartiQLLexer.g4:CASCADE
PartiQLLexer.g4:CASCADED
PartiQLLexer.g4:CATALOG
PartiQLLexer.g4:CHECK
PartiQLLexer.g4:CLOSE
PartiQLLexer.g4:COLLATE
PartiQLLexer.g4:COLLATION
PartiQLLexer.g4:COLUMN
PartiQLLexer.g4:COMMIT
PartiQLLexer.g4:CONNECT
PartiQLLexer.g4:CONNECTION
PartiQLLexer.g4:CONSTRAINTS
PartiQLLexer.g4:CONTINUE
PartiQLLexer.g4:CONVERT
PartiQLLexer.g4:CORRESPONDING
PartiQLLexer.g4:CURRENT
PartiQLLexer.g4:CURRENT_TIME
PartiQLLexer.g4:CURRENT_TIMESTAMP
PartiQLLexer.g4:CURSOR
PartiQLLexer.g4:DEALLOCATE
PartiQLLexer.g4:DECLARE
PartiQLLexer.g4:DEFAULT
PartiQLLexer.g4:DEFERRABLE
PartiQLLexer.g4:DEFERRED
PartiQLLexer.g4:DESCRIBE
PartiQLLexer.g4:DESCRIPTOR
PartiQLLexer.g4:DIAGNOSTICS
PartiQLLexer.g4:DISCONNECT
PartiQLLexer.g4:DOMAIN
PartiQLLexer.g4:END_EXEC
PartiQLLexer.g4:EXCEPTION
PartiQLLexer.g4:EXECUTE
PartiQLLexer.g4:EXTERNAL
PartiQLLexer.g4:FETCH
PartiQLLexer.g4:FOREIGN
PartiQLLexer.g4:FOUND
PartiQLLexer.g4:GET
PartiQLLexer.g4:GLOBAL
PartiQLLexer.g4:GO
PartiQLLexer.g4:GOTO
PartiQLLexer.g4:GRANT
PartiQLLexer.g4:IDENTITY
PartiQLLexer.g4:IMMEDIATE
PartiQLLexer.g4:INDICATOR
PartiQLLexer.g4:INITIALLY
PartiQLLexer.g4:INPUT
PartiQLLexer.g4:INSENSITIVE
PartiQLLexer.g4:INTERVAL
PartiQLLexer.g4:ISOLATION
PartiQLLexer.g4:KEY
PartiQLLexer.g4:LANGUAGE
PartiQLLexer.g4:LATERAL
PartiQLLexer.g4:LEVEL
PartiQLLexer.g4:LOCAL
PartiQLLexer.g4:MODULE
PartiQLLexer.g4:NAMES
PartiQLLexer.g4:NATIONAL
PartiQLLexer.g4:NATURAL
PartiQLLexer.g4:NCHAR
PartiQLLexer.g4:NEXT
PartiQLLexer.g4:NO
PartiQLLexer.g4:OF
PartiQLLexer.g4:ONLY
PartiQLLexer.g4:OPEN
PartiQLLexer.g4:OPTION
PartiQLLexer.g4:OUTPUT
PartiQLLexer.g4:OVERLAPS
PartiQLLexer.g4:PAD
PartiQLLexer.g4:PREPARE
PartiQLLexer.g4:PRESERVE
PartiQLLexer.g4:PRIMARY
PartiQLLexer.g4:PRIOR
PartiQLLexer.g4:PRIVILEGES
PartiQLLexer.g4:PROCEDURE
PartiQLLexer.g4:PUBLIC
PartiQLLexer.g4:READ
PartiQLLexer.g4:REFERENCES
PartiQLLexer.g4:RELATIVE
PartiQLLexer.g4:RESTRICT
PartiQLLexer.g4:REVOKE
PartiQLLexer.g4:ROLLBACK
PartiQLLexer.g4:ROWS
PartiQLLexer.g4:SCHEMA
PartiQLLexer.g4:SCROLL
PartiQLLexer.g4:SECTION
PartiQLLexer.g4:SESSION
PartiQLLexer.g4:SESSION_USER
PartiQLLexer.g4:SPACE
PartiQLLexer.g4:SQL
PartiQLLexer.g4:SQLCODE
PartiQLLexer.g4:SQLERROR
PartiQLLexer.g4:SQLSTATE
PartiQLLexer.g4:SYSTEM_USER
PartiQLLexer.g4:TEMPORARY
PartiQLLexer.g4:TO
PartiQLLexer.g4:TRANSACTION
PartiQLLexer.g4:TRANSLATE
PartiQLLexer.g4:TRANSLATION
PartiQLLexer.g4:UNIQUE
PartiQLLexer.g4:UNKNOWN
PartiQLLexer.g4:USAGE
PartiQLLexer.g4:USER
PartiQLLexer.g4:USING
PartiQLLexer.g4:VIEW
PartiQLLexer.g4:WHENEVER
PartiQLLexer.g4:WORK
PartiQLLexer.g4:WRITE
PartiQLLexer.g4:CARET
PartiQLLexer.g4:UNRECOGNIZED
PartiQLLexer.g4:LETTER
PartiQLLexer.g4:LETTER_NOT
05/27-12:40:26 ~/issues/g4-4632
$
  • The grammar contains a "symbol conflict". This is why it does not compile for Go. A symbol conflict is an identifier in a rule that causes a compilation error. The "parent" attribute is causing the problem. The fix to rename the attribute: parent= should be changed to parent_= (eight occurrences).

Correcting all these errors, we see the Go target work with this grammar.

PartiQLLexer.g4.txt

PartiQLParser.g4.txt

Input:
ex.txt

# A generated parser was built using trgen. Here the executable is called on input.
$ ./Test.exe ../examples/ex.txt
Go 0 ../examples/ex.txt success 0.025
Total Time: 0.027 s
05/27-13:33:33 ~/issues/g4-4632/Generated-Go

@h3n4l
Copy link
Contributor Author

h3n4l commented May 28, 2024

I am reworking the Go visitor code as there are some other issues. Though it generally works, there are some issues that need manual intervention, which is not ideal. It may take a week or so to get this ready for release. though.

Looks like you are using the Listener here though, which does work, so I woudl need to see your code to work out what is wrong here.

Please don't use images for text errors as they are hard to read.

Thank for your reply! I remove the image from issue description and put text instead.

@KvanTTT
Copy link
Member

KvanTTT commented May 28, 2024

options { caseInsentive=true; } has no meaning in a combined or parser grammar. Remove.

Incorrect. The option works in a combined grammar (for lexer rules). See testCaseInsensitiveInCombinedGrammar test for example.

Also, it could be actual in parser grammar if Token value comparators feature is implemented (not now).

@kaby76
Copy link
Contributor

kaby76 commented May 28, 2024

Incorrect. The option works in a combined grammar (for lexer rules).

Sorry, what I meant is for this grammar.

Even though PartiQL.g4 has a grammarType without "parser", it is essentially a parser grammar because options { tokensVocab=...; } is set, and there are no lexer rules. However, technically, it's still a combined grammar because one could add lexer rules to PartiQL.g4, run it through the Antlr4 Tool to not only generate a parser, but a second lexer class: PartiQLLexer.java. But then, it's unclear how to do the lexing because PartiQLTokens does not reference PartiQLLexer, PartiQLLexer does not reference PartQLTokens,new CommonTokenStream(lexer) can be passed only one lexer, and new PartQLParser(TokenStream input) takes only one token stream. It is also the case that the vocab and tables describing the rules in PartiQLTokens are not merged into PartiQLLexer, nor vice versa as PartiQLTokens was generated first. It does not work if the parse references both vocabs.

Rather than be exposed to this rabbit hole, a parser grammar should be declared as parser grammar PartiQLParser;.

@h3n4l
Copy link
Contributor Author

h3n4l commented May 30, 2024

The grammar contains a "symbol conflict". This is why it does not compile for Go. A symbol conflict is an identifier in a rule that causes a compilation error. The "parent" attribute is causing the problem.

First, I am curious about this: parent is neither a rule name in the parser nor a token name in the tokens. Is it reserved by ANTLR4?

Second, I found a related issue mentioned symbol conflict. However, why don't ANTLR tools raise any warning or error in this case?

BTW, I can get the correct go target after renamming parent= to parent_=. Thanks a lot!

@kaby76
Copy link
Contributor

kaby76 commented May 30, 2024

First, I am curious about this: parent is neither a rule name in the parser nor a token name in the tokens. Is it reserved by ANTLR4?

To see what is happening, let's rename all but the first of the attribute "parent=" to "parent_=" (in parser rule exprOr), then do a build.

$ make
bash build.sh
# example.com/myparser/parser
parser\partiql_parser.go:22846:9: cannot use s (variable of type *ExprOrBaseContext) as antlr.RuleContext value in return statement: *ExprOrBaseContext does not implement antlr.RuleContext (wrong type for method GetParent)
                have GetParent() IExprAndContext
                want GetParent() antlr.Tree
parser\partiql_parser.go:22893:13: cannot use NewExprOrBaseContext(p, localctx) (value of type *ExprOrBaseContext) as IExprOrContext value in assignment: *ExprOrBaseContext does not implement IExprOrContext (wrong type for method GetParent)
                have GetParent() IExprAndContext
                want GetParent() antlr.Tree
parser\partiql_parser.go:22902:3: impossible type assertion: localctx.(*ExprOrBaseContext)
        *ExprOrBaseContext does not implement IExprOrContext (wrong type for method GetParent)
                have GetParent() IExprAndContext
                want GetParent() antlr.Tree
mingw32-make: *** [makefile:3: build] Error 1
05/30-06:16:59 ~/issues/a4-4632/go/Generated-Go

For attribute parent, there has to be a function declared to access the attribute. Antlr Tool does this.

func (s *ExprOrBaseContext) GetParent() IExprAndContext { return s.parent }

func (s *ExprOrBaseContext) SetParent(v IExprAndContext) { s.parent = v }

For the renamed attribute parent_, the code is similar, but critically with a different name.

func (s *ExprAndBaseContext) GetParent_() IExprNotContext { return s.parent_ }

func (s *ExprAndBaseContext) SetParent_(v IExprNotContext) { s.parent_ = v }

However, the Antlr Go Runtime also defines functions GetParent() and SetParent().

// GetParent returns the combined text of all child nodes. This method only considers
// tokens which have been added to the parse tree.
//
// Since tokens on hidden channels (e.g. whitespace or comments) are not
// added to the parse trees, they will not appear in the output of this
// method.
func (prc *BaseParserRuleContext) GetParent() Tree {
return prc.parentCtx
}

func (prc *BaseParserRuleContext) SetParent(v Tree) {
if v == nil {
prc.parentCtx = nil
} else {
prc.parentCtx = v.(RuleContext)
}
}

Second, I found a related issue mentioned symbol conflict. However, why don't ANTLR tools raise any warning or error in this case?

The Antlr Tool cannot apriori warn there is a conflict because it doesn't do a build. It only "knows" when the symbol is in a list of symbol names that are known to cause build problems. That table is here:

protected static final HashSet<String> reservedWords = new HashSet<>(Arrays.asList(
// keywords
"break", "default", "func", "interface", "select",
"case", "defer", "go", "map", "struct",
"chan", "else", "goto", "package", "switch",
"const", "fallthrough", "if", "range", "type",
"continue", "for", "import", "return", "var",
// predeclared identifiers https://golang.org/ref/spec#Predeclared_identifiers
"bool", "byte", "complex64", "complex128", "error", "float32", "float64",
"int", "int8", "int16", "int32", "int64", "rune", "string",
"uint", "uint8", "uint16", "uint32", "uint64", "uintptr",
"true", "false", "iota", "nil",
"append", "cap", "close", "complex", "copy", "delete", "imag", "len",
"make", "new", "panic", "print", "println", "real", "recover",
"string",
// interface definition of RuleContext from runtime/Go/antlr/rule_context.go
"Accept", "GetAltNumber", "GetBaseRuleContext", "GetChild", "GetChildCount",
"GetChildren", "GetInvokingState", "GetParent", "GetPayload", "GetRuleContext",
"GetRuleIndex", "GetSourceInterval", "GetText", "IsEmpty", "SetAltNumber",
"SetInvokingState", "SetParent", "String",
// misc
"rule", "parserRule", "action",
// the use of start or stop abd others as a label name will cause the generation of a GetStart() or GetStop() method, which
// then clashes with the GetStart() or GetStop() method that is generated by the code gen for the rule. So, we need to
// convert it. This is not ideal as it will still probably confuse authors of parse listeners etc. but the code will
// compile. This is a proof of Hyrum's law.
"start", "stop", "exception"
));

That table does not contain "parent". It does contain "GetParent" however.

The Antlr tool does not warn because it just internally renames symbol conflicts before code generation. Other targets may still work with the original attribute name. The requirement for this solution was to not break existing parsers that use Antlr. We could have implemented a different solution to the problem by adding an option to generate code with a unique string appended to symbol names for other targets, which would have not broken old, working code for the original target. But that was not done here.

I have a script (https://github.com/kaby76/g4-scripts/blob/e5ca411ddbf8eb9fb97e986c99d124991e839899/find-known-conflicts.sh) that parses the Antlr4 grammar and checks all symbols against all known conflicts, but it's hard to keep it up to date. We can't predict when there is a conflict.

@h3n4l
Copy link
Contributor Author

h3n4l commented May 30, 2024

I get it. If we name it parent, the generated access code is GetParent(), which conflicts with the function in the embedded struct BaseParserRuleContext. Thank you for your attention to detail and patience.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

4 participants