Debugging Pyparsing Grammar

Question

I'm building a parser for an imaginary programming language called C-- (not the actual C-- language). I've gotten to the stage where I need to translate the language's grammar into something Pyparsing can accept. Unfortunatly when I come to parse my input string (which is correct and should not cause Pyparsing to error) it's not parsing correctly. I fear this is due to errors in my grammar, but as I'm starting Pyparsing for the first time, I can't seem to see where I'm going wrong. I've uploaded the grammar that I'm translating from here for people to have a read through. EDIT: Updated with the advice from Paul. This is the grammer I've currently got (the two top lines of Syntax definition are terribly bad of me I know): ``` # Lexical structure definition ifS = Keyword('if') elseS = Keyword('else') whileS = Keyword('while') returnS = Keyword('return') intVar = Keyword('int') voidKeyword = Keyword('void') sumdiff = Literal('+') | Literal('-') prodquot = Literal('*') | Literal('/') rela…

Accepted Answer

1) Change `Literal("if")` to `Keyword("if")` (and so on, down to `Literal("void")`), to prevent matching the leading "if" of a variable named `"ifactor"`.

2) `nums`, `alphas`, and `alphanums` are not expressions, they are strings, that can be used with the Word class to define some typical sets of characters when defining "words" like "a number is a word made up of nums", or "an identifier is a word that starts with an alpha, followed by zero or more alphanums." So instead of:

```
number = nums
identifier = alphas + OneOrMore(alphanums)
```

you want

```
number = Word(nums)
identifier = Word(alphas, alphanums)
```

3) Instead of `Combine`, I think you want `Group`. Use `Combine` when you want the matched tokens to be contiguous with no intervening whitespace, and will concatenate the tokens and return them as a single string. `Combine` is often used in cases like this:

```
realnum = Combine(Word(nums) + "." + Word(nums))
```

Without `Combine`, parsing `"3.14"` would return the list of strings `['3', '.', '14']`, so we add `Combine` so that the parsed result for realnum is `'3.14'` (which you could then pass to a parse action to convert to the actual floating value `3.14`). `Combine`s enforcement of no intervening whitespace also keeps us from accidentally parsing `'The answer is 3. 10 is too much.'` and thinking the `"3. 10"` represents a real number.

4) This should not cause your error, but your input string has lots of extra spaces. If you get your grammar working, you should be able to parse `"int x;"` just as well as `"int x ;"`.

Hope some of these hints get you going. Have you read any online pyparsing articles or tutorials? And please look through the online examples. You'll need to get a good grasp of how `Word`, `Literal`, `Combine`, etc. perform their individual parsing tasks.

5) You have mis-implemented the recursive definitions for term and statement. Instead of assigning `''` to them, write:

```
term = Forward()
statement = Forward()
```

Then when you go to actually define them with their recursive definitions, use the `<<` operator (and be sure to enclose the RHS in `()`'s).

```
term << (... term definition ...)
statement << (... statement definition ...)
```

You can find an example of a recursive parser here, and a presentation on basic pyparsing usage here - see the section titled "Parsing Lists" for more step-by-step on how the recursion is handled.

Debugging Pyparsing Grammar

Solution

Problem