Haskell/Parsec: How do you use the functions in Text.Parsec.Indent?

haskell, indentation, parsec, parsing

Solution

So, the first hint is to take a look at `IndentParser`

type IndentParser s u a = ParsecT s u (State SourcePos) a

I.e. it's a `ParsecT` keeping an extra close watch on `SourcePos`, an abstract container which can be used to access, among other things, the current column number. So, it's probably storing the current "level of indentation" in `SourcePos`. That'd be my initial guess as to what "level of reference" means.

In short, `indents` gives you a new kind of `Parsec` which is context sensitive—in particular, sensitive to the current indentation. I'll answer your questions out of order.

(2) The "level of reference" is the "belief" referred in the current parser context state of where this indentation level starts. To be more clear, let me give some test cases on (3).

(3) In order to start experimenting with these functions, we'll build a little test runner. It'll run the parser with a string that we give it and then unwrap the inner `State` part using an `initialPos` which we get to modify. In code

import Text.Parsec
import Text.Parsec.Pos
import Text.Parsec.Indent
import Control.Monad.State

testParse :: (SourcePos -> SourcePos) 
          -> IndentParser String () a 
          -> String -> Either ParseError a
testParse f p src = fst $ flip runState (f $ initialPos "") $ runParserT p () "" src

(Note that this is almost `runIndent`, except I gave a backdoor to modify the `initialPos`.)

Now we can take a look at `indented`. By examining the source, I can tell it does two things. First, it'll `fail` if the current `SourcePos` column number is less-than-or-equal-to the "level of reference" stored in the `SourcePos` stored in the `State`. Second, it somewhat mysteriously updates the `State` `SourcePos`'s line counter (not column counter) to be current.

Only the first behavior is important, to my understanding. We can see the difference here.

>>> testParse id indented ""
Left (line 1, column 1): not indented

>>> testParse id (spaces >> indented) "   "
Right ()

>>> testParse id (many (char 'x') >> indented) "xxxx"
Right ()

So, in order to have `indented` succeed, we need to have consumed enough whitespace (or anything else!) to push our column position out past the "reference" column position. Otherwise, it'll fail saying "not indented". Similar behavior exists for the next three functions: `same` fails unless the current position and reference position are on the same line, `sameOrIndented` fails if the current column is strictly less than the reference column, unless they are on the same line, and `checkIndent` fails unless the current and reference columns match.

`withPos` is slightly different. It's not just a `IndentParser`, it's an `IndentParser`-combinator—it transforms the input `IndentParser` into one that thinks the "reference column" (the `SourcePos` in the `State`) is exactly where it was when we called `withPos`.

This gives us another hint, btw. It lets us know we have the power to change the reference column.

(1) So now let's take a look at how `block` and `withBlock` work using our new, lower level reference column operators. `withBlock` is implemented in terms of `block`, so we'll start with `block`.

-- simplified from the actual source
block p = withPos $ many1 (checkIndent >> p)

So, `block` resets the "reference column" to be whatever the current column is and then consumes at least 1 parses from `p` so long as each one is indented identically as this newly set "reference column". Now we can take a look at `withBlock`

withBlock f a p = withPos $ do
  r1 <- a
  r2 <- option [] (indented >> block p)
  return (f r1 r2)

So, it resets the "reference column" to the current column, parses a single `a` parse, tries to parse an `indented` `block` of `p`s, then combines the results using `f`. Your implementation is almost correct, except that you need to use `withPos` to choose the correct "reference column".

Then, once you have `withBlock`, `withBlock' = withBlock (\_ bs -> bs)`.

(5) So, `indented` and friends are exactly the tools to doing this: they'll cause a parse to immediately fail if it's indented incorrectly with respect to the "reference position" chosen by `withPos`.

(4) Yes, don't worry about these guys until you learn how to use `Applicative` style parsing in base `Parsec`. It's often a much cleaner, faster, simpler way of specifying parses. Sometimes they're even more powerful, but if you understand `Monad`s then they're almost always completely equivalent.

(6) And this is the crux. The tools mentioned so far can only do indentation failure if you can describe your intended indentation using `withPos`. Quickly, I don't think it's possible to specify `withPos` based on the success or failure of other parses... so you'll have to go another level deeper. Fortunately, the mechanism that makes `IndentParser`s work is obvious—it's just an inner `State` monad containing `SourcePos`. You can use `lift :: MonadTrans t => m a -> t m a` to manipulate this inner state and set the "reference column" however you like.

Cheers!

Problem

Original source