Pipes.Binary.decode - what is the StateT for?

haskell, haskell-pipes

Solution

The `StateT (Producer a m r) m x` idiom comes from `pipes-parse`'s "Low-level Parsers". It typically means that the library is using `draw` and `unDraw` to pull values off a `Producer` and return them if they're unused. It's an essential component of parsing where failure might occur. It also requires the `StateT` layer to indicate that a pipe is being selectively drained and refilled in a stateful manner.

-- | Draw one element from the underlying Producer, 
-- returning Left if the Producer is empty
draw :: Monad m => StateT (Producer a m r) m (Either r a)

-- | Push back an element onto the underlying Producer
unDraw :: Monad m => a -> StateT (Producer a m r) m ()

So what does that mean for `decode` and `decodeMany`? If we look at some simplified types of those functions

-- for (Monad m, Binary b)

decode     :: StateT (Producer ByteString m r) m (Maybe b)
decodeMany :: Producer ByteString m r 
           -> Producer' b m (Either (Producer ByteString m r) r)

We first see that `decode` is `drawing` off enough `ByteString` chunks from a `Producer ByteString` statefully so as to try to parse a `b`. Since the chunk boundary on the `ByteString`s may not align with a parse boundary it's important to do this in `StateT` so that the leftover chunks can be `unDraw`-ed back into the `Producer`.

`decodeMany` builds atop `decode` and attempts to repeatedly `decode` `b`s off the input Producer returning a "continuation" `Producer` of leftover `ByteString`s on failure.

Long story short, due to a need to `unDraw` leftover `ByteString` chunks, we can just compose these things together into a chain with `(>->)`. If you want to do that, you can use something like `decodeMany` to transform a producer and then chain the result, but you'll want to handle error cases carefully.

Problem

I'm trying to write a basic network server using pipes and the assorted libraries that build on it. The intended flow would be: get bytestring from socket -> decode using binary -> server logic goes here -> send response to socket Which I figured would be something like: ``` fromSocket s 4096 >-> decode >-> serverLogic >-> toSocket s ``` pipes-binary has a `decode` and a `decodeMany`, but I am not sure I understand the difference, and I don't know how to use `decode`. Why does `decodeMany` take the upstream pipe as an argument instead of being chained off of it with `>->`? And how do you use `decode`, what is the `StateT` for and what should my pipe chain end up looking like?

Original source