Transparent (filter-like) gzip/gunzip in Go

go

Solution

You'll need to do two things to make it work

- Use an io.Pipe to connect the reader and writer together - you can't read and write from the same buffer

- Run the reading and writing in seperate goroutines. Because the first thing that gzip does is attempt to read the header you'll get a deadlock unless you have another go routine attemting to write it.

Here is what that looks like

Playground

func main() {
    s := []byte("Hello world!")
    fmt.Printf("%s\n", s)

    in, out := io.Pipe()

    gz := gzip.NewWriter(out)
    go func() {
        ungz, err := gzip.NewReader(in)
        fmt.Println("err: ", err)
        uncomp := make([]byte, 100)
        n, err2 := ungz.Read(uncomp)
        fmt.Println("err2: ", err2)
        fmt.Println("n: ", n)
        uncomp = uncomp[:n]
        fmt.Printf("%s\n", uncomp)
    }()
    gz.Write(s)
    gz.Flush()    
}

Problem

I'm trying, just for fun, to connect a gzip Writer directly to a gzip Reader, so I could write to the Writer and read from the Reader on the fly. I expected to read exactly what I wrote. I'm using gzip but I'd like to use this method also with crypto/aes, I suppose it should work very similar and it could be used with other reader/writers like jpeg, png... This is my best option, that is not working, but I hope you can see what I mean: http://play.golang.org/p/7qdUi9wwG7 ``` package main import ( "bytes" "compress/gzip" "fmt" ) func main() { s := []byte("Hello world!") fmt.Printf("%s\n", s) var b bytes.Buffer gz := gzip.NewWriter(&b) ungz, err := gzip.NewReader(&b) fmt.Println("err: ", err) gz.Write(s) gz.Flush() uncomp := make([]byte, 100) n, err2 := ungz.Read(uncomp) fmt.Println("err2: ", err2) fmt.Println("n: ", n) uncomp = uncomp[:n] fmt.Printf("%s\n", uncomp) } ``` It seems that `gzip.NewReader(&b)` is trying to read immediately and a EOF is returned.

Original source