Checking tarfile integrity in Python

Question

I'm working on converting my backup script from shell to Python. One of the features of my old script was to check the created tarfile for integrity by doing: gzip -t . This seems to be a bit tricky in Python. It seems that the only way to do this, is by reading each of the compressed TarInfo objects within the tarfile. Is there a way to check a tarfile for integrity, without extracting to disk, or keeping it in memory (in it's entirety)? Good people on #python on freenode suggested that I should read each TarInfo object chunk-by-chunk, discarding each chunk read. I must admit that I have no idea how to do this, seeing that I just started Python. Imagine that I have a tarfile of 30GB which contains files ranging from 1kb to 10GB... This is the solution that I started writing: ``` try: tardude = tarfile.open("zero.tar.gz") except: print "There was an error opening tarfile. The file might be corrupt or missing." for member_info in tardude.getmembers(): try: check = tardude.extractfile(m…

Accepted Answer

Just a minor improvement on Aya's answer to make things a little more idiomatic (although I'm removing some of the error checking to make the mechanics more visible):

```
BLOCK_SIZE = 1024

with tarfile.open("zero.tar.gz") as tardude:
    for member in tardude.getmembers():
        with tardude.extractfile(member.name) as target:
            for chunk in iter(lambda: target.read(BLOCK_SIZE), b''):
                pass
```

This really just removes the `while 1:` (sometimes considered a minor code smell) and the `if not data:` check. Also note that the use of `with` restricts this to Python 2.7+

Checking tarfile integrity in Python

Solution

Problem