Codec.Crypto.RSA: (decrypt . encrypt) /= id when PKCS#1 v1.5 padding is used?

encryption, haskell, padding, rsa

Solution

There's nothing wrong with your code, it's a bug in a used library.

The problem is that

generate_random_bytestring :: CryptoRandomGen g => g -> Int64 -> (ByteString, g)
generate_random_bytestring g 0 = (BS.empty, g)
generate_random_bytestring g x = (BS.cons' first rest, g'')
 where
  (rest, g')   = generate_random_bytestring g (x - 1)
  (first, g'') = throwLeft $ crandomR (1,255) g'

which is supposed to generate a random `ByteString` of given length with no 0-byte, doesn't.

Hacking the source of `Codec.Crypto.RSA` to test for it, I consistently get a 0-byte error very soon.

That means the decoded message is thought to be longer than it actually is and you get some rubbish in front of it.

The concrete bug is that `crandomR` sometimes produces a 0-byte due to a bug in `crandomR_Num`:

        Right (bs, g') ->
                let res = fromIntegral $ fromIntegral low + (bs2i bs .&. mask)
                in if res > high then go g' else Right (res, g')

Here, `mask` is `0xFF` (255), `low` is 1. If the generated unrestricted byte is 255, then

res = fromIntegral 256

which is 0 and thus not `> high`.

The bug should be has been fixed in the next release (0.4.1) of monadcryptorandom which is due soon already on hackage.

The OAEP methods are not affected, as far as I can see, because they use a different padding scheme to fill the chunks to the required length.

Problem

I'm using Codec.Crypto.RSA to encrypt a random string that is passed to an external processes over a socket in base64 representation. The external process (a ruby program using openssl for decryption) sometimes fails to decrypt the message. To debug this I set up a simple script in haskell that encrypts and decrypts a fixed message, all without base64 encoding/decoding. What baffles me is that this very simple program results in a failure after a few iterations. The decrypted ciphertext does not equal the original message, though the message is contained in the decryption (after some unprintable characters). Here's the code: ``` import Crypto.Random import qualified Codec.Crypto.RSA as RSA import qualified Data.ByteString.Lazy.Char8 as L m :: L.ByteString m = L.pack "11111222223333344444555556666600" main = do gen <- newGenIO :: IO SystemRandom let (pub, priv, _) = RSA.generateKeyPair gen 1024 doStuff pub priv doStuff pub priv = do gen <- newGenIO :: IO SystemRandom let (e,_) = RSA.encrypt' RSA.UsePKCS1_v1_5 gen pub m let d = RSA.decrypt' RSA.UsePKCS1_v1_5 priv e if (m == d) then do putStrLn "SUCCESS" doStuff pub priv else do putStrLn "FAILED" putStrLn $ "expected: " ++ show m putStrLn $ "got: " ++ show d ``` As the test suite for Codec.Crypto.RSA passes, there certainly must be a problem with my program. After replacing `RSA.encrypt' RSA.UsePKCS1_v1_5` with `RSA.encrypt` (defaulting to OAEP1) and `RSA.decrypt' RSA.UsePKCS1_v1_5` with `RSA.decrypt`, the failure is no longer triggered. Does anyone see what's wrong here? [1] I plan to use OAEP later, but the generated ciphertext cannot be decrypted with `echo ciphertext | openssl rsautl -oaep -inkey keypair.pem -decrypt` for some reason, but that's another problem. Update: To make OAEP work with OpenSSL one has to use SHA-1 as the hash function: ``` cryptOptions :: RSA.EncryptionOptions cryptOptions = RSA.UseOAEP sha1' (RSA.generate_MGF1 sha1') BS.empty where sha1' = bytestringDigest . sha1 -- then, to encrypt enc = RSA.encrypt' cryptOptions gen pubkey ```

Original source