Rotating video without rotating AVCaptureConnection and in the middle of AVAssetWriter session

avassetwriter, avfoundation, ios, objective-c, video

Solution

Method 1 isn't the preferred method according to Apple's documentation ("Physically rotating buffers does come with a performance cost, so only request rotation if it's necessary"). Method 2 worked for me but if I played my video on an app that doesn't support the transformation "metadata", the video isn't rotated properly. Method 3 is what I did.

I think it's crashing for you before you're trying to pass the image data directly from `vImageRotate...` to the `AVAssetWriterInputPixelBufferAdaptor`. You have to create a `CVPixelBufferRef` first. Here's my code:

Inside of `captureOutput:didOutputSampleBuffer:fromConnection:` I rotate the frame before writing it into the adaptor:

if ([self.videoWriterInput isReadyForMoreMediaData])
{
    // Rotate buffer first and then write to adaptor
    CMTime sampleTime = CMSampleBufferGetPresentationTimeStamp(sampleBuffer);
    CVPixelBufferRef rotatedBuffer = [self correctBufferOrientation:sampleBuffer];
    [self.videoWriterInputAdaptor appendPixelBuffer:rotatedBuffer withPresentationTime:sampleTime];
    CVBufferRelease(rotatedBuffer);
}

The referenced function that performs the vImage rotation is:

/* rotationConstant:
 *  0 -- rotate 0 degrees (simply copy the data from src to dest)
 *  1 -- rotate 90 degrees counterclockwise
 *  2 -- rotate 180 degress
 *  3 -- rotate 270 degrees counterclockwise
 */

- (CVPixelBufferRef)rotateBuffer:(CMSampleBufferRef)sampleBuffer withConstant:(uint8_t)rotationConstant
{
    CVImageBufferRef imageBuffer        = CMSampleBufferGetImageBuffer(sampleBuffer);
    CVPixelBufferLockBaseAddress(imageBuffer, 0);        

    OSType pixelFormatType              = CVPixelBufferGetPixelFormatType(imageBuffer);
    NSAssert(pixelFormatType == kCVPixelFormatType_32ARGB, @"Code works only with 32ARGB format. Test/adapt for other formats!");

    const size_t kAlignment_32ARGB      = 32;
    const size_t kBytesPerPixel_32ARGB  = 4;

    size_t bytesPerRow                  = CVPixelBufferGetBytesPerRow(imageBuffer);
    size_t width                        = CVPixelBufferGetWidth(imageBuffer);
    size_t height                       = CVPixelBufferGetHeight(imageBuffer);

    BOOL rotatePerpendicular            = (rotateDirection == 1) || (rotateDirection == 3); // Use enumeration values here
    const size_t outWidth               = rotatePerpendicular ? height : width;
    const size_t outHeight              = rotatePerpendicular ? width  : height;

    size_t bytesPerRowOut               = kBytesPerPixel_32ARGB * ceil(outWidth * 1.0 / kAlignment_32ARGB) * kAlignment_32ARGB;

    const size_t dstSize                = bytesPerRowOut * outHeight * sizeof(unsigned char);

    void *srcBuff                       = CVPixelBufferGetBaseAddress(imageBuffer);

    unsigned char *dstBuff              = (unsigned char *)malloc(dstSize);

    vImage_Buffer inbuff                = {srcBuff, height, width, bytesPerRow};
    vImage_Buffer outbuff               = {dstBuff, outHeight, outWidth, bytesPerRowOut};

    uint8_t bgColor[4]                  = {0, 0, 0, 0};

    vImage_Error err                    = vImageRotate90_ARGB8888(&inbuff, &outbuff, rotationConstant, bgColor, 0);
    if (err != kvImageNoError) 
    {
        NSLog(@"%ld", err);
    }

    CVPixelBufferUnlockBaseAddress(imageBuffer, 0);

    CVPixelBufferRef rotatedBuffer      = NULL;
    CVPixelBufferCreateWithBytes(NULL,
                                 outWidth,
                                 outHeight,
                                 pixelFormatType,
                                 outbuff.data,
                                 bytesPerRowOut,
                                 freePixelBufferDataAfterRelease,
                                 NULL,
                                 NULL,
                                 &rotatedBuffer);

    return rotatedBuffer;
}

void freePixelBufferDataAfterRelease(void *releaseRefCon, const void *baseAddress)
{
    // Free the memory we malloced for the vImage rotation
    free((void *)baseAddress);
}

Note: You may like to use enumeration for `rotationConstant`. Something like that (don't call this function with `MOVRotateDirectionUnknown`):

typedef NS_ENUM(uint8_t, MOVRotateDirection)
{
    MOVRotateDirectionNone = 0,
    MOVRotateDirectionCounterclockwise90,
    MOVRotateDirectionCounterclockwise180,
    MOVRotateDirectionCounterclockwise270,
    MOVRotateDirectionUnknown
};

Note: If you need `IOSurface` support, you should use `CVPixelBufferCreate` instead of `CVPixelBufferCreateWithBytes` and pass bytes data into it directly:

NSDictionary *pixelBufferAttributes = @{ (NSString *)kCVPixelBufferIOSurfacePropertiesKey : @{} };
CVPixelBufferCreate(kCFAllocatorDefault,
                    outWidth,
                    outHeight,
                    pixelFormatType,
                    (__bridge CFDictionaryRef)(pixelBufferAttributes),
                    &rotatedBuffer);

CVPixelBufferLockBaseAddress(rotatedBuffer, 0);
uint8_t *dest = CVPixelBufferGetBaseAddress(rotatedBuffer);
memcpy(dest, outbuff.data, bytesPerRowOut * outHeight);

CVPixelBufferUnlockBaseAddress(rotatedBuffer, 0);

Problem

Original source

Related problems

• How to directly rotate CVImageBuffer image in IOS 4 without converting to UIImage?