将CVImageBuffer转换为YUV420对象

Question

我想保持YUV420格式的流视频从相机，以避免在转换到灰度的惩罚，但我也想保留彩色分量。最终的目标是用像OpenCV这样的计算机视觉库进行处理。虽然我最终可能会选择BGRA，但我仍然希望能够用YUV测试一个工作解决方案。那么如何将像素格式为KCVPixelFormattype_420YPCBCR8BiplanarFullRange的CVImageBuffer转换为单个内存块呢？

拒绝的解决方案：

null

Answer 1

这将把数据填充到包含字节的NSObject中，具体取决于指定的像素格式。我继续，并提供了能力，以检测和malloc内存的BGRA或YUV像素格式。所以这个解决方案非常适合测试这两者。

- (void)captureOutput:(AVCaptureOutput *)captureOutput didOutputSampleBuffer:(CMSampleBufferRef)sampleBuffer fromConnection:(AVCaptureConnection *)connection
{
    CVImageBufferRef videoImageBuffer = CMSampleBufferGetImageBuffer(sampleBuffer);

    CVPixelBufferLockBaseAddress(videoImageBuffer, 0);

    void *baseAddress = NULL;
    NSUInteger totalBytes = 0;
    size_t width = CVPixelBufferGetWidth(videoImageBuffer);
    size_t height = 0;
    size_t bytesPerRow = CVPixelBufferGetBytesPerRow(videoImageBuffer);
    OSType pixelFormat = CVPixelBufferGetPixelFormatType(videoImageBuffer);
    if (pixelFormat == kCVPixelFormatType_420YpCbCr8BiPlanarVideoRange ||
        pixelFormat == kCVPixelFormatType_420YpCbCr8BiPlanarFullRange) {
        size_t planeCount = CVPixelBufferGetPlaneCount(videoImageBuffer);
        baseAddress = CVPixelBufferGetBaseAddressOfPlane(videoImageBuffer, 0);

        for (int plane = 0; plane < planeCount; plane++) {
            size_t planeHeight = CVPixelBufferGetHeightOfPlane(videoImageBuffer, plane);
            size_t bytesPerRow = CVPixelBufferGetBytesPerRowOfPlane(videoImageBuffer, plane);
            height += planeHeight;
            totalBytes += (int)planeHeight * (int)bytesPerRow;
        }
    } else if (pixelFormat == kCVPixelFormatType_32BGRA) {
        baseAddress = CVPixelBufferGetBaseAddress(videoImageBuffer);
        height = CVPixelBufferGetHeight(videoImageBuffer);
        size_t bytesPerRow = CVPixelBufferGetBytesPerRow(videoImageBuffer);
        totalBytes += (int)height * (int)bytesPerRow;
    }

    // Doesn't have to be an NSData object
    NSData *rawPixelData = [NSData dataWithBytes:baseAddress length:totalBytes];

    // Just a plain-ol-NSObject with the following properties
    NTNUVideoFrame *videoFrame = [[NTNUVideoFrame alloc] init];
    videoFrame.width = width;
    videoFrame.height = height;
    videoFrame.bytesPerRow = bytesPerRow;
    videoFrame.pixelFormat = pixelFormat; 
    // Alternatively if you switch rawPixelData to void *
    // videoFrame.rawPixelData = baseAddress;
    videoFrame.rawPixelData = rawPixelData;
    [self.delegate didUpdateVideoFrame:videoFrame];

    CVPixelBufferUnlockBaseAddress(videoImageBuffer, 0);
}

您需要记住的唯一一件事是，如果您计划切换线程，或者dispatch_async并且不使用nsdata，则需要malloc和memcpy基址。一旦解锁基址，像素数据将不再有效。

void *rawPixelData = malloc(totalBytes);
memcpy(rawPixelData, baseAddress, totalBytes);

此时您需要考虑在完成时调用该内存块的free的问题。