MediaMuxer将H264流写入mpg文件

Question

MediaMuxer已经让我抓狂两天了

情况：我通过UDP收到H264编码的1280x720视频流。h264流包含NALU 1切片和NALU 5关键帧（5之前总是NALU 7-SPS和NALU 8-PPS）。此流似乎稳定30fps，每秒至少有一个NALU 5关键帧。比特率可变，但小于4Mbps。MediaCodec成功解码流并将其呈现在表面视图中，以便该部分运行良好。

但现在我需要将H.264保存到本地mpg文件中。我用我拥有的所有MediaFormat信息设置了一个MediaMuxer，并将流中的样本数据提供给它。每个样本包含一帧（NALU 1或5），发送到MediaMuxer的第一个数据是一个关键帧（NALU 5）。演示时间根据帧数和帧率计算。所有涉及的方法都是从同一个线程调用的。

但是mpg文件永远不会创建。正如您在ByteBuffer中的数据下面的输出中看到的，数据确实以NALU标头开头，然后是不同大小的数据。MediaMuxer在计算帧时似乎“看到”了数据中的帧。那么这里有什么问题？

最低API是21，我已经用运行Android 5的三星Galaxy S4和运行Riegeos Oreo和牛轧糖的两台设备进行了测试。

以下是设置MediaMuxer的代码：

void setupMuxer(File f) throws IOException {
    if (DEBUG) Log.d(TAG, "Setup Muxer: " + f.getAbsolutePath() +" can write: " + f.canWrite());
    MediaFormat format = MediaFormat.createVideoFormat(MediaFormat.MIMETYPE_VIDEO_AVC, decoderWidth, decoderHeight);
    format.setInteger(MediaFormat.KEY_BIT_RATE, 4000000);
    format.setInteger(MediaFormat.KEY_FRAME_RATE, 29);
    format.setInteger(MediaFormat.KEY_I_FRAME_INTERVAL, 1);
    format.setByteBuffer("csd-0", ByteBuffer.wrap(sps)); // sps and pps have been retrieved from the stream's NAL 7/8
    format.setByteBuffer("csd-1", ByteBuffer.wrap(pps));
    format.setInteger(MediaFormat.KEY_MAX_INPUT_SIZE, 1920 * 1080);

    muxer = new MediaMuxer(f.getPath(), MediaMuxer.OutputFormat.MUXER_OUTPUT_MPEG_4);
    videoTrack = muxer.addTrack(format);
    muxer.start();
}

为每个（完整的）NALU 1和NALU 5调用此方法：

 void muxFrame(ByteBuffer buf, int frame) {
    MediaCodec.BufferInfo bufferInfo = new MediaCodec.BufferInfo();
    bufferInfo.offset = buf.arrayOffset();
    bufferInfo.size = buf.position() - bufferInfo.offset;
    bufferInfo.flags = (buf.get(4) & 0x1f) == 5 ? MediaCodec.BUFFER_FLAG_KEY_FRAME : 0;
    bufferInfo.presentationTimeUs = computePresentationTime(frame);


    if (DEBUG)
        Log.d(TAG, "muxFrame frame: " + frame + " size: " + bufferInfo.size + " NAL: " + (buf.get(4) & 0x1f) + " Flags: " + bufferInfo.flags + " PTS: " + bufferInfo.presentationTimeUs + " content: " + BitByteUtil.toByteString(buf.array(), buf.arrayOffset(), 8));

    try {
        muxer.writeSampleData(videoTrack, buf, bufferInfo);
    } catch (Exception e) {
        Log.w(TAG, "muxer failed", e);
    } finally {
    }
}

private static long computePresentationTime(int frameIndex) {
    return 42 + frameIndex * 1000000 / FRAME_RATE;
}

这是我的输出，如果MediaMuxer在消耗了100纳勒斯后停止。

05.651 D/VideoDecoderView: Setup Muxer: /storage/emulated/0/Pictures/test.mpg can write: true
05.656 I/MPEG4Writer: limits: 4294967295/0 bytes/us, bit rate: -1 bps and the estimated moov size 3317 bytes
06.263 D/VideoDecoderView: muxFrame frame: 2 size: 7257 NAL: 5 Flags: 1 PTS: 66708 content: 0:000 1:000 2:000 3:001 4:101 5:184 6:000 7:015 
06.264 I/MPEG4Writer: setStartTimestampUs: 66708
06.264 I/MPEG4Writer: Earliest track starting time: 66708
06.308 D/VideoDecoderView: muxFrame frame: 3 size: 8998 NAL: 1 Flags: 0 PTS: 100042 content: 0:000 1:000 2:000 3:001 4:065 5:224 6:034 7:020 
06.342 D/VideoDecoderView: muxFrame frame: 4 size: 13664 NAL: 1 Flags: 0 PTS: 133375 content: 0:000 1:000 2:000 3:001 4:065 5:224 6:066 7:020 
06.375 D/VideoDecoderView: muxFrame frame: 5 size: 13674 NAL: 1 Flags: 0 PTS: 166708 content: 0:000 1:000 2:000 3:001 4:065 5:224 6:098 7:020 
06.409 D/VideoDecoderView: muxFrame frame: 6 size: 13772 NAL: 1 Flags: 0 PTS: 200042 content: 0:000 1:000 2:000 3:001 4:065 5:224 6:130 7:020 
06.483 D/VideoDecoderView: muxFrame frame: 7 size: 13707 NAL: 1 Flags: 0 PTS: 233375 content: 0:000 1:000 2:000 3:001 4:065 5:224 6:162 7:020 
06.520 D/VideoDecoderView: muxFrame frame: 8 size: 13778 NAL: 1 Flags: 0 PTS: 266708 content: 0:000 1:000 2:000 3:001 4:065 5:224 6:194 7:020 
06.555 D/VideoDecoderView: muxFrame frame: 9 size: 13743 NAL: 1 Flags: 0 PTS: 300042 content: 0:000 1:000 2:000 3:001 4:065 5:224 6:226 7:020 
06.575 D/VideoDecoderView: muxFrame frame: 10 size: 7338 NAL: 5 Flags: 1 PTS: 333375 content: 0:000 1:000 2:000 3:001 4:101 5:184 6:000 7:015 
06.593 D/VideoDecoderView: muxFrame frame: 11 size: 9059 NAL: 1 Flags: 0 PTS: 366708 content: 0:000 1:000 2:000 3:001 4:065 5:224 6:034 7:020 
06.618 D/VideoDecoderView: muxFrame frame: 12 size: 13587 NAL: 1 Flags: 0 PTS: 400042 content: 0:000 1:000 2:000 3:001 4:065 5:224 6:066 7:020 
06.644 D/VideoDecoderView: muxFrame frame: 13 size: 13650 NAL: 1 Flags: 0 PTS: 433375 content: 0:000 1:000 2:000 3:001 4:065 5:224 6:098 7:020 
06.671 D/VideoDecoderView: muxFrame frame: 14 size: 13797 NAL: 1 Flags: 0 PTS: 466708 content: 0:000 1:000 2:000 3:001 4:065 5:224 6:130 7:020 
.... [snip]
09.620 D/VideoDecoderView: muxFrame frame: 97 size: 7212 NAL: 5 Flags: 1 PTS: 3233375 content: 0:000 1:000 2:000 3:001 4:101 5:184 6:000 7:015 
09.661 D/VideoDecoderView: muxFrame frame: 98 size: 8814 NAL: 1 Flags: 0 PTS: 3266708 content: 0:000 1:000 2:000 3:001 4:065 5:224 6:034 7:020 
09.692 D/VideoDecoderView: muxFrame frame: 99 size: 13566 NAL: 1 Flags: 0 PTS: 3300042 content: 0:000 1:000 2:000 3:001 4:065 5:224 6:066 7:020 
09.737 D/VideoDecoderView: muxFrame frame: 100 size: 13733 NAL: 1 Flags: 0 PTS: 3333375 content: 0:000 1:000 2:000 3:001 4:065 5:224 6:098 7:020 
09.771 D/VideoDecoderView: muxFrame frame: 101 size: 13771 NAL: 1 Flags: 0 PTS: 3366708 content: 0:000 1:000 2:000 3:001 4:065 5:224 6:130 7:020 
09.775 D/MPEG4Writer: Video track stopping. Stop source
09.775 I/MPEG4Writer: Received total/0-length     (100/1) buffers and encoded 100 frames. - Video
09.775 D/MPEG4Writer: Video track source stopping
09.775 D/MPEG4Writer: Video track source stopped
09.775 D/MPEG4Writer: Video track stopped. Stop source
09.775 D/MPEG4Writer: Stopping writer thread
09.776 D/MPEG4Writer: 0 chunks are written in the last batch
09.779 D/MPEG4Writer: Writer thread stopped
09.780 I/MPEG4Writer: Ajust the moov start time from 66708 us -> 66708 us
09.780 D/MPEG4Writer: Video track stopping. Stop source

Answer 1

@greeble31：你是对的。第一个日志条目清楚地说明了“图片”而不是“视频”。我花了几个小时看这个问题，却没有注意到一个简单的剪切

给自己的提示：编码两天