Home >Database >Mysql Tutorial >FFmpeg封装格式处理：视音频复用器（muxer）

FFmpeg封装格式处理：视音频复用器（muxer）

WBOYOriginal: 2016-06-07 16:05:032536browse

打算记录一下基于FFmpeg的封装格式处理方面的例子。包括了视音频分离，复用，封装格式转换。这是第3篇。本文记录一个基于FFmpeg的视音频复用器（Simplest FFmpeg muxer）。视音频复用器（Muxer）即是将视频压缩数据（例如H.264）和音频压缩数据（例如AAC）合

打算记录一下基于FFmpeg的封装格式处理方面的例子。包括了视音频分离，复用，封装格式转换。这是第3篇。

本文记录一个基于FFmpeg的视音频复用器（Simplest FFmpeg muxer）。视音频复用器（Muxer）即是将视频压缩数据（例如H.264）和音频压缩数据（例如AAC）合并到一个封装格式数据（例如MKV）中去。如图所示。在这个过程中并不涉及到编码和解码。

本文记录的程序将一个H.264编码的视频码流文件和一个MP3编码的音频码流文件，合成为一个MP4封装格式的文件。
，一共初始化了3个AVFormatContext，其中2个用于输入，1个用于输出。3个AVFormatContext初始化之后，通过avcodec_copy_context()函数可以将输入视频/音频的参数拷贝至输出视频/音频的AVCodecContext结构体。然后分别调用视频输入流和音频输入流的av_read_frame()，从视频输入流中取出视频的AVPacket，音频输入流中取出音频的AVPacket，分别将取出的AVPacket写入到输出文件中即可。其间用到了一个不太常见的函数av_compare_ts()，是比较时间戳用的。通过该函数可以决定该写入视频还是音频。

本文介绍的视音频复用器，输入的视频不一定是H.264裸流文件，音频也不一定是纯音频文件。可以选择两个封装过的视音频文件作为输入。程序会从视频输入文件中“挑”出视频流，音频输入文件中“挑”出音频流，再将“挑选”出来的视音频流复用起来。 PS1：对于某些封装格式（例如MP4/FLV/MKV等）中的H.264，需要用到名称为“h264_mp4toannexb”的bitstream filter。
PS2：对于某些封装格式（例如MP4/FLV/MKV等）中的AAC，需要用到名称为“aac_adtstoasc”的bitstream filter。

简单介绍一下流程中各个重要函数的意义：

avformat_open_input()：打开输入文件。
avcodec_copy_context()：赋值AVCodecContext的参数。
avformat_alloc_output_context2()：初始化输出文件。
avio_open()：打开输出文件。
avformat_write_header()：写入文件头。
av_compare_ts()：比较时间戳，决定写入视频还是写入音频。这个函数相对要少见一些。
av_read_frame()：从输入文件读取一个AVPacket。
av_interleaved_write_frame()：写入一个AVPacket到输出文件。
av_write_trailer()：写入文件尾。

代码

下面贴上代码：
<strong>[cpp]</strong> view plaincopy

```
/**
```

* 最简单的基于FFmpeg的视音频复用器

```
* Simplest FFmpeg Muxer
```
```
*
```
```
* 雷霄骅 Lei Xiaohua
```
```
* leixiaohua1020@126.com
```
```
* 中国传媒大学/数字电视技术
```

* Communication University of China / Digital TV Technology

```
* http://blog.csdn.net/leixiaohua1020
```
```
*
```

* 本程序可以将视频码流和音频码流打包到一种封装格式中。

* 程序中将MP3编码的音频码流和H.264编码（MPEG2TS封装中）的视频码流打包成

```
* MP4封装格式的文件。
```

* 需要注意的是本程序并不改变视音频的编码格式。

```
*
```

* This software mux a video bitstream and a audio bitstream

```
* together into a file.
```

* In this example, it mux a H.264 bitstream (in MPEG2TS) and

* a MP3 bitstream file together into MP4 format file.

```
*
```
```
*/
```
```
#include <stdio.h></stdio.h>
```
```
extern "C"
```
```
{
```
```
#include "libavformat/avformat.h"
```
```
};
```
```
/*
```

FIX: H.264 in some container format (FLV, MP4, MKV etc.) need

"h264_mp4toannexb" bitstream filter (BSF)

```
*Add SPS,PPS in front of IDR frame
```

*Add start code ("0,0,0,1") in front of NALU

H.264 in some container (MPEG2TS) don't need this BSF.

```
*/
```
```
//'1': Use H.264 Bitstream Filter
```
```
#define USE_H264BSF 0
```
```
/*
```

FIX:AAC in some container format (FLV, MP4, MKV etc.) need

```
"aac_adtstoasc" bitstream filter (BSF)
```
```
*/
```
```
//'1': Use AAC Bitstream Filter
```
```
#define USE_AACBSF 0
```
```
int main(int argc, char* argv[])
```
```
{
```
```
AVOutputFormat *ofmt = NULL;
```

//输入对应一个AVFormatContext，输出对应一个AVFormatContext

//（Input AVFormatContext and Output AVFormatContext）

AVFormatContext *ifmt_ctx_v = NULL, *ifmt_ctx_a = NULL,*ofmt_ctx = NULL;

```
AVPacket pkt;
```
```
int ret, i;
```

char *in_filename_v = "cuc_ieschool.ts";//输入文件名（Input file URL）

//char *in_filename_v = "cuc_ieschool.h264";

//char *in_filename_a = "cuc_ieschool.mp3";

```
//char *in_filename_a = "gowest.m4a";
```
```
//char *in_filename_a = "gowest.aac";
```
```
char *in_filename_a = "huoyuanjia.mp3";
```

char *out_filename = "cuc_ieschool.mp4";//输出文件名（Output file URL）

```
av_register_all();
```
```
//输入（Input）
```

if ((ret = avformat_open_input(&ifmt_ctx_v, in_filename_v, 0, 0))

```
printf( "Could not open input file.");
```
```
goto end;
```
```
}
```

if ((ret = avformat_find_stream_info(ifmt_ctx_v, 0))

printf( "Failed to retrieve input stream information");

```
goto end;
```
```
}
```

if ((ret = avformat_open_input(&ifmt_ctx_a, in_filename_a, 0, 0))

```
printf( "Could not open input file.");
```
```
goto end;
```
```
}
```

if ((ret = avformat_find_stream_info(ifmt_ctx_a, 0))

printf( "Failed to retrieve input stream information");

```
goto end;
```
```
}
```

printf("Input Information=====================\n");

av_dump_format(ifmt_ctx_v, 0, in_filename_v, 0);

av_dump_format(ifmt_ctx_a, 0, in_filename_a, 0);

printf("======================================\n");

```
//输出（Output）
```

avformat_alloc_output_context2(&ofmt_ctx, NULL, NULL, out_filename);

```
if (!ofmt_ctx) {
```

printf( "Could not create output context\n");

```
ret = AVERROR_UNKNOWN;
```
```
goto end;
```
```
}
```
```
ofmt = ofmt_ctx->oformat;
```
```
int videoindex_v=-1,videoindex_out=-1;
```
```
for (i = 0; i nb_streams; i++) {
```

//根据输入流创建输出流（Create output AVStream according to input AVStream）

if(ifmt_ctx_v->streams[i]->codec->codec_type==AVMEDIA_TYPE_VIDEO){

```
videoindex_v=i;
```

AVStream *in_stream = ifmt_ctx_v->streams[i];

AVStream *out_stream = avformat_new_stream(ofmt_ctx, in_stream->codec->codec);

```
if (!out_stream) {
```

printf( "Failed allocating output stream\n");

```
ret = AVERROR_UNKNOWN;
```
```
goto end;
```
```
}
```
```
videoindex_out=out_stream->index;
```

//复制AVCodecContext的设置（Copy the settings of AVCodecContext）

if (avcodec_copy_context(out_stream->codec, in_stream->codec)

printf( "Failed to copy context from input to output stream codec context\n");

```
goto end;
```
```
}
```
```
out_stream->codec->codec_tag = 0;
```

if (ofmt_ctx->oformat->flags & AVFMT_GLOBALHEADER)

out_stream->codec->flags |= CODEC_FLAG_GLOBAL_HEADER;

```
break;
```
```
}
```
```
}
```
```
int audioindex_a=-1,audioindex_out=-1;
```
```
for (i = 0; i nb_streams; i++) {
```

//根据输入流创建输出流（Create output AVStream according to input AVStream）

if(ifmt_ctx_a->streams[i]->codec->codec_type==AVMEDIA_TYPE_AUDIO){

```
audioindex_a=i;
```

AVStream *in_stream = ifmt_ctx_a->streams[i];

AVStream *out_stream = avformat_new_stream(ofmt_ctx, in_stream->codec->codec);

```
if (!out_stream) {
```

printf( "Failed allocating output stream\n");

```
ret = AVERROR_UNKNOWN;
```
```
goto end;
```
```
}
```
```
audioindex_out=out_stream->index;
```

//复制AVCodecContext的设置（Copy the settings of AVCodecContext）

if (avcodec_copy_context(out_stream->codec, in_stream->codec)

printf( "Failed to copy context from input to output stream codec context\n");

```
goto end;
```
```
}
```
```
out_stream->codec->codec_tag = 0;
```

if (ofmt_ctx->oformat->flags & AVFMT_GLOBALHEADER)

out_stream->codec->flags |= CODEC_FLAG_GLOBAL_HEADER;

```
break;
```
```
}
```
```
}
```
```
//输出一下格式------------------
```

printf("Output Information====================\n");

av_dump_format(ofmt_ctx, 0, out_filename, 1);

printf("======================================\n");

//打开输出文件（Open output file）

```
if (!(ofmt->flags & AVFMT_NOFILE)) {
```

if (avio_open(&ofmt_ctx->pb, out_filename, AVIO_FLAG_WRITE)

printf( "Could not open output file '%s'", out_filename);

```
goto end;
```
```
}
```
```
}
```
```
//写文件头（Write file header）
```

if (avformat_write_header(ofmt_ctx, NULL)

printf( "Error occurred when opening output file\n");

```
goto end;
```
```
}
```
```
int frame_index=0;
```
```
int64_t cur_pts_v=0,cur_pts_a=0;
```
```
//FIX
```
```
#if USE_H264BSF
```

AVBitStreamFilterContext* h264bsfc = av_bitstream_filter_init("h264_mp4toannexb");

```
#endif
```
```
#if USE_AACBSF
```

AVBitStreamFilterContext* aacbsfc = av_bitstream_filter_init("aac_adtstoasc");

```
#endif
```
```
while (1) {
```
```
AVFormatContext *ifmt_ctx;
```
```
int stream_index=0;
```
```
AVStream *in_stream, *out_stream;
```

//获取一个AVPacket（Get an AVPacket）

if(av_compare_ts(cur_pts_v,ifmt_ctx_v->streams[videoindex_v]->time_base,cur_pts_a,ifmt_ctx_a->streams[audioindex_a]->time_base)

```
ifmt_ctx=ifmt_ctx_v;
```
```
stream_index=videoindex_out;
```
```
if(av_read_frame(ifmt_ctx, &pkt) >= 0){
```
```
do{
```
```
if(pkt.stream_index==videoindex_v){
```
```
cur_pts_v=pkt.pts;
```
```
break;
```
```
}
```

}while(av_read_frame(ifmt_ctx, &pkt) >= 0);

```
}else{
```
```
break;
```
```
}
```
```
}else{
```
```
ifmt_ctx=ifmt_ctx_a;
```
```
stream_index=audioindex_out;
```
```
if(av_read_frame(ifmt_ctx, &pkt) >= 0){
```
```
do{
```
```
if(pkt.stream_index==audioindex_a){
```
```
cur_pts_a=pkt.pts;
```
```
break;
```
```
}
```

}while(av_read_frame(ifmt_ctx, &pkt) >= 0);

```
}else{
```
```
break;
```
```
}
```
```
}
```

in_stream = ifmt_ctx->streams[pkt.stream_index];

out_stream = ofmt_ctx->streams[stream_index];

```
//FIX
```
```
#if USE_H264BSF
```

av_bitstream_filter_filter(h264bsfc, in_stream->codec, NULL, &pkt.data, &pkt.size, pkt.data, pkt.size, 0);

```
#endif
```
```
#if USE_AACBSF
```

av_bitstream_filter_filter(aacbsfc, in_stream->codec, NULL, &pkt.data, &pkt.size, pkt.data, pkt.size, 0);

```
#endif
```
```
//FIX：No PTS (Example: Raw H.264)
```
```
//Simple Write PTS
```
```
if(pkt.pts==AV_NOPTS_VALUE){
```
```
//Write PTS
```

AVRational time_base1=in_stream->time_base;

```
//Duration between 2 frames (us)
```

int64_t calc_duration=(double)AV_TIME_BASE/av_q2d(in_stream->r_frame_rate);

```
//Parameters
```

pkt.pts=(double)(frame_index*calc_duration)/(double)(av_q2d(time_base1)*AV_TIME_BASE);

```
pkt.dts=pkt.pts;
```

pkt.duration=(double)calc_duration/(double)(av_q2d(time_base1)*AV_TIME_BASE);

```
frame_index++;
```
```
}
```
```
/* copy packet */
```
```
//转换PTS/DTS（Convert PTS/DTS）
```

pkt.pts = av_rescale_q_rnd(pkt.pts, in_stream->time_base, out_stream->time_base, (AVRounding)(AV_ROUND_NEAR_INF|AV_ROUND_PASS_MINMAX));

pkt.dts = av_rescale_q_rnd(pkt.dts, in_stream->time_base, out_stream->time_base, (AVRounding)(AV_ROUND_NEAR_INF|AV_ROUND_PASS_MINMAX));

pkt.duration = av_rescale_q(pkt.duration, in_stream->time_base, out_stream->time_base);

```
pkt.pos = -1;
```
```
pkt.stream_index=stream_index;
```

printf("Write 1 Packet. size:%5d\tpts:%8d\n",pkt.size,pkt.pts);

```
//写入（Write）
```

if (av_interleaved_write_frame(ofmt_ctx, &pkt)

```
printf( "Error muxing packet\n");
```
```
break;
```
```
}
```
```
av_free_packet(&pkt);
```
```
}
```
```
//写文件尾（Write file trailer）
```
```
av_write_trailer(ofmt_ctx);
```
```
#if USE_H264BSF
```
```
av_bitstream_filter_close(h264bsfc);
```
```
#endif
```
```
#if USE_AACBSF
```
```
av_bitstream_filter_close(aacbsfc);
```
```
#endif
```
```
end:
```
```
avformat_close_input(&ifmt_ctx_v);
```
```
avformat_close_input(&ifmt_ctx_a);
```
```
/* close output */
```

if (ofmt_ctx && !(ofmt->flags & AVFMT_NOFILE))

```
avio_close(ofmt_ctx->pb);
```
```
avformat_free_context(ofmt_ctx);
```
```
if (ret 
```
```
printf( "Error occurred.\n");
```
```
return -1;
```
```
}
```
```
return 0;
```

Statement：

The content of this article is voluntarily contributed by netizens, and the copyright belongs to the original author. This site does not assume corresponding legal responsibility. If you find any content suspected of plagiarism or infringement, please contact admin@php.cn

Previous article：MsSqlServerNext article：交易事实表、周期快照事实表和累积快照事实表

See more

FFmpeg封装格式处理：视音频复用器（muxer）

代码

Related articles