After conducting research, it has been found that the FFMPEG SDK provides two main points for controlling timestamps when writing video data: the AvPacket and the AvFrame. When using the function avcodec_encode_video, you pass a pointer to an AvFrame object, which represents an uncompressed video frame that will be encoded. The AvFrame contains a parameter called pts, which stands for Presentation Timestamp. This timestamp indicates the time at which the frame should be displayed during playback. Similarly, AvPackets also contain both pts and dts (Decoding Timestamp). To fully understand these parameters, it's important to first explain the three types of video compression frames: I, P, and B.
An I-frame is a key frame that does not depend on any other frame for decoding. A P-frame, or predictive frame, depends only on previous frames for decoding. A B-frame, or bidirectional frame, relies on both previous and future frames to reconstruct the image. Because B-frames require information from both before and after, their decoding order must be carefully managed. This is where the concepts of pts and dts come into play.
Pts is used to determine the display order of video frames, while dts determines the decoding order. In the absence of B-frames, the values of pts and dts are typically the same. However, when B-frames are present, they differ, as the decoding process may need to access frames that have not yet been encountered in the stream. For a deeper understanding of how this works, one can refer to the principles of MPEG encoding standards.
Now, let's take a closer look at the values of pts and dts stored in the AvPacket. These timestamps are essential for ensuring the correct sequence of decoding and playback. They are not actual timestamps of the video being played but rather represent the order in which frames should be processed. Each frame increments the value by 1, maintaining a consistent sequence.
However, there are cases where the video frame rate is not fixed—such as with RMVB videos—which can lead to synchronization issues between audio and video if the standard approach of incrementing pts and dts is used. This can result in audio and video becoming out of sync.
To solve audio-video synchronization issues:
The lTimeStamp represents the timestamp of the current video frame obtained via DirectShow. m_llframe_index keeps track of the number of frames that have been compressed so far. The first step involves using av_rescale to calculate the expected timestamp for the current frame being processed. If the calculated timestamp hasn't reached the actual timestamp provided by DirectShow, the frame is discarded. Otherwise, the compression process continues.
It is assumed here that no B-frames are involved. Since the video will be played back at a fixed frame rate, we compare the calculated timestamp based on that rate with the timestamp from DirectShow. If a delay is needed, the pts value is increased by two steps. Otherwise, it is set to 1, and dts is assigned accordingly.
__int64 x = av_rescale(m_llframe_index, AV_TIME_BASE*(int64_t)c->time_base.num, c->time_base.den);
If (x < lTimeStamp) { return TRUE; }
m_pVideoFrame2->pts = lTimeStamp;
m_pVideoFrame2->pict_type = 0;
int out_size = avcodec_encode_video(c, m_pvideo_outbuf, video_outbuf_size, m_pVideoFrame2);
/* if zero size, it means the image was buffered */
If (out_size > 0) {
AVPacket pkt;
av_init_packet(&pkt);
If (x < lTimeStamp) {
pkt.pts = pkt.dts = m_llframe_index;
pkt.duration = 0;
} else {
pkt.duration = (lTimeStamp - x)*c->time_base.den/1000000 + 1;
pkt.pts = m_llframe_index;
pkt.dts = pkt.pts;
m_llframe_index += pkt.duration;
}
/*pkt.pts = lTimeStamp * (__int64)frame_rate.den / 1000;*/
If (c->coded_frame && c->coded_frame->key_frame) {
pkt.flags |= PKT_FLAG_KEY;
}
pkt.stream_index = m_pVideoStream->index;
pkt.data = m_pvideo_outbuf;
pkt.size = out_size;
/* write the compressed frame into the media file */
ret = av_interleaved_write_frame(m_pAvFormatContext, &pkt);
} else {
ret = 0;
}
Solar Panel ,Flexible Solar Panels,Portable Solar Panels,Sunpower Solar Panels
zhejiang ttn electric co.,ltd , https://www.ttnpower.com