H.264 RTP Streaming
2016-12-02 15:08
337 查看
據RFC3984以RTP
封裝H.264 raw data來作video streaming.
1.H.264 raw data
以00 00 01 或 00 00 00 01作為開頭(Start Code),接著是8 bit NALU
NALU的format
封裝H.264 raw data來作video streaming.
1.H.264 raw data
以00 00 01 或 00 00 00 01作為開頭(Start Code),接著是8 bit NALU
NALU的format
+---------------+ |0|1|2|3|4|5|6|7| +-+-+-+-+-+-+-+-+ |F|NRI| Type | +---------------+ F : forbidden zero bit, 一定為0 NRI : nal_ref_idc, 表示資料的重要性, 00為最不重要. Type :nal_unit_type, H.264只定義1~23的範圍 一個H.264 raw data看起來像這樣 00 00 00 01 09 30 ...... 2.RTP header 因為一個H.264 video frame資料的大小往往會在數k bytes到數十K bytes, 在傳送封包時就會將資料切割分別封裝,也因此需要加入一些額外的參數讓 接收端可以正確組合被分割的video frame.這也是RFC3984最主要的目的. RTP header 中有三個參數要注意 timestamp : 以90KHz作為基準,以30 fps為例,timestamp遞增 90000 / 30. 實務上是以payload實際間隔時間作計算.同一個video frame的 分割資料timestamp是相同的 sequence : 每個RTP封包sequence number都遞增. mark bit : RTP封包封裝的是最後一個分割的video frame時mark bit 為 1. 2.Payload format 1~23 : Single NAL unit packet. RFC3984使用了H.264 NALU中未定義的type 24~29 (相當於增加H.264 nal_unit_type定義) 24 : STAP-A 單一時間組合 25 : STAP-B 單一時間組合 26 : MTAP16 多個時間組合 27 : MTAP32 多個時間組合 28 : FU-A 分割資料 29 : FU-B 分割資料 比較常見的是28,29. 3.Single NAL unit packet 當資料少於MTU的大小就用此方式封裝. H.264 raw data foramt 為 [Start code][NALU][Raw Data] 封裝時去掉Start Code即可.Format 如下 [RTP Header][NALU][Raw Date] 4.FU-A (Fragmentation unit) 當資料大於MTU以此方式分割 H.264 raw data foramt 為 [Start code][NALU][Raw Data] 去掉[Start code],[NALU],以不超過MTU大小分割[Raw Data] 以NALU產生FU indicator, FU Header. RFC 3984 0 1 2 3 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | FU indicator | FU header | | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | | | | FU payload | | | | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | :...OPTIONAL RTP padding | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ Figure 14. RTP payload format for FU-A The FU indicator octet has the following format: +---------------+ |0|1|2|3|4|5|6|7| +-+-+-+-+-+-+-+-+ |F|NRI| Type | +---------------+ The FU header has the following format: +---------------+ |0|1|2|3|4|5|6|7| +-+-+-+-+-+-+-+-+ |S|E|R| Type | +---------------+ [FU indicator] = (NALU & 0x60) | 28; [FU Header] = (start << 7) | (end << 6) | (NALU & 0x1f); format如下 [RTP Header][FU indicator][FU header][Raw Data Part 0] [RTP Header][FU indicator][FU header][Raw Data Part 1] [RTP Header][FU indicator][FU header][Raw Data Part 2] ... 5.FU-B (Fragmentation unit) RFC 3984 0 1 2 3 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | FU indicator | FU header | DON | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-| | | | FU payload | | | | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | :...OPTIONAL RTP padding | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ Figure 15. RTP payload format for FU-B format如下 [RTP Header][FU indicator][FU header][DON][Raw Data Part 0] [RTP Header][FU indicator][FU header][DON][Raw Data Part 1] [RTP Header][FU indicator][FU header][DON][Raw Data Part 2] ... 6.Encoding sample code with TI DSP /* Write the encoded frame to the network */ if (Buffer_getNumBytesUsed(hOutBuf)) { struct iovec data[3]; rtp_hdr_t rtp; rtp.version = 2; rtp.p = 0; rtp.x = 0; rtp.cc = 0; rtp.m = 0; rtp.pt = 96; rtp.seq = htons( sequence ); rtp.ts = htonl( timestamp ); rtp.ssrc = 10; data[0].iov_base = &rtp; data[0].iov_len = sizeof(rtp_hdr_t); unsigned char *ptr = (unsigned char *)Buffer_getUserPtr(hOutBuf); size_t len = Buffer_getNumBytesUsed(hOutBuf); size_t mtu = DEFAULT_MTU; /* Skip NAL Start Code 00 00 00 01 */ unsigned char nalType = ptr[4] & 0x1f; //printf("Processing Buffer with NAL TYPE=%d\n", nalType); if(len < (mtu - sizeof(rtp_hdr_t))) { /* Remove NAL Start Code 00 00 00 01 */ data[1].iov_base = (void *)ptr + 4; data[1].iov_len = len - 4; //printf("NAL Unit fit in one packet size=%d\n", len); /* only set the marker bit on packets containing access units */ if (IS_ACCESS_UNIT (nalType)) rtp.m = 1; writev(sfd, data, 2); sequence++; } else { int start = 1, end = 0; /* We keep 2 bytes for FU indicator and FU Header */ unsigned payload_len = mtu - sizeof(rtp_hdr_t) - 2; unsigned char nalHeader = ptr[4]; /* Remove NAL Start Code 00 00 00 01 and NAL Type*/ ptr += 5; len -= 5; //printf("Using FU-A fragmentation for data size=%d\n", len); while(end == 0) { unsigned char fu[2]; payload_len = len < payload_len ? len : payload_len; if (payload_len == len) end = 1; if (IS_ACCESS_UNIT (nalType)) rtp.m = end; /* FU indicator */ fu[0] = (nalHeader & 0x60) | 28; /* FU Header */ fu[1] = (start << 7) | (end << 6) | (nalHeader & 0x1f); rtp.seq = htons( sequence ); data[1].iov_base = fu; data[1].iov_len = 2; data[2].iov_base = (void *)ptr; data[2].iov_len = payload_len; writev(sfd, data, 3); start = 0; ptr += payload_len; len -= payload_len; sequence++; } } if (Time_delta(hTimestamp, &time) < 0) { printf("Failed to get timer delta\n"); goto cleanup; } //printf("Read time: %uus\n", (Uns)time); timestamp += (unsigned int)(0.09 * time); /*15 fps : 90000 / 15*/ }
相关文章推荐
- H.264 RTP Streaming
- H.264简单总结
- H.264与MPEG4两种压缩算法的比较
- 超线程技术以及H.264编码器中的并行运算分析
- 浅谈 H.264
- H.264 MV 学习笔记1:Inter Prediction 基本流程
- MPEG4 & H.264学习笔记之三 ------ 熵编码
- 利用FFmpeg编解码库解码h.264压缩数据的demo
- H.264:视频压缩编码
- H.264 编码技术发展状况以及与MPEG-4 的比较分析
- FFMPEG和H.264相关开发笔记
- [转] H.264简单总结 (quan整理)
- H.264 RTP payload 格式
- H.264句法和语法总结(十一)宏块层预测句法
- H.264名词
- 《H.264 and MPEG-4 video compression》读后感
- H.264 最新标准下载位置
- H.264码流结构
- H.264的CAVLC(编码.解码)过程详解