十 h264 RTP傳輸詳解(2)
上一章并沒有把打開文件分析文件的代碼找到，因為發現它隱藏得比較深，而且H264的Source又有多個，形成了連環計。所以此章中就將文件處理與H264的Source們并在一起分析吧。從哪里開始呢？從source開始吧!為什么要從它開始呢？我就想從這里開始，行了吧？
[cpp] view plain copyFramedSource* H264VideoFileServerMediaSubsession::createNewStreamSource( unsigned /*clientSessionId*/, unsigned& estBitrate) { estBitrate = 500; // kbps, estimate // Create the video source: ByteStreamFileSource* fileSource = ByteStreamFileSource::createNew(envir(), fFileName); if (fileSource == NULL) return NULL; fFileSize = fileSource->fileSize(); // Create a framer for the Video Elementary Stream: return H264VideoStreamFramer::createNew(envir(), fileSource); } 先創建一個ByteStreamFileSource，顯然這是一個從文件按字節讀取數據的source，沒什么可細說的。但是，打開文件，讀寫文件操作的確就在其中。最終來處理h264文件，分析其格式，解析出幀或nal的應是這個source: H264VideoStreamFramer。打開文件的地方找到了，但分析文件的代碼才是更有價值的。那我們只能來看H264VideoStreamFramer。H264VideoStreamFramer繼承自MPEGVideoStreamFramer，MPEGVideoStreamFramer繼承自FramedFilter,FramedFilter繼承自FramedSource。啊，中間又冒出個Filter?？吹剿?，是不是聯想到了DirectShow的filter？或者說photoshop中的filter？它們的意義應該都差不多吧？即插入到source和render(sink)之間的處理媒體數據的東東？如果這樣理解，還是更接近于photoshop中的概念。唉，說實話，我估計自己說的也不全對，反正就這樣認識吧，謬不了一千里。既然我們這樣認識了，那么我們就有理由相信可能會出現多個filter們一個連一個,然后高唱：手牽著腳腳牽著手一起向前走...H264VideoStreamFramer繼承自MPEGVideoStreamFramer，MPEGVideoStreamFramer比較簡單，它只是把一些工作交給了MPEGVideoStreamParser（又出來個parser，這可是個新東西哦，先不要管它吧），重點來看一下。構造函數：[cpp] view plain copyH264VideoStreamFramer::H264VideoStreamFramer(UsageEnvironment& env, FramedSource* inputSource, Boolean createParser, Boolean includeStartCodeInOutput) : MPEGVideoStreamFramer(env, inputSource), fIncludeStartCodeInOutput(includeStartCodeInOutput), fLastSeenSPS(NULL), fLastSeenSPSSize(0), fLastSeenPPS(NULL), fLastSeenPPSSize(0) { fParser = createParser ? new H264VideoStreamParser(this, inputSource, includeStartCodeInOutput) : NULL; fNextPResentationTime = fPresentationTimeBase; fFrameRate = 25.0; // We assume a frame rate of 25 fps, //unless we learn otherwise (from parsing a Sequence Parameter Set NAL unit) } 由于createParser肯定為真，所以主要內容是創建了H264VideoStreamParser對象（先不管這個parser）。其它的函數就沒什么可看的了，都集中在所保存的PPS與SPS上。看來分析工作移到了H264VideoStreamParser，Parser嘛，就是分析器。分析器的基類是StreamParser。StreamParser做了不少的工作，那我們就先搞明白StreamParser做了哪些工作吧，并且可能為繼承者提供什么樣的調用框架呢？.....我看完了，呵呵。直接說分析結果吧：StreamParser的主要工作是實現了對數據以位為單位進行訪問。因為在處理媒體格式時，按位分析是很常見的情況。這兩個函數skipBits(unsigned numBits)和unsigned getBits(unsigned numBits)很明顯是基于位的操作。unsigned char* fBank[2]這個變量中的兩個緩沖區被輪換使用。這個類中保存了一個Source，理所當然地它應該保存ByteStreamFileSource的實例，而不是FramedFilter的。那些getBytes()或getBits()最終會導致讀文件的操作。從文件讀取一次數據后，StreamParser::afterGettingBytes1（）被調用，StreamParser::afterGettingBytes1（）中做一點簡單的工作后便調用fClientContinueFunc這個回調函數。fClientContinueFunc可能指向Frame的函數體也可能是指向RtpSink的函數體－－因為Framer完全可以把RtpSink的函數體傳給Parser。至于到底指向哪個，只能在進一步分析之后才得知。下面再來分析StreamParser的兒子:MPEGVideoStreamParser。
[cpp] view plain copyMPEGVideoStreamParser::MPEGVideoStreamParser( MPEGVideoStreamFramer* usingSource, FramedSource* inputSource) : StreamParser(inputSource, FramedSource::handleClosure, usingSource, &MPEGVideoStreamFramer::continueReadProcessing, usingSource), fUsingSource(usingSource) { }MPEGVideoStreamParser的構造函數中有很多有意思的東西。首先參數usingSource是什么意思？表示正在使用這個Parser的Source? inputSource 很明確，就是能獲取數據的source，也就是 ByteStreamFileSource。而且很明顯的，StreamParser中保存的source是ByteStreamFileSource。從傳入給StreamParser的回調函數以及它們的參數可以看出，這些回調函數全是指向的StreamParser的子類的函數（為啥不用虛函數的方式？哦，回調函數全是靜態函數，不能成為虛函數）。這說明在每讀一次數據后，MPEGVideoStreamFramer::continueReadProcessing()被調用，在其中對幀進行界定和分析，完成后再調用RTPSink的相應函數，RTPSink中對幀進行打包和發送（還記得嗎，不記得了請回頭看以前的章節）。MPEGVideoStreamParser的fTo是RTPSink傳入的緩沖指針，其saveByte(),save4Bytes()是把數據從StreamParser的緩沖把數據復制到fTo中，是給繼承類使用的。saveToNextCode()是復制數據直到遇到一個同步字節串（比如h264中分隔nal的那一陀東東，當然此處的跟h264還不一樣），也是給繼承類使用的。純虛函數parse()很明顯是留繼承類去寫幀分析代碼的地方。registerReadInterest()被調用者用來告訴MPEGVideoStreamParser其接收幀的緩沖地址與容量。現在應該來分析一下MPEGVideoStreamFramer，以明確MPEGVideoStreamFramer與MPEGVideoStreamParser是怎樣配合的。MPEGVideoStreamFramer中用到Parser的重要的函數只有兩個，一是：
[cpp] view plain copyvoid MPEGVideoStreamFramer::doGetNextFrame() { fParser->registerReadInterest(fTo, fMaxSize); continueReadProcessing(); }很簡單，只是告訴了Parser保存幀的緩沖和緩沖的大小，然后執行continueReadProcessing()，那么來看一下continueReadProcessing()：
[cpp] view plain copyvoid MPEGVideoStreamFramer::continueReadProcessing() { unsigned acquiredFrameSize = fParser->parse(); if (acquiredFrameSize > 0) { // We were able to acquire a frame from the input. // It has already been copied to the reader's space. fFrameSize = acquiredFrameSize; fNumTruncatedBytes = fParser->numTruncatedBytes(); // "fPresentationTime" should have already been computed. // Compute "fDurationInMicroseconds" now: fDurationInMicroseconds = (fFrameRate == 0.0 || ((int) fPictureCount) < 0) ? 0 : (unsigned) ((fPictureCount * 1000000) / fFrameRate); fPictureCount = 0; // Call our own 'after getting' function. Because we're not a 'leaf' // source, we can call this directly, without risking infinite recursion. afterGetting(this); } else { // We were unable to parse a complete frame from the input, because: // - we had to read more data from the source stream, or // - the source stream has ended. } } 先利用Parser進行分析（應該是解析出一幀吧？），分析完成后，幀數據已到了MPEGVideoStreamFramer的緩沖fTo中。計算出幀的持續時間后，調用FrameSource的afterGetting()，最終會調用RTPSink的函數?？吹竭@里，可以總結一下，其實看來看去，Parser直正被外部使用的函數幾乎只有一個：parse()。下面可以看H264VideoStreamParser了。其實也很簡單，多了一些用于分析h264格式的函數，當然是非公開的，只供自己使用的。在哪里使用呢？當然是在parser()中使用。至于H264VideoStreamFramer前面已經說過了，沒什么太多的東西，所以就不看了?？偨Y起來也就是這樣：RTPSink向H264VideoStreamFramer要下一幀（其實h264中肯定不是一幀了，而是一個NAL Unit），H264VideoStreamFramer告訴H264VideoStreamParser輸出緩沖和容內的字節數，然后調用H264VideoStreamParser的parser()函數，parser()中調用ByteStreamFileSource從文件中讀取數據，直到parser()獲得完整的一幀，parser()返回，H264VideoStreamFramer進行一些自己的處理后把這一幀返回給了RTPSink（當然是以回調函數的方式返回）。還有一個東西，H264FUAFragmenter，被H264VideoRTPSink所使用，繼承自FramedFilter。它最初在RTPSink開始播放后創建，如下：[cpp] view plain copyBoolean H264VideoRTPSink::continuePlaying() { // First, check whether we have a 'fragmenter' class set up yet. // If not, create it now: if (fOurFragmenter == NULL) { fOurFragmenter = new H264FUAFragmenter(envir(), fsource, OutPacketBuffer::maxSize, ourMaxPacketSize() - 12/*RTP hdr size*/); fSource = fOurFragmenter; } // Then call the parent class's implementation: return MultiFramedRTPSink::continuePlaying(); } 并且它取代了H264VideoStreamFramer成為直接與RTPSink發生關系的source.如此一來，RTPSink要獲取幀時，都是從它獲取的．看它最主要的一個函數吧：[cpp] view plain copyvoid H264FUAFragmenter::doGetNextFrame() { if (fNumValidDataBytes == 1) { // We have no NAL unit data currently in the buffer. Read a new one: fInputSource->getNextFrame(&fInputBuffer[1], fInputBufferSize - 1, afterGettingFrame, this, FramedSource::handleClosure, this); } else { // We have NAL unit data in the buffer. There are three cases to consider: // 1. There is a new NAL unit in the buffer, and it's small enough to deliver // to the RTP sink (as is). // 2. There is a new NAL unit in the buffer, but it's too large to deliver to // the RTP sink in its entirety. Deliver the first fragment of this data, // as a FU-A packet, with one extra preceding header byte. // 3. There is a NAL unit in the buffer, and we've already delivered some // fragment(s) of this. Deliver the next fragment of this data, // as a FU-A packet, with two extra preceding header bytes. if (fMaxSize < fMaxOutputPacketSize) { // shouldn't happen envir() << "H264FUAFragmenter::doGetNextFrame(): fMaxSize (" << fMaxSize << ") is smaller than expected/n"; } else { fMaxSize = fMaxOutputPacketSize; } fLastFragmentCompletedNALUnit = True; // by default if (fCurDataOffset == 1) { // case 1 or 2 if (fNumValidDataBytes - 1 <= fMaxSize) { // case 1 memmove(fTo, &fInputBuffer[1], fNumValidDataBytes - 1); fFrameSize = fNumValidDataBytes - 1; fCurDataOffset = fNumValidDataBytes; } else { // case 2 // We need to send the NAL unit data as FU-A packets. Deliver the first // packet now. Note that we add FU indicator and FU header bytes to the front // of the packet (reusing the existing NAL header byte for the FU header). fInputBuffer[0] = (fInputBuffer[1] & 0xE0) | 28; // FU indicator fInputBuffer[1] = 0x80 | (fInputBuffer[1] & 0x1F); // FU header (with S bit) memmove(fTo, fInputBuffer, fMaxSize); fFrameSize = fMaxSize; fCurDataOffset += fMaxSize - 1; fLastFragmentCompletedNALUnit = False; } } else { // case 3 // We are sending this NAL unit data as FU-A packets. We've already sent the // first packet (fragment). Now, send the next fragment. Note that we add // FU indicator and FU header bytes to the front. (We reuse these bytes that // we already sent for the first fragment, but clear the S bit, and add the E // bit if this is the last fragment.) fInputBuffer[fCurDataOffset - 2] = fInputBuffer[0]; // FU indicator fInputBuffer[fCurDataOffset - 1] = fInputBuffer[1] & ~0x80; // FU header (no S bit) unsigned numBytesToSend = 2 + fNumValidDataBytes - fCurDataOffset; if (numBytesToSend > fMaxSize) { // We can't send all of the remaining data this time: numBytesToSend = fMaxSize; fLastFragmentCompletedNALUnit = False; } else { // This is the last fragment: fInputBuffer[fCurDataOffset - 1] |= 0x40; // set the E bit in the FU header fNumTruncatedBytes = fSaveNumTruncatedBytes; } memmove(fTo, &fInputBuffer[fCurDataOffset - 2], numBytesToSend); fFrameSize = numBytesToSend; fCurDataOffset += numBytesToSend - 2; } if (fCurDataOffset >= fNumValidDataBytes) { // We're done with this data. Reset the pointers for receiving new data: fNumValidDataBytes = fCurDataOffset = 1; } // Complete delivery to the client: FramedSource::afterGetting(this); } }如果輸入緩沖中沒有數據，調用fInputSource->getNextFrame()，fInputSource是H264VideoStreamFramer，H264VideoStreamFramer的getNextFrame()會調用H264VideoStreamParser的parser()，parser()又調用ByteStreamFileSource獲取數據，然后分析，parser()完成后會調用：
[cpp] view plain copyvoid H264FUAFragmenter::afterGettingFrame1( unsigned frameSize, unsigned numTruncatedBytes, struct timeval presentationTime, unsigned durationInMicroseconds) { fNumValidDataBytes += frameSize; fSaveNumTruncatedBytes = numTruncatedBytes; fPresentationTime = presentationTime; fDurationInMicroseconds = durationInMicroseconds; // Deliver data to the client: doGetNextFrame(); } 然后又調用回H264FUAFragmenter::doGetNextFrame()，此時輸入緩沖中有數據了，H264FUAFragmenter就進行分析處理．H264FUAFragmenter又對數據做了什么呢？新聞熱點
疑難解答
