Qidi 2017.02.23 （Markdown & Haroopad）
處理過音頻文件的工程師都知道音頻數據存在采樣率（Sample Rate）這個指標。在位深度（Bit Depth）一定的情況下，采樣率越高，理論上來說播放出來的聲音就越細膩，錄制的聲音也就越保真，反之亦然。
但在較早的Android系統版本上，不管音頻文件原來的采樣率幾何，統統都被重采樣（Resample）到44.1KHz進行播放，錄制的時候則是被固定為8KHz進行采樣。盡管這樣的處理方式被廣大音質愛好者所詬病，但在當時它確實是一種實現設備兼容的有效方法。
作為Android Audio BSP工程師，有必要了解系統實現Resample的過程。現在Android系統已經發布到了7.0版本，一起看看在最新的版本上這個Resample的過程是怎樣實現的吧。
我們知道在Android系統中，當應用層APP播放一個音頻文件時，Framework層的AudioPolicyService（APS）會接收上層APP傳遞來的音頻參數（例如格式、聲道、采樣率等），并調用AudioFlinger的createTrack()方法對應創建1個Track，再調用openOutput()方法來打開1個outputStream，然后使用這個outputStream來創建相應的Playback線程（依據應用場景可以是OffloadThread、DirectOutputThread、MixerThread），最終在Playback線程中匹配之前創建的Track，開始自APP至HAL的數據傳輸。
那么我們對Android Audio Resample過程的分析就從AudioFlinger開始。在AudioFlinger::openOutput()中可以看到，在Playback線程被成功創建之后，即被加入到mPlaybackThreads向量中進行管理了。具體代碼如下：
隨后Playback線程運行，對應的AudioFlinger::Playback::threadLoop()方法被執行，在該方法中調用了prepareTracks_l()函數。這個函數實際上是對應于AudioFlinger::MixerThread::prepareTracks_l()這個函數。threadLoop()函數代碼細節如下：
Resample的過程就發生在prepareTracks_l()函數中，所以我們來好好閱讀一下。在該函數中，通過一個for循環遍歷所有處于active狀態的track。每一次循環中，都要進行如下2步操作： 1. 通過reqSampleRate = track->mAudioTrackServerProxy->getSampleRate()來獲取硬件設備所支持的采樣率； 2. 之后調用mAudioMixer->setParameter(name, AudioMixer::RESAMPLE, AudioMixer::SAMPLE_RATE, (void*)(uintptr_t)reqSampleRate)，通過對比音頻文件采樣率和音頻設備支持的采樣率，判斷是否創建新的Resampler對象，或者從已有的Resampler對象列表中返回1個；
prepareTracks_l()函數代碼細節如下：
AudioFlinger::PlaybackThread::mixer_state AudioFlinger::MixerThread::prepareTracks_l( Vector< sp<Track> > *tracksToRemove){ ...... // find out which tracks need to be processed size_t count = mActiveTracks.size(); // 獲取處于active狀態的track的數量 ...... for (size_t i=0 ; i<count ; i++) { const sp<Track> t = mActiveTracks[i].promote(); if (t == 0) { continue; } // this const just means the local variable doesn't change Track* const track = t.get(); // 獲取對應的track ...... audio_track_cblk_t* cblk = track->cblk(); // The first time a track is added we wait // for all its buffers to be filled before processing it int name = track->name(); ...... if ((framesReady >= minFrames) && track->isReady() && !track->isPaused() && !track->isTerminated()) { ...... int param = AudioMixer::VOLUME; if (track->mFillingUpStatus == Track::FS_FILLED) { // no ramp for the first volume setting track->mFillingUpStatus = Track::FS_ACTIVE; if (track->mState == TrackBase::RESUMING) { track->mState = TrackBase::ACTIVE; param = AudioMixer::RAMP_VOLUME; } mAudioMixer->setParameter(name, AudioMixer::RESAMPLE, AudioMixer::RESET, NULL); // FIXME should not make a decision based on mServer } else if (cblk->mServer != 0) { // If the track is stopped before the first frame was mixed, // do not apply ramp param = AudioMixer::RAMP_VOLUME; } // compute volume for this track ...... // Delegate volume control to effect in track effect chain if needed ...... // XXX: these things DON'T need to be done each time mAudioMixer->setBufferProvider(name, track); mAudioMixer->enable(name); mAudioMixer->setParameter(name, param, AudioMixer::VOLUME0, &vlf); // 設置左聲道音量 mAudioMixer->setParameter(name, param, AudioMixer::VOLUME1, &vrf); // 設置右聲道音量 mAudioMixer->setParameter(name, param, AudioMixer::AUXLEVEL, &vaf); // 設置輔助聲道音量 mAudioMixer->setParameter( name, AudioMixer::TRACK, AudioMixer::FORMAT, (void *)track->format()); // 設置音頻數據格式 mAudioMixer->setParameter( name, AudioMixer::TRACK, AudioMixer::CHANNEL_MASK, (void *)(uintptr_t)track->channelMask()); // 設置音頻聲道數 mAudioMixer->setParameter( name, AudioMixer::TRACK, AudioMixer::MIXER_CHANNEL_MASK, (void *)(uintptr_t)mChannelMask); // limit track sample rate to 2 x output sample rate, which changes at re-configuration uint32_t maxSampleRate = mSampleRate * AUDIO_RESAMPLER_DOWN_RATIO_MAX; uint32_t reqSampleRate = track->mAudioTrackServerProxy->getSampleRate(); // 獲取音頻設備所支持的采樣率 if (reqSampleRate == 0) { reqSampleRate = mSampleRate; } else if (reqSampleRate > maxSampleRate) { reqSampleRate = maxSampleRate; } mAudioMixer->setParameter( name, AudioMixer::RESAMPLE, AudioMixer::SAMPLE_RATE, // 設置音頻采樣率(必要時會進行重采樣) (void *)(uintptr_t)reqSampleRate); AudioPlaybackRate playbackRate = track->mAudioTrackServerProxy->getPlaybackRate(); mAudioMixer->setParameter( name, AudioMixer::TIMESTRETCH, AudioMixer::PLAYBACK_RATE, // 設置播放碼率 &playbackRate); /* * Select the appropriate output buffer for the track. * * Tracks with effects go into their own effects chain buffer * and from there into either mEffectBuffer or mSinkBuffer. * * Other tracks can use mMixerBuffer for higher precision * channel accumulation. If this buffer is enabled * (mMixerBufferEnabled true), then selected tracks will accumulate * into it. * */ if (mMixerBufferEnabled && (track->mainBuffer() == mSinkBuffer || track->mainBuffer() == mMixerBuffer)) { mAudioMixer->setParameter( name, AudioMixer::TRACK, AudioMixer::MIXER_FORMAT, (void *)mMixerBufferFormat); // 設置緩沖區數據格式 mAudioMixer->setParameter( name, AudioMixer::TRACK, AudioMixer::MAIN_BUFFER, (void *)mMixerBuffer); // 分配主緩沖區 // TODO: override track->mainBuffer()? mMixerBufferValid = true; } else { ...... } mAudioMixer->setParameter( name, AudioMixer::TRACK, AudioMixer::AUX_BUFFER, (void *)track->auxBuffer()); // 分配副緩沖區 // reset retry count track->mRetryCount = kMaxTrackRetries; // If one track is ready, set the mixer ready if: // - the mixer was not ready during previous round OR // - no other track is not ready if (mMixerStatusIgnoringFastTracks != MIXER_TRACKS_READY || mixerStatus != MIXER_TRACKS_ENABLED) { mixerStatus = MIXER_TRACKS_READY; } } else { // 出現underrun，以及相應處理操作 ...... } } // Push the new FastMixer state if necessary ...... // Now perform the deferred reset on fast tracks that have stopped ...... // remove all the tracks that need to be... removeTracks_l(*tracksToRemove); ...... // sink or mix buffer must be cleared if all tracks are connected to an // effect chain as in this case the mixer will not write to the sink or mix buffer // and track effects will accumulate into it ...... // if any fast tracks, then status is ready ...... return mixerStatus;}在確認要使用的Resampler對象存在后，調用invalidateState(1 << name)使設置生效，開始執行重采樣。invalidateState()函數會調用AudioMixer::process_validate()，在該函數中首先通過語句t.hook = getTrackHook(TRACKTYPE_RESAMPLE, t.mMixerChannelCount, t.mMixerInFormat, t.mMixerFormat);獲取執行重采樣操作的函數，隨后通過state->hook = process_resampling;中的t.hook(&t, outTemp, numFrames, state->resampleTemp, aux)語句進行調用。 setParameter()函數代碼如下：
invalidateState()函數代碼如下：
void AudioMixer::invalidateState(uint32_t mask){ if (mask != 0) { mState.needsChanged |= mask; mState.hook = process__validate; // 使配置生效 }}process__validate()函數代碼如下：
void AudioMixer::process__validate(state_t* state){ ...... uint32_t en = state->enabledTracks; while (en) { ...... if (n & NEEDS_MUTE) { ...... } else { ...... if (n & NEEDS_RESAMPLE) { all16BitsStereoNoResample = false; resampling = true; t.hook = getTrackHook(TRACKTYPE_RESAMPLE, t.mMixerChannelCount, t.mMixerInFormat, t.mMixerFormat); // 獲取Resample時track對象需要執行的函數（查看getTrackHook()可以看到被獲取的函數是track__genericResample()） ALOGV_IF((n & NEEDS_CHANNEL_COUNT__MASK) > NEEDS_CHANNEL_2, "Track %d needs downmix + resample", i); } else { ...... } } } // select the processing hooks state->hook = process__nop; if (countActiveTracks > 0) { if (resampling) { if (!state->outputTemp) { state->outputTemp = new int32_t[MAX_NUM_CHANNELS * state->frameCount]; } if (!state->resampleTemp) { state->resampleTemp = new int32_t[MAX_NUM_CHANNELS * state->frameCount]; } state->hook = process__genericResampling; // 在需要重采樣操作的情況下，調用process_genericResampling()函數 } else { ...... } } ...... // Now that the volume ramp has been done, set optimal state and // track hooks for subsequent mixer process ......}process_genericResampling()函數代碼如下：
// generic code with resamplingvoid AudioMixer::process__genericResampling(state_t* state){ ...... uint32_t e0 = state->enabledTracks; while (e0) { // process by group of tracks with same output buffer // to optimize cache use ...... while (e1) { ...... // this is a little goofy, on the resampling case we don't // acquire/release the buffers because it's done by // the resampler. if (t.needs & NEEDS_RESAMPLE) { t.hook(&t, outTemp, numFrames, state->resampleTemp, aux); // 調用track__genericResample()函數執行Resample } else { ...... } } convertMixerFormat(out, t1.mMixerFormat, outTemp, t1.mMixerInFormat, numFrames * t1.mMixerChannelCount); }}至此，Android系統播放音頻時的Resample過程就分析完成了。
具體的Resample處理實質是數字信號處理，是個數學運算過程。Android系統中提供的算法有線性插值、三次插值、FIR濾波 3種。感興趣的工程師同仁可以自行查閱相關資料書籍，這里不對數字信號處理的細節進行討論。
新聞熱點
疑難解答
