声音捕获与OpenAL的iOS上(Sound capture with OpenAL on iOS)

2019-07-29 13:38发布

站内文章 / 移动开发
24 0 0

我试图做使用OpenAL的iOS的声音捕捉（我写一个跨平台的库，这就是为什么我避免特定IOS的方式来记录声音）。 开箱即用的OpenAL的捕获不起作用，但存在一个已知的解决方法： 开始捕捉之前打开输出上下文 。 该解决方案为我在iOS 5.0。

然而在iOS 5.1.1，解决办法不仅有助于为第一个样品我试图记录。 （将我的AudioSession到PlayAndRecord开始拍摄前，打开默认的输出设备。记录我的样品后，我关闭装置和开关会话回不管它是什么。）对于第二个样本，重新打开输出上下文不帮助没有声音被捕获。

是否存在已知的方式来处理这个问题呢？ 

// Here's what I do before starting the recording
oldAudioSessionCategory = [audioSession category];
[audioSession setCategory:AVAudioSessionCategoryPlayAndRecord error:nil];
[audioSession setActive:YES error:nil];
// We need to have an active context. If there is none, create one.
if (!alcGetCurrentContext()) {
    outputDevice = alcOpenDevice(NULL);
    outputContext = alcCreateContext(outputDevice, NULL);
    alcMakeContextCurrent(outputContext);
}

// Capture itself
inputDevice = alcCaptureOpenDevice(NULL, frequency, FORMAT, bufferSize);
....
alcCaptureCloseDevice(inputDevice);

// Restoring the audio state to whatever it had been before capture
if (outputContext) {
    alcDestroyContext(outputContext);
    alcCloseDevice(outputDevice);
}
[[AVAudioSession sharedInstance] setCategory:oldAudioSessionCategory 
                                 error:nil];

Answer 1:

下面是我用模拟捕获扩展的代码。 一些评论：

  1. 在该项目作为一个整体，OpenKD的用途，例如，线程原语。 你可能需要更换这些调用。
  2. 我只好打延迟在开始拍摄。 其结果是，我一直不断地读书声音输入，并把它扔了不需要的时候。 （这样的解决方案提出，例如， 在这里 ）。这，反过来，需要捕捉onResignActive通知，以便释放麦克风的控制。 您可能会或可能不会想用这样的杂牌。
  3. 代替alcGetIntegerv(device, ALC_CAPTURE_SAMPLES, 1, &res)我必须定义一个单独的函数， alcGetAvailableSamples 。

总之，这段代码是不太可能在您的项目使用的原样，但希望你可以把它调整到您的需要。 

#include <stdbool.h>
#include <stddef.h>
#include <stdint.h>
#include <KD/kd.h>
#include <AL/al.h>
#include <AL/alc.h>

#include <AudioToolbox/AudioToolbox.h>
#import <Foundation/Foundation.h>
#import <UIKit/UIKit.h>

#include "KD/kdext.h"

struct InputDeviceData {
    int id;
    KDThreadMutex *mutex;
    AudioUnit audioUnit;
    int nChannels;
    int frequency;
    ALCenum format;
    int sampleSize;
    uint8_t *buf;
    size_t bufSize;    // in bytes
    size_t bufFilledBytes;  // in bytes
    bool started;
};

static struct InputDeviceData *cachedInData = NULL;

static OSStatus renderCallback (void                        *inRefCon,
                                AudioUnitRenderActionFlags  *ioActionFlags,
                                const AudioTimeStamp        *inTimeStamp,
                                UInt32                      inBusNumber,
                                UInt32                      inNumberFrames,
                                AudioBufferList             *ioData);
static AudioUnit getAudioUnit();
static void setupNotifications();
static void destroyCachedInData();
static struct InputDeviceData *setupCachedInData(AudioUnit audioUnit, ALCuint frequency, ALCenum format, ALCsizei bufferSizeInSamples);
static struct InputDeviceData *getInputDeviceData(AudioUnit audioUnit, ALCuint frequency, ALCenum format, ALCsizei bufferSizeInSamples);

/** I only have to use NSNotificationCenter instead of CFNotificationCenter
 *  because there is no published name for WillResignActive/WillBecomeActive
 *  notifications in CoreFoundation.
 */
@interface ALCNotificationObserver : NSObject
- (void)onResignActive;
@end
@implementation ALCNotificationObserver
- (void)onResignActive {
    destroyCachedInData();
}
@end

static void setupNotifications() {
    static ALCNotificationObserver *observer = NULL;
    if (!observer) {
        observer = [[ALCNotificationObserver alloc] init];
        [[NSNotificationCenter defaultCenter] addObserver:observer selector:@selector(onResignActive) name:UIApplicationWillResignActiveNotification object:nil];
    }
}

static OSStatus renderCallback (void                        *inRefCon,
                                AudioUnitRenderActionFlags  *ioActionFlags,
                                const AudioTimeStamp        *inTimeStamp,
                                UInt32                      inBusNumber,
                                UInt32                      inNumberFrames,
                                AudioBufferList             *ioData) {
    struct InputDeviceData *inData = (struct InputDeviceData*)inRefCon;

    kdThreadMutexLock(inData->mutex);
    size_t bytesToRender = inNumberFrames * inData->sampleSize;
    if (bytesToRender + inData->bufFilledBytes <= inData->bufSize) {
        OSStatus status;
        struct AudioBufferList audioBufferList; // 1 buffer is declared inside the structure itself.
        audioBufferList.mNumberBuffers = 1;
        audioBufferList.mBuffers[0].mNumberChannels = inData->nChannels;
        audioBufferList.mBuffers[0].mDataByteSize = bytesToRender;
        audioBufferList.mBuffers[0].mData = inData->buf + inData->bufFilledBytes;
        status = AudioUnitRender(inData->audioUnit, 
                                 ioActionFlags, 
                                 inTimeStamp, 
                                 inBusNumber, 
                                 inNumberFrames, 
                                 &audioBufferList);
        if (inData->started) {
            inData->bufFilledBytes += bytesToRender;
        }
    } else {
        kdLogFormatMessage("%s: buffer overflow", __FUNCTION__);
    }
    kdThreadMutexUnlock(inData->mutex);

    return 0;
}

static AudioUnit getAudioUnit() {
    static AudioUnit audioUnit = NULL;

    if (!audioUnit) {
        AudioComponentDescription ioUnitDescription;

        ioUnitDescription.componentType          = kAudioUnitType_Output;
        ioUnitDescription.componentSubType       = kAudioUnitSubType_VoiceProcessingIO;
        ioUnitDescription.componentManufacturer  = kAudioUnitManufacturer_Apple;
        ioUnitDescription.componentFlags         = 0;
        ioUnitDescription.componentFlagsMask     = 0;

        AudioComponent foundIoUnitReference = AudioComponentFindNext(NULL,
                                                                     &ioUnitDescription);
        AudioComponentInstanceNew(foundIoUnitReference,
                                  &audioUnit);

        if (audioUnit == NULL) {
            kdLogMessage("Could not obtain AudioUnit");
        }
    }

    return audioUnit;
}

static void destroyCachedInData() {
    OSStatus status;
    if (cachedInData) {
        status = AudioOutputUnitStop(cachedInData->audioUnit);
        status = AudioUnitUninitialize(cachedInData->audioUnit);
        free(cachedInData->buf);
        kdThreadMutexFree(cachedInData->mutex);
        free(cachedInData);
        cachedInData = NULL;
    }
}

static struct InputDeviceData *setupCachedInData(AudioUnit audioUnit, ALCuint frequency, ALCenum format, ALCsizei bufferSizeInSamples) {
    static int idCount = 0;
    OSStatus status;
    int bytesPerFrame = (format == AL_FORMAT_MONO8) ? 1 :
                        (format == AL_FORMAT_MONO16) ? 2 :
                        (format == AL_FORMAT_STEREO8) ? 2 :
                        (format == AL_FORMAT_STEREO16) ? 4 : -1;
    int channelsPerFrame = (format == AL_FORMAT_MONO8) ? 1 :
                           (format == AL_FORMAT_MONO16) ? 1 :
                           (format == AL_FORMAT_STEREO8) ? 2 :
                           (format == AL_FORMAT_STEREO16) ? 2 : -1;
    int bitsPerChannel = (format == AL_FORMAT_MONO8) ? 8 :
                         (format == AL_FORMAT_MONO16) ? 16 :
                         (format == AL_FORMAT_STEREO8) ? 8 :
                         (format == AL_FORMAT_STEREO16) ? 16 : -1;

    cachedInData = malloc(sizeof(struct InputDeviceData));
    cachedInData->id = ++idCount;
    cachedInData->format = format;
    cachedInData->frequency = frequency;
    cachedInData->mutex = kdThreadMutexCreate(NULL);
    cachedInData->audioUnit = audioUnit;
    cachedInData->nChannels = channelsPerFrame;
    cachedInData->sampleSize = bytesPerFrame;
    cachedInData->bufSize = bufferSizeInSamples * bytesPerFrame;
    cachedInData->buf = malloc(cachedInData->bufSize);
    cachedInData->bufFilledBytes = 0;
    cachedInData->started = FALSE;

    UInt32 enableOutput        = 1;    // to enable output
    status = AudioUnitSetProperty(audioUnit,
                                  kAudioOutputUnitProperty_EnableIO,
                                  kAudioUnitScope_Input,
                                  1,
                                  &enableOutput, sizeof(enableOutput));

    struct AudioStreamBasicDescription basicDescription;
    basicDescription.mSampleRate = (Float64)frequency;
    basicDescription.mFormatID = kAudioFormatLinearPCM;
    basicDescription.mFormatFlags = kAudioFormatFlagIsSignedInteger | kAudioFormatFlagIsPacked;
    basicDescription.mBytesPerPacket = bytesPerFrame;
    basicDescription.mFramesPerPacket = 1;
    basicDescription.mBytesPerFrame = bytesPerFrame;
    basicDescription.mChannelsPerFrame = channelsPerFrame;
    basicDescription.mBitsPerChannel = bitsPerChannel;
    basicDescription.mReserved = 0;

    status = AudioUnitSetProperty(audioUnit, 
                                  kAudioUnitProperty_StreamFormat, // property key 
                                  kAudioUnitScope_Output,        // scope
                                  1,                             // 1 is output
                                  &basicDescription, sizeof(basicDescription));      // value

    AURenderCallbackStruct renderCallbackStruct;
    renderCallbackStruct.inputProc = renderCallback;
    renderCallbackStruct.inputProcRefCon = cachedInData;
    status = AudioUnitSetProperty(audioUnit, 
                                  kAudioOutputUnitProperty_SetInputCallback, // property key 
                                  kAudioUnitScope_Output,        // scope
                                  1,                             // 1 is output
                                  &renderCallbackStruct, sizeof(renderCallbackStruct));      // value

    status = AudioOutputUnitStart(cachedInData->audioUnit);

    return cachedInData;
}

static struct InputDeviceData *getInputDeviceData(AudioUnit audioUnit, ALCuint frequency, ALCenum format, ALCsizei bufferSizeInSamples) {
    if (cachedInData && 
        (cachedInData->frequency != frequency ||
         cachedInData->format != format ||
         cachedInData->bufSize / cachedInData->sampleSize != bufferSizeInSamples)) {
            kdAssert(!cachedInData->started);
            destroyCachedInData();
        }
    if (!cachedInData) {
        setupCachedInData(audioUnit, frequency, format, bufferSizeInSamples);
        setupNotifications();
    }

    return cachedInData;
}


ALC_API ALCdevice* ALC_APIENTRY alcCaptureOpenDevice(const ALCchar *devicename, ALCuint frequency, ALCenum format, ALCsizei buffersizeInSamples) {
    kdAssert(devicename == NULL);    

    AudioUnit audioUnit = getAudioUnit();
    struct InputDeviceData *res = getInputDeviceData(audioUnit, frequency, format, buffersizeInSamples);
    return (ALCdevice*)res->id;
}

ALC_API ALCboolean ALC_APIENTRY alcCaptureCloseDevice(ALCdevice *device) {
    alcCaptureStop(device);
    return true;
}

ALC_API void ALC_APIENTRY alcCaptureStart(ALCdevice *device) {
    if (!cachedInData || (int)device != cachedInData->id) {
        // may happen after the app loses and regains active status.
        kdLogFormatMessage("Attempt to start a stale AL capture device");
        return;
    }
    cachedInData->started = TRUE;
}

ALC_API void ALC_APIENTRY alcCaptureStop(ALCdevice *device) {
    if (!cachedInData || (int)device != cachedInData->id) {
        // may happen after the app loses and regains active status.
        kdLogFormatMessage("Attempt to stop a stale AL capture device");
        return;
    }
    cachedInData->started = FALSE;
}

ALC_API ALCint ALC_APIENTRY alcGetAvailableSamples(ALCdevice *device) {
    if (!cachedInData || (int)device != cachedInData->id) {
        // may happen after the app loses and regains active status.
        kdLogFormatMessage("Attempt to get sample count from a stale AL capture device");
        return 0;
    }
    ALCint res;
    kdThreadMutexLock(cachedInData->mutex);
    res = cachedInData->bufFilledBytes / cachedInData->sampleSize;
    kdThreadMutexUnlock(cachedInData->mutex);
    return res;
}

ALC_API void ALC_APIENTRY alcCaptureSamples(ALCdevice *device, ALCvoid *buffer, ALCsizei samples) {    
    if (!cachedInData || (int)device != cachedInData->id) {
        // may happen after the app loses and regains active status.
        kdLogFormatMessage("Attempt to get samples from a stale AL capture device");
        return;
    }
    size_t bytesToCapture = samples * cachedInData->sampleSize;
    kdAssert(cachedInData->started);
    kdAssert(bytesToCapture <= cachedInData->bufFilledBytes);

    kdThreadMutexLock(cachedInData->mutex);
    memcpy(buffer, cachedInData->buf, bytesToCapture);
    memmove(cachedInData->buf, cachedInData->buf + bytesToCapture, cachedInData->bufFilledBytes - bytesToCapture);
    cachedInData->bufFilledBytes -= bytesToCapture;
    kdThreadMutexUnlock(cachedInData->mutex);
}


Answer 2:

我发现了一个方法，使苹果公司的OpenAL的工作。 在我原来的代码片段，你需要调用alcMakeContextCurrent(NULL)之前alcDestroyContext(outputContext)



文章来源: Sound capture with OpenAL on iOS
标签： ios openal
举报

收藏的人(0)

Ta的文章 更多文章
登录 后发表评论
0条评论
还没有人评论过~