混合两个16位编码的立体声PCM样本，导致产生的音频中出现噪声和失真

Question

我从两个来源获得了两个不同的音频样本。

>

对于麦克风声音：

audioRecord =
         new AudioRecord(MediaRecorder.AudioSource.DEFAULT, 44100, AudioFormat.CHANNEL_IN_STEREO, AudioFormat.ENCODING_PCM_16BIT,
                 (AudioRecord.getMinBufferSize(44100, AudioFormat.CHANNEL_IN_STEREO, AudioFormat.ENCODING_PCM_16BIT)*5));

对于内部声音：

audioRecord = new AudioRecord.Builder()
                 .setAudioPlaybackCaptureConfig(config)
                 .setAudioFormat(new AudioFormat.Builder()
                         .setEncoding(AudioFormat.ENCODING_PCM_16BIT)
                         .setSampleRate(44100)
                         .setChannelMask(AudioFormat.CHANNEL_IN_STEREO)
                         .build())
                 .setBufferSizeInBytes((AudioRecord.getMinBufferSize(44100, AudioFormat.CHANNEL_IN_STEREO, AudioFormat.ENCODING_PCM_16BIT)*5))
                 .build();

对于从audioBook对象读取，我们创建单独的帧对象（自定义对象称为帧）-

private ByteBuffer pcmBuffer = ByteBuffer.allocateDirect(4096);
private Frame read() {
  pcmBuffer.rewind();
  int size = audioRecord.read(pcmBuffer, pcmBuffer.remaining());
  if (size <= 0) {
   return null;
  }
    return new Frame(pcmBuffer.array(),
           pcmBuffer.arrayOffset(), size);
}

我们创建了两个单独的LL（链接列表）来添加我们从读函数中获得的这些帧。

private LinkedList internalAudioQueue=新建LinkedList

public void onFrameReceived(Frame frame, boolean isInternalAudio) {
    if (isInternalAudio) {
        internalAudioQueue.add(frame);
    } else {
        microphoneAudioQueue.add(frame);
    }
    checkAndPoll();
}

每次我们在各自的LL中添加一个帧时，我们调用下面的checkAndPoll（）函数，并根据具体情况将帧传递给audioEncoder。

public void checkAndPoll() {
    Frame frame1 = internalAudioQueue.poll();
    Frame frame2 = microphoneAudioQueue.poll();
    if (frame1 == null && frame2 != null) {
        audioEncoder.inputPCMData(frame2);
    } else if (frame1 != null && frame2 == null) {
        audioEncoder.inputPCMData(frame1);
    } else if (frame1 != null && frame2 != null) {
        Frame frame = new Frame(PCMUtil.mix(frame1.getBuffer(), frame2.getBuffer(), frame1.getSize(), frame2.getSize(), false), frame1.getOrientation(), frame1.getSize());
        audioEncoder.inputPCMData(frame);
    }
}

现在，我们以ByteBuffer的形式混合来自两个源的音频样本，以这种方式从这个链接中获得Hendrik的帮助。

public static byte[] mix(final byte[] a, final byte[] b, final boolean bigEndian) {
    final byte[] aa;
    final byte[] bb;

    final int length = Math.max(a.length, b.length);
    // ensure same lengths
    if (a.length != b.length) {
        aa = new byte[length];
        bb = new byte[length];
        System.arraycopy(a, 0, aa, 0, a.length);
        System.arraycopy(b, 0, bb, 0, b.length);
    } else {
        aa = a;
        bb = b;
    }

    // convert to samples
    final int[] aSamples = toSamples(aa, bigEndian);
    final int[] bSamples = toSamples(bb, bigEndian);

    // mix by adding
    final int[] mix = new int[aSamples.length];
    for (int i=0; i<mix.length; i++) {
        mix[i] = aSamples[i] + bSamples[i];
        // enforce min and max (may introduce clipping)
        mix[i] = Math.min(Short.MAX_VALUE, mix[i]);
        mix[i] = Math.max(Short.MIN_VALUE, mix[i]);
    }

    // convert back to bytes
    return toBytes(mix, bigEndian);
}

private static int[] toSamples(final byte[] byteSamples, final boolean bigEndian) {
    final int bytesPerChannel = 2;
    final int length = byteSamples.length / bytesPerChannel;
    if ((length % 2) != 0) throw new IllegalArgumentException("For 16 bit audio, length must be even: " + length);
    final int[] samples = new int[length];
    for (int sampleNumber = 0; sampleNumber < length; sampleNumber++) {
        final int sampleOffset = sampleNumber * bytesPerChannel;
        final int sample = bigEndian
                ? byteToIntBigEndian(byteSamples, sampleOffset, bytesPerChannel)
                : byteToIntLittleEndian(byteSamples, sampleOffset, bytesPerChannel);
        samples[sampleNumber] = sample;
    }
    return samples;
}

private static byte[] toBytes(final int[] intSamples, final boolean bigEndian) {
    final int bytesPerChannel = 2;
    final int length = intSamples.length * bytesPerChannel;
    final byte[] bytes = new byte[length];
    for (int sampleNumber = 0; sampleNumber < intSamples.length; sampleNumber++) {
        final byte[] b = bigEndian
                ? intToByteBigEndian(intSamples[sampleNumber], bytesPerChannel)
                : intToByteLittleEndian(intSamples[sampleNumber], bytesPerChannel);
        System.arraycopy(b, 0, bytes, sampleNumber * bytesPerChannel, bytesPerChannel);
    }
    return bytes;
}

// from https://github.com/hendriks73/jipes/blob/master/src/main/java/com/tagtraum/jipes/audio/AudioSignalSource.java#L238
private static int byteToIntLittleEndian(final byte[] buf, final int offset, final int bytesPerSample) {
    int sample = 0;
    for (int byteIndex = 0; byteIndex < bytesPerSample; byteIndex++) {
        final int aByte = buf[offset + byteIndex] & 0xff;
        sample += aByte << 8 * (byteIndex);
    }
    return (short)sample;
}

// from https://github.com/hendriks73/jipes/blob/master/src/main/java/com/tagtraum/jipes/audio/AudioSignalSource.java#L247
private static int byteToIntBigEndian(final byte[] buf, final int offset, final int bytesPerSample) {
    int sample = 0;
    for (int byteIndex = 0; byteIndex < bytesPerSample; byteIndex++) {
        final int aByte = buf[offset + byteIndex] & 0xff;
        sample += aByte << (8 * (bytesPerSample - byteIndex - 1));
    }
    return (short)sample;
}

private static byte[] intToByteLittleEndian(final int sample, final int bytesPerSample) {
    byte[] buf = new byte[bytesPerSample];
    for (int byteIndex = 0; byteIndex < bytesPerSample; byteIndex++) {
        buf[byteIndex] = (byte)((sample >>> (8 * byteIndex)) & 0xFF);
    }
    return buf;
}

private static byte[] intToByteBigEndian(final int sample, final int bytesPerSample) {
    byte[] buf = new byte[bytesPerSample];
    for (int byteIndex = 0; byteIndex < bytesPerSample; byteIndex++) {
        buf[byteIndex] = (byte)((sample >>> (8 * (bytesPerSample - byteIndex - 1))) & 0xFF);
    }
    return buf;
}

我得到的混合样本既有失真又有噪音。无法弄清楚需要做些什么来去除它。非常感谢这里的任何帮助。提前谢谢！

Answer 1

我认为如果你在混合，你应该取两者的（加权）平均值。

如果您有一个128和128的样本，结果仍然是128，而不是256，这可能超出范围。

所以只需将代码更改为：

// mix by adding
final int[] mix = new int[aSamples.length];
for (int i=0; i<mix.length; i++) {
    // calculating the average
    mix[i] = (aSamples[i] + bSamples[i]) >> 1;
}

这对你有用吗？