使用谷歌语音到文本的实时转录

Question

我想用nodejs和googlespeech-to-text api制作一个实时转录应用程序。

我正在使用RecordRTC和socket。io将音频块获取到后端服务器。目前，我正在录制1秒长的块，并且转录工作正常，但它没有将其视为流，而是在处理每个块后发送响应。这意味着我要收回半句话，谷歌无法利用上下文帮助自己识别演讲。

我的问题是，如何让谷歌将我的块视为一个连续的流。或者是否有其他解决方案可以达到相同的结果？（正在现场录制麦克风音频，或非常接近现场）。

谷歌在他们的网站上有一个演示，它完全符合我的要求，所以应该可以做到这一点。

我的代码：（主要来自selfservicekiosk音频流媒体回购）

ss是插座。io流

服务器端

io.on("connect", (socket) => {
        socket.on("create-room", (data, cb) => createRoom(socket, data, cb))
        socket.on("disconnecting", () => exitFromRoom(socket))

        // getting the stream, it gets called every 1s with a blob
        ss(socket).on("stream-speech", async function (stream: any, data: any) {

            const filename = path.basename("stream.wav")
            const writeStream = fs.createWriteStream(filename)
           
            stream.pipe(writeStream)
            speech.speechStreamToText(
                stream,
                async function (transcribeObj: any) {
                    socket.emit("transcript", transcribeObj.transcript)
                }
            )
        })

async speechStreamToText(stream: any, cb: Function) {
        sttRequest.config.languageCode = "en-US"

        sttRequest = {
            config: {
                sampleRateHertz: 16000,
                encoding: "WEBM_OPUS",
                enableAutomaticPunctuation: true,
            },
            singleUtterance: false,
        }

        const stt = speechToText.SpeechClient()
        //setup the stt stream
        const recognizeStream = stt
            .streamingRecognize(sttRequest)
            .on("data", function (data: any) {
                //this gets called every second and I get transciption chunks which usually have close to no sense
                console.log(data.results[0].alternatives)
            })
            .on("error", (e: any) => {
                console.log(e)
            })
            .on("end", () => {
                //this gets called every second. 
                console.log("on end")
            })

        stream.pipe(recognizeStream)
        stream.on("end", function () {
            console.log("socket.io stream ended")
        })
    }

客户端

const sendBinaryStream = (blob: Blob) => {
    const stream = ss.createStream()
    ss(socket).emit("stream-speech", stream, {
        name: "_temp/stream.wav",
        size: blob.size,
    })
    ss.createBlobReadStream(blob).pipe(stream)
}

useEffect(() => {
        let recorder: any
        if (activeChat) {
            navigator.mediaDevices.getUserMedia({ audio: true, video: false }).then((stream) => {
                streamRef.current = stream
                recorder = new RecordRTC(stream, {
                    type: "audio",
                    mimeType: "audio/webm",
                    sampleRate: 44100,
                    desiredSampleRate: 16000,
                    timeSlice: 1000,
                    numberOfAudioChannels: 1,
                    recorderType: StereoAudioRecorder,
                    ondataavailable(blob: Blob) {
                        sendBinaryStream(blob)
                    },
                })
                recorder.startRecording()
            })
        }
        return () => {
            recorder?.stopRecording()
            streamRef.current?.getTracks().forEach((track) => track.stop())
        }
    }, [])

感谢您的帮助！

Answer 1

我有完全相同的问题！

也许谷歌的演示是使用node-rece-lpcm16与SoX：https://cloud.google.com/speech-to-text/docs/streaming-recognize?hl=en