我如何通过麦克风为我网页上的用户自动填写表格？

Question

我有一个带烧瓶的网页表单。目前，用户需要手动将他们的信息输入网页。然后，它被附加到一个表中，一旦单击提交，它们将被重定向到该表。设置基本上是：视频自动播放并向用户提问，用户手动填写他们的答案，一旦点击提交，他们会看到他们的答案被附加到一个表格中。

我想减少页面的混乱，让用户可以口头回答视频问题。我已经阅读了getusermedia、webSocket和WebRTCs，但我对它们感到困惑。我到处都找过了，youtube，reddit，等等。具体来说，这里，这里，这里，这里没有太多运气。

我正在考虑一个简单的for循环，带有语音识别器，在dict中使用不同的变量，然后按原样传递数据，但我不确定如何将麦克风动作与前端连接起来。前端不是所有数据所在的地方，所以我们需要一个http请求来获取和分析它？这是我的密码：

主要的py:

from flask import render_template, Flask, request
import os
from vaderSentiment.vaderSentiment import SentimentIntensityAnalyzer as SIA
import nltk
import io
import os
from nltk.corpus import stopwords
import speech_recognition as sr

app = Flask(__name__, static_folder = 'static')

# # set the stopwords to be the english version
# stop_words = set(stopwords.words("english"))
# # create the recognizer
# r = sr.Recognizer()
# # define the microphone
# mic = sr.Microphone(device_index=0)
# r.energy_threshold = 300
# # vader sentiment analyzer for analyzing the sentiment of the text
# sid = SIA()
# user = []
# location = []
# state = []
# info = [user, location, state]
# # patient.name?


@app.route("/", methods=["GET", "POST"])
def home():
    user = request.values.get('name')
    location = request.values.get('location')
    state = request.values.get('state')
    # if request.method == "POST":
        # with mic as source:
        #     holder = []
        #     for x in info:
        #         audio_data = r.listen(source)
        #         r.adjust_for_ambient_noise(source)
        #         text = r.recognize_google(audio_data, language = 'en-IN')
        #         holder.append(text.lower())
        #         if x == "state":
        #             ss = sid.polarity_scores(holder)
        #             if ss == "neg":
        #                 x.append(str("sad"))
        #             else:
        #                 x.append(str("not sad"))
        #         else:
        #             filtered_words = [words for words in holder if not words in stop_words] # this filters out the stopwords
        #             x.append(filtered_words.lower())

        # return redirect(url_for('care', user = user))

    return render_template('index.html', user = user, location=location, state=state)

@app.route("/care", methods=["POST"])
def care():
    user = request.values.get('name')
    location = request.values.get('location')
    state = request.values.get('state')
    return render_template('list.html', user = user, location=location, state=state)


if __name__ == "__main__":
    #app.run(debug=True)    
    app.run(debug=True, threaded=True)

指数html:

{% extends "base.html" %}
{% block content %}

<!---------Therapist Section--------->
    <section id="therapist">
        <div class="container" id="therapist_container">
            <script>
              window.onload = function() {
            </script>
            <div id="button">
              <button type="button" class="btn btn-primary" id="therapist-button" data-toggle="modal" data-target="#myModal">Talk with Delphi</button>
            </div>
            
            <!-- Modal -->
            <div class="modal fade" id="myModal" tabindex="-1" role="dialog" aria-labelledby="vid1Title" aria-hidden="true">
              <div class="modal-dialog modal-dialog-centered" role="document">
                <div class="modal-content">
                  <div class="modal-body">
                    <video width="100%" id="video1">
                      <source src="./static/movie.mp4" type="video/mp4">
                    </video>
                    <form action="/care" method="POST">
                      <input type="text" name="name" placeholder="what's your name?" id="name">
                      <input type="text" name="location" placeholder="Where are you?" id="location">
                      <input type="text" name="state" placeholder="how can I help?" id="state">
                      <input id="buttonInput" class="btn btn-success form-control" type="submit" value="Send">
                    </form>
                  </div>
                </div>
              </div>
            </div>
            <script>
              $('#myModal').on('shown.bs.modal', function () {
              $('#video1')[0].play();
              })
              $('#myModal').on('hidden.bs.modal', function () {
                $('#video1')[0].pause();
              })
              video = document.getElementById('video1');
              video.addEventListener('ended',function(){       
              window.location.pathname = '/care';})

              function callback(stream) {
                  var context = new webkitAudioContext();
                  var mediaStreamSource = context.createMediaStreamSource(stream);
              }

              $(document).ready(function() {
                  navigator.webkitGetUserMedia({audio:true}, callback);
              }

            </script>
        </div>
    </section>
{% endblock content %}

list.html：

{% extends "base.html" %}
{% block content %}

<!----LIST------>
<section id="care_list">
    <div class="container" id="care_list_container">
        <h1 class="jumbotron text-center" id="care_list_title">{{ user }} Care Record</h1>
        <div class="container">
            <table class="table table-hover"> 
                <thead>
                  <tr>
                    <th scope="col">Session #</th>
                    <th scope="col">Length</th>
                    <th scope="col">Location</th>
                    <th scope="col">State</th> 
                  </tr>
                </thead>
                <tbody>
                  <tr>
                    <th scope="row">1</th>
                    <td>{{ length }}</td>
                    <td>{{ location }}</td>
                    <td>{{ state }}</td>
                  </tr>
                  <tr>
                    <th scope="row">2</th>
                    <td></td>
                    <td></td>
                    <td></td>
                  </tr>
                  <tr>
                    <th scope="row">3</th>
                    <td colspan="2"></td>
                    <td></td>
                  </tr>
                </tbody>
              </table>
        <ul class="list-group list-group-flush" id="care_list">
            <li class="list-group-item">Please email [e-mail] for help.</li>
        </ul>
    </div> 
</section> 
{% endblock content %}

Answer 1

实际上，您不能使用flask进行语音识别。Flask是一个后端框架，在托管它的服务器上运行。因为您希望用户所说的话能够被识别，所以需要使用客户端的东西，即使用JavaScript。您可以使用本教程完成任务。

Answer 2

这比我们想象的要容易得多，就像我们创建newarray（），有一个codenewspeechrecognition（）来创建语音到文本转换器。不需要外部库来执行此操作。以下是代码：-

            /* JS comes here */
            function SpeechRecog() {
                var output = document.getElementById("output");
                var action = document.getElementById("action");
                var SpeechRecognition = SpeechRecognition || webkitSpeechRecognition;
                var recognition = new SpeechRecognition();
            
                // This runs when the speech recognition service starts
                recognition.onstart = function() {
                    action.innerHTML = "<small>listening, please speak...</small>";
                };
                
                recognition.onspeechend = function() {
                    action.innerHTML = "<small>stopped listening, hope you are done...</small>";
                    recognition.stop();
                }
              
                // This runs when the speech recognition service returns result
                recognition.onresult = function(event) {
                    var transcript = event.results[0][0].transcript;
                    var confidence = event.results[0][0].confidence;
                    output.value=transcript;
                };
              
                 // start recognition
                 recognition.start();
            }

css lang-css prettyprint-override">button{
  color:white;
  background:blue;
  border:none;
  padding:10px;margin:5px;
  border-radius:1em;
}
input{
  padding:.5em;margin:.5em;
}

<p>I'm Aakash1282,<br> Are you lazy, here is voice writer for your</p>
<p><button type="button" onclick="SpeechRecog()">Write By Voice</button> &nbsp; <span id="action"></span></p>
        <input type="text" id="output">