Python-docs-samples: Unable to install required packages in NAO Robot (based on Gentoo OS)

[chan71]

In which file did you encounter the issue?

https://github.com/GoogleCloudPlatform/python-docs-samples/blob/master/speech/grpc/requirements.txt

Did you change the file? If so, how?

No

Describe the issue

This is a sequel of the issues #723 that was closed 2 days back. I could have added this under same issue had it not been closed as both issues are related. #723 is about compiler errors occurred during running pip install -r requirements.txt.

This issue is about what we have observed when the requirements were installed in NAO Robot. We got into few issues as emerge package manager is not available in the NAO OS (supposed to be similar to opennao VM).

These are the steps followed in the installation.

1. Check pipversion =9.0.1 - OK
2. Install virtualenv- OK
3. Install portaudiousing emerge– Fail as emerge is not available in usr/bin
4. Install pyaudiousing emerge– Not Started yet
5. Install package as per requirements.txt
   grpcio==1.0.4 – Not Started yet
   PyAudio==0.2.9 – Not Started yet
   grpc-google-cloud-speech-v1beta1==0.14.0 – Not Started yet
   six==1.10.0 – Not Started yet
   requests==2.12.4 – Not Started yet
   google-auth==0.5.0 – Not Started yet
6. python transcribe_streaming.py – Not Started yet.

Then we copied the portaudiolibrary (portaudio-19_pre20071207.tbz2) from opennao vm to Robot and ran pip install.

(env) nao [0] ~/googlespeech $ pip install -r requirements.txt
Collecting PyAudio==0.2.9 (from -r requirements.txt (line 2))
/var/persistent/home/nao/googlespeech/env/lib/python2.7/site-packages/pip/_vendor/requests/packages/urllib3/util/ssl_.py:318: SNIMissingWarning: An HTTPS request has been made, but the SNI (Subject Name Indication) extension to TLS is not available on this platform. This may cause the server to present an incorrect TLS certificate, which can cause validation failures. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/security.html#snimissingwarning.
  SNIMissingWarning
/var/persistent/home/nao/googlespeech/env/lib/python2.7/site-packages/pip/_vendor/requests/packages/urllib3/util/ssl_.py:122: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/security.html#insecureplatformwarning.
  InsecurePlatformWarning
  Using cached PyAudio-0.2.9.tar.gz
Building wheels for collected packages: PyAudio
  Running setup.py bdist_wheel for PyAudio ... error
  Complete output from command /var/persistent/home/nao/googlespeech/env/bin/python2.7 -u -c "import setuptools, tokenize;__file__='/var/volatile/tmp/pip-build-SqC1D8/PyAudio/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" bdist_wheel -d /tmp/tmp4BLE1Upip-wheel- --python-tag cp27:
  running bdist_wheel
  running build
  running build_py
  creating build
  creating build/lib.linux-i686-2.7
  copying src/pyaudio.py -> build/lib.linux-i686-2.7
  running build_ext
  building '_portaudio' extension
  creating build/temp.linux-i686-2.7
  creating build/temp.linux-i686-2.7/src
  i686-pc-linux-gnu-gcc -pthread -fPIC -I/usr/include/python2.7 -c src/_portaudiomodule.c -o build/temp.linux-i686-2.7/src/_portaudiomodule.o
  unable to execute i686-pc-linux-gnu-gcc: No such file or directory
  error: command 'i686-pc-linux-gnu-gcc' failed with exit status 1

  ----------------------------------------
  Failed building wheel for PyAudio
  Running setup.py clean for PyAudio
Failed to build PyAudio
Installing collected packages: PyAudio
  Running setup.py install for PyAudio ... error
    Complete output from command /var/persistent/home/nao/googlespeech/env/bin/python2.7 -u -c "import setuptools, tokenize;__file__='/var/volatile/tmp/pip-build-SqC1D8/PyAudio/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" install --record /tmp/pip-dluM0A-record/install-record.txt --single-version-externally-managed --compile --install-headers /var/persistent/home/nao/googlespeech/env/include/site/python2.7/PyAudio:
    running install
    running build
    running build_py
    creating build
    creating build/lib.linux-i686-2.7
    copying src/pyaudio.py -> build/lib.linux-i686-2.7
    running build_ext
    building '_portaudio' extension
    creating build/temp.linux-i686-2.7
    creating build/temp.linux-i686-2.7/src
    i686-pc-linux-gnu-gcc -pthread -fPIC -I/usr/include/python2.7 -c src/_portaudiomodule.c -o build/temp.linux-i686-2.7/src/_portaudiomodule.o
    unable to execute i686-pc-linux-gnu-gcc: No such file or directory
    error: command 'i686-pc-linux-gnu-gcc' failed with exit status 1

    ----------------------------------------
Command "/var/persistent/home/nao/googlespeech/env/bin/python2.7 -u -c "import setuptools, tokenize;__file__='/var/volatile/tmp/pip-build-SqC1D8/PyAudio/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" install --record /tmp/pip-dluM0A-record/install-record.txt --single-version-externally-managed --compile --install-headers /var/persistent/home/nao/googlespeech/env/include/site/python2.7/PyAudio" failed with error code 1 in /var/volatile/tmp/pip-build-SqC1D8/PyAudio/

Ignoring this error (should not have been done) we ran the python transcribe_streaming.py and ended up getting below error.

We checked if PyAudioversion is correct using pip freeze and found the following packages present in Robot.

(env) nao [err 1] ~/googlespeech $ pip freeze
enum34==1.1.6
futures==3.0.5
google-auth==0.5.0
googleapis-common-protos==1.5.0
grpc-google-cloud-speech-v1beta1==0.14.0
grpcio==1.0.4
httplib2==0.9.2
oauth2client==3.0.0
protobuf==3.1.0.post1
pyasn1==0.1.9
pyasn1-modules==0.0.8
requests==2.12.4
rsa==3.4.2
six==1.10.0

My question is how to run pip install -r requirements.txt without an error so that PyAudio is installed properly. And I hope it will solve this issue. Appreciate any support.

[jerjou]

Hm... interesting. It looks like other folk have encountered this issue with portaudio on Nao as well. From the looks of your trace, it seems like pyaudio is attempting to build an extension in addition to the actual portaudio install - indeed, when I look at my virtualenv, I see a portaudio library within it:

$ find venv -type f | grep portaudio
venv/lib/python2.7/site-packages/_portaudio.so

Assuming the Nao virtual machine that you're building stuff on is the same architecture as the Nao that you're loading your code onto, I'd suggest just doing the pip install on your VM, and then copying the entire virtualenv onto your Nao (in addition to the portaudio binary install that you're doing). This way you don't even have to do the pip install on your Nao, which should sidestep the whole compilation problem. ie:

vm:~$ virtualenv venv
vm:~$ source venv/bin/activate
(venv) vm:~$ pip install -r requirements.txt
...

and then copy the entire venv directory onto your Nao. Then, on your Nao:

nao:~$ source venv/bin/activate
(venv) nao:~$ python speech_streaming.py

[theacodes]

If you copy the venv, be sure to run virtualenv --relocatable ENV or better yet use pip wheel.

[chan71]

@jerjou thanks for the suggestions. We followed a similar path yesterday to copy portaudio and pyaudio files from NAO VM to NAO. These are the files we copied.

/usr/include  
-rw-r--r-- 1 root root 43092 Dec 16 18:13 portaudio.h
drwxr-xr-x 2 root root  4096 Dec 16 18:14 portaudiocpp

/usr/include/portaudiocpp 
-rw-r--r-- 1 root root 1645 Dec 16 18:14 AutoSystem.hxx
-rw-r--r-- 1 root root 1165 Dec 16 18:14 BlockingStream.hxx
-rw-r--r-- 1 root root 1602 Dec 16 18:14 CFunCallbackStream.hxx
-rw-r--r-- 1 root root 1502 Dec 16 18:14 CallbackInterface.hxx
-rw-r--r-- 1 root root  993 Dec 16 18:14 CallbackStream.hxx
-rw-r--r-- 1 root root 2601 Dec 16 18:14 CppFunCallbackStream.hxx
-rw-r--r-- 1 root root 2788 Dec 16 18:14 Device.hxx
-rw-r--r-- 1 root root 2366 Dec 16 18:14 DirectionSpecificStreamParameters.hxx
-rw-r--r-- 1 root root 3201 Dec 16 18:14 Exception.hxx
-rw-r--r-- 1 root root 1909 Dec 16 18:14 HostApi.hxx
-rw-r--r-- 1 root root 1427 Dec 16 18:14 InterfaceCallbackStream.hxx
-rw-r--r-- 1 root root 3422 Dec 16 18:14 MemFunCallbackStream.hxx
-rw-r--r-- 1 root root 5339 Dec 16 18:14 PortAudioCpp.hxx
-rw-r--r-- 1 root root  831 Dec 16 18:14 SampleDataFormat.hxx
-rw-r--r-- 1 root root 2185 Dec 16 18:14 Stream.hxx
-rw-r--r-- 1 root root 2605 Dec 16 18:14 StreamParameters.hxx
-rw-r--r-- 1 root root 2704 Dec 16 18:14 System.hxx
-rw-r--r-- 1 root root 1665 Dec 16 18:14 SystemDeviceIterator.hxx
-rw-r--r-- 1 root root 1542 Dec 16 18:14 SystemHostApiIterator.hxx

/usr/lib
-rw-r--r--  1 root root  138912 Dec 16 18:16 libportaudio.a
-rw-r--r--  1 root root     865 Dec 16 18:16 libportaudio.la
-rwxr-xr-x  1 root root  108840 Dec 16 18:16 libportaudio.so.2.0.0
-rw-r--r--  1 root root   99890 Dec 16 18:16 libportaudiocpp.a
-rw-r--r--  1 root root     896 Dec 16 18:17 libportaudiocpp.la
-rwxr-xr-x  1 root root   55028 Dec 16 18:17 libportaudiocpp.so.0.0.12
drwxr-xr-x  2 root root    4096 Dec 16 18:18 pkgconfig
lrwxrwxrwx  1 root root      21 Dec 16 19:16 libportaudio.so -> libportaudio.so.2.0.0
lrwxrwxrwx  1 root root      21 Dec 16 19:16 libportaudio.so.2 -> libportaudio.so.2.0.0
lrwxrwxrwx  1 root root      25 Dec 16 19:17 libportaudiocpp.so -> libportaudiocpp.so.0.0.12
lrwxrwxrwx  1 root root      25 Dec 16 19:17 libportaudiocpp.so.0 -> libportaudiocpp.so.0.0.12

/usr/lib/pkgconfig 
-rw-r--r--  1 root root   239 Dec 16 18:18 portaudio-2.0.pc
-rw-r--r--  1 root root   248 Dec 16 18:18 portaudiocpp.pc

Files copied are as below:-

_portaudio.so
pyaudio.py
pyaudio.pyc
PyAudio-0.2.9-py2.7.egg-info/PKG-INFO  SOURCES.txt  dependency_links.txt  installed-files.txt  top_level.txt

Now it is possible to play an audio file using pyaudio in NAO using following script.

import pyaudio
import wave
import sys

CHUNK = 1024

if len(sys.argv) < 2:
    print("Plays a wave file.\n\nUsage: %s filename.wav" % sys.argv[0])
    sys.exit(-1)

wf = wave.open(sys.argv[1], 'rb')

p = pyaudio.PyAudio()

stream = p.open(format=p.get_format_from_width(wf.getsampwidth()),
                channels=wf.getnchannels(),
                rate=wf.getframerate(),
                output=True)

data = wf.readframes(CHUNK)

while data != '':
    stream.write(data)
    data = wf.readframes(CHUNK)

stream.stop_stream()
stream.close()

p.terminate()

However, NAO mic is still not identified correctly when tried with this script.

import pyaudio
import wave

CHUNK = 1024
FORMAT = pyaudio.paInt16
CHANNELS = 1
RATE = 44100
RECORD_SECONDS = 5
WAVE_OUTPUT_FILENAME = "output.wav"

p = pyaudio.PyAudio()

stream = p.open(format=FORMAT,
                channels=CHANNELS,
                rate=RATE,
                input=True,
                input_device_index=0,
                frames_per_buffer=CHUNK)

print("* recording")

frames = []

for i in range(0, int(RATE / CHUNK * RECORD_SECONDS)):
    data = stream.read(CHUNK)
    frames.append(data)

print("* done recording")

stream.stop_stream()
stream.close()
p.terminate()

wf = wave.open(WAVE_OUTPUT_FILENAME, 'wb')
wf.setnchannels(CHANNELS)
wf.setsampwidth(p.get_sample_size(FORMAT))
wf.setframerate(RATE)
wf.writeframes(b''.join(frames))
wf.close()

Error description:

(env) nao [0] ~/googlespeech $ python pyaudio_test2.py
Expression 'parameters->channelCount <= maxChans' failed in 'src/hostapi/alsa/pa_linux_alsa.c', line: 924
Expression 'ValidateParameters( inputParameters, hostApi, StreamDirection_In )' failed in 'src/hostapi/alsa/pa_linux_alsa.c', line: 1961
Traceback (most recent call last):
  File "pyaudio_test2.py", line 18, in <module>
    frames_per_buffer=CHUNK)
  File "/var/persistent/home/nao/googlespeech/env/lib/python2.7/site-packages/pyaudio.py", line 750, in open
    stream = Stream(self, *args, **kwargs)
  File "/var/persistent/home/nao/googlespeech/env/lib/python2.7/site-packages/pyaudio.py", line 441, in __init__
    self._stream = pa.open(**arguments)
IOError: [Errno -9998] Invalid number of channels

Then we listed the IO devices attached to NAO.

import pyaudio

p = pyaudio.PyAudio()
device_count = p.get_device_count()
for i in range(0, device_count):
   print("Name: " + p.get_device_info_by_index(i)["name"])
   print("Index: " + str(p.get_device_info_by_index(i)["index"]))
   print("\n")
(env) nao [err 1] ~/googlespeech $ python pyaudio_no_dev.py

Name: HDA Intel MID: AD198x Analog (hw:0,0)
Index: 0

Name: AD1989A_outputs
Index: 1

Name: aldmix
Index: 2

Name: alduplex
Index: 3

We further investigated by changing input_device_index parameter p.open(...) method and also removed it all together hoping that the default input device will be taken. None worked.

When input devices are listed, we figured out that none has been identified.

>>>import pyaudio
>>>pa = pyaudio.PyAudio()
>>>pa.get_default_input_device_info()
Traceback (most recent call last):
 File "<stdin>", line 1, in <module>
 File "/var/persistent/home/nao/googlespeech/env/lib/python2.7/site-packages/pyaudio.py", line 949, in get_default_input_device_info
   device_index = pa.get_default_input_device()
IOError: No Default Input Device Available

However, output devices are correctly listed and that may be why NAO was able to play the audio but failed to recognize the mic.

>>>pa.get_default_output_device_info()
{'defaultSampleRate': 44100.0, 'defaultLowOutputLatency': 0.011609977324263039, 'defaultLowInputLatency': -1.0, 'maxInputChannels': 0L, 'structVersion': 2L, 'hostApi': 0L, 'index': 0L, 'defaultHighOutputLatency': 0.046439909297052155, 'maxOutputChannels': 2L, 'name': u'HDA Intel MID: AD198x Analog (hw:0,0)', 'defaultHighInputLatency': -1.0}

It looks like we have all files required to run the python transcribe_streaming.py in NAO now. But the index of the input device is not picked up for some reason. Any suggestions would be appreciated.

[chan71]

Furthermore, can arecord be used instead of pyaudio / portaudio for streaming with google speech API? We have tried this approach for other streaming APIs with success. Is there any sample written using arecord that can be reused streaming audio to google speech?

[chan71]

I commented pyaudio references in the record_audio(rate, chunk) method with the arecord code given below.

def record_audio(rate, chunk):
    """Opens a recording stream in a context manager."""
    # Create a thread-safe buffer of audio data
    buff = queue.Queue()

#    audio_interface = pyaudio.PyAudio()
#    audio_stream = audio_interface.open(
#        format=pyaudio.paInt16,
        # The API currently only supports 1-channel (mono) audio
        # https://goo.gl/z757pE
#        channels=1, rate=rate,
#        input=True, frames_per_buffer=chunk,
        # Run the audio stream asynchronously to fill the buffer object.
        # This is necessary so that the input device's buffer doesn't overflow
        # while the calling thread makes network requests, etc.
#        stream_callback=functools.partial(_fill_buffer, buff),
#    )

    reccmd = ["arecord", "-f", "S16_LE", "-r", "16000", "-t", "raw"]
    p = subprocess.Popen(reccmd, stdout=subprocess.PIPE)

    for num in range(1,10000):
        data = p.stdout.read(1024)
        print data

        try: buff.put(bytearray(data))
        except ssl.SSLError: pass

    p.kill()

    yield _audio_data_generator(buff)

#    audio_stream.stop_stream()
#    audio_stream.close()
    # Signal the _audio_data_generator to finish
    buff.put(None)
#    audio_interface.terminate()
# [END audio_stream]

Couldn't quite get it to work but hope this is a starting point. Appreciate your comments.

[jerjou]

Hm... yeah - we use pyaudio mainly because it's cross-platform, so we can have one sample for linux, osx, and windows. I'm not familiar with the Nao environment, but if it's got arecord / alsa, you might try using pyalsaaudio instead, especially if arecord is successfully identifying your input device. In principle you should be able to subprocess arecord and pipe the raw bytes to the buffer, though it might be a bit cleaner to use pyalsaaudio.

Anyway, your approach seems sound, and a good idea since portaudio isn't identifying your input device (perhaps due to having been compiled in the VM that has no input devices? Probably shouldn't be, but I dunno). It looks like your code is still in the "confirming this approach works" mode (you'll want to fill buff in a separate thread, for one), so I say go for it. All buff expects is raw samples in 1-channel, little-endian format at the specified rate, so it doesn't matter if it comes from pyaudio or directly from alsa.

Let us know how it goes!

[chan71]

I got transcribe_streaming.py to work with arecord and ubuntu 16.04 on Oracle VirtualBox (host windows 8). I could not get the CentOS 6.6. to be configured to recognize input device and hence ubuntu was chosen for testing with arecord.

I could run the modified script for 1 minute, but it crashed with an error afterwards. Can you explain what the Client GRPC deadline is and how to adjust it to keep a continuous conversation going?

(env) ck@ck-VirtualBox:~/nao-testing$ python transcribe_streaming_arecord.py 
about to start recording
recording in progress
inside function
Recording raw data 'stdin' : Signed 16 bit Little Endian, Rate 16000 Hz, Mono
Verizon account
 I want performance
 I want to pretend
 hello hello
 performance
 hello    
Traceback (most recent call last):
  File "transcribe_streaming_arecord.py", line 224, in <module>
    main()
  File "transcribe_streaming_arecord.py", line 215, in main
    listen_print_loop(recognize_stream)
  File "transcribe_streaming_arecord.py", line 164, in listen_print_loop
    raise RuntimeError('Server error: ' + resp.error.message)
RuntimeError: Server error: Client GRPC deadline too short. Should be at least: 3 * audio-duration + 5 seconds. Current deadline is: 185.99952546987333 second(s). Required at least: 187 second(s).

Modified file is attached as text.
transcribe_streaming_arecord.txt

[jerjou]

Yup! See #517

[SilviaSantano]

@chan71 according to http://doc.aldebaran.com/2-4/naoqi/audio/alaudiodevice.html#alaudiodevice the configuration 44100 one channel is not possible, I think this might be your problem. I got your script to work using 16000 and one channel with portaudio