Mavsdk: Trouble installing Python wrapper

Created on 5 Dec 2018  ·  78Comments  ·  Source: mavlink/MAVSDK

Hello again, I have another issue.
The backend installed OK after some effort. When installing the Python wrapper everything appeared to install OK. But when I tried to run the takeoff_and_landing script it threw this error:

joe@joe-VirtualBox:~/src/Firmware/DronecodeSDK-Python$ examples/takeoff_and_land.py
Traceback (most recent call last):
File "examples/takeoff_and_land.py", line 5, in 
from dronecode_sdk import connect as dronecode_sdk_connect
ModuleNotFoundError: No module named 'dronecode_sdk'

What next?
Joe

PS. I asked this same question in another thread that was closed, so it's probably best to create a new thread.

question
Source
picture joe-marrone

All 78 comments

Did you follow these steps? https://github.com/Dronecode/DronecodeSDK-Python#get-the-python-wrapper

And if any of them was not successful, could you please paste the output. You can paste code in between ```.

julianoes picture julianoes on 6 Dec 2018

Update: I think the pip3 install didn't work the first time so I reinstalled it and then tried running the script again, but now I get a different error.

Here is the entire build process taken from 

joe@joe-VirtualBox:~/src/Firmware/DronecodeSDK-Python$ ./other/tools/run_protoc.sh
[+] Installing the DronecodeSDK autogenerator
Processing /home/joe/src/Firmware/DronecodeSDK-Python/proto/pb_plugins
Collecting protobuf (from dcsdkgen===0.1a0)
  Using cached https://files.pythonhosted.org/packages/c2/f9/28787754923612ca9bfdffc588daa05580ed70698add063a5629d1a4209d/protobuf-3.6.1-cp36-cp36m-manylinux1_x86_64.whl
Collecting jinja2 (from dcsdkgen===0.1a0)
  Using cached https://files.pythonhosted.org/packages/7f/ff/ae64bacdfc95f27a016a7bed8e8686763ba4d277a78ca76f32659220a731/Jinja2-2.10-py2.py3-none-any.whl
Collecting setuptools (from protobuf->dcsdkgen===0.1a0)
  Using cached https://files.pythonhosted.org/packages/e7/16/da8cb8046149d50940c6110310983abb359bbb8cbc3539e6bef95c29428a/setuptools-40.6.2-py2.py3-none-any.whl
Collecting six>=1.9 (from protobuf->dcsdkgen===0.1a0)
  Using cached https://files.pythonhosted.org/packages/67/4b/141a581104b1f6397bfa78ac9d43d8ad29a7ca43ea90a2d863fe3056e86a/six-1.11.0-py2.py3-none-any.whl
Collecting MarkupSafe>=0.23 (from jinja2->dcsdkgen===0.1a0)
  Using cached https://files.pythonhosted.org/packages/08/04/f2191b50fb7f0712f03f064b71d8b4605190f2178ba02e975a87f7b89a0d/MarkupSafe-1.1.0-cp36-cp36m-manylinux1_x86_64.whl
Installing collected packages: setuptools, six, protobuf, MarkupSafe, jinja2, dcsdkgen
  Running setup.py install for dcsdkgen ... done
Successfully installed MarkupSafe-1.1.0 dcsdkgen-0.1a0 jinja2-2.10 protobuf-3.6.1 setuptools-40.6.2 six-1.11.0
You are using pip version 8.1.1, however version 18.1 is available.
You should consider upgrading via the 'pip install --upgrade pip' command.
[+] Done
[+] Generating plugins from 
./other/tools/run_protoc.sh: line 11: /home/joe/src/Firmware/DronecodeSDK-Python/other/tools/../..//dronecode_sdk/plugins/__init__.py: Permission denied
/home/joe/src/Firmware/DronecodeSDK-Python/other/tools/../..//dronecode_sdk/generated/telemetry_pb2.py: Permission deniedsed: can't read /home/joe/src/Firmware/DronecodeSDK-Python/other/tools/../..//dronecode_sdk/generated/telemetry_pb2_grpc.py: No such file or directory
 -> [+] Generated protobuf and gRPC bindings for telemetry
/home/joe/src/Firmware/DronecodeSDK-Python/other/tools/../..//dronecode_sdk/plugins/Telemetry.py: Permission deniedmv: cannot stat '/home/joe/src/Firmware/DronecodeSDK-Python/other/tools/../..//dronecode_sdk/plugins/Telemetry.py': No such file or directory
./other/tools/run_protoc.sh: line 52: /home/joe/src/Firmware/DronecodeSDK-Python/other/tools/../..//dronecode_sdk/plugins/__init__.py: Permission denied
 -> [+] Generated plugin for telemetry
/home/joe/src/Firmware/DronecodeSDK-Python/other/tools/../..//dronecode_sdk/generated/action_pb2.py: Permission deniedsed: can't read /home/joe/src/Firmware/DronecodeSDK-Python/other/tools/../..//dronecode_sdk/generated/action_pb2_grpc.py: No such file or directory
 -> [+] Generated protobuf and gRPC bindings for action
/home/joe/src/Firmware/DronecodeSDK-Python/other/tools/../..//dronecode_sdk/plugins/Action.py: Permission deniedmv: cannot stat '/home/joe/src/Firmware/DronecodeSDK-Python/other/tools/../..//dronecode_sdk/plugins/Action.py': No such file or directory
./other/tools/run_protoc.sh: line 52: /home/joe/src/Firmware/DronecodeSDK-Python/other/tools/../..//dronecode_sdk/plugins/__init__.py: Permission denied
 -> [+] Generated plugin for action
/home/joe/src/Firmware/DronecodeSDK-Python/other/tools/../..//dronecode_sdk/generated/camera_pb2.py: Permission deniedsed: can't read /home/joe/src/Firmware/DronecodeSDK-Python/other/tools/../..//dronecode_sdk/generated/camera_pb2_grpc.py: No such file or directory
 -> [+] Generated protobuf and gRPC bindings for camera
/home/joe/src/Firmware/DronecodeSDK-Python/other/tools/../..//dronecode_sdk/plugins/Camera.py: Permission deniedmv: cannot stat '/home/joe/src/Firmware/DronecodeSDK-Python/other/tools/../..//dronecode_sdk/plugins/Camera.py': No such file or directory
./other/tools/run_protoc.sh: line 52: /home/joe/src/Firmware/DronecodeSDK-Python/other/tools/../..//dronecode_sdk/plugins/__init__.py: Permission denied
 -> [+] Generated plugin for camera
/home/joe/src/Firmware/DronecodeSDK-Python/other/tools/../..//dronecode_sdk/generated/gimbal_pb2.py: Permission deniedsed: can't read /home/joe/src/Firmware/DronecodeSDK-Python/other/tools/../..//dronecode_sdk/generated/gimbal_pb2_grpc.py: No such file or directory
 -> [+] Generated protobuf and gRPC bindings for gimbal
/home/joe/src/Firmware/DronecodeSDK-Python/other/tools/../..//dronecode_sdk/plugins/Gimbal.py: Permission deniedmv: cannot stat '/home/joe/src/Firmware/DronecodeSDK-Python/other/tools/../..//dronecode_sdk/plugins/Gimbal.py': No such file or directory
./other/tools/run_protoc.sh: line 52: /home/joe/src/Firmware/DronecodeSDK-Python/other/tools/../..//dronecode_sdk/plugins/__init__.py: Permission denied
 -> [+] Generated plugin for gimbal
/home/joe/src/Firmware/DronecodeSDK-Python/other/tools/../..//dronecode_sdk/generated/core_pb2.py: Permission deniedsed: can't read /home/joe/src/Firmware/DronecodeSDK-Python/other/tools/../..//dronecode_sdk/generated/core_pb2_grpc.py: No such file or directory
 -> [+] Generated protobuf and gRPC bindings for core
/home/joe/src/Firmware/DronecodeSDK-Python/other/tools/../..//dronecode_sdk/plugins/Core.py: Permission deniedmv: cannot stat '/home/joe/src/Firmware/DronecodeSDK-Python/other/tools/../..//dronecode_sdk/plugins/Core.py': No such file or directory
./other/tools/run_protoc.sh: line 52: /home/joe/src/Firmware/DronecodeSDK-Python/other/tools/../..//dronecode_sdk/plugins/__init__.py: Permission denied
 -> [+] Generated plugin for core
/home/joe/src/Firmware/DronecodeSDK-Python/other/tools/../..//dronecode_sdk/generated/calibration_pb2.py: Permission deniedsed: can't read /home/joe/src/Firmware/DronecodeSDK-Python/other/tools/../..//dronecode_sdk/generated/calibration_pb2_grpc.py: No such file or directory
 -> [+] Generated protobuf and gRPC bindings for calibration
/home/joe/src/Firmware/DronecodeSDK-Python/other/tools/../..//dronecode_sdk/plugins/Calibration.py: Permission deniedmv: cannot stat '/home/joe/src/Firmware/DronecodeSDK-Python/other/tools/../..//dronecode_sdk/plugins/Calibration.py': No such file or directory
./other/tools/run_protoc.sh: line 52: /home/joe/src/Firmware/DronecodeSDK-Python/other/tools/../..//dronecode_sdk/plugins/__init__.py: Permission denied
 -> [+] Generated plugin for calibration
/home/joe/src/Firmware/DronecodeSDK-Python/other/tools/../..//dronecode_sdk/generated/discovery_pb2.py: Permission deniedsed: can't read /home/joe/src/Firmware/DronecodeSDK-Python/other/tools/../..//dronecode_sdk/generated/discovery_pb2_grpc.py: No such file or directory
 -> [+] Generated protobuf and gRPC bindings for discovery
/home/joe/src/Firmware/DronecodeSDK-Python/other/tools/../..//dronecode_sdk/plugins/Discovery.py: Permission deniedmv: cannot stat '/home/joe/src/Firmware/DronecodeSDK-Python/other/tools/../..//dronecode_sdk/plugins/Discovery.py': No such file or directory
./other/tools/run_protoc.sh: line 52: /home/joe/src/Firmware/DronecodeSDK-Python/other/tools/../..//dronecode_sdk/plugins/__init__.py: Permission denied
 -> [+] Generated plugin for discovery
/home/joe/src/Firmware/DronecodeSDK-Python/other/tools/../..//dronecode_sdk/generated/mission_pb2.py: Permission deniedsed: can't read /home/joe/src/Firmware/DronecodeSDK-Python/other/tools/../..//dronecode_sdk/generated/mission_pb2_grpc.py: No such file or directory
 -> [+] Generated protobuf and gRPC bindings for mission
/home/joe/src/Firmware/DronecodeSDK-Python/other/tools/../..//dronecode_sdk/plugins/Mission.py: Permission deniedmv: cannot stat '/home/joe/src/Firmware/DronecodeSDK-Python/other/tools/../..//dronecode_sdk/plugins/Mission.py': No such file or directory
./other/tools/run_protoc.sh: line 52: /home/joe/src/Firmware/DronecodeSDK-Python/other/tools/../..//dronecode_sdk/plugins/__init__.py: Permission denied
 -> [+] Generated plugin for mission
[+] Done
joe@joe-VirtualBox:~/src/Firmware/DronecodeSDK-Python$ pip3 install -e .
Obtaining file:///home/joe/src/Firmware/DronecodeSDK-Python
    Complete output from command python setup.py egg_info:
    running egg_info
    /home/joe/.local/lib/python3.6/site-packages/setuptools/dist.py:470: UserWarning: Normalizing '0.1.0a' to '0.1.0a0'
      normalized_version,
    error: [Errno 13] Permission denied

    ----------------------------------------
Command "python setup.py egg_info" failed with error code 1 in /home/joe/src/Firmware/DronecodeSDK-Python/
You are using pip version 8.1.1, however version 18.1 is available.
You should consider upgrading via the 'pip install --upgrade pip' command.
joe@joe-VirtualBox:~/src/Firmware/DronecodeSDK-Python$ sudo pip3 install -e .
[sudo] password for joe: 
The directory '/home/joe/.cache/pip/http' or its parent directory is not owned by the current user and the cache has been disabled. Please check the permissions and owner of that directory. If executing pip with sudo, you may want sudo's -H flag.
The directory '/home/joe/.cache/pip' or its parent directory is not owned by the current user and caching wheels has been disabled. check the permissions and owner of that directory. If executing pip with sudo, you may want sudo's -H flag.
Obtaining file:///home/joe/src/Firmware/DronecodeSDK-Python
Requirement already satisfied (use --upgrade to upgrade): protobuf>=3.5.1 in /home/joe/.local/lib/python3.6/site-packages (from dronecode-sdk===0.1.0a0)
Requirement already satisfied (use --upgrade to upgrade): aiogrpc>=1.5 in /home/joe/.local/lib/python3.6/site-packages (from dronecode-sdk===0.1.0a0)
Requirement already satisfied (use --upgrade to upgrade): grpcio>=1.11.0 in /home/joe/.local/lib/python3.6/site-packages (from dronecode-sdk===0.1.0a0)
Requirement already satisfied (use --upgrade to upgrade): setuptools in /home/joe/.local/lib/python3.6/site-packages (from protobuf>=3.5.1->dronecode-sdk===0.1.0a0)
Requirement already satisfied (use --upgrade to upgrade): six>=1.9 in /home/joe/.local/lib/python3.6/site-packages (from protobuf>=3.5.1->dronecode-sdk===0.1.0a0)
Installing collected packages: dronecode-sdk
  Running setup.py develop for dronecode-sdk
Successfully installed dronecode-sdk-0.1.0a0
You are using pip version 8.1.1, however version 18.1 is available.
You should consider upgrading via the 'pip install --upgrade pip' command.
joe@joe-VirtualBox:~/src/Firmware/DronecodeSDK-Python$ examples/takeoff_and_land.py
Traceback (most recent call last):
  File "examples/takeoff_and_land.py", line 7, in <module>
    drone = dronecode_sdk_connect(host="127.0.0.1")
  File "/home/joe/src/Firmware/DronecodeSDK-Python/dronecode_sdk/__init__.py", line 49, in connect
    globals()[plugin](plugin_manager)
KeyError: 'Action'
joe@joe-VirtualBox:~/s

I've edited your post and put ``` around your code, so now I can actually read it.

joe-marrone picture joe-marrone on 6 Dec 2018

Sorry about my last post. I don't know why the text has strikeouts, but I must have hit a key by mistake.

Joe

joe-marrone picture joe-marrone on 6 Dec 2018

Joe again. In the process of building the wrapper I got several comments that I was using pip version 8.1.1 and that v18.1 is available. When I tried to update this is what I got:

 joe@joe-VirtualBox:~/src/Firmware/DronecodeSDK-Python$ pip install --upgrade pipRequirement already up-to-date: pip in /usr/local/lib/python2.7/dist-packages (18.1)

Can anyone explain why this happened?

Thanks,
Joe

Edit: adding more of these ```.

joe-marrone picture joe-marrone on 6 Dec 2018

So, you really have to use pip3. As long as you do things with pip (which is usually pip2) it won't work.

And we can see that it goes wrong early:

/home/joe/src/Firmware/DronecodeSDK-Python/other/tools/../..//dronecode_sdk/plugins/__init__.py: Permission denied
/home/joe/src/Firmware/DronecodeSDK-Python/other/tools/../..//dronecode_sdk/generated/telemetry_pb2.py: Permission denied
sed: can't read /home/joe/src/Firmware/DronecodeSDK-Python/other/tools/../..//dronecode_sdk/generated/telemetry_pb2_grpc.py: No such file or directory

First thing I see is that your path is a bit weird. DronecodeSDK-Python is somehow inside your Firmware folder. Is that on purpose?
If not I'd probably delete that folder and go back to src and start over there.

Also, somehow there are permission problems, I would recommend you try to fix these first:

cd DronecodeSDK-Python
sudo chown -R $USER
julianoes picture julianoes on 6 Dec 2018

I'll reorg the file system, but where exactly does DronecodeSDK-Python belong?

joe-marrone picture joe-marrone on 6 Dec 2018

Just in src I'd say where you have the different repositories.

julianoes picture julianoes on 6 Dec 2018

@julianoes: Some progress today. It was not my intention to install the Python wrapper in the Firmware directory, so not sure why I thought it had to be there. I deleted it and rebuilt the backend in a new directory. Then I installed the Python wrapper in the src directory, and it installed OK. But then when I ran the takeoff_and_land.py script, this is the result:

joe@joe-VirtualBox:~/src/DronecodeSDK-Python$ examples/takeoff_and_land.py
Traceback (most recent call last):
  File "examples/takeoff_and_land.py", line 29, in <module>
    loop.run_until_complete(run())
  File "/usr/lib/python3.6/asyncio/base_events.py", line 473, in run_until_complete
    return future.result()
  File "examples/takeoff_and_land.py", line 12, in run
    arm_result, _ = await drone.action.arm()
  File "/home/joe/src/DronecodeSDK-Python/dronecode_sdk/plugins/action.py", line 64, in arm
    response = await self._stub.Arm(action_pb2.ArmRequest())
  File "/home/joe/.local/lib/python3.6/site-packages/aiogrpc/channel.py", line 40, in __call__
    return await fut
grpc._channel._Rendezvous: <_Rendezvous of RPC that terminated with:
    status = StatusCode.UNAVAILABLE
    details = "Connect Failed"
    debug_error_string = "{"created":"@1544125638.371795599","description":"Failed to create subchannel","file":"src/core/ext/filters/client_channel/client_channel.cc","file_line":2706,"referenced_errors":[{"created":"@1544125638.371791535","description":"Pick Cancelled","file":"src/core/ext/filters/client_channel/lb_policy/pick_first/pick_first.cc","file_line":242,"referenced_errors":[{"created":"@1544125638.371758821","description":"Connect Failed","file":"src/core/ext/filters/client_channel/subchannel.cc","file_line":867,"grpc_status":14,"referenced_errors":[{"created":"@1544125638.370876362","description":"Failed to connect to remote host: OS Error","errno":111,"file":"src/core/lib/iomgr/tcp_client_posix.cc","file_line":207,"os_error":"Connection refused","syscall":"connect","target_address":"ipv4:127.0.0.1:50051"}]}]}]}"

I would appreciate your looking at this to see what might be the problem.

joe-marrone picture joe-marrone on 6 Dec 2018

You need to run the C++ backend (which you successfully compiled, I believe). From the "DronecodeSDK" folder, that's something like:

$ ./build/default/backend/backend_bin

The backend will connect to your drone/SITL instance, and the python SDK will connect to the backend.

JonasVautherin picture JonasVautherin on 6 Dec 2018

@julianoes : I did run the backend program and it linked to SITL. But the takeoff_and_land.py script did not execute. I have DronecodeSDK and DronecodeSDK-Python in different folders. Is this the problem?

home/joe/DronecodeSDK home/joe/src/DronecodeSDK-Python

Joe

joe-marrone picture joe-marrone on 7 Dec 2018

Re: last post:

The files should read:

home/joe/DronecodeSDK
home/joe/src/DronecodeSDK-Python

Should the DronecodeSDK and DronecodeSDK-Python files be in the same directory?
Joe

joe-marrone picture joe-marrone on 7 Dec 2018

No that's correct.

Is backend_bin running when you start the Python file?

julianoes picture julianoes on 7 Dec 2018

With PX4 SITL running, I start the backend and can see the link established . I can see the backend giving mirror commands to what is on the SITL screen. In another terminal I launch examples/takeoff_and_land.py. This results in the following error.

joe@joe-VirtualBox:~/src/DronecodeSDK-Python$ examples/takeoff_and_land.py
Traceback (most recent call last):
  File "examples/takeoff_and_land.py", line 29, in <module>
    loop.run_until_complete(run())
  File "/usr/lib/python3.6/asyncio/base_events.py", line 473, in run_until_complete
    return future.result()
  File "examples/takeoff_and_land.py", line 12, in run
    arm_result, _ = await drone.action.arm()
  File "/home/joe/src/DronecodeSDK-Python/dronecode_sdk/plugins/action.py", line 64, in arm
    response = await self._stub.Arm(action_pb2.ArmRequest())
  File "/home/joe/.local/lib/python3.6/site-packages/aiogrpc/channel.py", line 40, in __call__
    return await fut
grpc._channel._Rendezvous: <_Rendezvous of RPC that terminated with:
    status = StatusCode.UNIMPLEMENTED
    details = ""
    debug_error_string = "{"created":"@1544197359.037579442","description":"Error received from peer","file":"src/core/lib/surface/call.cc","file_line":1036,"grpc_message":"","grpc_status":12}"

Joe

joe-marrone picture joe-marrone on 7 Dec 2018

Ok cool, so now we're one step closer. Thanks for doing the debugging with us.
@JonasVautherin coult it be that we forgot to update some Python stuff after proto changes? :fearful:

julianoes picture julianoes on 7 Dec 2018

Just to keep the conversation going, I have some other files that you might want to see. First is the backend chatter when communicating with jMAVSim.

joe@joe-VirtualBox:~/DronecodeSDK$ ./build/default/backend/src/backend_bin
[10:35:34|Info ] Waiting to discover system... (connection_initiator.h:57)
[10:35:34|Info ] New device on: 127.0.0.1:14580 (udp_connection.cpp:208)
[10:35:34|Debug] New: System ID: 1 Comp ID: 1 (dronecore_impl.cpp:292)
[10:35:34|Debug] Component Autopilot added. (system_impl.cpp:339)
[10:35:34|Debug] MAVLink: emergency: Accel #0 fail:  TIMEOUT! (system_impl.cpp:277)
[10:35:34|Debug] MAVLink: emergency: Gyro #0 fail:  TIMEOUT! (system_impl.cpp:277)
[10:35:34|Debug] MAVLink: emergency: Accel #1 fail:  TIMEOUT! (system_impl.cpp:277)
[10:35:34|Debug] MAVLink: emergency: Accel #0 fail:  TIMEOUT! (system_impl.cpp:277)
[10:35:34|Debug] MAVLink: emergency: Gyro #0 fail:  TIMEOUT! (system_impl.cpp:277)
[10:35:34|Debug] Found 1 component(s). (system_impl.cpp:466)
[10:35:34|Debug] Discovered 5283920058631409231 (system_impl.cpp:468)
[10:35:34|Info ] System discovered [UUID: 5283920058631409231] (connection_initiator.h:61)
[10:35:34|Info ] Server set to listen on 0.0.0.0:50051 (grpc_server.cpp:39)
[10:35:34|Info ] Server started (grpc_server.cpp:23)
[10:35:34|Warn ] command failed (521). (mavlink_commands.cpp:189)

[10:39:19|Debug] MAVLink: emergency: Accel #0 fail:  TIMEOUT! (system_impl.cpp:277)
[10:39:19|Debug] MAVLink: emergency: Gyro #0 fail:  TIMEOUT! (system_impl.cpp:277)
[10:42:44|Debug] MAVLink: emergency: Accel #0 fail:  TIMEOUT! (system_impl.cpp:277)
[10:42:44|Debug] MAVLink: emergency: Gyro #0 fail:  TIMEOUT! (system_impl.cpp:277)
[10:42:45|Debug] MAVLink: emergency: Accel #0 fail:  TIMEOUT! (system_impl.cpp:277)
[10:42:45|Debug] MAVLink: emergency: Gyro #0 fail:  TIMEOUT! (system_impl.cpp:277)
[10:43:25|Debug] MAVLink: info: ARMED by command line (system_impl.cpp:277)
[10:43:25|Debug] MAVLink: info: [logger] file: ./log/2018-12-07/15_43_25.ulg (system_impl.cpp:277)
[10:43:25|Debug] MAVLink: info: Using minimum takeoff altitude: 2.50 m (system_impl.cpp:277)
[10:43:25|Debug] MAVLink: info: Takeoff detected (system_impl.cpp:277)
[10:44:47|Debug] MAVLink: info: Landing at current position (system_impl.cpp:277)
[10:44:53|Debug] MAVLink: info: Landing detected (system_impl.cpp:277)
[10:44:53|Debug] MAVLink: info: DISARMED by auto disarm on land (system_impl.cpp:277)
[10:48:53|Debug] MAVLink: emergency: Accel #0 fail:  TIMEOUT! (system_impl.cpp:277)
[10:48:53|Debug] MAVLink: emergency: Gyro #0 fail:  TIMEOUT! (system_impl.cpp:277)
[10:49:15|Debug] MAVLink: emergency: Accel #0 fail:  TIMEOUT! (system_impl.cpp:277)
[10:49:15|Debug] MAVLink: emergency: Gyro #0 fail:  TIMEOUT! (system_impl.cpp:277)
[10:49:31|Debug] MAVLink: emergency: Accel #0 fail:  TIMEOUT! (system_impl.cpp:277)
[10:49:31|Debug] MAVLink: emergency: Gyro #0 fail:  TIMEOUT! (system_impl.cpp:277)
[10:52:31|Debug] MAVLink: emergency: Accel #0 fail:  TIMEOUT! (system_impl.cpp:277)
[10:52:31|Debug] MAVLink: emergency: Gyro #0 fail:  TIMEOUT! (system_impl.cpp:277)
[10:57:01|Debug] MAVLink: emergency: Accel #0 fail:  TIMEOUT! (system_impl.cpp:277)
[10:57:01|Debug] MAVLink: emergency: Gyro #0 fail:  TIMEOUT! (system_impl.cpp:277)
[10:58:01|Debug] MAVLink: emergency: Accel #0 fail:  TIMEOUT! (system_impl.cpp:277)
[10:58:01|Debug] MAVLink: emergency: Gyro #0 fail:  TIMEOUT! (system_impl.cpp:277)
[11:03:13|Debug] MAVLink: emergency: Accel #0 fail:  TIMEOUT! (system_impl.cpp:277)
[11:03:13|Debug] MAVLink: emergency: Gyro #0 fail:  TIMEOUT! (system_impl.cpp:277)
[11:06:24|Debug] MAVLink: emergency: Accel #0 fail:  TIMEOUT! (system_impl.cpp:277)
[11:06:24|Debug] MAVLink: emergency: Gyro #0 fail:  TIMEOUT! (system_impl.cpp:277)
[11:07:01|Debug] MAVLink: emergency: Accel #0 fail:  TIMEOUT! (system_impl.cpp:277)
[11:07:01|Debug] MAVLink: emergency: Gyro #0 fail:  TIMEOUT! (system_impl.cpp:277)
[11:14:25|Debug] MAVLink: emergency: Accel #0 fail:  TIMEOUT! (system_impl.cpp:277)
[11:14:25|Debug] MAVLink: emergency: Gyro #0 fail:  TIMEOUT! (system_impl.cpp:277)
[11:14:28|Debug] MAVLink: emergency: Accel #0 fail:  TIMEOUT! (system_impl.cpp:277)
[11:14:28|Debug] MAVLink: emergency: Gyro #0 fail:  TIMEOUT! (system_impl.cpp:277)
[11:15:34|Debug] MAVLink: emergency: Accel #0 fail:  TIMEOUT! (system_impl.cpp:277)
[11:15:34|Debug] MAVLink: emergency: Gyro #0 fail:  TIMEOUT! (system_impl.cpp:277)
[11:16:17|Debug] MAVLink: emergency: Accel #0 fail:  TIMEOUT! (system_impl.cpp:277)
[11:16:17|Debug] MAVLink: emergency: Gyro #0 fail:  TIMEOUT! (system_impl.cpp:277)
[11:16:17|Debug] MAVLink: emergency: Accel #0 fail:  TIMEOUT! (system_impl.cpp:277)
[11:16:17|Debug] MAVLink: emergency: Gyro #0 fail:  TIMEOUT! (system_impl.cpp:277)
[11:16:59|Debug] MAVLink: emergency: Accel #0 fail:  TIMEOUT! (system_impl.cpp:277)
[11:16:59|Debug] MAVLink: emergency: Gyro #0 fail:  TIMEOUT! (system_impl.cpp:277)
[11:17:40|Debug] MAVLink: emergency: Accel #0 fail:  TIMEOUT! (system_impl.cpp:277)
[11:17:40|Debug] MAVLink: emergency: Gyro #0 fail:  TIMEOUT! (system_impl.cpp:277)
[11:27:30|Debug] MAVLink: emergency: Accel #0 fail:  TIMEOUT! (system_impl.cpp:277)
[11:27:30|Debug] MAVLink: emergency: Gyro #0 fail:  TIMEOUT! (system_impl.cpp:277)
[11:33:00|Debug] MAVLink: emergency: Accel #0 fail:  TIMEOUT! (system_impl.cpp:277)
[11:33:00|Debug] MAVLink: emergency: Gyro #0 fail:  TIMEOUT! (system_impl.cpp:277)
[11:33:07|Debug] MAVLink: emergency: Accel #0 fail:  TIMEOUT! (system_impl.cpp:277)
[11:33:07|Debug] MAVLink: emergency: Gyro #0 fail:  TIMEOUT! (system_impl.cpp:277)
[11:33:33|Debug] MAVLink: emergency: Accel #0 fail:  TIMEOUT! (system_impl.cpp:277)
[11:33:33|Debug] MAVLink: emergency: Gyro #0 fail:  TIMEOUT! (system_impl.cpp:277)
[11:33:41|Debug] MAVLink: emergency: Accel #0 fail:  TIMEOUT! (system_impl.cpp:277)
[11:33:41|Debug] MAVLink: emergency: Gyro #0 fail:  TIMEOUT! (system_impl.cpp:277)
[11:35:43|Debug] MAVLink: emergency: Accel #0 fail:  TIMEOUT! (system_impl.cpp:277)
[11:35:43|Debug] MAVLink: emergency: Gyro #0 fail:  TIMEOUT! (system_impl.cpp:277)
[11:35:45|Debug] MAVLink: emergency: Accel #0 fail:  TIMEOUT! (system_impl.cpp:277)
[11:35:45|Debug] MAVLink: emergency: Gyro #0 fail:  TIMEOUT! (system_impl.cpp:277)
[11:37:28|Debug] MAVLink: emergency: Accel #0 fail:  TIMEOUT! (system_impl.cpp:277)
[11:37:28|Debug] MAVLink: emergency: Gyro #0 fail:  TIMEOUT! (system_impl.cpp:277)
[11:38:00|Debug] MAVLink: emergency: Accel #0 fail:  TIMEOUT! (system_impl.cpp:277)
[11:38:00|Debug] MAVLink: emergency: Gyro #0 fail:  TIMEOUT! (system_impl.cpp:277)

The next one is the SITL output and as you will see I am getting a lot of sensor error messages.

pxh> commander takeoff
pxh> INFO  [logger] Start file log (type: mission)
INFO  [logger] Opened mission log file: ./mission_log/2018-12-07/15_01_07.ulg
INFO  [commander] Takeoff detected
WARN  [commander_tests] Failsafe enabled: no datalink
INFO  [navigator] RTL HOME activated
INFO  [navigator] RTL: climb to 499 m (10 m above home)
INFO  [navigator] RTL: return at 499 m (10 m above home)
INFO  [navigator] RTL: descend to 498 m (10 m above home)
INFO  [navigator] RTL: land at home
INFO  [commander] Landing detected
INFO  [logger] closed logfile, bytes written: 3758
INFO  [logger] closed logfile, bytes written: 87068877
pxh> ERROR [sensors] Accel #0 fail:  TIMEOUT!
ERROR [sensors] Sensor Accel #0 failed. Reconfiguring sensor priorities.
WARN  [sensors] Remaining sensors after failover event 0: Accel #0 priority: 1
WARN  [sensors] Remaining sensors after failover event 0: Accel #1 priority: 1
ERROR [sensors] Gyro #0 fail:  TIMEOUT!
ERROR [sensors] Sensor Gyro #0 failed. Reconfiguring sensor priorities.
WARN  [sensors] Remaining sensors after failover event 0: Gyro #0 priority: 1
WARN  [ekf2] accel id changed, resetting IMU bias
WARN  [ekf2] accel id changed, resetting IMU bias

Joe

joe-marrone picture joe-marrone on 7 Dec 2018

Right, so all these timeouts are probably due to the fact that the simulation doesn't run particularly smooth in a VirtualBox. You can try to give the VirtualBox instance more cores and make sure the extensions and guest additions are installed so that the graphics support is best and you can start SITL with HEADLESS=1 make posix gazebo which won't start the GUI.
Generally it's not recommended to run the SITL simulation in a virtual environment due to performance reasons.

julianoes picture julianoes on 8 Dec 2018

I updated DronecodeSDK-Python, so it now has the latest proto files. @joe-marrone: Would you mind trying to pull DronecodeSDK-Python from master again and try to run it?

JonasVautherin picture JonasVautherin on 9 Dec 2018
👍1 
@JonasVautherin :

I deleted the existing DronecodeSDK-Python files and reinstalled as per instructions on this page. Then I started the SITL and ran takeoff_and_land.py in the wrapper window, but it failed again as shown.

 joe@joe-VirtualBox:~/src/DronecodeSDK-Python$ examples/takeoff_and_land.py
Traceback (most recent call last):
  File "examples/takeoff_and_land.py", line 29, in <module>
    loop.run_until_complete(run())
  File "/usr/lib/python3.6/asyncio/base_events.py", line 473, in run_until_complete
    return future.result()
  File "examples/takeoff_and_land.py", line 12, in run
    arm_result, _ = await drone.action.arm()
  File "/home/joe/src/DronecodeSDK-Python/dronecode_sdk/plugins/action.py", line 141, in arm
    response = await self._stub.Arm(request)
  File "/home/joe/.local/lib/python3.6/site-packages/aiogrpc/channel.py", line 40, in __call__
    return await fut
grpc._channel._Rendezvous: <_Rendezvous of RPC that terminated with:
    status = StatusCode.UNIMPLEMENTED
    details = ""
    debug_error_string = "{"created":"@1544539645.231817305","description":"Error received from peer","file":"src/core/lib/surface/call.cc","file_line":1036,"grpc_message":"","grpc_status":12}"
>

When you say run the install from master, is that different than what I did?

Thanks
Joe

joe-marrone picture joe-marrone on 11 Dec 2018

Also, you mention that it is not recommended to run SITL in a virtual machine. I am working on getting a dedicated system to use for this, and then I will build the entire ecosystem from scratch and avoid the issues associated with a virtual machine. But that may take a few days.

Joe

joe-marrone picture joe-marrone on 11 Dec 2018

It all comes down to how beefy your computer is and if you need the visualization, and how well the GPU support works on your host and client inside VirtualBox.

julianoes picture julianoes on 11 Dec 2018

@joe-marrone: just to be sure: you run everything (python frontend, c++ backend, SITL) on the very same VM instance? And they have network access (i.e. a client on the VM can actually access a server on the VM)?

And did you build the latest develop backend (DronecodeSDK, not DronecodeSDK-Python)?

What your python client says is that the function you try to call in the backend is not implemented. Which suggests that you are indeed running a backend, but that it is not the latest develop.

JonasVautherin picture JonasVautherin on 11 Dec 2018

@JonasVautherin : I have been using ubuntu 16.04.5 LTS up to this point. Is it recommended to upgrade to ubuntu 18.04.x LTS at this time?

joe-marrone picture joe-marrone on 11 Dec 2018

@joe-marrone I think both should work. If something of PX4 doesn't work in 18.04 yet, we'll just have to fix it asap.

julianoes picture julianoes on 11 Dec 2018

And it really looks like everything works fine, but the function that the python client tries to call is just not implemented on the backend (but it seems like it can connect to it, otherwise the error would be different).

Can you double-check that you built backend_bin from the develop branch of DronecodeSDK, and not from master?

JonasVautherin picture JonasVautherin on 12 Dec 2018

@joe-marrone I think both should work. If something of PX4 doesn't work in 18.04 yet, we'll just have to fix it asap.

@dagar Have we a plan/timeframe for moving to 18.04. Any known issues?

hamishwillee picture hamishwillee on 12 Dec 2018

@JonasVautherin : I just installed ubuntu 16.04.5 on my machine as the primary OS ... no more virtual machine. I then reinstalled the entire ecosystem starting with the development toolchain, then built the SDK from source, then the backend. The Python wrapper appeared to install OK. However, when I tried to start PX4 and SITL, the GUI did not start. This is what I got, after finding out that make posix_sitl_default jmavsim had been deprecated.

joe-pl@joe-PL:~/src/Firmware$ make posix_sitl_default jmavsim 
Makefile:198:  posix_sitl_default has been deprecated and will be removed, please use px4_sitl_default!
make px4_sitl_default
make[1]: Entering directory '/home/joe-pl/src/Firmware'
ninja: Entering directory `/home/joe-pl/src/Firmware/build/px4_sitl_default'
[1/1] Generating ../../logs
make[1]: Leaving directory '/home/joe-pl/src/Firmware'
joe-pl@joe-PL:~/src/Firmware$ make px4_sitl_default
ninja: Entering directory `/home/joe-pl/src/Firmware/build/px4_sitl_default'
[1/1] Generating ../../logs
joe-pl@joe-PL:~/src/Firmware$

On the virtual machine I was able to run the GUI, but now I can't. Something must have gone wrong. Can you make nay suggestions?

Thanks,
Joe

joe-marrone picture joe-marrone on 13 Dec 2018

I want to mention that some of my issues might be because I put things in the wrong directory. There are some steps where I'm not sure which directory to use, so it would be helpful if this information was included with each step. I notice that a few steps do list the directory, but others I have been guessing at.

Thanks
Joe

joe-marrone picture joe-marrone on 13 Dec 2018

then built the SDK from source, then the backend

You still haven't answered my question. Do you build the backend from the develop branch?

However, when I tried to start PX4 and SITL, the GUI did not start.

Can you make clean && make posix jmavsim? Don't you get a compilation error there?

JonasVautherin picture JonasVautherin on 13 Dec 2018

@JonasVautherin : This is the process I used for the backend. I don't know whether this is from the develop branch.

Build and run backend

First, we need to build and run the backend which is the gRPC server that the Python wrapper will connect to.

Note: it is planned to automate this step using package managers like brew, apt-get, etc. .

git clone https://github.com/Dronecode/DronecodeSDK --recursive
cd DronecodeSDK
make BUILD_BACKEND=1

For more help on this step, check the docs on how to build from source.

Once this is built, start PX4 SITL and run the backend:

./build/default/backend/src/backend_bin

By default, the backend will connect using MAVLink on UDP port 14540 which is running by default when PX4 is run in SITL (software in the loop simulation). To change the connection port, check this line in the backend. For now, the backend is limited to UDP even though the core supports UDP, TCP, and serial.

Joe

joe-marrone picture joe-marrone on 13 Dec 2018

I tried make clean && make posix jmavsim and immediately the SITL launched and the GUI appeared. Next I tried takeoff_and_land.py by typing it at the > prompt, but it said "invalid command". So I tried entering it at the DronecodeSDK and Firmware prompts, but it said "command not found". Maybe this is caused by the fact that the backend has not installed properly.

Joe

joe-marrone picture joe-marrone on 13 Dec 2018

Next I tried takeoff_and_land.py by typing it at the > prompt, but it said "invalid command".

Does that mean you tried to run a Python scipt in the pxh aka PX4 shell? That's a bit adventurous. :smile:

You would start the Python script in another shell/terminal.

julianoes picture julianoes on 13 Dec 2018

In the example folder I found takeoff_and_land.cpp, but no Python version. That's probably why it doesn't run.

Joe

joe-marrone picture joe-marrone on 13 Dec 2018

UPDATE: I reinstalled the backend build, and it took. This allowed me to verify that the backend was talking to SITL. Then I installed the wrapper again, and this installed OK. But running the takeoff and land script failed as shown.

joe-pl@joe-PL:~/src/DronecodeSDK/DronecodeSDK-Python$ examples/takeoff_and_land.py
  File "examples/takeoff_and_land.py", line 13
    print(f"-- Arm result: {arm_result}")
                                       ^
SyntaxError: invalid syntax

Right now It would appear that the install is good except for running the script.

Joe

joe-marrone picture joe-marrone on 13 Dec 2018

Right now It would appear that the install is good except for running the script.

It seems so, indeed, as it apparently passed the arming call.

Can you show me the output of:

$ python --version

And can you try:

$ python3 examples/takeoff_and_land.py
JonasVautherin picture JonasVautherin on 13 Dec 2018

@JonasVautherin : Here are the results:

joe-pl@joe-PL:~$ python --version
Python 2.7.12

joe-pl@joe-PL:~/src/DronecodeSDK/DronecodeSDK-Python$ python3 examples/takeoff_and_land.py
  File "examples/takeoff_and_land.py", line 13
    print(f"-- Arm result: {arm_result}")
                                       ^
SyntaxError: invalid syntax



md5-62e1911d38fa1dc4a96c2db85b35e67a



joe-pl@joe-PL:~$ python3
Python 3.5.2 (default, Nov 12 2018, 13:43:14) 
[GCC 5.4.0 20160609] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>>

I must have a path wrong, which has been my concern all along. Putting things in the right directory has been my biggest problem. Thanks for helping narrow down the problem. Now how do I get Python3 in the right place?

Thanks,
Joe

joe-marrone picture joe-marrone on 14 Dec 2018

Once the backend is built, does is have to be started each time I run a mission? In other words, right now I am starting the SITL in this order:

  1. Launch PX4/SITL

/src/Firmware/make PX4_sitl_default jmavsim

  1. Start backend

/src/DronecodeSDK/.build/default/backend/src/backend_bin

  1. Run Python script

/src/DronecodeSDK/DronecodeSDK-Python/examples/python_script.py

So do I have to start Backend each time? If so, then I would like to have a shell script do all the work.

-Joe

joe-marrone picture joe-marrone on 14 Dec 2018
👍1

So do I have to start Backend each time?

Yes you do. At some point it will be done automatically, but not yet :-).

Clearly its calling for Python 2 and not 3. Yet Python 3 exists on my system.

Not sure yet. So python links to python2, and I would expect python3 to link to... well, python3.

Can you show the output of:

$ python3 --version
JonasVautherin picture JonasVautherin on 14 Dec 2018 
joe-pl@joe-PL:~$ python3 --version
Python 3.5.2
joe-pl@joe-PL:~$
joe-marrone picture joe-marrone on 14 Dec 2018

Ooooh, formatted string literals come with python 3.6... I believe we need python 3.6 for other things in asyncio, but let's try to fix this first.

Try to change the print line for something like:

print("-- Arm result: " + arm_result)
JonasVautherin picture JonasVautherin on 15 Dec 2018

So do I have to start Backend each time?

Yes you do. At some point it will be done automatically, but not yet :-).

If the docs don't say that, they should :-)

hamishwillee picture hamishwillee on 16 Dec 2018
👍1

@JonasVautherin : Is there any action I should take now, or do I wait for a fix? Also, there is mention of Python 3.6, but my version is 3.5.2. Should I upgrade?

Thanks,
Joe

joe-marrone picture joe-marrone on 17 Dec 2018

Should I upgrade?

I would definitely try that, if I were you! Maybe this tutorial can help you!

Let us know how it goes!

JonasVautherin picture JonasVautherin on 17 Dec 2018

@JonasVautherin : I successfully installed Python 3.6.7 as per the tutorial. But then I could not run the backend as shown here:

joe-pl@joe-PL:~/src/DronecodeSDK/build/default/backend/src$ backend_bin
Traceback (most recent call last):
  File "/usr/lib/python3.6/dbm/gnu.py", line 4, in <module>
    from _gdbm import *
ModuleNotFoundError: No module named '_gdbm'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/CommandNotFound/CommandNotFound.py", line 7, in <module>
    import dbm.gnu as gdbm
  File "/usr/lib/python3.6/dbm/gnu.py", line 6, in <module>
    raise ImportError(str(msg) + ', please install the python3-gdbm package')
ImportError: No module named '_gdbm', please install the python3-gdbm package

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/lib/command-not-found", line 27, in <module>
    from CommandNotFound.util import crash_guard
  File "/usr/lib/python3/dist-packages/CommandNotFound/__init__.py", line 3, in <module>
    from CommandNotFound.CommandNotFound import CommandNotFound
  File "/usr/lib/python3/dist-packages/CommandNotFound/CommandNotFound.py", line 9, in <module>
    import gdbm
ModuleNotFoundError: No module named 'gdbm'
Error in sys.excepthook:
Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/apport_python_hook.py", line 63, in apport_excepthook
    from apport.fileutils import likely_packaged, get_recent_crashes
  File "/usr/lib/python3/dist-packages/apport/__init__.py", line 5, in <module>
    from apport.report import Report
  File "/usr/lib/python3/dist-packages/apport/report.py", line 30, in <module>
    import apport.fileutils
  File "/usr/lib/python3/dist-packages/apport/fileutils.py", line 23, in <module>
    from apport.packaging_impl import impl as packaging
  File "/usr/lib/python3/dist-packages/apport/packaging_impl.py", line 23, in <module>
    import apt
  File "/usr/lib/python3/dist-packages/apt/__init__.py", line 23, in <module>
    import apt_pkg
ModuleNotFoundError: No module named 'apt_pkg'

Original exception was:
Traceback (most recent call last):
  File "/usr/lib/python3.6/dbm/gnu.py", line 4, in <module>
    from _gdbm import *
ModuleNotFoundError: No module named '_gdbm'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/CommandNotFound/CommandNotFound.py", line 7, in <module>
    import dbm.gnu as gdbm
  File "/usr/lib/python3.6/dbm/gnu.py", line 6, in <module>
    raise ImportError(str(msg) + ', please install the python3-gdbm package')
ImportError: No module named '_gdbm', please install the python3-gdbm package

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/lib/command-not-found", line 27, in <module>
    from CommandNotFound.util import crash_guard
  File "/usr/lib/python3/dist-packages/CommandNotFound/__init__.py", line 3, in <module>
    from CommandNotFound.CommandNotFound import CommandNotFound
  File "/usr/lib/python3/dist-packages/CommandNotFound/CommandNotFound.py", line 9, in <module>
    import gdbm
ModuleNotFoundError: No module named 'gdbm'

Could it be that Python 3.6 broke something?

Joe

joe-marrone picture joe-marrone on 17 Dec 2018

@joe-marrone: Let me try to reproduce the setup on Ubuntu 16.04, I'll come back to you ASAP.

JonasVautherin picture JonasVautherin on 17 Dec 2018

Could it be that Python 3.6 broke something?

Make sure to re-install the requirements, and run ./other/tools/run_protoc.sh again now that you have updated Python.

julianoes picture julianoes on 18 Dec 2018

Below are the steps I followed for installing DronecodeSDK-Python on an Ubuntu 16.04 Docker container (see the Dockerfile here). Make sure to adapt those steps to your system if needed, and to read the output of those commands to make sure they don't fail.

  1. Install prerequisites (in your case I believe you already have them):
$ apt install curl git software-properties-common
  1. Clone the repository:
$ git clone https://github.com/dronecode/dronecodesdk-python --recursive <path/to/dronecodesdk_python>
  1. Install Python 3.6:
$ add-apt-repository ppa:deadsnakes/ppa
$ apt update
$ apt install python3.6

Note that it won't replace python or python3, so python3 will still point to python3.5. You can either change that (there are tutorials for this) or always call python3.6 explicitly (which I do in these instructions).

  1. I had to install pip3.6 manually:
$ curl https://bootstrap.pypa.io/get-pip.py | python3.6
  1. Install dcsdkgen, needed to generate the Python SDK:
$ cd <path/to/dronecodesdk_python>/proto/pb_plugins
$ pip3.6 install -r requirements.txt
$ pip3.6 install -e .
  1. Now we can generate the Python SDK:

Note the sed instruction below: it is replacing python3 with python3.6 in the generation script, because python3 points to python3.5 on Ubuntu 16.04, as mentioned above.

$ cd <path/to/dronecodesdk_python>
$ pip3.6 install -r requirements.txt -r requirements-dev.txt
$ sed -i 's/python3/python3.6/g' other/tools/run_protoc.sh
$ other/tools/run_protoc.sh
  1. Finally, install the SDK:
$ cd <path/to/dronecodesdk_python>
$ pip3.6 install -e .

If everything goes right, you should be able to run the example, with:

$ python3.6 examples/takeoff_and_land.py
JonasVautherin picture JonasVautherin on 18 Dec 2018

@JonasVautherin : Thanks for this new information. Before I start this, I have a question: do I have to install Docker on my system? Is it necessary in order to use the process you have provided?

I have not installed Docker on my system, unless it was already there. And I have no experience using it, so I am wondering if I can get the job done without it?

Thanks again for all your help.
Joe

joe-marrone picture joe-marrone on 18 Dec 2018

Sorry I was not clear: I gave the Dockerfile just for the record, because I did all that in Docker. But the instructions above should just work in your computer, no need for Docker at all. It could be that you already have the first 3 steps done, so you could skip them.

JonasVautherin picture JonasVautherin on 18 Dec 2018

@JonasVautherin : Here we go again. Tried to follow your instructions and right away I got this error. Evidently I lack the module apt_pkg

joe-pl@joe-PL:/$ add-apt-repository ppa:deadsnakes/ppa
Traceback (most recent call last):
  File "/usr/bin/add-apt-repository", line 11, in <module>
    from softwareproperties.SoftwareProperties import SoftwareProperties, shortcut_handler
  File "/usr/lib/python3/dist-packages/softwareproperties/SoftwareProperties.py", line 27, in <module>
    import apt_pkg
ModuleNotFoundError: No module named 'apt_pkg'
Error in sys.excepthook:
Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/apport_python_hook.py", line 63, in apport_excepthook
    from apport.fileutils import likely_packaged, get_recent_crashes
  File "/usr/lib/python3/dist-packages/apport/__init__.py", line 5, in <module>
    from apport.report import Report
  File "/usr/lib/python3/dist-packages/apport/report.py", line 30, in <module>
    import apport.fileutils
  File "/usr/lib/python3/dist-packages/apport/fileutils.py", line 23, in <module>
    from apport.packaging_impl import impl as packaging
  File "/usr/lib/python3/dist-packages/apport/packaging_impl.py", line 23, in <module>
    import apt
  File "/usr/lib/python3/dist-packages/apt/__init__.py", line 23, in <module>
    import apt_pkg
ModuleNotFoundError: No module named 'apt_pkg'

Original exception was:
Traceback (most recent call last):
  File "/usr/bin/add-apt-repository", line 11, in <module>
    from softwareproperties.SoftwareProperties import SoftwareProperties, shortcut_handler
  File "/usr/lib/python3/dist-packages/softwareproperties/SoftwareProperties.py", line 27, in <module>
    import apt_pkg
ModuleNotFoundError: No module named 'apt_pkg'

What's next?
Joe

joe-marrone picture joe-marrone on 18 Dec 2018

https://stackoverflow.com/questions/13708180/python-dev-installation-error-importerror-no-module-named-apt-pkg

apt-get install python3-apt
julianoes picture julianoes on 18 Dec 2018

It looks like the apt_pkg issue has been solved. Then I started to go thru the steps and got hung up on
curl https://bootstrap.pypa.io/get-pip.py | python3.6 This throws the error:

joe-pl@joe-PL:~$ sudo curl https://bootstrap.pypa.io/get-pip.py | python3.6
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100 1622k  100 1622k    0     0  3223k      0 --:--:-- --:--:-- --:--:-- 3226k
Collecting pip
  Using cached https://files.pythonhosted.org/packages/c2/d7/90f34cb0d83a6c5631cf71dfe64cc1054598c843a92b400e55675cc2ac37/pip-18.1-py2.py3-none-any.whl
Installing collected packages: pip
  Found existing installation: pip 8.1.1
    Uninstalling pip-8.1.1:
Could not install packages due to an EnvironmentError: [Errno 13] Permission denied: '/usr/bin/pip'
Consider using the `--user` option or check the permissions.

I researched this error and found a few ways around it using
sudo apt-get install python-virtualenv and virtualenv --python=/usr/bin/python3.6 venv
but this did not fix the problem.

I will continue to look for a fix, but do you have any other ideas?
anks,
Joe
Th

joe-marrone picture joe-marrone on 18 Dec 2018

I guess I would run sudo curl https://bootstrap.pypa.io/get-pip.py | python3.6.

JonasVautherin picture JonasVautherin on 18 Dec 2018

Same thing happens running
sudo curl https://bootstrap.pypa.io/get-pip.py | python3.6
It's got something to do with permissions, so I looked at some of the files involved, and I have rwx on them, so this seems to be OK.

Joe

joe-marrone picture joe-marrone on 18 Dec 2018

Oh, my bad. Probably more curl https://bootstrap.pypa.io/get-pip.py | sudo python3.6.

Basically the goal is to run python3.6 get-pip.py, so you could also download the file manually and run it then.

JonasVautherin picture JonasVautherin on 18 Dec 2018

@JonasVautherin : Sorry, but cannot get past step 4
$ curl https://bootstrap.pypa.io/get-pip.py | python3.6

I added sudo to the last part of the command as Julian suggested, but I still get errors related to permissions. I have looked at all the directories on the /home/joe-pl/.cache/pip/http path and cannot find the offending directory. Here's the terminal output for this:

joe-pl@joe-PL:~$ curl https://bootstrap.pypa.io/get-pip.py | sudo python3.6
[sudo] password for joe-pl:   % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
Sorry, try again.
[sudo] password for joe-pl: 
100 1622k  100 1622k    0     0  28828      0  0:00:57  0:00:57 --:--:-- 28829
The directory '/home/joe-pl/.cache/pip/http' or its parent directory is not owned by the current user and the cache has been disabled. Please check the permissions and owner of that directory. If executing pip with sudo, you may want sudo's -H flag.
The directory '/home/joe-pl/.cache/pip' or its parent directory is not owned by the current user and caching wheels has been disabled. check the permissions and owner of that directory. If executing pip with sudo, you may want sudo's -H flag.
Collecting pip
  Downloading https://files.pythonhosted.org/packages/c2/d7/90f34cb0d83a6c5631cf71dfe64cc1054598c843a92b400e55675cc2ac37/pip-18.1-py2.py3-none-any.whl (1.3MB)
    100% |████████████████████████████████| 1.3MB 1.7MB/s 
Installing collected packages: pip
  Found existing installation: pip 18.1
    Uninstalling pip-18.1:
      Successfully uninstalled pip-18.1
Successfully installed pip-18.1

-Joe

joe-marrone picture joe-marrone on 19 Dec 2018

Successfully installed pip-18.1

Looks like it worked to me :sweat_smile:. Above lines say that the cache has been disabled, but well, should not be an issue.

What if you do $ pip3.6 --version now? Does it exist?

JonasVautherin picture JonasVautherin on 19 Dec 2018

joe-pl@joe-PL:~$ pip3.6 --version pip 18.1 from /usr/local/lib/python3.6/dist-packages/pip (python 3.6)

Looks like it's there. Now when I go to step 5
$ cd <path/to/dronecodesdk_python>/proto/pb_plugins
$ pip3.6 install -r requirements.txt
here's what happens:

joe-pl@joe-PL:/src/DronecodeSDK/DronecodeSDK-Python/proto/pb_plugins$ pip3.6 install -r requirements.txt
Collecting protobuf (from -r requirements.txt (line 1))
  Downloading https://files.pythonhosted.org/packages/c2/f9/28787754923612ca9bfdffc588daa05580ed70698add063a5629d1a4209d/protobuf-3.6.1-cp36-cp36m-manylinux1_x86_64.whl (1.1MB)
    100% |████████████████████████████████| 1.1MB 940kB/s 
Requirement already satisfied: jinja2 in /usr/lib/python3/dist-packages (from -r requirements.txt (line 2)) (2.8)
Requirement already satisfied: setuptools in /usr/lib/python3/dist-packages (from protobuf->-r requirements.txt (line 1)) (20.7.0)
Requirement already satisfied: six>=1.9 in /usr/lib/python3/dist-packages (from protobuf->-r requirements.txt (line 1)) (1.10.0)
Requirement already satisfied: MarkupSafe in /usr/lib/python3/dist-packages (from jinja2->-r requirements.txt (line 2)) (0.23)
Installing collected packages: protobuf
Could not install packages due to an EnvironmentError: [Errno 13] Permission denied: '/usr/local/lib/python3.6/dist-packages/protobuf-3.6.1-py3.6-nspkg.pth'
Consider using the `--user` option or check the permissions.

For some files it says 'Requirement already satisfied ' yet down at the end it says "could not install packages ... error 13". So at this point I can't tell what's been done and what's not.

-Joe

joe-marrone picture joe-marrone on 19 Dec 2018

[Errno 13] Permission denied: '/usr/local/lib/python3.6/dist-packages/protobuf-3.6.1-py3.6-nspkg.pth'

It is trying install the packages on your system (/usr/local/lib/python3.6/...), and your user doesn't have the permission for that. Try sudo:

sudo pip3.6 install -r requirements.txt
JonasVautherin picture JonasVautherin on 20 Dec 2018

I think steps 1-5 are complete, so I tried step 6 in the instructions above and this was the result:

joe-pl@joe-PL:/src/DronecodeSDK/DronecodeSDK-Python/proto/pb_plugins$ sudo pip3.6 install -r requirements.txt -r requirements-dev.txt
The directory '/home/joe-pl/.cache/pip/http' or its parent directory is not owned by the current user and the cache has been disabled. Please check the permissions and owner of that directory. If executing pip with sudo, you may want sudo's -H flag.
The directory '/home/joe-pl/.cache/pip' or its parent directory is not owned by the current user and caching wheels has been disabled. check the permissions and owner of that directory. If executing pip with sudo, you may want sudo's -H flag.
Could not open requirements file: [Errno 2] No such file or directory: 'requirements-dev.txt'

Still trying to get this all resolved.
Thanks,
Joe

joe-marrone picture joe-marrone on 21 Dec 2018

Re: the last post.
The file requirements-dev.txt is not in the directory:

joe-pl@joe-PL:/src/DronecodeSDK/DronecodeSDK-Python/proto/pb_plugins$ ls
dcsdkgen  README.rst  requirements.txt  setup.py  test

The other file is there. Any idea why the file needed for this pip3 command is not there?

-Joe

joe-marrone picture joe-marrone on 21 Dec 2018

You are in the directory DronecodeSDK-Python/proto/pb_plugins but you should be just in DronecodeSDK-Python/.

julianoes picture julianoes on 21 Dec 2018
👍1

Continued progress:
I found the requirements-dev.txt file in a different directory, and then the command line worked. Turned out I was using the wrong directory.

-Joe

joe-marrone picture joe-marrone on 21 Dec 2018
👍1

Continued progress:
I found the requirements-dev.txt file in a different directory, and then the command line worked. Turned out I was using the wrong directory.

-Joe

joe-marrone picture joe-marrone on 21 Dec 2018

Now that everything is place, I tried the script, but it didn't work.

joe-pl@joe-PL:~/src/DronecodeSDK/DronecodeSDK-Python$ python3.6 examples/takeoff_and_land.py
Traceback (most recent call last):
  File "examples/takeoff_and_land.py", line 7, in <module>
    drone = dronecode_sdk_connect(host="127.0.0.1")
  File "/src/DronecodeSDK/DronecodeSDK-Python/dronecode_sdk/__init__.py", line 49, in connect
    globals()[plugin](plugin_manager)
KeyError: 'Action'

Any other ideas about what's wrong?

Thanks,
Joe

joe-marrone picture joe-marrone on 21 Dec 2018

:thinking:

@xzvf: Would you by any chance have an idea about that KeyError: 'Action'?

JonasVautherin picture JonasVautherin on 21 Dec 2018

Happy New Year.
I was wondering if anything had been done about the KeyError: 'Action' error?

Joe

joe-marrone picture joe-marrone on 3 Jan 2019

Hi @joe-marrone! Happy new year, and sorry for the delay!

Can you show me the output of $ tree from /src/DronecodeSDK/DronecodeSDK-Python/dronecode_sdk?

JonasVautherin picture JonasVautherin on 12 Jan 2019

@JonasVautherin : Here it is. Obviously its missing, so I will add it.
joe-pl@joe-PL:~/src/DronecodeSDK/DronecodeSDK-Python/dronecode_sdk$ tree The program 'tree' is currently not installed. You can install it by typing: sudo apt install tree
Joe

joe-marrone picture joe-marrone on 14 Jan 2019

Right. Please paste the output of it after you have installed it ($ sudo apt install tree) :-).

JonasVautherin picture JonasVautherin on 15 Jan 2019

OK, here is the tree output.

`
joe-pl@joe-PL:~/src/DronecodeSDK/DronecodeSDK-Python/dronecode_sdk$ tree
.
├── async_plugin_manager.py
├── _base.py
├── generated
│   ├── action_pb2_grpc.py
│   ├── action_pb2.py
│   ├── calibration_pb2_grpc.py
│   ├── calibration_pb2.py
│   ├── camera_pb2_grpc.py
│   ├── camera_pb2.py
│   ├── core_pb2_grpc.py
│   ├── core_pb2.py
│   ├── discovery_pb2_grpc.py
│   ├── discovery_pb2.py
│   ├── gimbal_pb2_grpc.py
│   ├── gimbal_pb2.py
│   ├── info_pb2_grpc.py
│   ├── info_pb2.py
│   ├── __init__.py
│   ├── mission_pb2_grpc.py
│   ├── mission_pb2.py
│   ├── telemetry_pb2_grpc.py
│   └── telemetry_pb2.py
├── __init__.py
└── plugins
└── __init__.py

2 directories, 23 files
`

Joe

joe-marrone picture joe-marrone on 15 Jan 2019

~/src/DronecodeSDK/DronecodeSDK-Python/dronecode_sdk/__plugins__ should not be empty. It means that run_protoc.sh failed.

Can you verify that the python calls in run_protoc.sh are python3.6 (and not python3), and run run_protoc.sh again?

JonasVautherin picture JonasVautherin on 15 Jan 2019

This is the run_protoc.sh file in my system. I installed this several weeks ago when I started this thread.

Joe

`

!/bin/bash

SCRIPT_DIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" && pwd )"
WORK_DIR="${SCRIPT_DIR}/../../"
PROTO_DIR="${WORK_DIR}/proto"
GENERATED_DIR="${WORK_DIR}/dronecode_sdk/generated"
PLUGIN_DIR="${WORK_DIR}/dronecode_sdk/plugins"
PLUGIN_INIT="${PLUGIN_DIR}/__init__.py"
export TEMPLATE_PATH="${WORK_DIR}/other/templates/"

function generate {
echo -e "# -- coding: utf-8 --\n" > $PLUGIN_INIT

for PROTO_FILE in `find ${PROTO_DIR} -name "*.proto" -type f`; do

    # Generate bindings for each file individually
    python3 -m grpc_tools.protoc -I${GENERATED_DIR} \
                                 --proto_path=$(dirname ${PROTO_FILE}) \
                                 --python_out=${GENERATED_DIR} \
                                 --grpc_python_out=${GENERATED_DIR} \
                                 ${PROTO_FILE}

    # For some reason the import is broken with Python3.5.x and works fine
    # with Python3.6.x, set an absolute path and everything is fine
    PROTO_IMPORT_NAME="$(basename -- ${PROTO_FILE%.*})_pb2"

    # We need to create the .original backup files, otherwise we're not compatible with
    # BSD sed.
    sed -i'.sedoriginal' -e "s/import ${PROTO_IMPORT_NAME}/from . import ${PROTO_IMPORT_NAME}/" \
        "${GENERATED_DIR}/${PROTO_IMPORT_NAME}_grpc.py"
    # Clean up the backup files.
    find ${GENERATED_DIR} -name '*.sedoriginal' -delete
    echo " -> [+] Generated protobuf and gRPC bindings for ${PROTO_IMPORT_NAME%_*}"

    # Generate plugin
    python3 -m grpc_tools.protoc -I$(dirname ${PROTO_FILE}) \
                                 --plugin=protoc-gen-custom=$(which dcsdkgen) \
                                 --custom_out=${PLUGIN_DIR} \
                                 --custom_opt=py \
                                 ${PROTO_FILE}

    WANTED_PLUGIN_NAME="$(echo ${PROTO_FILE} | sed "s#.*/\(.*\).proto#\1#g").py"
    # protoc generates java like filenames, we don't want that with python

    # @TODO: cleanup this script and figure out a way to make it cross-os friendly perhaps rewrite in python
    # capilalization as trivial as this needs a workaround for macos bash 3.2
    # this solution avoids using bash ^ substitution (from bash >4.0) and is using python 3 instead
    CAPITALIZED_PLUGIN_NAME=$(python3 -c "import sys;print(sys.argv[1].capitalize())" "$WANTED_PLUGIN_NAME")
    mv ${PLUGIN_DIR}/${CAPITALIZED_PLUGIN_NAME} ${PLUGIN_DIR}/${WANTED_PLUGIN_NAME}
    # Add to imports
    echo "from .${WANTED_PLUGIN_NAME%.py} import *" >> $PLUGIN_INIT
    echo " -> [+] Generated plugin for ${PROTO_IMPORT_NAME%_*}"

done

}
function install_dcsdkgen {
cd ${PROTO_DIR}/pb_plugins
if [[ "$VIRTUAL_ENV" != "" ]]
then
pip3 install .
else
pip3 install --user .
fi
}
echo "[+] Installing the DronecodeSDK autogenerator"
install_dcsdkgen
echo "[+] Done"
echo "[+] Generating plugins from "
generate
echo "[+] Done"
`

joe-marrone picture joe-marrone on 16 Jan 2019

Ok. So part of what you did before was to install python3.6 and pip3.6. On your system, you should now have multiple versions of Python. python is some Python 2 (maybe 2.7?), python3 is Python 3.X (maybe 3.5?), and python3.6 is Python 3.6 (the one you just installed).

The script run_protoc.sh calls python3 and pip3, because on more modern systems like Ubuntu 18.04, that would be fine. But on your Ubuntu 16.04, you need the script to explicitly call python3.6 and pip3.6.

So you should open run_protoc.sh, and replace python3 with python3.6 everywhere. And also replace pip3 with pip3.6 everywhere. Then save your changes and run run_protoc.sh.

JonasVautherin picture JonasVautherin on 17 Jan 2019

I made the changes and ran run_protoc.sh. It installed OK. But now I have another issue preventing me from using the simulator. It seems that my ubuntu 16.04 install does not like the graphics card on my PC because of incompatibilities with GNOME, so I am using the standard graphics shell from Microsoft, which does not run the sim. Never a dull day!

I tried some NVidia driver updates but they didn't work either. I'm kind of stuck right now, and not sure how to proceed. Do you have any suggestions?

Thanks for all your help so far.

Joe

joe-marrone picture joe-marrone on 17 Jan 2019

If you're using Gazebo, you can try to run it without graphics:

HEADLESS=1 make px4_sitl gazebo
julianoes picture julianoes on 17 Jan 2019
👍1

If you haven't installed Gazebo yet, and have Docker, you can also try to run it directly from this docker image (have a look at the documentation there).

JonasVautherin picture JonasVautherin on 17 Jan 2019

Unfortunately, I have not used either Gazebo or Docker yet.
Joe

joe-marrone picture joe-marrone on 18 Jan 2019

I see. I guess you may want to try Gazebo (see Julian's comment above).

I'm closing this issue now, as the Python wrapper got installed.

JonasVautherin picture JonasVautherin on 20 Jan 2019
Was this page helpful?
0 / 5 - 0 ratings

Related issues

helloE9 picture helloE9  ·  6Comments
itrigitech picture itrigitech  ·  5Comments
dangviethieu picture dangviethieu  ·  6Comments
julianoes picture julianoes  ·  6Comments
julianoes picture julianoes  ·  3Comments