buildozer is choosing odd venv, and using a different python?

Created on 29 Apr 2020 · 11Comments · Source: kivy/buildozer

Okay, so on Debian Buster (Stable). I've tested a ton of things, and used different versions of buildozer, including directly from github master.zip.

My virtualenv is at ~/venv-3.6/ (but it has python 3.7 installed in it...)
It's active, and "which python" shows it there, and python --version shows 3.7.3.
(I don't personally care for this version; it's just what was at /usr/bin/python3 on this debian system).

So:

What causes buildozer to run: _virtualenv --python=python3 venv_
What is it doing when it then sources "_venv/bin/activate_"? Where did that come from? I went ahead and symlinked ~/venv to ~/venv-3.6 just for kicks, but it didn't change anything.
Why is it then accessing the system _/usr/lib/python3.7/runpy.py_ ?

I have no env vars for $PYTHONPATH or anything set. I have no clue what's going on here.

[INFO]:    <- directory context /home/jaggz/kivy/tests/splash/.buildozer/android/pla
tform/python-for-android                                                              
[INFO]:    Postbuilding android for armeabi-v7a                                       
[INFO]:    Postbuilding kivy for armeabi-v7a                                          
[INFO]:    # Installing pure Python modules                                           
[INFO]:    *** PYTHON PACKAGE / PROJECT INSTALL STAGE ***                             
[INFO]:    The requirements (certifi) don't have recipes, attempting to install them w
ith pip
[INFO]:    If this fails, it may mean that the module has compiled components and need
s a recipe.
[INFO]:    -> directory context /home/jaggz/kivy/tests/splash/.buildozer/android/pla
tform/build-armeabi-v7a/build
[INFO]:    -> running virtualenv --python=python3 venv                                
[INFO]:    Upgrade pip to latest version                                              
[INFO]:    -> running bash -c source venv/bin/activate && pip install -U pip          
Traceback (most recent call last):                                                    
  File "/usr/lib/python3.7/runpy.py", line 193, in _run_module_as_main                
    "__main__", mod_spec)                                                             
  File "/usr/lib/python3.7/runpy.py", line 85, in _run_code                           
    exec(code, run_globals)                                                           
  File "/home/jaggz/kivy/tests/splash/.buildozer/android/platform/python-for-androi$
/pythonforandroid/toolchain.py", line 1199, in <module>                               
    main()                                                                            
  File "/home/jaggz/kivy/tests/splash/.buildozer/android/platform/python-for-androi$
/pythonforandroid/entrypoints.py", line 18, in main                                   
    ToolchainCL()                                                                     
  File "/home/jaggz/kivy/tests/splash/.buildozer/android/platform/python-for-androi$
/pythonforandroid/toolchain.py", line 671, in __init__                                
    getattr(self, args.subparser_name.replace('-', '_'))(args)                        
  File "/home/jaggz/kivy/tests/splash/.buildozer/android/platform/python-for-androi$
/pythonforandroid/toolchain.py", line 155, in wrapper_func                            
    build_dist_from_args(ctx, dist, args)                                             
  File "/home/jaggz/kivy/tests/splash/.buildozer/android/platform/python-for-androi$
/pythonforandroid/toolchain.py", line 209, in build_dist_from_args                    
    args, "ignore_setup_py", False                                                    
  File "/home/jaggz/kivy/tests/splash/.buildozer/android/platform/python-for-androi$
/pythonforandroid/build.py", line 605, in build_recipes                               
    ignore_setup_py=ignore_project_setup_py                                           
  File "/home/jaggz/kivy/tests/splash/.buildozer/android/platform/python-for-androi$
/pythonforandroid/build.py", line 791, in run_pymodules_install                       
    ), _env=copy.copy(base_env))                                                      
  File "/home/jaggz/kivy/tests/splash/.buildozer/android/platform/python-for-androi$
/pythonforandroid/logger.py", line 179, in shprint                                    
    output = command(*args, **kwargs)                                                 
  File "/home/jaggz/venv-3.6/lib/python3.7/site-packages/sh.py", line 1474, in __ca$
l__                                                                                   
    extracted_call_args, kwargs = self._extract_call_args(kwargs)                     
  File "/home/jaggz/venv-3.6/lib/python3.7/site-packages/sh.py", line 1377, in _ext$
act_call_args                                                                         
    raise TypeError("Invalid special arguments:\n\n%s\n" % exc_msg)                   
TypeError: Invalid special arguments:                                                 

  'env': env must be a dict. Got environ({'PROFILEHOME': '',

Source

jaggzh

Most helpful comment

That's not what I meant. I mean from your buildozer.spec file. Use p4a.branch = develop

AndreMiras on 30 Apr 2020

👍3 ❤1

All 11 comments

Yes this just got fixed upstream via https://github.com/kivy/python-for-android/pull/2160
Please buildozer android update

AndreMiras on 29 Apr 2020

$ buildozer android update
Results in the following. Separate issue?

```[INFO]: <- directory context /home/jaggz/kivy/tests/splash/.buildozer/android/platform/python-for-android
working: Already using interpreter /home/jaggz/venv-3.6/bin/python3
working: Using base prefix '/usr'
working: New python executable in /home/jaggz/kivy/tes...(and 78 more)
working: Not overwriting existing python script /home/br...(and 207 more)
working: Installing setuptools, pip, wheel...
working: done.
Traceback (most recent call last):
File "/usr/lib/python3.7/runpy.py", line 193, in _run_module_as_main
"__main__", mod_spec)
File "/usr/lib/python3.7/runpy.py", line 85, in _run_code
exec(code, run_globals)
File "/home/jaggz/kivy/tests/splash/.buildozer/android/platform/python-for-android/pythonforandroid/toolchain.py", line 1199, in
main()
File "/home/jaggz/kivy/tests/splash/.buildozer/android/platform/python-for-android/pythonforandroid/entrypoints.py", line 18, in main
ToolchainCL()
File "/home/jaggz/kivy/tests/splash/.buildozer/android/platform/python-for-android/pythonforandroid/toolchain.py", line 671, in __init__
getattr(self, args.subparser_name.replace('-', '_'))(args)
File "/home/jaggz/kivy/tests/splash/.buildozer/android/platform/python-for-android/pythonforandroid/toolchain.py", line 155, in wrapper_func
build_dist_from_args(ctx, dist, args)
File "/home/jaggz/kivy/tests/splash/.buildozer/android/platform/python-for-android/pythonforandroid/toolchain.py", line 209, in build_dist_from_args
args, "ignore_setup_py", False
File "/home/jaggz/kivy/tests/splash/.buildozer/android/platform/python-for-android/pythonforandroid/build.py", line 605, in build_recipes
ignore_setup_py=ignore_project_setup_py
File "/home/jaggz/kivy/tests/splash/.buildozer/android/platform/python-for-android/pythonforandroid/build.py", line 791, in run_pymodules_install
), _env=copy.copy(base_env))
File "/home/jaggz/kivy/tests/splash/.buildozer/android/platform/python-for-android/pythonforandroid/logger.py", line 179, in shprint
output = command(args, *kwargs)
File "/home/jaggz/venv-3.6/lib/python3.7/site-packages/sh.py", line 1474, in __call__
extracted_call_args, kwargs = self._extract_call_args(kwargs)
File "/home/jaggz/venv-3.6/lib/python3.7/site-packages/sh.py", line 1377, in _extract_call_args
raise TypeError("Invalid special arguments:\n\n%s\n" % exc_msg)
TypeError: Invalid special arguments:
{p...}

Command failed: /home/jaggz/venv-3.6/bin/python3 -m pythonforandroid.toolchain create --dist_name=myapp --bootstrap=sdl2 --requirements=python3,kivy --arch armeabi-v7a --copy-libs --color=always --storage-dir="/home/jaggz/kivy/tests/splash/.buildozer/android/platform/build-armeabi-v7a" --ndk-api=21

ENVIRONMENT:

SHELL = '/bin/bash'

```

jaggzh on 29 Apr 2020

Yes if p4a was installed from pip, try removing it first. Also maybe manually delete .buildozer/ directory from your project dir rm -rf ~/kivy/tests/splash/.buildozer to be sure to be sure and try again

AndreMiras on 29 Apr 2020

Okay, still getting those errors:

[INFO]:    Postbuilding android for armeabi-v7a
[INFO]:    Postbuilding kivy for armeabi-v7a
[INFO]:    # Installing pure Python modules
[INFO]:    *** PYTHON PACKAGE / PROJECT INSTALL STAGE ***
[INFO]:    The requirements (certifi) don't have recipes, attempting to install them with pip
[INFO]:    If this fails, it may mean that the module has compiled components and needs a recipe.
[INFO]:    -> directory context /home/jaggz/kivy/tests/splash/.buildozer/android/platform/build-armeabi-v7a/build
[INFO]:    -> running virtualenv --python=python3 venv

           working: Already using interpreter /home/jaggz/venv-3.6/bin/python3
           working: Using base prefix '/usr'
           working: New python executable in /home/jaggz/kivy/tes...(and 78 more)
           working: Also creating executable in /home/jaggz/kivy/...(and 80 more)
           working: Installing setuptools, pip, wheel...
           working: done.

[INFO]:    Upgrade pip to latest version
[INFO]:    -> running bash -c source venv/bin/activate && pip install -U pip
Traceback (most recent call last):
  File "/usr/lib/python3.7/runpy.py", line 193, in _run_module_as_main
    "__main__", mod_spec)
  File "/usr/lib/python3.7/runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "/home/jaggz/kivy/tests/splash/.buildozer/android/platform/python-for-android/pythonforandroid/toolchain.py", line 1199, in <module>
    main()
  File "/home/jaggz/kivy/tests/splash/.buildozer/android/platform/python-for-android/pythonforandroid/entrypoints.py", line 18, in main
    ToolchainCL()
  File "/home/jaggz/kivy/tests/splash/.buildozer/android/platform/python-for-android/pythonforandroid/toolchain.py", line 671, in __init__
    getattr(self, args.subparser_name.replace('-', '_'))(args)
  File "/home/jaggz/kivy/tests/splash/.buildozer/android/platform/python-for-android/pythonforandroid/toolchain.py", line 155, in wrapper_func
    build_dist_from_args(ctx, dist, args)
  File "/home/jaggz/kivy/tests/splash/.buildozer/android/platform/python-for-android/pythonforandroid/toolchain.py", line 209, in build_dist_from_args
    args, "ignore_setup_py", False
  File "/home/jaggz/kivy/tests/splash/.buildozer/android/platform/python-for-android/pythonforandroid/build.py", line 605, in build_recipes
    ignore_setup_py=ignore_project_setup_py
  File "/home/jaggz/kivy/tests/splash/.buildozer/android/platform/python-for-android/pythonforandroid/build.py", line 791, in run_pymodules_install
    ), _env=copy.copy(base_env))
  File "/home/jaggz/kivy/tests/splash/.buildozer/android/platform/python-for-android/pythonforandroid/logger.py", line 179, in shprint
    output = command(*args, **kwargs)
  File "/home/jaggz/venv-3.6/lib/python3.7/site-packages/sh.py", line 1474, in __call__
    extracted_call_args, kwargs = self._extract_call_args(kwargs)
  File "/home/jaggz/venv-3.6/lib/python3.7/site-packages/sh.py", line 1377, in _extract_call_args
    raise TypeError("Invalid special arguments:\n\n%s\n" % exc_msg)
TypeError: Invalid special arguments:

{p...}

# Command failed: /home/jaggz/venv-3.6/bin/python3 -m pythonforandroid.toolchain create --dist_name=myapp --bootstrap=sdl2 --requirements=python3,kivy --arch armeabi-v7a --copy-libs --color=always --storage-dir="/home/jaggz/kivy/tests/splash/.buildozer/android/platform/build-armeabi-v7a" --ndk-api=21
# ENVIRONMENT:
#     SHELL = '/bin/bash'

jaggzh on 29 Apr 2020

Which p4a.branch are you in? You should usually share your buildozer.spec file too

AndreMiras on 29 Apr 2020

[Edit]
"which python3.7" shows /usr/bin/python3.7

[Original post]
After pip removing python-for-android, I did not install it manually again. I just went right to buildozer android update.
Note that "which python3" shows:
/home/brandon/venv-3.6/bin/python3
"which pip":
/home/brandon/venv-3.6/bin/pip
In that log the error shows it's using:
/usr/lib/python3.7/runpy.py

My buildozer.spec:

[app]

# (str) Title of your application
title = My Application

# (str) Package name
package.name = myapp

# (str) Package domain (needed for android/ios packaging)
package.domain = org.test

# (str) Source code where the main.py live
source.dir = .

# (list) Source files to include (let empty to include all the files)
source.include_exts = py,png,jpg,kv,atlas,gif,mp4,mkv

# (list) List of inclusions using pattern matching
#source.include_patterns = assets/*,images/*.png

# (list) Source files to exclude (let empty to not exclude anything)
#source.exclude_exts = spec

# (list) List of directory to exclude (let empty to not exclude anything)
#source.exclude_dirs = tests, bin

# (list) List of exclusions using pattern matching
#source.exclude_patterns = license,images/*/*.jpg

# (str) Application versioning (method 1)
version = 0.1

# (str) Application versioning (method 2)
# version.regex = __version__ = ['"](.*)['"]
# version.filename = %(source.dir)s/main.py

# (list) Application requirements
# comma separated e.g. requirements = sqlite3,kivy
#requirements = python3,kivy,ffpyplayer,ffpyplayer_codecs
requirements = python3,kivy
#requirements = python3,kivy==1.11.0

# (str) Custom source folders for requirements
# Sets custom source for any requirements with recipes
# requirements.source.kivy = ../../kivy

# (list) Garden requirements
#garden_requirements =

# (str) Presplash of the application
#presplash.filename = %(source.dir)s/data/presplash.png

# (str) Icon of the application
#icon.filename = %(source.dir)s/data/icon.png

# (str) Supported orientation (one of landscape, sensorLandscape, portrait or all)
orientation = portrait

# (list) List of service to declare
#services = NAME:ENTRYPOINT_TO_PY,NAME2:ENTRYPOINT2_TO_PY

#
# OSX Specific
#

#
# author = © Copyright Info

# change the major version of python used by the app
osx.python_version = 3

# Kivy version to use
osx.kivy_version = 1.9.1

#
# Android specific
#

# (bool) Indicate if the application should be fullscreen or not
fullscreen = 0

# (string) Presplash background color (for new android toolchain)
# Supported formats are: #RRGGBB #AARRGGBB or one of the following names:
# red, blue, green, black, white, gray, cyan, magenta, yellow, lightgray,
# darkgray, grey, lightgrey, darkgrey, aqua, fuchsia, lime, maroon, navy,
# olive, purple, silver, teal.
#android.presplash_color = #FFFFFF

# (list) Permissions
#android.permissions = INTERNET

# (int) Target Android API, should be as high as possible.
#android.api = 27

# (int) Minimum API your APK will support.
#android.minapi = 21

# (int) Android SDK version to use
#android.sdk = 20

# (str) Android NDK version to use
#android.ndk = 17c

# (int) Android NDK API to use. This is the minimum API your app will support, it should usually match android.minapi.
#android.ndk_api = 21

# (bool) Use --private data storage (True) or --dir public storage (False)
#android.private_storage = True

# (str) Android NDK directory (if empty, it will be automatically downloaded.)
#android.ndk_path = /home/jaggz/Android/Sdk/ndk-bundle

# (str) Android SDK directory (if empty, it will be automatically downloaded.)
#android.sdk_path = /home/jaggz/Android/Sdk/

# (str) ANT directory (if empty, it will be automatically downloaded.)
#android.ant_path = /home/jaggz/projects/android-studio/lib/ant/

# (bool) If True, then skip trying to update the Android sdk
# This can be useful to avoid excess Internet downloads or save time
# when an update is due and you just want to test/build your package
# android.skip_update = False

# (bool) If True, then automatically accept SDK license
# agreements. This is intended for automation only. If set to False,
# the default, you will be shown the license when first running
# buildozer.
# android.accept_sdk_license = False

# (str) Android entry point, default is ok for Kivy-based app
#android.entrypoint = org.renpy.android.PythonActivity

# (str) Android app theme, default is ok for Kivy-based app
# android.apptheme = "@android:style/Theme.NoTitleBar"

# (list) Pattern to whitelist for the whole project
#android.whitelist =

# (str) Path to a custom whitelist file
#android.whitelist_src =

# (str) Path to a custom blacklist file
#android.blacklist_src =

# (list) List of Java .jar files to add to the libs so that pyjnius can access
# their classes. Don't add jars that you do not need, since extra jars can slow
# down the build process. Allows wildcards matching, for example:
# OUYA-ODK/libs/*.jar
#android.add_jars = foo.jar,bar.jar,path/to/more/*.jar

# (list) List of Java files to add to the android project (can be java or a
# directory containing the files)
#android.add_src =

# (list) Android AAR archives to add (currently works only with sdl2_gradle
# bootstrap)
#android.add_aars =

# (list) Gradle dependencies to add (currently works only with sdl2_gradle
# bootstrap)
#android.gradle_dependencies =

# (list) add java compile options
# this can for example be necessary when importing certain java libraries using the 'android.gradle_dependencies' option
# see https://developer.android.com/studio/write/java8-support for further information
# android.add_compile_options = "sourceCompatibility = 1.8", "targetCompatibility = 1.8"

# (list) Gradle repositories to add {can be necessary for some android.gradle_dependencies}
# please enclose in double quotes 
# e.g. android.gradle_repositories = "maven { url 'https://kotlin.bintray.com/ktor' }"
#android.add_gradle_repositories =

# (list) packaging options to add 
# see https://google.github.io/android-gradle-dsl/current/com.android.build.gradle.internal.dsl.PackagingOptions.html
# can be necessary to solve conflicts in gradle_dependencies
# please enclose in double quotes 
# e.g. android.add_packaging_options = "exclude 'META-INF/common.kotlin_module'", "exclude 'META-INF/*.kotlin_module'"
#android.add_gradle_repositories =

# (list) Java classes to add as activities to the manifest.
#android.add_activites = com.example.ExampleActivity

# (str) OUYA Console category. Should be one of GAME or APP
# If you leave this blank, OUYA support will not be enabled
#android.ouya.category = GAME

# (str) Filename of OUYA Console icon. It must be a 732x412 png image.
#android.ouya.icon.filename = %(source.dir)s/data/ouya_icon.png

# (str) XML file to include as an intent filters in <activity> tag
#android.manifest.intent_filters =

# (str) launchMode to set for the main activity
#android.manifest.launch_mode = standard

# (list) Android additional libraries to copy into libs/armeabi
#android.add_libs_armeabi = libs/android/*.so
#android.add_libs_armeabi_v7a = libs/android-v7/*.so
#android.add_libs_arm64_v8a = libs/android-v8/*.so
#android.add_libs_x86 = libs/android-x86/*.so
#android.add_libs_mips = libs/android-mips/*.so

# (bool) Indicate whether the screen should stay on
# Don't forget to add the WAKE_LOCK permission if you set this to True
#android.wakelock = False

# (list) Android application meta-data to set (key=value format)
#android.meta_data =

# (list) Android library project to add (will be added in the
# project.properties automatically.)
#android.library_references =

# (list) Android shared libraries which will be added to AndroidManifest.xml using <uses-library> tag
#android.uses_library =

# (str) Android logcat filters to use
#android.logcat_filters = *:S python:D

# (bool) Copy library instead of making a libpymodules.so
#android.copy_libs = 1

# (str) The Android arch to build for, choices: armeabi-v7a, arm64-v8a, x86, x86_64
android.arch = armeabi-v7a

#
# Python for android (p4a) specific
#

# (str) python-for-android fork to use, defaults to upstream (kivy)
#p4a.fork = kivy

# (str) python-for-android branch to use, defaults to master
#p4a.branch = master

# (str) python-for-android git clone directory (if empty, it will be automatically cloned from github)
#p4a.source_dir =

# (str) The directory in which python-for-android should look for your own build recipes (if any)
#p4a.local_recipes =

# (str) Filename to the hook for p4a
#p4a.hook =

# (str) Bootstrap to use for android builds
# p4a.bootstrap = sdl2

# (int) port number to specify an explicit --port= p4a argument (eg for bootstrap flask)
#p4a.port =


#
# iOS specific
#

# (str) Path to a custom kivy-ios folder
#ios.kivy_ios_dir = ../kivy-ios
# Alternately, specify the URL and branch of a git checkout:
ios.kivy_ios_url = https://github.com/kivy/kivy-ios
ios.kivy_ios_branch = master

# Another platform dependency: ios-deploy
# Uncomment to use a custom checkout
#ios.ios_deploy_dir = ../ios_deploy
# Or specify URL and branch
ios.ios_deploy_url = https://github.com/phonegap/ios-deploy
ios.ios_deploy_branch = 1.7.0

# (str) Name of the certificate to use for signing the debug version
# Get a list of available identities: buildozer ios list_identities
#ios.codesign.debug = "iPhone Developer: <lastname> <firstname> (<hexstring>)"

# (str) Name of the certificate to use for signing the release version
#ios.codesign.release = %(ios.codesign.debug)s


[buildozer]

# (int) Log level (0 = error only, 1 = info, 2 = debug (with command output))
log_level = 2

# (int) Display warning if buildozer is run as root (0 = False, 1 = True)
warn_on_root = 1

# (str) Path to build artifact storage, absolute or relative to spec file
# build_dir = ./.buildozer

# (str) Path to build output (i.e. .apk, .ipa) storage
# bin_dir = ./bin

#    -----------------------------------------------------------------------------
#    List as sections
#
#    You can define all the "list" as [section:key].
#    Each line will be considered as a option to the list.
#    Let's take [app] / source.exclude_patterns.
#    Instead of doing:
#
#[app]
#source.exclude_patterns = license,data/audio/*.wav,data/images/original/*
#
#    This can be translated into:
#
#[app:source.exclude_patterns]
#license
#data/audio/*.wav
#data/images/original/*
#


#    -----------------------------------------------------------------------------
#    Profiles
#
#    You can extend section / key with a profile
#    For example, you want to deploy a demo version of your application without
#    HD content. You could first change the title to add "(demo)" in the name
#    and extend the excluded directories to remove the HD content.
#
#[app@demo]
#title = My Application (demo)
#
#[app:source.exclude_patterns@demo]
#images/hd/*
#
#    Then, invoke the command line with the "demo" profile:
#
#buildozer --profile demo android debug

jaggzh on 29 Apr 2020

Try with p4a.branch = develop because the fix is on p4a develop

AndreMiras on 29 Apr 2020

Okay, I did:
$ pip install git+git://github.com/kivy/python-for-android.git@develop
$ rm -rf .buildozer
$ buildozer android update

It actually gives me the same TypeError again, and the env must be a dict error returned.

  File "/home/jaggz/venv-3.6/lib/python3.7/site-packages/sh.py", line 1377, in _extr
act_call_args                                                                         
    raise TypeError("Invalid special arguments:\n\n%s\n" % exc_msg)
TypeError: Invalid special arguments:

  'env': env must be a dict.

jaggzh on 30 Apr 2020

That's not what I meant. I mean from your buildozer.spec file. Use p4a.branch = develop

AndreMiras on 30 Apr 2020

👍3 ❤1

Changing p4a.branch = develop inbuildozer.spec helped me today with exactly same issue.