Python-for-android: requests module not compiling in buildozer when used with Python3crystax

[VineethRaghavan]

The recipe for the requests module gives a conflict error with regard to hostpython2 when I try to compile it as a dependency of kivy with python3crystax on buildozer. Please fix it.

Here are the logs:

kivy@kivyvm:~/Desktop$ python3.6 -m buildozer -v android debug
# Check configuration tokens
# Ensure build layout
# Check configuration tokens
# Preparing build
# Check requirements for android
# Run 'dpkg --version'
# Cwd None
Debian 'dpkg' package management program version 1.18.10 (amd64).
This is free software; see the GNU General Public License version 2 or
later for copying conditions. There is NO warranty.
# Search for Git (git)
#  -> found at /usr/bin/git
# Search for Cython (cython)
#  -> found at /usr/local/bin/cython
# Search for Java compiler (javac)
#  -> found at /usr/lib/jvm/java-8-openjdk-amd64/bin/javac
# Search for Java keytool (keytool)
#  -> found at /usr/lib/jvm/java-8-openjdk-amd64/jre/bin/keytool
# Install platform
# Run 'pip install -q --user "appdirs" "colorama>=0.3.3" "sh>=1.10" "jinja2" "six"'
# Cwd None
# Apache ANT found at /home/kivy/.buildozer/android/platform/apache-ant-1.9.4
# Android SDK found at /home/kivy/.buildozer/android/platform/android-sdk-20
# Android NDK found at /media/sf_windows_desktop/crystax-ndk-10.3.2
# Check application requirements
# Check garden requirements
# Compile platform
# Run '/usr/bin/python3.6 -m pythonforandroid.toolchain create --dist_name=app --bootstrap=sdl2 --requirements=python3crystax==3.6,requests,pyrebase,kivy==master,git+https://gitlab.com/kivymd/KivyMD.git,pyrebase --arch armeabi-v7a --copy-libs --color=always --storage-dir=/home/kivy/Desktop/.buildozer/android/platform/build'
# Cwd /home/kivy/Desktop/.buildozer/android/platform/python-for-android-master
[INFO]:    Recipe python3crystax: version "3.6" requested
[INFO]:    Recipe kivy: version "master" requested
[INFO]:    Will compile for the following archs: armeabi-v7a
[INFO]:    Found Android API target in $ANDROIDAPI
[INFO]:    Available Android APIs are (19)
[INFO]:    Requested API target 19 is available, continuing.
[INFO]:    Found NDK dir in $ANDROIDNDK
[INFO]:    Got NDK version from $ANDROIDNDKVER
[WARNING]: NDK version was set as r9c, but checking the NDK dir claims it is 10.3.2.
[WARNING]: The build will try to continue, but it may fail and you should check that your setting is correct.
[WARNING]: If the NDK dir result is correct, you don't need to manually set the NDK ver.
[INFO]:    Using Crystax NDK r9c
[INFO]:    Found virtualenv at /usr/local/bin/virtualenv
[INFO]:    ccache is missing, the build will not be optimized in the future.
[INFO]:    Found the following toolchain versions: ['4.9', '5', 'clang3.6', 'clang3.7']
[INFO]:    Picking the latest gcc toolchain, here 5
[WARNING]: Missing executable: autoconf is not installed
[WARNING]: Missing executable: automake is not installed
[WARNING]: Missing executable: libtoolize is not installed
[INFO]:    No existing dists meet the given requirements!
[INFO]:    No dist exists that meets your requirements, so one will be built.
[INFO]:    Bootstrap requires recipes ['sdl2', ('python2', 'python3crystax')]
[INFO]:    Loaded recipe sdl2 (depends on [('python2', 'python3crystax'), 'sdl2_image', 'sdl2_mixer', 'sdl2_ttf'], conflicts ['sdl', 'pygame', 'pygame_bootstrap_components'])
[INFO]:    Loaded recipe kivy (depends on [('sdl2', 'pygame'), 'pyjnius', ('python2', 'python3crystax')])
[INFO]:    No recipe named git+https://gitlab.com/kivymd/KivyMD.git; will attempt to install with pip
[INFO]:    No recipe named pyrebase; will attempt to install with pip
[INFO]:    Loaded recipe requests (depends on ['hostpython2', 'setuptools'])
[INFO]:    Loaded recipe python3crystax (depends on ['hostpython3crystax'], conflicts ['python2', 'python3'])
[INFO]:    Loaded recipe sdl2_image (depends on [])
[INFO]:    Loaded recipe sdl2_mixer (depends on [])
[INFO]:    Loaded recipe sdl2_ttf (depends on [])
[INFO]:    Loaded recipe pyjnius (depends on [('python2', 'python3crystax'), ('sdl2', 'sdl', 'genericndkbuild'), 'six'])
[INFO]:    Loaded recipe hostpython2 (depends on [], conflicts ['hostpython3'])
[INFO]:    Loaded recipe setuptools (depends on ['python2'])
[INFO]:    Loaded recipe hostpython3crystax (depends on [], conflicts ['hostpython2'])
[WARNING]: hostpython3crystax conflicts with hostpython2, but both have been included or pulled into the requirements.
[WARNING]: Due to this conflict the build cannot continue, exiting.
# Command failed: /usr/bin/python3.6 -m pythonforandroid.toolchain create --dist_name=app --bootstrap=sdl2 --requirements=python3crystax==3.6,requests,pyrebase,kivy==master,git+https://gitlab.com/kivymd/KivyMD.git,pyrebase --arch armeabi-v7a --copy-libs --color=always --storage-dir=/home/kivy/Desktop/.buildozer/android/platform/build
# 
# Buildozer failed to execute the last command
# The error might be hidden in the log above this error
# Please read the full log, and search for it before
# raising an issue with buildozer itself.
# In case of a bug report, please add a full log with log_level = 2
kivy@kivyvm:~/Desktop$ 

And here is the spec file:

[app]

# (str) Title of your application
title = app

# (str) Package name
package.name = app

# (str) Package domain (needed for android/ios packaging)
package.domain = org.app

# (str) Source code where the main.py live
source.dir = .

# (list) Source files to include (let empty to include all the files)
source.include_exts = py,png,jpg,kv,atlas

# (list) List of inclusions using pattern matching
#source.include_patterns = assets/*,images/*.png

# (list) Source files to exclude (let empty to not exclude anything)
#source.exclude_exts = spec

# (list) List of directory to exclude (let empty to not exclude anything)
#source.exclude_dirs = tests, bin

# (list) List of exclusions using pattern matching
#source.exclude_patterns = license,images/*/*.jpg

# (str) Application versioning (method 1)
version = 0.1

# (str) Application versioning (method 2)
# version.regex = __version__ = ['"](.*)['"]
# version.filename = %(source.dir)s/main.py

# (list) Application requirements
# comma seperated e.g. requirements = sqlite3,kivy
requirements = python3crystax==3.6, requests, pyrebase, kivy==master, git+https://gitlab.com/kivymd/KivyMD.git

# (str) Custom source folders for requirements
# Sets custom source for any requirements with recipes
# requirements.source.kivy = ../../kivy

# (list) Garden requirements
#garden_requirements =

# (str) Presplash of the application
#presplash.filename = %(source.dir)s/data/presplash.png

# (str) Icon of the application
#icon.filename = %(source.dir)s/data/icon.png

# (str) Supported orientation (one of landscape, portrait or all)
orientation = portrait

# (list) List of service to declare
#services = NAME:ENTRYPOINT_TO_PY,NAME2:ENTRYPOINT2_TO_PY

#
# OSX Specific
#

#
# author = © Copyright Info

#
# Android specific
#

# (bool) Indicate if the application should be fullscreen or not
fullscreen = 1

# (list) Permissions
android.permissions = INTERNET

# (int) Android API to use
#android.api = 19

# (int) Minimum API required
#android.minapi = 9

# (int) Android SDK version to use
#android.sdk = 20

# (str) Android NDK version to use
#android.ndk = 9c

# (bool) Use --private data storage (True) or --dir public storage (False)
#android.private_storage = True

# (str) Android NDK directory (if empty, it will be automatically downloaded.)
android.ndk_path = /media/sf_windows_desktop/crystax-ndk-10.3.2/

# (str) Android SDK directory (if empty, it will be automatically downloaded.)
#android.sdk_path =

# (str) ANT directory (if empty, it will be automatically downloaded.)
#android.ant_path =

# (str) python-for-android git clone directory (if empty, it will be automatically cloned from github)
#android.p4a_dir =

# (list) python-for-android whitelist
#android.p4a_whitelist =

# (bool) If True, then skip trying to update the Android sdk
# This can be useful to avoid excess Internet downloads or save time
# when an update is due and you just want to test/build your package
# android.skip_update = False

# (str) Android entry point, default is ok for Kivy-based app
#android.entrypoint = org.renpy.android.PythonActivity

# (list) List of Java .jar files to add to the libs so that pyjnius can access
# their classes. Don't add jars that you do not need, since extra jars can slow
# down the build process. Allows wildcards matching, for example:
# OUYA-ODK/libs/*.jar
#android.add_jars = foo.jar,bar.jar,path/to/more/*.jar

# (list) List of Java files to add to the android project (can be java or a
# directory containing the files)
#android.add_src =

# (str) python-for-android branch to use, if not master, useful to try
# not yet merged features.
#android.branch = master

# (str) OUYA Console category. Should be one of GAME or APP
# If you leave this blank, OUYA support will not be enabled
#android.ouya.category = GAME

# (str) Filename of OUYA Console icon. It must be a 732x412 png image.
#android.ouya.icon.filename = %(source.dir)s/data/ouya_icon.png

# (str) XML file to include as an intent filters in <activity> tag
#android.manifest.intent_filters =

# (list) Android additionnal libraries to copy into libs/armeabi
#android.add_libs_armeabi = libs/android/*.so
#android.add_libs_armeabi_v7a = libs/android-v7/*.so
#android.add_libs_x86 = libs/android-x86/*.so
#android.add_libs_mips = libs/android-mips/*.so

# (bool) Indicate whether the screen should stay on
# Don't forget to add the WAKE_LOCK permission if you set this to True
#android.wakelock = False

# (list) Android application meta-data to set (key=value format)
#android.meta_data =

# (list) Android library project to add (will be added in the
# project.properties automatically.)
#android.library_references =

# (str) Android logcat filters to use
#android.logcat_filters = *:S python:D

# (bool) Copy library instead of making a libpymodules.so
#android.copy_libs = 1

#
# iOS specific
#

# (str) Path to a custom kivy-ios folder
#ios.kivy_ios_dir = ../kivy-ios

# (str) Name of the certificate to use for signing the debug version
# Get a list of available identities: buildozer ios list_identities
#ios.codesign.debug = "iPhone Developer: <lastname> <firstname> (<hexstring>)"

# (str) Name of the certificate to use for signing the release version
#ios.codesign.release = %(ios.codesign.debug)s


[buildozer]

# (int) Log level (0 = error only, 1 = info, 2 = debug (with command output))
log_level = 1

# (int) Display warning if buildozer is run as root (0 = False, 1 = True)
warn_on_root = 1

# (str) Path to build artifact storage, absolute or relative to spec file
# build_dir = ./.buildozer

# (str) Path to build output (i.e. .apk, .ipa) storage
# bin_dir = ./bin

#    -----------------------------------------------------------------------------
#    List as sections
#
#    You can define all the "list" as [section:key].
#    Each line will be considered as a option to the list.
#    Let's take [app] / source.exclude_patterns.
#    Instead of doing:
#
#[app]
#source.exclude_patterns = license,data/audio/*.wav,data/images/original/*
#
#    This can be translated into:
#
#[app:source.exclude_patterns]
#license
#data/audio/*.wav
#data/images/original/*
#


#    -----------------------------------------------------------------------------
#    Profiles
#
#    You can extend section / key with a profile
#    For example, you want to deploy a demo version of your application without
#    HD content. You could first change the title to add "(demo)" in the name
#    and extend the excluded directories to remove the HD content.
#
#[app@demo]
#title = My Application (demo)
#
#[app:source.exclude_patterns@demo]
#images/hd/*
#
#    Then, invoke the command line with the "demo" profile:
#
#buildozer --profile demo android debug

[KeyWeeUsr]

Please paste the log and spec file.

[VineethRaghavan]

I have pasted the log and spec file

[GhostNappa]

I had the same problem adding requests to my python3 android kivy app. I solved it by removing the 'requests' and 'idna' recipes from my local p4a repo used by buildozer.
Then added:
requests, urllib3, chardet, certifi, idna
to buildozer.spec requirements. The apk can now use requests.
What is the purpose of the requests recipe? Does it pull in these dependencies? It doesn't look like it to me, but I don't understand recipes well. Except that you only need them when the module isn't pure python. Requests does not appear to need a recipe, unless it can make it easy to pull in dependencies, same for idna.
In my experience, all it did was prevent me from being able to use requests with python3. I initially tried to change the dependencies in the requests recipe to need (python3crystax or python2) but to no avail. But as I said, I don't understand recipes.

[MaximeLibert]

Hi , i have the same problem , buildozer wouldnt debug if i add requests in the requirements. However i don't understand your solve to the problem ... Where do i find my p4a repo ?

[GhostNappa]

The p4a repo is inside the .buildozer folder for the project you are trying to package (sorry I can't check where exactly right now). However, if you also want to package for iOS do not try my solution. I can't remember why but it didn't work for iOS. I ended up not using requests as a result.

Do have a look at #1075... I never did somehow...

[MaximeLibert]

I only want to package for Android.

Here is my Buildozer.spec
(sorry for this cant find the command )

`[app]

(str) Title of your application

title = La Fabrique des voisins

(str) Package name

package.name = myapp3

(str) Package domain (needed for android/ios packaging)

package.domain = org.test

(str) Source code where the main.py live

source.dir = .

(list) Source files to include (let empty to include all the files)

source.include_exts = py,png,jpg,kv,atlas,ttf

(list) List of inclusions using pattern matching

source.include_patterns = assets/,images/.png

(list) Source files to exclude (let empty to not exclude anything)

source.exclude_exts = spec

(list) List of directory to exclude (let empty to not exclude anything)

source.exclude_dirs = tests, bin

(list) List of exclusions using pattern matching

source.exclude_patterns = license,images//.jpg

(str) Application versioning (method 1)

version = 0.1

(str) Application versioning (method 2)

version.regex = __version__ = '"['"]

version.filename = %(source.dir)s/main.py

(list) Application requirements

comma seperated e.g. requirements = sqlite3,kivy

requirements = kivy ,python3crystax , openssl, futures, requests

(str) Custom source folders for requirements

Sets custom source for any requirements with recipes

requirements.source.kivy = ../../kivy

(list) Garden requirements

garden_requirements = mapview

(str) Presplash of the application

presplash.filename = %(source.dir)s/data/icon.png

(str) Icon of the application

icon.filename = %(source.dir)s/data/icon1.png

(str) Supported orientation (one of landscape, portrait or all)

orientation = portrait

(list) List of service to declare

services = NAME:ENTRYPOINT_TO_PY,NAME2:ENTRYPOINT2_TO_PY

#

OSX Specific

#

#

author = © Copyright Info

change the major version of python used by the app

osx.python_version = 3

Kivy version to use

osx.kivy_version = 1.9.1

#

Android specific

#

(bool) Indicate if the application should be fullscreen or not

fullscreen = 0

(string) Presplash background color (for new android toolchain)

Supported formats are: #RRGGBB #AARRGGBB or one of the following names:

red, blue, green, black, white, gray, cyan, magenta, yellow, lightgray,

darkgray, grey, lightgrey, darkgrey, aqua, fuchsia, lime, maroon, navy,

olive, purple, silver, teal.

android.presplash_color = #FFFFFF

(list) Permissions

android.permissions = INTERNET

(int) Android API to use

android.api = 19

(int) Minimum API required

android.minapi = 9

(int) Android SDK version to use

android.sdk = 20

(str) Android NDK version to use

android.ndk = 10.3.2

(bool) Use --private data storage (True) or --dir public storage (False)

android.private_storage = True

(str) Android NDK directory (if empty, it will be automatically downloaded.)

android.ndk_path = ~/crystax-ndk-10.3.2

(str) Android SDK directory (if empty, it will be automatically downloaded.)

android.sdk_path =

(str) ANT directory (if empty, it will be automatically downloaded.)

android.ant_path =

(bool) If True, then skip trying to update the Android sdk

This can be useful to avoid excess Internet downloads or save time

when an update is due and you just want to test/build your package

android.skip_update = False

(str) Android entry point, default is ok for Kivy-based app

android.entrypoint = org.renpy.android.PythonActivity

(list) Pattern to whitelist for the whole project

android.whitelist =

(str) Path to a custom whitelist file

android.whitelist_src =

(str) Path to a custom blacklist file

android.blacklist_src =

(list) List of Java .jar files to add to the libs so that pyjnius can access

their classes. Don't add jars that you do not need, since extra jars can slow

down the build process. Allows wildcards matching, for example:

OUYA-ODK/libs/*.jar

android.add_jars = foo.jar,bar.jar,path/to/more/*.jar

(list) List of Java files to add to the android project (can be java or a

directory containing the files)

android.add_src =

(list) Android AAR archives to add (currently works only with sdl2_gradle

bootstrap)

android.add_aars =

(list) Gradle dependencies to add (currently works only with sdl2_gradle

bootstrap)

android.gradle_dependencies =

(list) Java classes to add as activities to the manifest.

android.add_activites = com.example.ExampleActivity

(str) python-for-android branch to use, defaults to stable

p4a.branch = stable

(str) OUYA Console category. Should be one of GAME or APP

If you leave this blank, OUYA support will not be enabled

android.ouya.category = GAME

(str) Filename of OUYA Console icon. It must be a 732x412 png image.

android.ouya.icon.filename = %(source.dir)s/data/ouya_icon.png

(str) XML file to include as an intent filters in tag

android.manifest.intent_filters =

(list) Android additionnal libraries to copy into libs/armeabi

android.add_libs_armeabi = libs/android/*.so

android.add_libs_armeabi_v7a = libs/android-v7/*.so

android.add_libs_x86 = libs/android-x86/*.so

android.add_libs_mips = libs/android-mips/*.so

(bool) Indicate whether the screen should stay on

Don't forget to add the WAKE_LOCK permission if you set this to True

android.wakelock = False

(list) Android application meta-data to set (key=value format)

android.meta_data =

(list) Android library project to add (will be added in the

project.properties automatically.)

android.library_references =

(str) Android logcat filters to use

android.logcat_filters = *:S python:D

(bool) Copy library instead of making a libpymodules.so

android.copy_libs = 1

(str) The Android arch to build for, choices: armeabi-v7a, arm64-v8a, x86

android.arch = armeabi-v7a

#

Python for android (p4a) specific

#

(str) python-for-android git clone directory (if empty, it will be automatically cloned from github)

p4a.source_dir =

(str) The directory in which python-for-android should look for your own build recipes (if any)

p4a.local_recipes =

(str) Filename to the hook for p4a

p4a.hook =

(str) Bootstrap to use for android builds

p4a.bootstrap = sdl2

(int) port number to specify an explicit --port= p4a argument (eg for bootstrap flask)

p4a.port =

#

iOS specific

#

(str) Path to a custom kivy-ios folder

ios.kivy_ios_dir = ../kivy-ios

(str) Name of the certificate to use for signing the debug version

Get a list of available identities: buildozer ios list_identities

ios.codesign.debug = "iPhone Developer: ()"

(str) Name of the certificate to use for signing the release version

ios.codesign.release = %(ios.codesign.debug)s

[buildozer]

(int) Log level (0 = error only, 1 = info, 2 = debug (with command output))

log_level = 1

(int) Display warning if buildozer is run as root (0 = False, 1 = True)

warn_on_root = 1

(str) Path to build artifact storage, absolute or relative to spec file

build_dir = ./.buildozer

(str) Path to build output (i.e. .apk, .ipa) storage

bin_dir = ./bin

-----------------------------------------------------------------------------

List as sections

#

You can define all the "list" as [section:key].

Each line will be considered as a option to the list.

Let's take [app] / source.exclude_patterns.

Instead of doing:

#

[app]

source.exclude_patterns = license,data/audio/.wav,data/images/original/

#

This can be translated into:

#

[app:source.exclude_patterns]

license

data/audio/*.wav

data/images/original/*

#

-----------------------------------------------------------------------------

Profiles

#

You can extend section / key with a profile

For example, you want to deploy a demo version of your application without

HD content. You could first change the title to add "(demo)" in the name

and extend the excluded directories to remove the HD content.

#

[app@demo]

title = My Application (demo)

#

[app:source.exclude_patterns@demo]

images/hd/*

#

Then, invoke the command line with the "demo" profile:

#

buildozer --profile demo android debug

`

[MaximeLibert]

Oh and ty for your answer men !

[MaximeLibert]

OK i slept on it and i figured it out !! yourt solution actualy works !!

I solved it by removing the 'requests' and 'idna' recipes from my local p4a repo used by buildozer.
Then added:
requests, urllib3, chardet, certifi, idna

for the next persons to look for this issue

go to the repo where your buildozer.spec is then open the hiden folder named ".buildozer" -> android -> platform -> python-for-android-new-toolchain->pythonforandroid->recipies

and here you delete the requestes and idna folder

Then you add requests, urllib3, chardet, certifi, idna to the requirements lin in the buidozer.spec file

THANK YOU SO MUCH @GhostNappa

[VineethRaghavan]

Thanks a lot!
I guess this solves the issue then.
I'm closing this issue

[urbanonymous]

This is gold