Kiwix-android: WiFI Hostpot HTTP service mode

[Xeddius]

I feel this would be an important and incredibly useful feature given that most android devices can create wireless hotspots that others can connect to. Given the low power consumption of most android devices I feel this could also be helpful in small villages/towns or third world countries where access to more powerful devices is limited and often prohibitively expensive. The desktop edition of Kiwix comes packaged with the server, so I was confused as to why the android version doesn't have it.

[mhutti1]

@kelson42, @Xeddius suggested a wrapper app for kiwix serve using its arm build. Have you thought about this or would it be easier to implement it using kiwix-lib?

[Xeddius]

@mhutti1 I've been testing kiwix-serve on android via adb and so far it is completely functional after an hour of heavy usage.
kiwix-serve and other executables are being run at /data/local/tmp and the kiwix database is loaded from /sdcard/Kiwix/ the browser on my computer is able to use everything as if it was being hosted on any other machine. I think it may almost be trivial to pack it in with the apk.

An example design/layout for a companion app:

[kelson42]

@Xeddius How do you transform your mobile device in a WIFI hotspot which route all http requests to the kiwix-serve?

[kelson42]

@mhutti1 Regarding the implementation, I'm pretty much against a system call to start an external binary. @mgautierfr This might be the ultime reason why to integrate kiwix-serve code - as an option - within the kiwix-lib.

[Xeddius]

@kelson42 Most android devices can host a wireless access point. Every android device I have owned has had this feature. If you look at the IP of the android device and add the port you can connect to it just like you would when accessing kiwix-serve on a laptop on the local network.

Edit: To host the server properly I believe you'd have to choose a different port as default. I think android only allows executable access to port 80 for root processes. So port 8080 or 9080 should work fine.

Edit2: And if you implement a nameserver or captive-portal option you could route any traffic or url to the phone itself. thus removing the need to know the IP.

[kelson42]

@Xeddius AFAIK, "Edit2" is impossible to do properly :( Which makes the whole idea less worth :(

[okkebal]

This would be a great idea, even without a nameserver. The applications should say something like

---

Kiwix server is now available on: http://192.168.178.0.1:8080

To use Kiwix on another phone:
1 Create a wireless hotspot on this phone
2 Connect another phone to your wireless hotspot.
3 Open an internet browser on the other phone
4 In the adress bar type in: _http://192.168.178.0.1:8080_ and press enter

You can connect as many phones as you like, just repeat steps 2-4.

[Stop]

It should only show this when Kiwix server is running. If Kiwix server is not running it should explain what the app is for:

---

Welcome to Kiwix serve: share your Kiwix library with others without using the internet!

(nice to haves*)

The server is now Offline

Choose a port number (optional) : 8080

[Start]

The library selection should come from the Android app and should be the same as your "get content -on device" list: so just sharing your entire library.xml. Kiwix-serve itself should handle the file selection trough the selector-accordion, or whatever the new interface is.
The port changer should be there so people can troubleshoot. 8080, or some other optimal number, should be pre-filled. This port-changer would not be necessary if you could somehow make a smart routine that finds and unused port. But that would be another nice to have.

*Nice to haves:

• It would be nice to see which .zim files you are about to share. So below the "Welcome" line it would say: "Share these libraries: Wikipedia (nl) 2017-08-02, WikiSource (en) 2017-08-02" etc. maybe in a small, green font.
• It would be even nicer if that list was a checkbox-list, so you can choose which zim files to share, and which ones not to share. In that case no green font, but standard checkbox-silders in a list.

What do you think?

[Xeddius]

@kelson42 I have to agree with @okkebal. Even without a nameserver it would still be extremely useful. A nameserver wouldn't be impossible, just incredibly difficult, Bind9 has arm binaries. Besides, the nameserver is more of a longshot/side goal. The kiwix-serve program works on my android device and thus by extension so should a native server in the app.

[kelson42]

I had a look again a bit and it looks like we can pretty much do everything with the Hotspot from Kiwix itself: https://developer.android.com/reference/android/net/wifi/WifiManager.html. @mhutti1 We should assume the kiwix-lib is able to create the HTTP server, that means the rest would be in Kiwix for Android which should:

• Deal with DHCP request
• Deal with DNS requests
• Forward all HTTP traffic to an a web server listening on an internal socker or loopback port.

[sakchhams]

@kelson42 Since the concept relies on using local WiFi hotspot, it can be safely assumed that it's users are in close proximity. So a simple QR code display on the host device which can be scanned on other devices to access content should suffice. This would save the could that would be needed for handling DNS requests and would add an additional layer of security as well.

[kelson42]

@sakchhams Why should we do something "complicated" like this if we can avoid it?

[sakchhams]

@kelson42 I agree it's some work. But it's more convenient as well. Scanning QR codes is easier than typing IP addresses.

[kelson42]

@sakchhams this is the exaclty purpose of a DNS to avoid to type IP addresses... and this is more convenient than having a whole classroom ruching to scan a QRcode (which might not be so easy if you do not have a mobile phone).

[RohanBh]

Not taking sides here, but some smartphones have poor camera quality because of which it's not possible to scan the QR code.

[sakchhams]

@kelson42 Yes I agree. Makes total sense. I meant to suggest this as an alternative to be used if the DNS implementation fails. @RohanBh has a valid point as well.

[mrjohnc]

I'm trying to find programmers to help with this on Twitter (fingers crossed), retweets very welcome https://twitter.com/mrjohnc/status/979428219140149249

[jvonau]

I have experience with kiwix from my time spent with internet-in-a-box. The dhcp and dns part would be handled by android's hotspot feature by default, you can also make use of avahi(.local) if that is available on android. How is the http server started, bound to the lookback only? The $64,000 question does the android kernel have required iptables modules available for the redirect or can you have kiwix-serve bind to all available interfaces?

[kelson42]

@jvonau Good question indeed :) I hope we can start a daemon bound to the wireless network interface.

[mrjohnc]

Is there anything a non programmer can do to help make this happen? Very happy to beta test etc

[dschwen]

https://github.com/NanoHttpd/nanohttpd is a BSD licensed Java http server for Android.

[kelson42]

@dschwen The HTTP side is the least problem IMO as we have already a HTTP daemon running (soon in the C++ core). The challenge is all the WIFI/DNS/Networking configuration to get users connection via WIFI works properly and then the HTTP requests automatically routed to our HTTP daemon running. I'm not so familiar with nanohttpd, but it looks like this does not solve these "problems".

@mrjohnc Advertising is a good start ;) Otherwise find the money or find the developer.

[sakchhams]

@jvonau Yes, the server is bound to loopback only. However, incoming connections are redirected by default so that is not a problem. The $64,000 answer; No, android production builds do not include iptable modules. There are ways to set that up on android but none without root so I believe we'd have to think of something else.

[dschwen]

How do ftp server apps for transferring data to and from your phone to it then? For example https://github.com/wolpi/prim-ftpd

Is the problem that those all operate in an existing WiFi network rather than in hotspot mode?

[Xeddius]

@dschwen Kiwix-Serve already has a built in webserver, if we integrate the server into kiwix-lib we can just update the app to load the server on a port, worst case why not just make it so the kiwix app can also connect to a server running in the same network? That'd solve the issues with qrcodes and ip addresses. Plus, android supports "network service discovery" natively.

To clarify,

1. Make the server part of the kiwix-lib and able to launch from the android app.
2. Add a "Connect to local kiwix server." tab/button/option that then lets others connect using the same app running on the other devices.
3. The connections can use androids native "Network Service Discovery" library to find the server running on the hosted instance of kiwix.
4. The networking and hotspot part is already handled by android, no need to re-invent the wheel there.
5. The IP can be shown on the app hosting the server so that other devices (Like a laptop, wii, etc.) can connect to it with ease by just typing it in a web browser. EX: "Enter this into your browser/device http://192.168.42.1:8081"
6. If avahi can be implimented it could be as simple as using http://Phone-Name.local:8080/

[Xeddius]

I suggest we integrate this feature with the server as it'll make it easy for others to download a copy of the wiki databases. ZIM file transfer between two devices...

[mrjohnc]

Hi all

Is there progress on this issue? We are working on getting a table for Wikipedia at UNESCO Mobile Learning Week (the largest UN ICT in education conference) in March and this would be an amazing thing to demonstrate

https://en.unesco.org/mlw

[mhutti1]

Not as of this moment no. We are working on getting multi zim support and better downloading as our highest priorities.

[TheBrokenRail]

The FTC robotics app has a similur feature where it will start a hotspot which you can log on to and then go to self hosted website to edit code, the app is open source (most of the code is in the libs folder), so maybe that could help.

https://github.com/ftctechnh/ftc_app

[kelson42]

@mrjohnc No direct progress on this. https://github.com/kiwix/kiwix-lib/issues/138 is a requirement/blocker for this and it does not belong to the top of the priorities of the Kiwix Library. IMO, this might take quite a bit of time to get all of this working fine (many technical challenges in different areas of expertise) if nobody clearly decides to make the necessary investment in time/money/skills to make it work like it should.

[iadeelzafar]

I've cloned some sample apps that use NanoHttpd but they're unable to host the server on the device using wifi hotspot. The http server can only be hosted on the android device when it's connected to a wifi network.
After an android device has hosted a wireless access point, how can we host a webserver on it? Is it even possible? I tried to figure out a solution but couldn't find any valid resource to refer to. Please enlighten me if anyone here has knowledge about this.

[iadeelzafar]

This is the closest resource that I've found relevant to our use case.
https://circuits4you.com/2016/12/16/esp8266-web-server-ap/

[macgills]

I may have a fundamental misunderstanding of the problem here but we want a kiwix app user who has already downloaded files to easily be able to share them with another kiwix app user? https://developer.android.com/training/connect-devices-wirelessly/wifi-direct if so then this seems like the avenue we would want to take

[iadeelzafar]

@macgills no, it's not that. Although, I've found and implemented the solution. Thanks anyway!

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be now be reviewed manually. Thank you for your contributions.

[mrjohnc]

@kelson42 why was the issue closed? Has it been implemented?

[abdulwd]

Yes, it's implemented in #1327. You can pull the code from the develop branch and test/use it.

[mrjohnc]

@abdulwd amazing :)

[kelson42]

@mrjohnc Will be shipped with Kiwix for Android 3.0 during September.