Kolibri: 'Server 500' response when trying to view any of the H5P content

[intelliant01]

Observed behavior

Server 500 response within the content viewport when trying to view any H5P content imported from H5P Samples channel.

Expected behavior

H5P interactive content should have loaded correctly.

User-facing consequences

Can't use the content.

Errors and logs

Logs herein -
kolibri.txt

Steps to reproduce

1. Enable h5p_viewer plugin
2. Restart Kolibri
3. Import content from H5P Samples channel
4. Navigate in Learn to view content

Context

Firefox on Windows 10.

Some information from Device - Info:
Version: 0.13.2 OS: Windows-8-6.2.9200 Python: 3.4.3 Installer: Windows Server: Kolibri internal server

[intelliant01]

This works fine in the following environment -

Version:           0.13.2
OS:                Linux-5.3.7-050307-generic-x86_64-with-Ubuntu-18.04-bionic
Python:            3.6.9
Installer:         kolibri(apt) with kolibri-server Version: 0.3.8~beta2-0ubuntu1
Server:            nginx/1.14.0 (Ubuntu)

[rtibbles]

Thanks for the feedback @intelliant01 - sounds like there is an issue with this on Windows, very helpful to know.

[indirectlylit]

@mrpau-dev would you mind investigating?

[ivanistheone]

FYI, the command for Step 1 is

kolibri plugin enable kolibri.plugins.h5p_viewer

and Step 2 restart is needed for the plugin-initialization-code to run.

And the channel with H5P samples has the token piman-fapum

[mrpau-richard]

I reproduced this in Windows 7, Upgrading the Python version from 3.4 to 3.6 will fix the issue on Windows.

[rtibbles]

Do you have a fuller traceback @mrpau-richard - presumably we could fix the underlying code as well?

[cpauya]

For reference, here's the last few lines of the kolibri.txt log file attached by @intelliant01 above:

INFO 2020-04-28 08:06:44,128 cherrypy.access.115203152 127.0.0.1 - - "GET /static/kolibri.plugins.h5p_viewer.main/kolibri.plugins.h5p_viewer.main-0.13.2.js HTTP/1.1" 200 5730 "http://127.0.0.1:8080/en/learn/" "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:75.0) Gecko/20100101 Firefox/75.0"
ERROR 2020-04-28 08:06:44,171 exception Internal Server Error: /zipcontent/dda40b9ee291ef85d0911ccba551aba4.h5p/
Traceback (most recent call last):
  File "C:\Python34\lib\site-packages\kolibri\dist\django\core\handlers\exception.py", line 41, in inner
    response = get_response(request)
  File "C:\Python34\lib\site-packages\kolibri\dist\django\core\handlers\base.py", line 187, in _get_response
    response = self.process_exception_by_middleware(e, request)
  File "C:\Python34\lib\site-packages\kolibri\dist\django\core\handlers\base.py", line 185, in _get_response
    response = wrapped_callback(request, *callback_args, **callback_kwargs)
  File "C:\Python34\lib\site-packages\kolibri\dist\django\views\generic\base.py", line 68, in view
    return self.dispatch(request, *args, **kwargs)
  File "C:\Python34\lib\site-packages\kolibri\dist\django\views\generic\base.py", line 88, in dispatch
    return handler(request, *args, **kwargs)
  File "C:\Python34\lib\site-packages\kolibri\dist\django\utils\decorators.py", line 67, in _wrapper
    return bound_func(*args, **kwargs)
  File "C:\Python34\lib\site-packages\kolibri\dist\django\views\decorators\http.py", line 99, in inner
    response = func(request, *args, **kwargs)
  File "C:\Python34\lib\site-packages\kolibri\dist\django\utils\decorators.py", line 63, in bound_func
    return func.__get__(self, type(self))(*args2, **kwargs2)
  File "C:\Python34\lib\site-packages\kolibri\core\content\api.py", line 75, in wrapper_func
    response = some_func(*args, **kwargs)
  File "C:\Python34\lib\site-packages\kolibri\dist\django\views\decorators\clickjacking.py", line 58, in wrapped_view
    resp = view_func(*args, **kwargs)
  File "C:\Python34\lib\site-packages\kolibri\core\content\decorators.py", line 14, in wrapper_func
    response = some_func(request, *args, **kwargs)
  File "C:\Python34\lib\site-packages\kolibri\core\content\views.py", line 269, in get
    response = get_h5p(zf, embedded_filepath)
  File "C:\Python34\lib\site-packages\kolibri\core\content\views.py", line 151, in get_h5p
    h5pdata = json.loads(zf.open(zf.getinfo("h5p.json")).read())
  File "C:\Python34\lib\json\__init__.py", line 312, in loads
    s.__class__.__name__))
TypeError: the JSON object must be str, not 'bytes'

This has the same exception to this issue: https://github.com/learningequality/kolibri/issues/6615 - I'm not sure if they are related or the same @indirectlylit.

@mrpau-richard You may be right that upgrading to Python 3.6 fixes the issue. The Python 3.6 release notes mentions json improvements very similar to this issue.

I reproduced this in Windows 7, Upgrading the Python version from 3.4 to 3.6 will fix the issue on Windows.

Please note however that we had an issue using Kolibri on Windows 10 with Python 3.6 last year: https://github.com/learningequality/kolibri/issues/5733

I see two approaches to this:

1. Test if upgrading to Python 3.6 indeed fixes this issue (and the other one too) and make a PR for such. We hope to enter beta testing soon so we can thoroughly test this "upgrade fix" too.
2. Fix the underlying code as @rtibbles suggested.

Or we can do both 😺

[rtibbles]

I think both sounds good to me - but mostly because 3.4 has reached EOL, so we should upgrade the windows installer anyway!

[rtibbles]

I can fix 2, if you guys can tackle 1.

[rtibbles]

Hrm, investigating further, I'm not sure why upgrading to Python 3.6 fixes this - as in both 3.4 and 3.6, the read returns a byte object, rather than a string, so I think this is more of a Py2/3 compatibility issue than anything else.

[mrpau-richard]

@rtibbles I tested your PR for this issue and the error is still there.

```ERROR 2020-05-28 06:22:54,600 exception Internal Server Error: /zipcontent/03617f6c3a5d476cd6f692c611331c6e.h5p/
Traceback (most recent call last):
File "C:\Python34\lib\site-packages\kolibri\dist\djangocore\handlers\exception.py", line 41, in inner
response = get_response(request)
File "C:\Python34\lib\site-packages\kolibri\dist\djangocore\handlers\base.py", line 187, in _get_response
response = self.process_exception_by_middleware(e, request)
File "C:\Python34\lib\site-packages\kolibri\dist\djangocore\handlers\base.py", line 185, in _get_response
response = wrapped_callback(request, callback_args, *callback_kwargs)
File "C:\Python34\lib\site-packages\kolibri\dist\django\views\generic\base.py", line 68, in view
return self.dispatch(request, args, *kwargs)
File "C:\Python34\lib\site-packages\kolibri\dist\django\views\generic\base.py", line 88, in dispatch
return handler(request, args, *kwargs)
File "C:\Python34\lib\site-packages\kolibri\dist\django\utils\decorators.py", line 67, in _wrapper
return bound_func(args, *kwargs)
File "C:\Python34\lib\site-packages\kolibri\dist\django\views\decorators\http.py", line 99, in inner
response = func(request, args, *kwargs)
File "C:\Python34\lib\site-packages\kolibri\dist\django\utils\decorators.py", line 63, in bound_func
return func.__get__(self, type(self))(args2, *kwargs2)
File "C:\Python34\lib\site-packages\kolibricorecontent\api.py", line 75, in wrapper_func
response = some_func(args, *kwargs)
File "C:\Python34\lib\site-packages\kolibri\dist\django\views\decoratorsclickjacking.py", line 58, in wrapped_view
resp = view_func(args, *kwargs)
File "C:\Python34\lib\site-packages\kolibricorecontent\decorators.py", line 14, in wrapper_func
response = some_func(request, args, *kwargs)
File "C:\Python34\lib\site-packages\kolibricorecontent\views.py", line 280, in get
response = get_h5p(zf, embedded_filepath)
File "C:\Python34\lib\site-packages\kolibricorecontent\views.py", line 156, in get_h5p
h5pdata = load_json_from_zipfile(zf, "h5p.json")
File "C:\Python34\lib\site-packages\kolibricorecontent\views.py", line 78, in load_json_from_zipfile
return json.load(f)
File "C:\Python34\libjson__init__.py", line 268, in load
parse_constant=parse_constant, object_pairs_hook=object_pairs_hook, **kw)
File "C:\Python34\libjson__init__.py", line 312, in loads
s.__class__.__name__))
TypeError: the JSON object must be str, not 'bytes'

```

kolibri.txt

I think this explain why this issue is fix on Python 3.6
https://docs.python.org/3/whatsnew/3.6.html#json

We will work on upgrading the Windows installer Python to 3.6 to fix this.

[rtibbles]

OK, this must be a windows specific thing in terms of how the zipfiles are handled then, because on Linux the issues I replicated were common to 3.4 and 3.6.