Tribler: "TypeError: Odd-length string" on entering a channel

[vi]

Tribler version/branch+revision:

v7.3.0-beta6

Operating system and version:

Linux

Steps to reproduce the behavior:

1. Open Triber
2. Seach for some channel
3. Click on it

Expected behavior:

Channel content displayed

Actual behavior:

Pop up error message is displayed twice, then there is empty channel page.

Relevant log file output:

Traceback (most recent call last):
  File "/usr/lib/python2.7/dist-packages/twisted/web/server.py", line 194, in process
    resrc = self.site.getResourceFor(self)
  File "/usr/lib/python2.7/dist-packages/twisted/web/server.py", line 853, in getResourceFor
    return resource.getChildForRequest(self.resource, request)
  File "/usr/lib/python2.7/dist-packages/twisted/web/resource.py", line 98, in getChildForRequest
    resource = resource.getChildWithDefault(pathElement, request)
  File "/usr/lib/python2.7/dist-packages/twisted/web/resource.py", line 201, in getChildWithDefault
    return self.getChild(path, request)
  File "/home/tribler/tribler/Tribler/Core/Modules/restapi/metadata_endpoint.py", line 110, in getChild
    return ChannelPublicKeyEndpoint(self.session, path)
  File "/home/tribler/tribler/Tribler/Core/Modules/restapi/metadata_endpoint.py", line 159, in __init__
    self.channel_pk = unhexlify(path)
TypeError: Odd-length string

[vi]

Also shutting down stuck on "Shutting down Torrent Checker".

[ichorid]

Hmm... Can't reproduce it on beta6.

@vi , could you please copy paste or make a screenshot of your "Debug/Requests" screen immediately after you trigger the bug? Please, Make sure that the whole request line fits on the screenshot, or better yet, just copy-paste the last lines of the "Requests" table.

If you don't see the "Debug" tab, you can enable it in the settings window, "Debug" tab.

[vi]

[ichorid]

@vi , I still cannot reproduce it on our systems. The error is produced by the endpoint trying to parse the URL metadata/channels//30.../torrents. It seems like on your system either unhexlify('') results in an error, or Twisted URL parser splits the URL string in a wrong way. To figure it out, you have to put something like

     print "PK_PATH \"%s\" %s" % (path, type(path))

before line 159 in metadata_endpoint.py, run Tribler from console and look for the printed line. https://github.com/Tribler/tribler/blob/6cf5e3e3727ff09d103c41f18e6d93e669653847/Tribler/Core/Modules/restapi/metadata_endpoint.py#L158

Thank you for your involvement! :+1:

[vi]

PK_PATH "3069278397749299861" <type 'str'>

The string is indeed odd-length:

$ echo 3069278397749299861 | fold -w2
30
69
27
83
97
74
92
99
86
1

[vi]

What are "(Legacy)" channels? I'm only finding those ones. And all of them are stated to be updated "yesterday"

Is there an id of some non-legacy channel to test if the bug is buggy for a non-legacy channels?

[ichorid]

Legacy channels are channels converted from the previous version of Tribler, that is 7.2.2. These channels have no public key, their contents are added as so-called free-for-all (FFA) gossip elements that are not signed either. Legacy channels are never gossiped around actively, though their contents can be sent over to other peers on remote search requests. Legacy channels will be cleaned up in one of the next versions of Tribler (though their contents will remain in the database).

Non-legacy channels are channels that have a 'heart' symbols in the table. You don't see a lot of them because we changed the GigaChannel community ID in beta6, and that means this beta can't benefit from GigaChannels from the previous betas.

[ichorid]

Could you please tell us what Linux distribution and Twisted version do you use? To see the Twisted version, you can run python in the console and type the following:

import twisted
twisted.version

[vi]

$ python
Python 2.7.16 (default, Apr  6 2019, 01:42:57) 
[GCC 8.3.0] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import twisted
>>> twisted.version
Version('Twisted', 18, 9, 0)

[ichorid]

@vi, thanks for the info! I installed the same version of Twisted as used by you. However, I was not able to reproduce the bug (I'm running it on Ubuntu 19.04). I guess the problem is specific to your distribution's codepage/python version or something like that. Although URLs with double slashes are perfectly legit, it seems that some systems do not handle it well. Also, they look ugly. I'll change legacy channels public key identifier to 00, so the endpoint will be able to handle it even in case of buggy URL handlers.

[vi]

I guess the problem is specific to your distribution's codepage/python version or something like that.

Does Tribler have specific version ranges for dependencies?

Part of Python packages are from system, part are just "pip installed" to get though some yet another error after Tribler update.

[ichorid]

We never established any precise dependency version ranges. The stuff just worked for us all these years. For Windows, we pack all dependencies in a single binary, so that is not a problem there.

I guess yours is the first case we see some incompatibility problem on Linux (aside from typical encoding problems).

Anyways, 7.4 will move to Python 3. Further down the road we ditch Twisted in favor of asyncio, so things will really change in a few months.

[ichorid]

@vi apparently, we have a bug in our debug panel! The requests list swallows non-printable characters, so the problem is not in your setup. Could you please send us your ~/.Tribler/sqlite/metadata.db and ~/.Tribler/sqlite/tribler.sdb for analysis?

[ichorid]

If you don't want to send us your databases, you can modify the following line in your Tribler install:
https://github.com/Tribler/tribler/blob/6cf5e3e3727ff09d103c41f18e6d93e669653847/TriblerGUI/debug_window.py#L303

to

item.setText(0, "%s %s %s" % (method, repr(endpoint), repr(data)))

And then trigger the bug and post here the requests list screenshot as you did earlier.

[vi]

Here is the screenshot after the patch:

Also created a torrent with metadata database:

magnet:?xt=urn:btih:5d3064f73c106d42ee84e31565a8a51c43c75240&dn=TribMeta

Seeding may happen in about 7 hours.

[ichorid]

Thanks! However, the screenshot shows that the URL is correct, and there are no non-printable characters. We'll see for sure when we will receive your database.

[ichorid]

Maybe it would be faster if you shared the database through, e.g. https://void.cat/ ?

[vi]

Uploaded to void.cat:

metadata.db https://void.cat/#Znz1NLdWqs67Xrjgg47C0Cxb8Ku:bffa76ffeed698ac2c497ae1bfb3cfb6:d1392e07df3e5c65e7b90fb0fcad0d45
tribler.sdb https://void.cat/#YrxQevJoEXeZ3LihH5LuDlxoTE1:bb7e8573d0e823736d3d6fb3868bb124:5187959925d7025be6a458ff34887a0a

But maybe still try downloading the magnet later to test if Tribler really works for uploads for me.

[ichorid]

Thanks! Already added your torrent for downloading.

[ichorid]

I've received your database and checked it. I can't reproduce the problem. This means that the real problem is in your pip dependencies or something like that.

I'll create a workaround for this problem by making a stand-in string for empty public keys in our REST URLs. Most probably, it will be available in RC1. However, you can track my progress with this bug on
https://github.com/Tribler/tribler/pull/4670

As soon as that PR is merged, you can check out the git version and try it for yourself, if you don't want to wait 'till RC1.

Thanks for you help!

[vi]

While creating that "TribMeta" magnet in Tribler, I was asked for name and description of the torrent.

It has been created with the download name based on filesystem directory name, and I don't see neither that name nor description anywhere.

[ichorid]

@vi , please, file a separate issue for this latest problem you reported.