Lbry-desktop: Source file error when editing existing claim

[tzarebczan]

The Issue

When copying data from an existing claim, you should not need to select the source file again since the original blobs should be used in the updated claim. Thanks @reillysmith for finding this one.

Steps to reproduce

1. Type existing claim name in Publish screen
2. Use "Use data from my existing claim" option to copy claim data
3. Publish and get error message about source
   

Expected behaviour

Edit claim without adding source

Actual behaviour

Error about source

System Configuration

• LBRY Daemon version: 0.14.2
• LBRY App version: 0.14.3
• LBRY Installation ID:
• Operating system: Windows

Anything Else

Screenshots

[kauffj]

I didn't test, but I doubt #464 fixes this. I actually think that error message is coming from the daemon, though I agree it's likely an app issue.

[tzarebczan]

Discussion with Kay regarding changing of SD-hash. Want to test this scenario out.

tomzarebczan
When we allow for claim updates, is it normal that you can either use the same file (same sd-hash) or pick a new file (new sd-hash?) What would happen if someone downloaded the file and then it's updated by the owner - it will have the same claim id but different contents...does that screw things up somehow?

kaykurokawa 
Yes, its normal. I think we solved this issue, but I would not be too suprised if there are still lingering problems.

tomzarebczan 
I haven't actually tried it, was just thinking out loud. I could see how it would be problematic. I'll try it in practice and see what happens sometime. I just feel like 1 claimid can point to 2 sets of blobs...so unless we are tracking that carefully, could lead to issues.

[tzarebczan]

@btzr-io - any update on this one? I know you made some progress, but did you get stuck?

[tzarebczan]

Discussion with @btzr-io:

btzr [28 minutes ago
I know  is the app forces you to add a path every time otherwise you get an error:
`error occurred when attempting to publish your file: source`

jiggytom [27 minutes ago] 
yep, and that's exactly what we are trying to solve. So it cold be a backend issue, or maybe we aren't passing something correctly when a path is not selected.
btzr [19 minutes ago] 
that's where I'm stuck, how / why does it stores the path if isn't need it?
jiggytom 
[13 minutes ago] 
Yea I'm not sure, sorry. I think it still needs the path, but if the path is the same, don't re-upload the file. What happens now is that the content gets re-uploaded as new data even though nothing changed with the file itself. Should be able to update claim and leave file intact. Want me to  comment on the issue
btzr [5 minutes ago] 
`https://lbryio.github.io/lbry/#publish`

btzr [4 minutes ago] 
it says  -> `'file_path'(optional)`
btzr [3 minutes ago] 
>  If no path is given but a metadata dict is provided, the source from the given metadata will be used.
(edited)
jiggytom 
[3 minutes ago] 
yea, so it is an app issue then
jiggytom 
[3 minutes ago] 
or should be...just need to check if its an update and if the file is not selected (optional), don't pass/check for the file_path?
btzr [2 minutes ago] 
https://github.com/lbryio/lbry-app/blob/master/ui/js/component/publishForm/view.jsx#L118
btzr [< 1 minute ago] 
I guess this is what don't get passed -> `metadata dict`, not sure what it's that yet ?

[kauffj]

@btzr-io @tzarebczan to debug further:

1. Open console.
2. Perform broken publish.
3. Observe request. Is it making it to daemon properly? Is it returning errors?
4. If daemon is erroring, what is being passed? Is it passing a signature that the API docs report as valid?
5. If yes, it's a bug in the daemon and has to be fixed there. But could just as easily be app-side.

[btzr-io]

KeyError: 'source'

2017-08-28 19:28:08,168 INFO     lbrynet.daemon.Daemon:1851: Publish: {'claim_address': None, 'name': u'fullprice', 'bid': 1, 'claim_dict': {'version': '_0_0_1', 'claimType': 'streamType', 'stream': {'version': '_0_0_1', 'metadata': {u'description': u'Lbry dev - fixing full price format', u'license': u'Public Domain', u'title': u'Lbry dev - fix price ', u'language': u'en', 'version': '_0_1_0', u'nsfw': False, u'licenseUrl': None}}}, 'file_path': None, 'change_address': None}
2017-08-28 19:28:08,187 ERROR    lbrynet.daemon.auth.server:353: Failed to process publish: 'source'
Traceback (most recent call last):
  File "site-packages/twisted/internet/defer.py", line 1299, in _inlineCallbacks
  File "site-packages/lbrynet/core/Wallet.py", line 934, in claim_name
  File "site-packages/lbryschema/claim.py", line 139, in load_dict
  File "site-packages/lbryschema/encoding.py", line 36, in decode_fields
KeyError: 'source'

Python raises a KeyError whenever a dict() object is requested (using the format a = adict[key]) and the key is not in the dictionary.

https://wiki.python.org/moin/KeyError

[btzr-io]

@kauffj As far as I know when there is no path provided it should use source but it never get passed ^^

[btzr-io]

https://github.com/lbryio/lbry/blob/master/lbrynet/daemon/Publisher.py#L41

[btzr-io]

Also the API docs aren't really clear about this:
https://lbryio.github.io/lbry/#publish

If provided, a lbry stream of this file will be used in 'sources'.
If no path is given but a metadata dict is provided, the source
from the given metadata will be used.

[kauffj]

@tzarebczan can you confirm this exists purely daemon side by determining a precise API call that fails when it shouldn't? Then we can open this on the daemon.

[tzarebczan]

@kauffj sure, will take a look tomorrow. I thought it was fixed already, had a comment on slack.

[btzr-io]

@kauffj Already mention this but here is it again,
daemon expects a fallback source when no file_path is passed,
the app only pass file_path, check this possible fix: 7a853f1

[kauffj]

But if the daemon says a source isn't required, a source shouldn't be required. I'm not inclined to add a fix for this app side (which also later ought to be removed), when we can just fix it in the right place.

[tzarebczan]

Can confirm I am seeing the same as @btzr-io

Original publish:
{"jsonrpc":"2.0","method":"publish","params":{"name":"lbryio","bid":0.01,"metadata":{"title":"test pub 1","language":"en","license":"","licenseUrl":null,"nsfw":false},"file_path":"C:\\Users\\Tom-LaptopVM\\Downloads\\lbryio.jpg"},"id":126}

daemon log:
2017-08-30 08:12:44,973 INFO lbrynet.daemon.Daemon:1838: Publish: {'claim_address': None, 'name': u'lbryio', 'bid': 0.01, 'claim_dict': {'version': '_0_0_1', 'claimType': 'streamType', 'stream': {'version': '_0_0_1', 'metadata': {u'language': u'en', u'license': u'', u'title': u'test pub 1', 'version': '_0_1_0', u'nsfw': False, u'licenseUrl': None}}}, 'file_path': u'C:\\Users\\Tom-LaptopVM\\Downloads\\lbryio.jpg', 'change_address': None}

Update publish:
{"jsonrpc":"2.0","method":"publish","params":{"name":"lbryio","bid":0.02,"metadata":{"title":"test pub 2","description":"update to publish","language":"en","license":"","licenseUrl":null,"nsfw":false}},"id":211}

daemon log:

2017-08-30 08:18:48,490 INFO     lbrynet.daemon.Daemon:1838: Publish: {'claim_address': None, 'name': u'lbryio', 'bid': 0.02, 'claim_dict': {'version': '_0_0_1', 'claimType': 'streamType', 'stream': {'version': '_0_0_1', 'metadata': {u'description': u'update to publish', u'license': u'', u'title': u'test pub 2', u'language': u'en', 'version': '_0_1_0', u'nsfw': False, u'licenseUrl': None}}}, 'file_path': None, 'change_address': None}
2017-08-30 08:18:48,605 ERROR    lbrynet.daemon.auth.server:353: Failed to process publish: 'source'
Traceback (most recent call last):
  File "site-packages\twisted\internet\defer.py", line 1299, in _inlineCallbacks
  File "site-packages\lbrynet\core\Wallet.py", line 933, in claim_name
  File "site-packages\lbryschema\claim.py", line 139, in load_dict
  File "site-packages\lbryschema\encoding.py", line 36, in decode_fields
KeyError: 'source'

@kauffj I think the daemon is saying that it does require the source to be specified (from the original claim) if a new file isn't selected. Are you saying that the daemon should just figure that part out on its own as opposed to filling it from the app?

[btzr-io]

@kauffj where does it says the source isn't required?

If provided, a lbry stream of this file will be used in 'sources'.
If no path is given but a metadata dict is provided, the source
from the given metadata will be used.

Publisher.py#L41

       if 'source' not in claim_dict['stream']:
            claim_dict['stream']['source'] = {}

so what happens when both are missing? :confused:

[kauffj]

@btzr-io you're right, my mistake.