Uppy: Pass metadata to backend

[jazkh]

@goto-bus-stop
Hello,
how can I pass meta data information to backend tus node server through uppy tus endpoint ?
I want to get that information on my backend server such as file original name, mime type and size so I can rename the file to its original name or whatever filename is set in metaField option in uppy.

On my backend, I get only file id, name (encoded), type (encoded).
server.on(EVENTS.EVENT_UPLOAD_COMPLETE, (event) => { console.log(`Upload complete for file ${event.file.id}`); });

[goto-bus-stop]

According to the docs the EVENT_UPLOAD_COMPLETE event has a file.upload_metadata property. The upload_metadata property is encoded tus-style.

Something like this should work to decode the property:

server.on(EVENTS.EVENT_UPLOAD_COMPLETE, (event) => {
  const metadata = {}
  event.file.upload_metadata.split(',').forEach((data) => {
    const [name, encoded] = data.split(' ')
    metadata[name] = Buffer.from(encoded, 'base64').toString('utf8')
  })
})

[jazkh]

@goto-bus-stop
thank you. But how do I decode a URL which contains something like this c939c38adcf1873299837894214a35eb ?
http://192.168.22.124:3000/upload/files/c939c38adcf1873299837894214a35eb

If I try to decode it using base64, I get some weird characters s��su������7��x�^ߗ�

[jazkh]

@goto-bus-stop

In my console log I am getting HEAD http://192.168.22.124:3000/upload/files/c939c38adcf1873299837894214a35eb 404 (Not Found).
This is what I have been doing:
Renaming the file on backend (node js and express js) after upload is completed by decoding the name as you suggested. But then I get HEAD 404 (Not Found) on my client end (using Reactjs).
My client side is expecting the encoded c939c38adcf1873299837894214a35eb name in the HEAD, but it has already been changed / renamed after completion.

I think I am doing something wrong here. Can you pls help with this one as well?

Also, let me know, if you want me to open "New issue" for this question.

[arturi]

@Acconut maybe you could assist here?

[jazkh]

Kindly help me with the above questions. I want the client side to expect the HEAD with the renamed filename. And that file has been renamed by decoding the original filename.

Also if I don't do any file renaming on the backend and leave it default. This is the behavior I get then:

1. I upload a file and send it to the backend.
2. Then i delete it
3. Again I upload a file and send it.
4. I get HEAD 404 Not Found for the first uploaded file which I have deleted.

In my GET method on backend (node and express) i am only responding with the file url by res.sendFile to the client side so it can display the uploaded file.

[Acconut]

But how do I decode a URL which contains something like this c939c38adcf1873299837894214a35eb ?

Why are you trying to decode the URL? It's just a random string and is not Base64-encoded. Therefore you just get some garbage when trying the decode. The values of the Tus-Metadata headers are Base64-encoded as @goto-bus-stop said and can be accessed using file.upload_metadata.

Renaming the file on backend (node js and express js) after upload is completed by decoding the name as you suggested. But then I get HEAD 404 (Not Found) on my client end (using Reactjs).
My client side is expecting the encoded c939c38adcf1873299837894214a35eb name in the HEAD, but it has already been changed / renamed after completion.

When you are renaming the uploaded files without copying them to another place before renaming, the tus server is not able to find the files at the original upload URL. That's simply not possible. You first have to copy the files somewhere else if you want the upload URL to continue working.

I get HEAD 404 Not Found for the first uploaded file which I have deleted.

This is the expected behavior. You deleted the file and the upload cannot be served anymore. This makes sense, don't you agree?

[jazkh]

@Acconut
But why is it looking for the old deleted file when i upload a new file?
I think it should just forget about the old file once it has been 'successfully' uploaded and do not care whatever happens to that file on the backend.

[Acconut]

But why is it looking for the old deleted file when i upload a new file?
I think it should just forget about the old file once it has been 'successfully' uploaded and do not care whatever happens to that file on the backend.

tus-js-client notices that you uploaded the file before and attempts to resume this uploads. That's why it sends the HEAD request to find out if it can resume (see the resume option: https://github.com/tus/tus-js-client#tusdefaultoptions). If the HEAD response is not successful, it will start a completely new upload. You can force latter behavior by disabling the option.

[jazkh]

@Acconut

If I do not delete/rename/move the file and when I try to reupload the same file I get this error:

In my uppy dashboard:
Upload failed. Please press Retry to upload again
tus: invalid or missing offset value, originated from request (response code: 200, response text: )

In my console log:
Refused to get unsafe header "Upload-Offset" upload.js:430

The above behavior happens when I change the location from /data/ to /files/ on my backend GET method res.sendFile:
/files/ is the location where uppy client files land with some random strings.

/data is where I copy the file from /files location and rename it.

app.get('/upload/files/:fileid', (req, res) => { const { fileid } = req.params; res.sendFile(__dirname + /files/ + fileid); });

If I use /data/ in GET method res.sendFile, I get HEAD 404 Not Found.
res.sendFile(__dirname + /data/ + fileid);

I have used this get method so I can display it on the client side.

[jazkh]

@kvz @Acconut
Do I need to include something in my GET method to meet client side requirement?

[jazkh]

When I remove the GET method from backend and try to reupload the same file again without deleting the old one, then there are _no errors_ such as HEAD 404 or Refused to get unsafe header "Upload-Offset"
It shows file uploaded Complete in dashboard.
But the file is actually never duplicated on the backend server.
And I am getting this warning on my client side console log:

Warning: Encountered two children with the same key, uppy-pexelsphotojpg-image/jpeg-135010-1524562149793. Keys should be unique so that components maintain their identity across updates. Non-unique keys may cause children to be duplicated and/or omitted — the behavior is unsupported and could change in a future version.

And If I delete the old file, then it uploads and console log shows error for previously uploaded file:
HEAD 410 (Gone)

[jazkh]

should I change the GET location?

[Acconut]

Do I need to include something in my GET method to meet client side requirement?

tus-js-client does not ever make a GET request. So from the tus side, you don't have to. But I am not sure whether Uppy has some requirements. Likely @arturi or @goto-bus-stop can answer that.

When I remove the GET method from backend and try to reupload the same file again without deleting the old one, then there are no errors such as HEAD 404 or Refused to get unsafe header "Upload-Offset"
It shows file uploaded Complete in dashboard.
But the file is actually never duplicated on the backend server.

This is exactly the behavior I explained above:

tus-js-client notices that you uploaded the file before and attempts to resume this uploads. That's why it sends the HEAD request to find out if it can resume (see the resume option: https://github.com/tus/tus-js-client#tusdefaultoptions). If the HEAD response is not successful, it will start a completely new upload. You can force latter behavior by disabling the option.

[jazkh]

@Acconut

This is exactly the behavior I explained above

But I get this behavior only when I remove the GET method from the backend.
I hope somebody from uppy team will explain it further.
@goto-bus-stop @arturi

[Acconut]

But I get this behavior only when I remove the GET method from the backend.
I hope somebody from uppy team will explain it further.

Ok, since tus-js-client does not do GET requests, this might be caused by Uppy but I cannot help there much :)

[goto-bus-stop]

Renaming the file on backend (node js and express js) after upload is completed by decoding the name as you suggested.

I suggested decoding the _upload_metadata_ property, not the name :)

Uppy does not have additional requirements beyond what tus-js-client has.

As I understand it the HEAD 404 response is expected when you move the file, and should not affect the upload further.

When I remove the GET method from backend and try to reupload the same file again without deleting the old one, then there are no errors such as HEAD 404 or Refused to get unsafe header "Upload-Offset"

It looks like you were overriding the /upload/files/:fileId route from the tus node server, which may be why you get the error about Upload-Offset. I think that is also why it works correctly when you remove that route, you're probably not supposed to override it.

I'm a bit lost—it seems like the problem is no longer with reading metadata on the server is it? I'm struggling to piece together what is going wrong / unexpected from the comments so far.