Dvc: Add support for ftp server

[nicolalandro]

Exist any support for FTP server?

UPDATES: See https://github.com/iterative/dvc/issues/1696#issuecomment-474344514 and https://github.com/iterative/dvc/issues/1696#issuecomment-535569113 for most current conclusions for this one.

[ghost]

Hi, @nicolalandro ! DVC currently doesn''t support FTP, as there are some operations that would be difficult to perform, like computing checksums. However, the SSH remote uses SFTP for pulling/uploading information :)

[efiop]

@nicolalandro would you like to use FTP specifically, or SFTP would do too? If so, as @mroutis said, we do support it the same way as any ssh server(see SSH example at https://dvc.org/doc/commands-reference/remote-add ). Computing checksums is not a problem since it is only needed for advanced scenarios and is not required for simple push/pull/fetch.

[nicolalandro]

@nicolalandro would you like to use FTP specifically, or SFTP would do too? If so, as @mroutis said, we do support it the same way as any ssh server(see SSH example at https://dvc.org/doc/commands-reference/remote-add ). Computing checksums is not a problem since it is only needed for advanced scenarios and is not required for simple push/pull/fetch.

I Ask for FTP beacuse I can find FTP free solution easily, so it is possible to use in a free environment for study that can be near to a production.

[efiop]

@nicolalandro What do you mean by "free" though? Could you please share more info about your scenario?

[nicolalandro]

@nicolalandro What do you mean by "free" though? Could you please share more info about your scenario?

I want to deploy it on Heroku, in order to be visible to other people, but I train the model from my machine. I should use free ftp server for example altervista, and from heroku I can read and show, serving it to the people I want, without spend money (There are some limitation but is only study case).

[efiop]

@nicolalandro So sorry, notification got lost in the flow :slightly_frowning_face:

Great point! We'll take a look at it in the future. If you'd like to contribute a patch for this yourself, it is pretty simple, please take a look at dvc/remote/ssh as an example and let us know if you need any help. :slightly_smiling_face:

Thank you for the feedback!

[ei-grad]

Just my 50 cents - Google Cloud Storage, Amazon S3 and Azure all have free 5Gb usage tier. Amazon's and Azure is limited up to 12 month, but Google's is not - https://cloud.google.com/storage/pricing-summary/:

Store up to 5GB/month in US regional storage locations for free during and after your free trial while you learn Cloud Storage. These usage limits do not expire, but are subject to change.

Also, Heroku has https://elements.heroku.com/addons/cloudcube with free 5Mb plan, if it could be enough.

[shcheklein]

@nicolalandro will any of these free options @ei-grad mentioned work for you?

[nicolalandro]

I think Heroku is good but 5Mb are not enogth. So I thing Google Cloud Storage is better. I will try later this opportunity.

[efiop]

@nicolalandro Any news? 🙂

Seems like there is not much interest from others in this feature and there is already a viable workaround of mounting your ftp storage to a local directory and using it as a local remote. Considering that, I'll close this issue for now until we see more interest or someone willing to implement it himself.

[efiop]

Reopening since people seem to be interested in box.com https://github.com/iterative/dvc/issues/1940

[jorgeorpinel]

Seems like a p3-nice-to-have to me?

[efiop]

@jorgeorpinel Still in the process of removing p4 label, it will be gone soon.

[efiop]

@jorgeorpinel Please don't delete comments.

[jorgeorpinel]

I think I should be free to hide or delete my own comments when I think they're no longer relevant. Maybe something to chat about on Slack or an all-hands later to set the policy. Unhid yours though, sorry 🙂

[dmpetrov]

@efiop, @mroutis and @jorgeorpinel did someone asked for ftp in the last half a year? We should probably close the issue if not. Or prioritize it if users ask.

[efiop]

@dmpetrov Indeed, not much activity here as well as in box.com ticket. Closed that one, closing this one as well. Thanks for the heads up!

[mihajenko]

I use FTP+curl to provision models to persistent volume as part of a prototype pipeline for inference APIs. We are planning a transition to cloud services so this isn't a problem for me personally, but I do feel like FTP is one of the easier ways to setup on-prem. There are also plenty of cheap storage services using FTP, benefiting those with limited resources.

[efiop]

Hi @mihajenko !

Thank you for your comment! :slightly_smiling_face: If anyone needs FTP specifically, we will be happy to accept a patch for that (it is extremely easy to create it, but we really need active users so it is not a dead feature). Also, if someone is interested in this feature but is not willing to contribute, please leave a comment here, so we know there is interest and could consider reopening this issue and implementing it ourselves. In the meantime, there is a great variety of other clouds supported (like sFTP/s3/etc) and not enough current interest that we've received over the years, so we won't reopen it for now.