Data.table: [R-Forge #2622] Add command "fwrite" to faster save csv files

[arunsrinivasan]

Submitted by: Ma Jom; Assigned to: Nobody; R-Forge link

I really love the new command "fread" as it really speeds up the work with large files. It would be great to see a corresponding command for writing CSV files.

Currently, I use a combination of write.csv and as.matrix to speed up this process a bit:
http://stackoverflow.com/questions/10505605/speeding-up-the-performance-of-write-table

[abfleishman]

Is this in the works? I am dealing with the same problem

[mattdowle]

@abfleishman @robbyjo We're not working on it currently. Contributions are more than welcome and I'm happy to guide anyone on how to do it. Or, funding would help a lot.

[skanskan]

I agree, data.table is missing a fwrite functionality.
It would be a great addition.

[minhhahl]

+1 for fwrite function

[stanasa]

+1 for fwrite function

[danielsjf]

I would also really appreciate an fwrite function!

[MichaelChirico]

Of course I'm all in for this.

[arunsrinivasan]

At this point, as I briefly touched upon during UseR 2015, there's no motivation or incentive to implement fwrite for neither of us. To implement it efficiently is a lot of work, and it comes out of our free time. With the possibility of it being just _reimplemented_ as well in another package (like readr) at some point in time, it doesn't seem like a good investment of our time.

If anyone else is interested in investing their time, please feel free to contact us.

[oseiskar]

We have developed a faster function for writing CSV files and achieved > 2x speedups (on a 64-bit Windows machine with an SSD) See this Gist for a proof of concept. I can start working on a pull request with more features if you are interested.

[smartinsightsfromdata]

@oseiskar code seems very promising.
@mattdowle @arunsrinivasan would you consider a pr for a fwrite on this basis?

[Bohdan-Khomtchouk]

Not having fwrite to complement fread is like not having write.csv to complement read.csv (or write.table/read.table)

[migueltp]

+1 for fwrite function

[julja]

+1 for fwrite function

[mattdowle]

@oseiskar @smartinsightsfromdata Sorry I didn't see your comments above in Oct or Nov at the time. Of course (!) we would be delighted to accept PR. But I already wrote above in Oct 2014 that "Contributions are more than welcome" in case that was not already clear, so I don't know why you asked rather than just submitting the PR.
Maybe it was because of our fears expressed above that the effort would be wasted. You may be better off contributing to the so called Hadley-verse since that is what most people (well, on twitter and blogs anyway) jump up and down about. Hadley has the midas touch if that's what you want. But if you wanted to contribute fwrite to data.table, of course it would be accepted. Or create your own new package of course.

[mattdowle]

fwrite PR merged! Thanks @oseiskar !!

[MichaelChirico]

Just took the liberty of updating the SO "fast writing" post linked at the head of this thread.

Some more clean-up: @mattdowle you mention the lack of fwrite in this answer, not sure if you prefer to update yourself

[mattdowle]

Thanks @MichaelChirico. Please do go ahead and update my answers too - I don't mind at all as it saves time. Thanks! Nice detailed answer btw with extensive benchmark.

[mattdowle]

Work on fwrite continued here: #1664