Zfs: NEX-3514 Implement persistent L2ARC

[kernelOfTruth]

Reminder for using the persistent l2arc implementation from Nexenta

(there doesn't seem to be any activity related to that on Illumos ?)

https://github.com/Nexenta/illumos-nexenta/commit/f0a9ee9eeed31410aecb7bc0172cec615698aecd NEX-3514 Implement persistent L2ARC

https://github.com/Nexenta/illumos-nexenta/commit/887263262dff3397c305be7129b6fb394189de8e NEX-3514 Implement persistent L2ARC - fix build breakage in libzpool.

https://github.com/Nexenta/illumos-nexenta/commit/9498bb1539167ca88ff9237851a2ac9047859a5a NEX-3514 Implement persistent L2ARC - fix build breakage in libzpool (v2).

https://github.com/Nexenta/illumos-nexenta/commits/master?page=4

@grahamperrin thanks for pointing that out

references:

https://github.com/zfsonlinux/zfs/pull/2672 3525 persistent l2arc

https://github.com/zfsonlinux/zfs/issues/925 allow for persistent l2arc

[kerberizer]

illumos 3525 is now in code review.

[error10]

This feature would be extraordinarily helpful to have. Is there something us mere mortals can do to help move this forward?

[kerberizer]

Indeed, it's been already 6 months since the last activity on the illumos code review request. My guess is that @skiselkov has simply been too busy with other, more pressing matters, but I do hope he'll eventually catch up on this too. Obviously, once the patch reaches reasonable maturity on illumos, someone would have to also port it to Linux. With my pathetic programming skills, I'm afraid I can't personally offer much help in terms of development, but once we reach that stage, I'd eagerly participate in the testing.

[skiselkov]

@kerberizer Sorry I couldn't get to it so far, but it's definitely not dropped. We've been running this in Nexenta for a few months now and it seems working well, so it's really just a matter of me working on the changes requested by Matt and getting it into open-zfs (from which ZoL should get it naturally as part of upstreaming merging).

[cwedgwood]

@behlendorf this is a duplicate of #925 - might want to merge/close as you think fit.

[behlendorf]

@cwedgwood thanks for pointing this out. I'm closing #925 as a duplicate.

Updated progress at https://reviews.csiden.org/r/267/

[skiselkov]

@ptx0 The work is ongoing. I'm currently focusing on getting TRIM upstreamed, pL2ARC will be next after that. TRIM is nearing the end of its protracted upstreaming effort, so it'll go in hopefully "Very Soon(tm)".

[justinlynn]

@skiselkov This appears stuck upstream on https://reviews.csiden.org/r/267. Hate to bother you, but do you have any context for the current state?

[kernelOfTruth]

CC: @skiselkov @ahrens @lundman

[clefru]

I am offering a symbolic bounty of 0.025 BTC (~100$ at current price) for the committer of this feature expiring 2017/11/01. To claim, please mail me a BTC address.

[justinlynn]

Interestingly, it appears the TRIM change they've blocked on has no pending open review issues (at least at https://reviews.csiden.org/r/263/) but the last update was months ago. Additionally, the TLS cert has expired... so, is that review site even in use? @skiselkov

[sienar1]

Any updates on this feature?

[wolfgangwuerfl]

Are there any news/updates on this important feature? The community would really appreciate any status updates. Great to see ZFS on linux and thanks for your work!

[clefru]

I am happy to post a new (still only symbolic) bounty for the committer of this feature of 0.05 BTC (~400$ at current price). Thanks to @Rudd-O for sponsoring half. Expiry: 2018/02/01.

[mailinglists35]

has anybody tried work by @yshui ?
https://github.com/yshui/zfs/commit/2a65c1e3e08b423af5885459d0cf119212a2f8a8
does it still apply to current master?

[wolfgangwuerfl]

I just did a cherry pick and got 2 minor merge errors, but in module/zfs/arc.c the merge errors are plenty. Maybe @behlendorf or @skiselkov can give an update about when we can expect this killer-feature in zfsonlinux in the future. Thx.

[gf-mse]

+1 on the waiting list )

[jumbi77]

I am waiting too. @skiselkov I would like to ask you if you can give any updates about the progress? Above you said pers. l2arc is the next thing after TRIM, so do you have any timeline? Many thanks, we really appreciate your work!

[djkazic]

:+1: this is a huge frustration for my usecase -- takes hours to warm the pool cache and persistent l2arc would be amazing.

[msLinuxNinja]

Bump for awareness, having persistent L2ARC will help considerably with use cases like homelab servers and office storage servers that get rebooted often. 😄

[drescherjm]

I don't recall seeing any activity from @kernelOfTruth in some time. At least on github.

[yshui]

I think the current consensus is that we are waiting for illumos #3525 to be merged? @behlendorf

[gf-mse]

By the looks of it, @ahrens did a code review "2 years, 7 months ago", then Saso ( @skiselkov ) resolved those issues, and then Matthew did a review again, "2 years 5 months ago", at which point the ball seems to be on Saso's side.

Probably it would help to port 3525 here under GitHub, although I'm not sure how essential that is.

PS. At that time @ahrens was of course the main reviewer, but not the only one -- see e.g. https://reviews.csiden.org/r/267/#comment690 and other comments by Sean Doran under r/267

[ahrens]

@gf-mse That's my understanding as well.

[wolfgangwuerfl]

From what i have seen, @skiselkov isnt still any longer active in the openzfs (on linux) project.
For me it would be interesting when we can expect the feature in (hopefully near) future.

[skiselkov]

@wolfgangwuerfl I'm around and I read all these messages. But I'm swamped with work on internal stuff that I don't have time to get this push finalized :( I want to make time some time soon, but I can't make promises. If there are any volunteers that would be willing to take the push over, I'd be happy to provide any info necessary.

[wolfgangwuerfl]

Thanks for the info and your amazing work, guess, i as i PHP Developer would be not much of a help ;-)

[gf-mse]

@skiselkov I suppose passing over all the necessary information would cost more trouble than just doing it oneself. But if I can be of any help -- please let me know ( an experienced C programmer but zero acquaintance with ZFS code ); I suppose it is possible to delegate simple related tasks, if there are any.

[yshui]

@skiselkov I have previously made attempts on porting 3525 to ZoL (#2672, #1823). If there is anything I could do, just let me know.

[CySlider]

Also would love to see this as I am a desktop user that values his data :-)

But I wonder about one thing. Would this also persist the ARC? Is it mirrored on the L2ARC? My current understanding is, that this is not the case. And it would also not make much sense to do this, if data is lost after reboot anyway.

Unfortunately the ARC would have the most relevant data, that would get lost during reboot. No?

[gf-mse]

@CySlider please see http://www.brendangregg.com/blog/2008-07-22/zfs-l2arc.html ( and http://www.c0t0d0s0.org/archives/5329-Some-insight-into-the-read-cache-of-ZFS-or-The-ARC.html if really interested )

In short: L2ARC is a usual disk-level cache for ARC which is memory. So yes you're correct, but it kinda makes sense to address the biggest cache which takes longer to populate in the first place. For a desktop user, I would suggest hibernation which would restore the in-memory cache as well. But that would be an off-topic to discuss here.

[gf-mse]

@yshui How hard it would be to rebase your PR to Ubuntu 16.04 ' standard v. 0.6.5 ?
We have ~0.5+ Pb of data sitting on that version, so I could do some tests with real-life data and see how it goes for different patterns. Meanwhile, I will try to get some understanding of the code ( will probably take another year or so ). If Saso will still be busy by then, we will then have some ground for another code review. Does this sound as a plan?

[gf-mse]

@kpande offtopic, but nice to know. thanks.

hibernation doesn't work with zfs-only systems.