Zfs: (DDT) ZAP is inefficient when ashift=12?

[nwf]

I have two pools with ashift=12 and DDT turned on and have noticed that their size-on-disk is enormous:

dedup: DDT entries 58011991, size 1192 on disk, 168 in core
dedup: DDT entries 20163506, size 1051 on disk, 169 in core

I don't recall ever seeing on-disk-size values that large with an ashift=9 pool, even when it had lots of data, so I ran a small experiment. I created two pools thusly:

for i in 9 12; do
  dd if=/dev/zero of=/tmp/ashift$i bs=1M count=0 seek=4096
  zpool create -o ashift=$i ashift$i /tmp/ashift$i
  zfs set dedup=sha256 ashift$i
done

And then proceeded to fill them up with dd if=/dev/urandom. Here are various samples taken with zpool status -D after filling more and more of the disk:

64M: (2.9x factor)

ashift12: dedup: DDT entries 512, size 288 on disk, 88 in core
ashift9: dedup: DDT entries 512, size 99 on disk, 64 in core

192M: (1.2x factor)

ashift12: dedup: DDT entries 1536, size 336 on disk, 106 in core
ashift9: dedup: DDT entries 1536, size 279 on disk, 160 in core

320M: (1.7x factor)

ashift12: dedup: DDT entries 2560, size 475 on disk, 153 in core
ashift9: dedup: DDT entries 2560, size 274 on disk, 155 in core

448M: (1.8x factor)

ashift12: dedup: DDT entries 3584, size 462 on disk, 147 in core
ashift9: dedup: DDT entries 3584, size 259 on disk, 144 in core

576M: (1.7x factor)

ashift12: dedup: DDT entries 4608, size 421 on disk, 135 in core
ashift9: dedup: DDT entries 4608, size 254 on disk, 136 in core

832M: (1.7x factor)

ashift12: dedup: DDT entries 6656, size 483 on disk, 155 in core
ashift9: dedup: DDT entries 6656, size 280 on disk, 153 in core

As you can see, the ashift12 pool is consistently using a good bit more space to store its DDT entries than the ashift9 pool. Why? Shouldn't they be roughly the same?

Is there more information I could provide?

[nwf]

I am curious why this was closed? So far as I know it's still maybe an issue that suggests that ZAP structures could be revised to take ashift into account?

[nwf]

ashift=9 is not helpful for AF devices, I'm afraid. And I recognize that ZAP layout changes are complicated, but I don't think that merits writing off the idea entirely. Anyway.

[behlendorf]

I've reopened this and have added the feature label to make it easier to track. It would be nice to look in to how we can optimize the space usage for this case when someone has the time.

[pcd1193182]

It's possible that the log-structured DDT proposal would help with this to some extent (in addition to its other virtues). I'd link the issue/bug report, but I'm not sure where it is.

[rincebrain]

@pcd1193182 I don't think it ever got one; AFAIK mahrens mentioned it in the "dedup doesn't have to suck" talk, there's the PoC branch on his Github tree, and that's it.