Druid: Confusing directory creation error on RealtimeTuningConfig construction

[gianm]

Creating a RealtimeTuningConfig can cause funky errors like "Failed to create directory within 10000 attempts (tried 1470845895301-0 to 1470845895301-9999)" if a default basePersistDirectory can't be created. The funny thing is that a lot of the time, this default basePersistDirectory isn't actually needed, because it will either never be used (overlord/supervisor) or it will shortly be replaced using withBasePersistDirectory.

Probably the best way to fix this is to avoid creating the directory on construction, and instead create it only if it's actually needed. This would defer the error to where it actually matters and would also avoid the creation of needless temp directories that are going to be replaced anyway.

See also https://groups.google.com/d/topic/druid-user/BahFn4H93UU/discussion.

[neynt]

For anyone who found this issue because of an error like Failed to create directory within 10000 attempts (tried 1477086112673-0 to 1477086112673-9999): the problem is possibly that in your jvm.config files, java.io.tmpdir is set to a directory that doesn't exist. Creating this directory fixed the problem for me.

[jihoonson]

Fixed in https://github.com/apache/incubator-druid/pull/5648.