Max Value for bucket ID

[reed_kelly]

I haven't been able to find this in the docs. The bucket ID is the last number in the name of a bucket directory. For example, 2768 is the bucket ID in the following folder name.

e.g. db_1363148387_1363078186_2768

To resolve bucket ID conflicts, we frequently add a digit to the ends of the filenames. If we do this too often, the bucket IDs could get large.

Does anyone know the maximum value that the bucket ID can have?

[ekost · Accepted solution]

On Splunk 7.0.1, *nix, x86_64, single-instance, the bucket id: db_1521417585_1518757502_999999999 was searchable at the cli, but search failed at db_1521417585_1518757502_9999999991.
The only log entry I found notable and consistent required DEBUG mode:

04–11-2018 19:20:07.650 +0000 DEBUG DatabaseDirectoryManager - Skipping path='/opt/splunk/var/lib/splunk/defaultdb/db/db_1521417585_1518757502_9999999991'

Additional errors might be noted elsewhere, but I didn't catch them.

[msearles_splunk]

The bucket ID has a theoretical maximum of 2,147,483,647 (it's an int, so 2^31).

[znaesh]

It's an issue indeed. When we append some digits to avoid collisions, we must choose a big number (eg. 1000), but there indeed is a limit, and the bucket dirs beyond that limit are not included in .bucketManifest and not sought. And there is even no warning in logs.

[w531t4]

found myself asking this question earlier today -- same. i'm interested in knowing the answer.

[gkanapathy]

good question.