Faker: Null byte errors with postgresql

[zyrthofar]

Describe the bug

I'm having an issue related to Faker::String.random when using a Postgresql database. The pg extension sometimes complains about null bytes in the string, and the test fails.

To Reproduce

A Rails project using a Postgresql database, and a model with a string attribute.

In a rails console, after a require 'faker',

MyModel.create!(string_attr: Faker::String.random)

Usually takes a few times, but the record creation will eventually fail with an ArgumentError (string contains null byte).

Expected behavior

Faker::String not to create strings with null bytes. This, in turn, will not prevent the record creation.

Additional context

The rationale is that null bytes are not really possible when users give string values (as opposed to, say, a tab character). As far as I understand UTF-8, a null byte is a bit like an "absence of character" (?).

This is the only thing preventing me from using the true power of Faker when generating random strings.

How do people usually use Faker in this situation? I would prefer not to manually remove these null bytes, eg. Faker::String.random.tr("\u0000", '').

Please tell me if I'm looking at this the wrong way. Would another method on Faker::String be useful for when every kind of characters is wanted (eg. Faker::String.utf8)? Or perhaps a Faker::Binary?

Thank you.

[Zeragamba]

I noticed this issue in the rails repo that sounds similar to your issue.

[zyrthofar]

That does sound like the same issue, and no fault of Faker.

I was in fact wondering

• if \u0000 had a place inside randomly-generated Faker::Strings;
• if another faker generator would be interesting for user-provided strings; or
• if there is anything I can do to keep using Faker::Strings without the tr hack described above.

The way I see this, user-provided strings cannot contain \u0000 characters (I don't know if this is really true? A request could be fabricated with a string containing any bytes), and this null byte would be more relevant when reading input from a file, in which case a Faker::Binary could be used to generate byte-by-byte random data.

I can make a pull request if anyone thinks this makes sense - otherwise, I'd like to understand the use-case behind null bytes in string.

Thank you for your input!

[stympy]

Null bytes do show up in input strings in Rails apps... we see it regularly at Honeybadger. :)

[zyrthofar]

@stympy So I take it you don't use a postgresql database? What do you recommend I do in this situation? I'm about to use Faker::Lorem instead, or something that will only have "normal" characters.

[stympy]

I do... we escape that character in particular before inserting user strings into Postgres.

[zyrthofar]

Thank you for your time. I'll probably do something similar.