Related to #102
A json export would be very helpful for those who wish to preserve the data and formatting of the messages, yet still keep a compact, text-based, easily-parsable file type.
I tried parsing the CSV format using jq and some really ugly regex hackery, but I hit some problems with the contained unicode/multibyte characters.
Exporting as proper JSON would solve my problems to a very large extent, since that'd be ASCII only with escape codes for unicode (i.e. \u0000 for the binary zero).
Until then I'll mess with my locale settings and hope to get everything working.…
Edit: I got it working with a little perl:
<input.csv perl -MJSON::XS -0777 -ne 'while(m{(?<author>[^";]+|""?(?:[^"]|"")*""?);(?<date>[^";]+|""?(?:[^"]|"")*""?);(?<content>[^";]+|""?(?:[^"]|"")*""?);(?<attachments>[^";]+|""?(?:[^"]|"")*""?);\n}g) {print(encode_json(\%+))}' | jq 'def uncsv: (if startswith("\"") then gsub("\"\"";"\"") | sub("^\"";"") | sub("\"$";"") else . end); .date |= uncsv | .attachments |= uncsv | .author |= uncsv | .content |= uncsv'
However my output still has a few encoding problems.
This might be because I work on both GNU/Linux and Windows boxes with Discord, because I can only get the encoding right per message, not by converting the input.
I think a big benefit of JSON output would be directly saving the message object, keeping all original data the way Discord returned it.
Merged JSON renderer to master.
Please check out the artifacts from the latest CI build and report any errors you may find.
Most helpful comment
I think a big benefit of JSON output would be directly saving the message object, keeping all original data the way Discord returned it.