Bentobox: UTF-8 Encoding

[wellnesscookie]

Description

Challenges that contain some " č ć ž š đ " characters in both description and title convert into " ? " after several server restarts

Steps to reproduce the behavior:

1. Have database settings set as: https://i.imgur.com/alwaqum.png
2. Restart a few times, they will convert. (At least we think it's occuring on restarts)

Server / BentoBox information:

• Database being used:
  MySQL
• OS:
  Debian GNU/Linux 9
• Java Version:
  Java(TM) SE Runtime Environment 18.3 (build 10.0.1+10)
  Java HotSpot(TM) 64-Bit Server VM 18.3 (build 10.0.1+10, mixed mode)

[20:23:04] [main/INFO]: [CHAT] Running PAPER 1.13.2.
[20:23:04] [main/INFO]: [CHAT] BentoBox version: 1.3.0-SNAPSHOT
[20:23:04] [main/INFO]: [CHAT] Loaded Game Worlds:
[20:23:04] [main/INFO]: [CHAT] skyblock (skyblock)
[20:23:04] [main/INFO]: [CHAT] Loaded Add-Ons
[20:23:04] [main/INFO]: [CHAT] BSkyBlock 1.3.0-SNAPSHOT
[20:23:04] [main/INFO]: [CHAT] WelcomeWarps 1.2-SNAPSHOT
[20:23:04] [main/INFO]: [CHAT] Challenges 0.6.0-SNAPSHOT
[20:23:04] [main/INFO]: [CHAT] Level 1.2.1-SNAPSHOT
[20:23:04] [main/INFO]: [CHAT] Biomes 0.5.0-SNAPSHOT
[20:23:04] [main/INFO]: [CHAT] SerbCraftAddon 2.0

Expected behavior

• Have it not converting if the encoding is correctly set.
• It should be "Slatka kuća" (http://prntscr.com/mosoku), but I have to change it manually everytime it converts back.

Additional context

• It doesn't seem that they all convert in a moment, it takes a while and it does randomly.

[BONNe]

This is more BentoBox issue, then Challenges addon.

[tastybento]

I'll do some investigation to see if the database storage/retrieval is doing this.

[tastybento]

@wellnesscookie When you create the database, do you have a COLLATE option for utf8mb4_unicode_ci? Apparently, utf8mb4 should be used rather than utf8 because the older utf8 has a MySQL-specific limitation preventing the use of characters numbered above 0xFFFD. I don't know if this is the issue you are having or not, but please try it if you can.

Source: https://stackoverflow.com/questions/766809/whats-the-difference-between-utf8-general-ci-and-utf8-unicode-ci

[baticag]

We are using utf8_unicode_ci for most of the tables.
You can consider to add this query when connecting to MySQL: "SET NAMES utf8" inside plugin.

[tastybento]

@baticag Thanks. I fixed it for MySQL. For MariaDB it looks like it needs a different setup so I need to have that one tested. But the MySQL one works now.

[tastybento]

MariaDB actually works as-is so there's no need for a fix there. Closing now.