Ejabberd: How to change permanently the storage type for database tables/nodes

[Emilio-Gonzalez]

What version of ejabberd are you using?

17.08

What operating system (version) are you using?

CENTOS 7

How did you install ejabberd (source, package, distribution)?

.RPM package

What did not work as expected? Are there error messages in the log? What
was the unexpected behavior? What was the expected result?

I need to change the storage type for database tables/nodes.
For example, from _disc and ram copy_ to _ram_copy_
How can I change it permanently?
Each time Ejabberd restarts, the table/node returns to the default storage type.

I'm changing it at the Ejabberd admin webpage o via Mnesia shell, but the result is always the same

[zinid]

Create ejabberd.schema inside Mnesia directory. It's a simiple YAML file. Set something like:

table_name:
  storage_type: ram_copies

[Emilio-Gonzalez]

Could you tell me, please, where is the Mnesia directory?
I can´t find it.

/opt/ejabberd/database/ejabberd@localhost ?

[zinid]

Yes, you have some *.DAT files in there.

[Emilio-Gonzalez]

Ok, I have tested it an it works!!

Thanks.

[Emilio-Gonzalez]

Now there is a problem creating an ejabberd cluster.
It doesnt´work, ejabberd fails to start.

[debug] <0.175.0>@ejabberd_mnesia:schema:201 No custom Mnesia schema for table 'ejabberd_commands' found

[error] <0.258.0> CRASH REPORT Process <0.258.0> with 0 neighbours exited with reason: {aborted,{no_exists,ejabberd_commands}} in gen_server:init_it/6 line 352

[error] <0.248.0> Supervisor ejabberd_sup had child ejabberd_commands started with ejabberd_commands:start_link() at undefined exit with reason {aborted,{no_exists,ejabberd_commands}} in context start_error

[error] <0.61.0> CRASH REPORT Process <0.61.0> with 0 neighbours exited with reason: {{shutdown,{failed_to_start_child,ejabberd_commands,{aborted,{no_exists,ejabberd_commands}}}},{ejabberd_app,start,[normal,[]]}} in application_master:init/4 line 134

[info] <0.31.0> Application ejabberd exited with reason: {{shutdown,{failed_to_start_child,ejabberd_commands,{aborted,{no_exists,ejabberd_commands}}}},{ejabberd_app,start,[normal,[]]}}

[zinid]

I don't think this is related.

[Emilio-Gonzalez]

Each Ejabberd runs correctly alone, with and without the new ejabberd.schema file.

The cluster runs ok without the new file, but it doesn´t with it.

Ejabberd is looking for other custom table schema:
_No custom Mnesia schema for table 'ejabberd_commands' found_

[zinid]

Show mnesia:info() from both nodes

[Emilio-Gonzalez]

Before creating the cluster

_Node_1_
(ejabberd@node_1)1> mnesia:info().
---> Processes holding locks <---
---> Processes waiting for locks <---
---> Participant transactions <---
---> Coordinator transactions <---
---> Uncertain transactions <---
---> Active tables <---
pubsub_node : with 0 records occupying 298 words of mem
route : with 6 records occupying 417 words of mem
ejabberd_commands: with 134 records occupying 49279 words of mem
mod_register_ip: with 0 records occupying 298 words of mem
caps_features : with 0 records occupying 5464 bytes on disc
bosh : with 0 records occupying 298 words of mem
acl : with 18 records occupying 736 words of mem
access : with 36 records occupying 1186 words of mem
shaper : with 2 records occupying 320 words of mem
carboncopy : with 0 records occupying 298 words of mem
last_activity : with 0 records occupying 5464 bytes on disc
pubsub_index : with 0 records occupying 298 words of mem
vcard_search : with 0 records occupying 298 words of mem
motd_users : with 0 records occupying 5464 bytes on disc
reg_users_counter: with 2 records occupying 322 words of mem
sr_user : with 0 records occupying 298 words of mem
schema : with 40 records occupying 5319 words of mem
roster_version : with 0 records occupying 5464 bytes on disc
pubsub_orphan : with 0 records occupying 298 words of mem
session : with 0 records occupying 298 words of mem
pubsub_last_item: with 0 records occupying 298 words of mem
offline_msg : with 0 records occupying 5464 bytes on disc
private_storage: with 0 records occupying 5464 bytes on disc
motd : with 0 records occupying 5464 bytes on disc
sr_group : with 0 records occupying 298 words of mem
oauth_token : with 0 records occupying 5464 bytes on disc
pubsub_item : with 0 records occupying 5464 bytes on disc
muc_room : with 0 records occupying 298 words of mem
privacy : with 0 records occupying 5464 bytes on disc
pubsub_state : with 0 records occupying 88 words of mem
iq_response : with 0 records occupying 298 words of mem
passwd : with 0 records occupying 5464 bytes on disc
temporarily_blocked: with 0 records occupying 298 words of mem
muc_registered : with 0 records occupying 298 words of mem
s2s : with 0 records occupying 298 words of mem
session_counter: with 0 records occupying 298 words of mem
muc_online_room: with 0 records occupying 88 words of mem
vcard : with 0 records occupying 5464 bytes on disc
route_multicast: with 0 records occupying 298 words of mem
roster : with 0 records occupying 5464 bytes on disc
===> System info in version "4.14.3", debug level = none <===
opt_disc. Directory "/opt/ejabberd/database/ejabberd@localhost" is used.
use fallback at restart = false
running db nodes = ['ejabberd@node_1']
stopped db nodes = []
master node tables = []
remote = []
ram_copies = [access,acl,bosh,carboncopy,ejabberd_commands,
iq_response,mod_register_ip,muc_online_room,
pubsub_last_item,pubsub_node,reg_users_counter,route,
route_multicast,s2s,session,session_counter,shaper,
temporarily_blocked]
disc_copies = [muc_registered,muc_room,pubsub_index,pubsub_orphan,
pubsub_state,schema,sr_group,sr_user,vcard_search]
disc_only_copies = [caps_features,last_activity,motd,motd_users,oauth_token,
offline_msg,passwd,privacy,private_storage,pubsub_item,
roster,roster_version,vcard]
[{'ejabberd@node_1',disc_copies}] = [muc_registered,pubsub_state,
muc_room,sr_group,pubsub_orphan,
schema,sr_user,vcard_search,
pubsub_index]
[{'ejabberd@node_1',disc_only_copies}] = [roster,vcard,passwd,
privacy,pubsub_item,
oauth_token,caps_features,
motd,private_storage,
offline_msg,roster_version,
motd_users,last_activity]
[{'ejabberd@node_1',ram_copies}] = [shaper,route_multicast,
muc_online_room,mod_register_ip,
session_counter,s2s,
temporarily_blocked,iq_response,
bosh,acl,access,pubsub_last_item,
session,reg_users_counter,
carboncopy,ejabberd_commands,
route,pubsub_node]
66 transactions committed, 4 aborted, 0 restarted, 6 logged to disc
0 held locks, 0 in queue; 0 local transactions, 0 remote
0 transactions waits for other nodes: []
ok

_Node_2_
(ejabberd@node_2)1> mnesia:info().
---> Processes holding locks <---
---> Processes waiting for locks <---
---> Participant transactions <---
---> Coordinator transactions <---
---> Uncertain transactions <---
---> Active tables <---
pubsub_node : with 0 records occupying 298 words of mem
route : with 6 records occupying 417 words of mem
ejabberd_commands: with 134 records occupying 49279 words of mem
mod_register_ip: with 0 records occupying 298 words of mem
caps_features : with 0 records occupying 5464 bytes on disc
bosh : with 0 records occupying 298 words of mem
acl : with 18 records occupying 736 words of mem
access : with 36 records occupying 1186 words of mem
shaper : with 2 records occupying 320 words of mem
carboncopy : with 0 records occupying 298 words of mem
last_activity : with 0 records occupying 5464 bytes on disc
pubsub_index : with 0 records occupying 298 words of mem
vcard_search : with 0 records occupying 298 words of mem
motd_users : with 0 records occupying 5464 bytes on disc
reg_users_counter: with 2 records occupying 322 words of mem
sr_user : with 0 records occupying 298 words of mem
schema : with 40 records occupying 5319 words of mem
roster_version : with 0 records occupying 5464 bytes on disc
pubsub_orphan : with 0 records occupying 298 words of mem
session : with 0 records occupying 298 words of mem
pubsub_last_item: with 0 records occupying 298 words of mem
offline_msg : with 0 records occupying 5464 bytes on disc
private_storage: with 0 records occupying 5464 bytes on disc
motd : with 0 records occupying 5464 bytes on disc
sr_group : with 0 records occupying 298 words of mem
oauth_token : with 0 records occupying 5464 bytes on disc
pubsub_item : with 0 records occupying 5464 bytes on disc
muc_room : with 0 records occupying 298 words of mem
privacy : with 0 records occupying 5464 bytes on disc
pubsub_state : with 0 records occupying 88 words of mem
iq_response : with 0 records occupying 298 words of mem
passwd : with 0 records occupying 5464 bytes on disc
temporarily_blocked: with 0 records occupying 298 words of mem
muc_registered : with 0 records occupying 298 words of mem
s2s : with 0 records occupying 298 words of mem
session_counter: with 0 records occupying 298 words of mem
muc_online_room: with 0 records occupying 88 words of mem
vcard : with 0 records occupying 5464 bytes on disc
route_multicast: with 0 records occupying 298 words of mem
roster : with 0 records occupying 5464 bytes on disc
===> System info in version "4.14.3", debug level = none <===
opt_disc. Directory "/opt/ejabberd/database/ejabberd@localhost" is used.
use fallback at restart = false
running db nodes = ['ejabberd@node_2']
stopped db nodes = []
master node tables = []
remote = []
ram_copies = [access,acl,bosh,carboncopy,ejabberd_commands,
iq_response,mod_register_ip,muc_online_room,
pubsub_last_item,pubsub_node,reg_users_counter,route,
route_multicast,s2s,session,session_counter,shaper,
temporarily_blocked]
disc_copies = [muc_registered,muc_room,pubsub_index,pubsub_orphan,
pubsub_state,schema,sr_group,sr_user,vcard_search]
disc_only_copies = [caps_features,last_activity,motd,motd_users,oauth_token,
offline_msg,passwd,privacy,private_storage,pubsub_item,
roster,roster_version,vcard]
[{'ejabberd@node_2',disc_copies}] = [muc_registered,pubsub_state,
muc_room,sr_group,pubsub_orphan,
schema,sr_user,vcard_search,
pubsub_index]
[{'ejabberd@node_2',disc_only_copies}] = [roster,vcard,passwd,
privacy,pubsub_item,
oauth_token,caps_features,
motd,private_storage,
offline_msg,roster_version,
motd_users,last_activity]
[{'ejabberd@node_2',ram_copies}] = [shaper,route_multicast,
muc_online_room,mod_register_ip,
session_counter,s2s,
temporarily_blocked,iq_response,
bosh,acl,access,pubsub_last_item,
session,reg_users_counter,
carboncopy,ejabberd_commands,
route,pubsub_node]
66 transactions committed, 4 aborted, 0 restarted, 6 logged to disc
0 held locks, 0 in queue; 0 local transactions, 0 remote
0 transactions waits for other nodes: []
ok

[Emilio-Gonzalez]

Executing the cluster:
sudo /opt/ejabberd-17.08/bin/ejabberdctl join_cluster 'ejabberd@node_2'

Error: {{shutdown,
{failed_to_start_child,ejabberd_commands,
{aborted,{no_exists,ejabberd_commands}}}},
{ejabberd_app,start,[normal,[]]}}

[Emilio-Gonzalez]

After executing the cluster:

_Node_1_
(ejabberd@node_1)1> mnesia:info().
---> Processes holding locks <---
---> Processes waiting for locks <---
---> Participant transactions <---
---> Coordinator transactions <---
---> Uncertain transactions <---
---> Active tables <---
pubsub_node : with 0 records occupying 298 words of mem
route : with 6 records occupying 441 words of mem
ejabberd_commands: with 0 records occupying 298 words of mem
carboncopy : with 0 records occupying 298 words of mem
last_activity : with 0 records occupying 5464 bytes on disc
pubsub_index : with 0 records occupying 298 words of mem
vcard_search : with 0 records occupying 298 words of mem
motd_users : with 0 records occupying 5464 bytes on disc
reg_users_counter: with 2 records occupying 322 words of mem
sr_user : with 0 records occupying 298 words of mem
roster_version : with 0 records occupying 5464 bytes on disc
pubsub_orphan : with 0 records occupying 298 words of mem
session : with 0 records occupying 298 words of mem
pubsub_last_item: with 0 records occupying 298 words of mem
offline_msg : with 0 records occupying 5464 bytes on disc
private_storage: with 0 records occupying 5464 bytes on disc
motd : with 0 records occupying 5464 bytes on disc
access : with 0 records occupying 298 words of mem
caps_features : with 0 records occupying 5432 bytes on disc
sr_group : with 0 records occupying 298 words of mem
oauth_token : with 0 records occupying 5464 bytes on disc
pubsub_item : with 0 records occupying 5464 bytes on disc
muc_room : with 0 records occupying 298 words of mem
acl : with 0 records occupying 298 words of mem
privacy : with 0 records occupying 5464 bytes on disc
bosh : with 0 records occupying 298 words of mem
pubsub_state : with 0 records occupying 88 words of mem
iq_response : with 0 records occupying 298 words of mem
passwd : with 0 records occupying 5464 bytes on disc
temporarily_blocked: with 0 records occupying 298 words of mem
muc_registered : with 0 records occupying 298 words of mem
s2s : with 0 records occupying 298 words of mem
session_counter: with 0 records occupying 298 words of mem
mod_register_ip: with 0 records occupying 298 words of mem
muc_online_room: with 0 records occupying 88 words of mem
vcard : with 0 records occupying 5464 bytes on disc
route_multicast: with 0 records occupying 298 words of mem
shaper : with 0 records occupying 298 words of mem
roster : with 0 records occupying 5464 bytes on disc
schema : with 40 records occupying 5668 words of mem
===> System info in version "4.14.3", debug level = none <===
opt_disc. Directory "/opt/ejabberd/database/ejabberd@localhost" is used.
use fallback at restart = false
running db nodes = ['ejabberd@node_2','ejabberd@node_1']
stopped db nodes = []
master node tables = []
remote = []
ram_copies = [access,acl,bosh,carboncopy,ejabberd_commands,
iq_response,mod_register_ip,muc_online_room,
pubsub_last_item,pubsub_node,reg_users_counter,route,
route_multicast,s2s,session,session_counter,shaper,
temporarily_blocked]
disc_copies = [muc_registered,muc_room,pubsub_index,pubsub_orphan,
pubsub_state,schema,sr_group,sr_user,vcard_search]
disc_only_copies = [caps_features,last_activity,motd,motd_users,oauth_token,
offline_msg,passwd,privacy,private_storage,pubsub_item,
roster,roster_version,vcard]
[{'ejabberd@node_1',disc_copies},
{'ejabberd@node_2',disc_copies}] = [schema,muc_registered,
pubsub_state,muc_room,sr_group,
pubsub_orphan,sr_user,
vcard_search,pubsub_index]
[{'ejabberd@node_1',disc_only_copies}] = [caps_features]
[{'ejabberd@node_1',disc_only_copies},
{'ejabberd@node_2',disc_only_copies}] = [roster,vcard,passwd,
privacy,pubsub_item,
oauth_token,motd,
private_storage,
offline_msg,roster_version,
motd_users,last_activity]
[{'ejabberd@node_1',ram_copies}] = [shaper,mod_register_ip,bosh,acl,
access,ejabberd_commands]
[{'ejabberd@node_1',ram_copies},
{'ejabberd@node_2',ram_copies}] = [route_multicast,muc_online_room,
session_counter,s2s,
temporarily_blocked,iq_response,
pubsub_last_item,session,
reg_users_counter,carboncopy,
route,pubsub_node]
44 transactions committed, 2 aborted, 2 restarted, 120 logged to disc
0 held locks, 0 in queue; 0 local transactions, 0 remote
0 transactions waits for other nodes: []
ok

_Node_2_
(ejabberd@node_2)1> mnesia:info().
---> Processes holding locks <---
---> Processes waiting for locks <---
---> Participant transactions <---
---> Coordinator transactions <---
---> Uncertain transactions <---
---> Active tables <---
pubsub_node : with 0 records occupying 298 words of mem
route : with 6 records occupying 417 words of mem
ejabberd_commands: with 134 records occupying 49279 words of mem
carboncopy : with 0 records occupying 298 words of mem
last_activity : with 0 records occupying 5464 bytes on disc
pubsub_index : with 0 records occupying 298 words of mem
vcard_search : with 0 records occupying 298 words of mem
motd_users : with 0 records occupying 5464 bytes on disc
reg_users_counter: with 2 records occupying 322 words of mem
sr_user : with 0 records occupying 298 words of mem
roster_version : with 0 records occupying 5464 bytes on disc
pubsub_orphan : with 0 records occupying 298 words of mem
session : with 0 records occupying 298 words of mem
pubsub_last_item: with 0 records occupying 298 words of mem
offline_msg : with 0 records occupying 5464 bytes on disc
private_storage: with 0 records occupying 5464 bytes on disc
motd : with 0 records occupying 5464 bytes on disc
access : with 36 records occupying 1186 words of mem
caps_features : with 0 records occupying 5464 bytes on disc
sr_group : with 0 records occupying 298 words of mem
oauth_token : with 0 records occupying 5464 bytes on disc
pubsub_item : with 0 records occupying 5464 bytes on disc
muc_room : with 0 records occupying 298 words of mem
acl : with 18 records occupying 736 words of mem
privacy : with 0 records occupying 5464 bytes on disc
bosh : with 0 records occupying 298 words of mem
pubsub_state : with 0 records occupying 88 words of mem
iq_response : with 0 records occupying 298 words of mem
passwd : with 0 records occupying 5464 bytes on disc
temporarily_blocked: with 0 records occupying 298 words of mem
muc_registered : with 0 records occupying 298 words of mem
s2s : with 0 records occupying 298 words of mem
session_counter: with 0 records occupying 298 words of mem
mod_register_ip: with 0 records occupying 298 words of mem
muc_online_room: with 0 records occupying 88 words of mem
vcard : with 0 records occupying 5464 bytes on disc
route_multicast: with 0 records occupying 298 words of mem
shaper : with 2 records occupying 320 words of mem
roster : with 0 records occupying 5464 bytes on disc
schema : with 40 records occupying 5668 words of mem
===> System info in version "4.14.3", debug level = none <===
opt_disc. Directory "/opt/ejabberd/database/ejabberd@localhost" is used.
use fallback at restart = false
running db nodes = ['ejabberd@node_1','ejabberd@node_2']
stopped db nodes = []
master node tables = []
remote = []
ram_copies = [access,acl,bosh,carboncopy,ejabberd_commands,
iq_response,mod_register_ip,muc_online_room,
pubsub_last_item,pubsub_node,reg_users_counter,route,
route_multicast,s2s,session,session_counter,shaper,
temporarily_blocked]
disc_copies = [muc_registered,muc_room,pubsub_index,pubsub_orphan,
pubsub_state,schema,sr_group,sr_user,vcard_search]
disc_only_copies = [caps_features,last_activity,motd,motd_users,oauth_token,
offline_msg,passwd,privacy,private_storage,pubsub_item,
roster,roster_version,vcard]
[{'ejabberd@node_1',disc_copies},
{'ejabberd@node_2',disc_copies}] = [schema,muc_registered,
pubsub_state,muc_room,sr_group,
pubsub_orphan,sr_user,
vcard_search,pubsub_index]
[{'ejabberd@node_1',disc_only_copies},
{'ejabberd@node_2',disc_only_copies}] = [roster,vcard,passwd,
privacy,pubsub_item,
oauth_token,motd,
private_storage,
offline_msg,roster_version,
motd_users,last_activity]
[{'ejabberd@node_1',ram_copies},
{'ejabberd@node_2',ram_copies}] = [route_multicast,muc_online_room,
session_counter,s2s,
temporarily_blocked,iq_response,
pubsub_last_item,session,
reg_users_counter,carboncopy,
route,pubsub_node]
[{'ejabberd@node_2',disc_only_copies}] = [caps_features]
[{'ejabberd@node_2',ram_copies}] = [shaper,mod_register_ip,bosh,acl,
access,ejabberd_commands]
130 transactions committed, 4 aborted, 0 restarted, 127 logged to disc
0 held locks, 0 in queue; 0 local transactions, 0 remote
0 transactions waits for other nodes: []
ok

[zinid]

How does your ejabberd.schema file looks like? Is it the same on both nodes?

[Emilio-Gonzalez]

Yes, it is:

pubsub_node:
storage_type: ram_copies

[Emilio-Gonzalez]

Now, I have made a test and works fine, the cluster starts ok.
So it seems the problem is related to the custom schema file.

I created a new ejabberd.schema file with all the tables with their storage type, like this:
_access:
storage_type: ram_copies
acl:
storage_type: ram_copies
bosh:
storage_type: ram_copies
caps_features:
storage_type: ram_copies
carboncopy:
storage_type: ram_copies
ejabberd_commands:
storage_type: ram_copies
iq_response:
storage_type: ram_copies
last_activity:
storage_type: ram_copies
mod_register_ip:
storage_type: ram_copies
motd:
storage_type: ram_copies
motd_users:
storage_type: ram_copies
muc_online_room:
storage_type: ram_copies
muc_registered:
storage_type: ram_copies
muc_room:
storage_type: ram_copies
oauth_token:
storage_type: ram_copies
offline_msg:
storage_type: ram_copies
passwd:
storage_type: ram_copies
privacy:
storage_type: ram_copies
private_storage:
storage_type: ram_copies
pubsub_index:
storage_type: ram_copies
pubsub_item:
storage_type: ram_copies
pubsub_last_item:
storage_type: ram_copies
pubsub_node:
storage_type: ram_copies
pubsub_orphan:
storage_type: ram_copies
pubsub_state:
storage_type: ram_copies
reg_users_counter:
storage_type: ram_copies
roster:
storage_type: ram_copies
roster_version:
storage_type: ram_copies
route:
storage_type: ram_copies
route_multicast:
storage_type: ram_copies
s2s:
storage_type: ram_copies
schema:
storage_type: ram_copies
session:
storage_type: ram_copies
session_counter:
storage_type: ram_copies
shaper:
storage_type: ram_copies
sr_group:
storage_type: ram_copies
sr_user:
storage_type: ram_copies
temporarily_blocked:
storage_type: ram_copies
vcard:
storage_type: ram_copies
vcard_search:
storage_type: ram_copies_

[zinid]

o_O, everything in RAM...

[Emilio-Gonzalez]

Yes, it is only a test to know if the cluster starts.
And it does.

As I can see, Ejabberd is lookiing for other table/nodes custom storage types in the ejabberd.schema file.
If they are missing, it fails starting.

[zinid]

As I can see, Ejabberd is lookiing for other table/nodes custom storage types in the ejabberd.schema file. If they are missing, it fails starting.

No, I don't see this in the code.

[Emilio-Gonzalez]

But it happens

[zinid]

Maybe there is a problem in join_cluster command, dunno, I'm not the author of that code.

[Emilio-Gonzalez]

It could be, of course

[zinid]

I'm calling for @cromain :)

[Emilio-Gonzalez]

Could you tell me what different options can I use in this custom ejabberd.schema file?
How are they written?

Provisionally, I'll make a file with all the correct table/nodes storage types and continue working with the cluster.

[zinid]

Here they are: https://github.com/processone/ejabberd/blob/17.08/src/ejabberd_mnesia.erl#L232
Hope it understandable :)

[Emilio-Gonzalez]

Yes, it is.
_Type == ram_copies;
Type == disc_copies;
Type == disc_only_copies;_

Can I supose that _ram_copies_ is only for RAM, _disc_only_copies_ is only for DISC and _disc_copies_ is for BOTH?
Or what should I use for both options (ram+disc)?

[zinid]

Yes, you got it correctly, you should use disc_copies for ram+disc.

[lock[bot]]

This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.