C-toxcore: Add alias support

[ovalseven8]

At the moment, every client implements an own system how to save aliases for contacts. Because every client should support aliases (they are really important because of how Tox works), it would makes sense to add it to the .tox file.

If you use client X with aliases and export the Tox profile because you'd like to use it on another device, everybody expect that the names in the contact list are the same as before. But that's not the case currently.

Pros:
:white_check_mark: It will be much easier for client developers to implement alias support
:white_check_mark: Because of the consistent save system, import/export of a profile will be easier and more comfortable for the users
:white_check_mark: Global alias support and no client patchwork rug solution
:white_check_mark: Toxic gets alias support -> https://github.com/JFreegman/toxic/issues/382

---

As far as I can remember, @JFreegman agreed to implement it when he has time. :) Unfortunately, he didn't start because there was no chance to get it in irungentoo/toxcore. But because we have a new repository now, it's time to discuss it again, I think.

[GrayHatter]

:-1:

[nurupo]

It's not very likely this will be in toxcore (the messenger layer of toxcore), as my understanding is that we try to keep toxcore minimal, providing only minimum of the necessary functionality. Aliases can be implemented on top of toxcore, so they not necessarily should be implemented in toxcore, they can be implemented in either a client or as a separate C library that all clients use. If implemented in clients, some kind of standardization would be needed, so that clients all agree on a export/import format of the aliases (and perhaps what aliases are and how they should be implemented). As for a library, there has been an idea for a high-level/extra client library that implements some goodies on top of toxcore, making life of client developers easier. This could be one of the things that library could do.

[ovalseven8]

Thing is, if I transfer my .tox file from one device to the other: Actually, I expect that the aliases are backuped too.

In case there won't be alias support, a solution to the problem is needed.

[cleverca22]

what about more of a generic property system that lets you assign key=value pairs to a friend?

[GrayHatter]

@cleverca22 that would be the obvious solution, but why is that the job of toxcore to make it easy for one client to be replaced by another?

[aaannndddyyy]

If another library will be introduced ta makes use of toxcore and is used by virtually all clients, you could simply have a new file, not a .tox but .bla
This would contain the .tox data AND the additional data you want to store. That library would then exfract the bare .tox file from the .bla file and on each change of the .tox file keep both in sync.
This would require that the lib is aware of all changes that are being performed. But that is a minimal change tomtoxcore or none at all if done well, and yet you'd get a single file that you can use for migrating and backing up not only your own keys and contacts, but a lot of additional information, like profile data, nicknames, notes, .....without changing and bloating toxcore.

[SkyzohKey]

See https://github.com/RicinApp/Ricin/issues/45

@GrayHatter

why is that the job of toxcore to make it easy for one client to be replaced by another?

Cuz TCS, and because the better tox clients will be compatible, the better it'll be for end-user, and we both know that end-user expect things to "just works".

[GrayHatter]

Cuz TCS,

You just made my point for me. tacos exists because people wanted clients to get along, but didn't think core should enforce any of those.

[ovalseven8]

In the .tox file, all your contacts are already saved. I don't see any reason why not to extend the .tox file so that additionally to the contact you can (optionally!) save an alias matched to the contact/Tox ID.

[SkyzohKey]

Bump, this is actually needed and should be fairly simple to do I guess (despite my poor C level :/).
@GrayHatter what bad with adding this ? That's a feature enhancement, meaning you can keep your contacts aliases when you change client. Having to create aliases again on client change is frustrating for the end-user. Do we want user to be frustrated while using Tox ?

[GrayHatter]

@SkyzohKey As I've said on irc, don't actually care what the end user wants. I care about the best interests of Toxcore. That said, Toxcore should make communication from system A to remote system B easy. Not transferring of client info from one client to another on the same system.

I'm still against alias support in Toxcore.

[ovalseven8]

Don't get me wrong, but your argumentation really sucks.

1. Do you want to build software for human beings or for nobody? If one doesn't care about what the end user wants, the whole point of creating software is useless. Except you create it only for yourself.
2. In the .tox file the contacts are already saved! The .tox file is actually your profile. And if you transfer your .tox file WITH your contacts you expect that the aliases are transferred too.
3. To (optionally) add the alias to the matching Tox ID in the .tox file improves UX a lot because profile import/export is much easier. And: From my point of view, the alias is definitely not client info! It's global information needed for the user to use Tox in a better way.

[zetok]

@ovalseven8 No. Read what @nurupo wrote.

It's basically the right idea, except for the lib being written in C.

A simple start (working version 0.0.0) would be to provide a library that parses input file (bytes) into 2 things: client data, and toxcore data.

Input bytes – that would be json.

Storage format

• version as number in json
• toxcore data (raw bytes) stored as base64 in json
• client data stored as json key=value.

version

Allows client to know whether given format version is supported (by toxcore it uses, or by json encoded data).

toxcore data

toxcore will get what it wants, for raw bytes and STR8C were made to be together

client data

Could be a json object that client is supposed to handle; could be separated into parts like "common" (working across clients) and "client-name-specific", or something.

Clients would be supposed to modify only their own part and common one.

Language choice: Rust

_As you are aware_ C is a horrible language to work with when it comes to code correctness/security.
The problems with languages other than C include interoperability (almost every language can bind to C) and performance.

Rust makes it easy to write correct, memory safe code, yada yada and at the same time avoids C flaws. Not that performance really matters in this case, that's just FYI. Native support for C FFI is where things are at. Meaning, you have safe Rust code and C interface that you can work with.

Interface exposed by lib

I was thinking about 4-5 functions exposed by lib.

/// Returns `false if not toxj data or unsupported version.
pub fn is_toxj(bytes: &[u8]) -> bool;

pub fn decode_client_data(bytes: &[u8]) -> Result<Vec<u8>>;

pub fn encode_data(toxcore_data: &[u8], client_data: &[u8]) -> Result<Vec<u8>>;

pub fn decode_toxcore_data(bytes: &[u8]) -> Vec<u8>;

That could be a Rust version, client_data possibly could be of type Json, in which case there would be a failure if data provided by client to the lib wouldn't be json. But perhaps binary data is fine too.

Don't worry, on top of Rust API the C FFI would be layered on, with all the horrible stuff that C requires.

Data flow

https://gist.github.com/zetok/7d670045bf0f999395ec2ebee5a8b77c (gh doesn't like embedding .svg)
(rust-powered)

Opinions?

[ovalseven8]

@zetok: Well, your suggestions does not conflict with my request. It's basically a new, restructured .tox file containing the aliases.

[sudden6]

@zetok The idea fails again at compatibility of minimal clients that don't want any extra library.

My idea would be to extend the *.tox file format so it can incorporate a json encoded data blob at the end. This way simple clients can just read the first part with toxcore specific stuff and advanced clients would read the json data and use it. The json data could then follow a common specification like @zetok described it.

EDIT: To enforce the blob specification and make clients life easier there could be an official helper library, but it would be optional to use it.

[isotoxin]

Btw, current .tox file is very ineffective. It spends 2216 (as I remember) bytes per friend. Why so much? Because it saves all variable-sized fields (eg. strings) with maximum size. Json, raw text json, will be more effective then tox binary format. Paradox...

[zetok]

@sudden6

The idea fails again at compatibility of minimal clients that don't want any extra library.

The way I see it, you'd want to have either compatibility or minimalism.

There's nothing that stops client from providing users with an option to export "raw" toxcore data.

My idea would be to extend the *.tox file format so it can incorporate a json encoded data blob at the end. (…)

Since I'm implementing the current toxcore file format in zetox, I just gotta say this: it's a bad idea to try to extend it.

It should be burned with fire instead. Sadly, due to backwards compatibility requirement that won't happen.

Instead, slow & graceful replacement has to happen, and proposed by me idea should help with that.

Any other ideas how that could be achieved?

[Renha]

I've said that before,
.toxz:
zip(or anything)-archive, containing .tox file for almost-full backward compatibility, and any amount of .json files as per-client settings and variable-length data, supposed to be saved in profile. Even relatively big files could be stored - user avatar, for example.