Telethon: Any plans to upgrade to MTProto 2.0

[matiniamirhossein]

Hello,
Thanks for your awesome library.

Is there any plan to upgrade to MTProto v2.0 ?

[Lonami]

@matiniamirhossein eventually.

[matiniamirhossein]

@lonami I don`t know how to thank you for this library; Implementing MTProto myself takes a lot of time and effort and also MadelineProto is a mess.

I didn't see much changed from MTProto v1.0 to 2.0 version.

I hope you update soon. Thanks <3

[Lonami]

I think we can keep this open until it's implemented, because it has to be done. Also, thanks for the appreciation!

[danog]

Just a quick question, why do you think my client is a mess?
I am honestly interested, I am always open to constructive criticism.

[matiniamirhossein]

@danog Your library needs a lot of dependencies, needs whole php compilation and uses lot's of memory. Actually my friends are using your library and I'm thankful of you but I prefer this one because it's not heavy.

Madeline proto memory usage snapshot:

I'm not allowed to paste code here but it's 10 lines only in the while loop. inserts a few to mysql.

[danog]

LMFAO I have never seen such high memory usage values on any instance of my client, probably you didn't use the getUpdates offset properly, not deleting old updates.
About the dependencies, any MTProto client (including this one) requires dependecies to handle big numbers, cryptography and VoIP, and compilation of native extensions is actually not needed, since native fallbacks are always used, where possible.

[Lonami]

dependecies to handle big numbers

Built into the language.

cryptography

Mostly AES, RSA could be easily get cut away.

VoIP

Not supported! But indeed would use libtgvoip.

[ingria]

Sorry for the offtop, but:

I have never seen such high memory usage values

I've experienced such memory issues too, before I switched to Telethon 🤷‍♂️ .

That happens because your library is serializing literally everything, even what's not really need to be serialized. As the PHP script is (usually) not intended to run as a daemon, one have to perform that resource-consuming _ioread-parsing-serialization-iowrite_ cycle very often.

[matiniamirhossein]

@ingria @danog @Lonami I think madeline should definitely switch to mysql instead of serializing. Telethon does a good job in saving entities and you can disable saving entities but I think We should all use a mysql or sqlite3 database. That way the process is done on mysql. No more memory usage by php or python. and simply instead of parsing whole file and keep it into memory each time (Consider 500 groups with 5-13k members) we query database each time to only get the needed result.

[Lonami]

I thought of that too (using Python's sqlite). We could have a single database for all sessions, maybe in the home directory. But would this work out with several instances running at the same time accessing the database? Another option is a database per session, with just two tables, session (with the required information and a single row) and entities.

[matiniamirhossein]

@Lonami Definitely the way to go is database per session. I tried using a database file with multiple instances (with threading and multiprocess library) even with mysql which is thread safe pretty much you must create one connection per instance/thread.

[ingria]

I'd personally vote against that. Caching should be your app's responsibility, not the underlying lib's.

Maybe in something called telethon-cli that would be appropriate, along with convenient readline input, json output, sockets, etc.

[Lonami]

Caching should be your app's responsibility, not the underlying lib's.

Okay, but there are a lot of people that would love to run .get_entity('username') without having to worry about FloodWaitError for a whole day. It's a lot more convenient to remember the username rather than the user ID (which I need to save anyway if you want to be able to get users by ID). So, I don't think letting every application reimplement this is a good idea, more when I can easily hook-up saving them in a straightforward way on the library. And yet, yes. You can disable this and re-implement it in your own application.

[ingria]

Yeah, of course it's convenient, yet not correct. Because username is not permanent and tend to change sometimes. Sometimes people even switch usernames between different entities – e. g. between the channel and the user.

That's why there's no permanent caching in official telegram apps. Just saying.

[Lonami]

There isn't? The Android application can show all dialogs, people names, usernames and phones, even when you're connected to the network. I know they can change and it won't be 100% reliable but it's still convenient, while allowing of course a feature to force a re-fetch. And if all the updates are caught, the values will be updated to be correct. Just like I presume official clients do.

[Lonami]

MtProto 2.0 is actually trivial. Thanks again @delivrance, I was too lazy to understand the official documentation which actually had nice pseudocode anyway.

[delivrance]

Python code ain't pseudocode :^)