Cherrytree: Bringing CherryTree to C++

[txe]

Hi @giuspen
As my first step, I consider porting menus.py. It looks like a good task to start, so I can begin working on it if you don't mind.

[giuspen]

Hi @txe, that is perfect but please note that the gtk ui manager used in pygtk2 is now deprecated, so the porting should not be exactly one-one, please have a look at https://developer.gnome.org/gtkmm-tutorial/stable/chapter-menus-and-toolbars.html.en and generally at https://developer.gnome.org/gtkmm-tutorial/stable/index.html.en for examples /recommended techniques. Furthermore if you download the latest gtkmm3 source code you can build the demo code with "make check". Cheers!

[txe]

@giuspen
I've made a menu using gtkmm approach, but there's no way to add menu icons through Gtk::Builder or Gio::Menu (looks like they miss this functionality because menu icons are not recommended any longer under the GNOME HIG)
I checked other Gtk3 projects and it looks like they either use deprecated Gtk::UIManager or generate GtkMenu manually.
I'd like to see icons in menu, so maybe I will try to the approach with GtkMenu? You can see how it's done in inkscape

[giuspen]

@txe agree with you on the icons, if this is possible only via non deprecated GtkMenu let's go for it

[txe]

Hi @giuspen, after #446 and #447 can you point out what is better to do the next?

[giuspen]

Hi @txe it depends if you want to keep it simple for now or delve into difficult stuff. If you want to do something simple but time consuming that would be the preferences dialog or also the various right click menus. An example of what could be done straight away is the export to txt and html since all the content is there. Or the right click menus. Or you rather start to manage editing the contents? It's up to what you would like to work mostly

[txe]

Hmm, to make it consistent, I'll take popup menus (done: #448)

[txe]

I'm going to work on the preferences dialog (done: #451)

[txe]

Hi @giuspen
While I worked on porting code, I had to read 'legacy' python code, new C++ code and also code from other projects, e.g. inkscape. And I found out that CamelCase code is not readable as should be. While I could easy scan through python code with underscores, I had to spend time just to 'parse' CamelCase function/variable names and longer the name (more words) harder to read.

Why did you decide to use other code style in the new project? Could you consider returning to the old code style or some other style like in inscape (although, they are not always consistent)

[giuspen]

Hi @txe, the reason for camel case is that it is what I'm used to in my company. In fact all the style is nicked from my company. I like that it's nicely differentiating the non gtkmm library methods against the methods belonging to the library. Be careful about inkscape because that was a C/GTK project, only subsequently gradually migrated to Gtkmm. In fact I consider the underscores naming more C style. On the other hand I'm not against using the underscores; especially if this is very important to you. More important to me is the great work that you are doing in the porting so I'm happy to change route in that (except for the class names). You are free to rename what you wish adopting the underscores.

[txe]

Big thanks, @giuspen! I really appreciate it and will try not to abuse your goodwill. About the next steps, I'm going to port some smaller functionality, the last one was rather huge and it wore me out a bit. Your advice will be welcome!

[txe]

BTW @giuspen, I wonder how you see the project architecture, to be more precise, where better keep code related to 'actions'. I thought about keeping it in cp_app/ct_main_win/ct_treestore/etc, but it looks like this will make these classes too mess, so I got another idea:

• keep general\helper functions and signal handlers in CtApp, CtMainWin, CtTreeStore,
• keep actions in special action classes, for example, CtEditorAction, CtTreeAction and etc., while object of these classes will get references to CtApp, CtMainMain and will be able to use them.

```C++
class CtApp {
private:
CtTreeAction _ctTreeAction;

void somewhere() {
       _ctTreeAction.init(this, main_win(), _treestore, something_else);
}

}

class CtTreeAction {
void add_node();
void delete_node();
// etc
}
````
What do you think? It doesn't sound too odd? Also, maybe you get some ideas you can share?

[giuspen]

@txe I agree it is tidier to keep the actions separate. In the past it happened me already to have separate files for the callbacks. But it may be sufficient to have the body of the actions in separate dedicated .cc files pointing to the same header class and group the actions nicely in the class header. This also applies generally if a class .cc files gets too big to split into logical source files. My view is to avoid the spreading of too many classes layers where not strictly necessary, but anyway if you want to try your approach on part of the code I will give you feedback later and we can rework if necessary. About the next step I had in mind was to allow the links to work if you want to give it a try but if you have other preferences feel free. I will try to get back on the gtkmm code in the early mornings of the next week (uk time)

[txe]

OK, I see your point although it is quite unusual to have separate .cc. Can you come up with a suitable name for the class, nothing good comes to my mind? And then, it will be like like ct_classname.h, ct_classname_editor.cc, ct_classname_tree.cc and etc. (so they will be nicely sorted in a IDE project tree)?
The next thing I want to do is adding/delete/duplicate nodes if the current codebase allows it.

[giuspen]

Exactly, same prefix of the header and an additional specification just like your example. You can start with the node add/delete/duplicate that should be fine, and maybe I'll start to work on the save, save as and export to Cherrytree document

[txe]

Hi @giuspen what do you think about using fmt in the project? it's handy and fast and never failed me.

[giuspen]

Hi @txe seems fine to me I'm just wondering if it will be working fine with the utf-8 Glib::ustring and i18n

[txe]

Hi @giuspen
I'm not sure if commands from Edit\Formatting\Search do not depend on some missing sourceview features or another core stuff and thus cannot be implemented.
What is the status of the source view, can I do something for it?

[giuspen]

Hi @txe, the visualisation is all there already so you can surely start working on the search. About the replace and generally the edit, that can still be done even if the undo/redo state machine is not there and neither the possibility to save the modifications to disk.

I raised an important question at https://stackoverflow.com/questions/54529359/gtkmm-3-text-view-anchored-in-text-view-cannot-get-cursor-inside and also wrote to the gtk mailing list but got nothing, this is a critical thing if anyone knows of a channel where that could be advertised please go for it.

[txe]

It looks like C++11 regex doesn't support a multiline option (it has been supported since C++17). I think it's OK to keep the built-in regex for now, and later we can switch to something else like PCRE.

[giuspen]

I think we can go C++17, we are currently C++14 but I was already thinking to go 17 in case any of the new features would have been useful to us

[txe]

Sure, then I'll switch to C++17

[txe]

Hi @giuspen, it looks like there are several issues with regex although I likely have a solution to them:

1. std::regex_search doesn't have start_from(offset) input parameter to start regex not from the string beginning. Using substring or first&last input iterators is not easy because of offset converting. I fixed it by using Glib::Regex that can do everything we need and actually looks more mature than std::regex.
2. TextBuffer works with symbols and uses positions in symbols, regex works with bytes and returns positions in bytes. So, TextBuffer returns a utf-8 string where symbols can be both one or two bytes and, actually, Glib::Regex can use this string without problem. The problem is that it's not easy to convert a byte index into a symbol index and vice versa (OMG, why they cannot always use two bytes per symbol?). As I see Glib::ustring doesn't have means for the converting, so we have to parse the string by hand just to find out these indexes.

I checked other Gtk projects for the solution and maybe GtkSourceSearchContext resolves all problems (although it is not in gtkmm). What do you think?

[txe]

I've ported mostly all find code, so maybe I can write and use converting index functions for now, and someday I'll rewrite code to use GtkSourceSearch

[giuspen]

Hi @txe, I think a gtk solution rather than a gtkmm solution is perfectly acceptable, then it's up to you. Myself raised a bug to https://gitlab.gnome.org/GNOME/gtk/issues/1663 and subscribed to the mailing list but still got no valuable feedback. I will look at the gtk source code myself and try to figure out what from gtk2 to gtk3 broke the functionality. What concerns me is that if indeed it's a bug an there's no workaround we may have to support building against gtk2 for the beginning, what do you think?

[txe]

1. You may try to rewrite the demo without gtkmm to exclude that gtkmm doesn't make a mess.
2. I don't know how easy to build gtk libraries from source, so maybe installing older\unstable version of gtk3/2 helps to find out when the bug was introduced.

[giuspen]

Yep, I will try since the very first gtk 3.0 and see if the bug was already there.

[txe]

Hi @giuspen, I took a look at the codebox bug and, in fact, it's quite easy to fix.
When you click on textViewNested, it gets focused and actually may show cursor, but the problem is the click event goes further for some reason and gets caught by textViewBase, that thinks it is its click and thus gets focused. So, the different between gtk2.4 & gtk3.0 in propagating the event further or not.
The solution is to stop the event like that:

textViewNested.signal_button_press_event().connect([](GdkEventButton* event){
        if (/* left button click*/) return true;
        return false;
});

[giuspen]

Hi @txe thanks so much you must be an angel sent from somewhere

[txe]

FIY, I'm a bit busy later, but I think I can finish formatting actions and lists this week.

[giuspen]

Cool @txe, I'm releasing 0.38.8 in the next hours and subsequently I will work on document write starting from xml

[txe]

Hi @giuspen, I'm working on clipboard.py and can't figure out from where an argument from_codebox comes in copy and cut functions.

[giuspen]

Hi @txe, it is easier to spot that if you run a 'find in files' with the pattern 'clipboard_handler' since 'copy' and 'cut' goes everywhere, in fact not a lucky choice.
Doing that you canfind, in core.py
self.sourceview.connect("copy-clipboard", self.clipboard_handler.copy, False)
self.sourceview.connect("cut-clipboard", self.clipboard_handler.cut, False)
And in codeboxes.py
anchor.sourceview.connect("copy-clipboard", self.dad.clipboard_handler.copy, True)
anchor.sourceview.connect("cut-clipboard", self.dad.clipboard_handler.cut, True)

[txe]

I saw them, but thought True/False meant whether this signal handler should be called before or after the default signal like in gtkmm. Never expect them to be data for a callback. Thanks for help!

[txe]

Hi @giuspen , congratulations for finishing xml document writing! 👍
In case you going to work on the next piece, I'm currently dealing with clipboard.py as well as exports.py, clipboard heavily depends on xml and plain text exports.

[giuspen]

Hi @txe, I was thinking to firstly test the write to xml to actually work as expected and then start with write to sqlite but I'm happy to change plans if you need anything

[txe]

That's perfectly fine! I just wanted to make sure we wouldn't waste time working on the same stuff.

[txe]

Just to be sure, there is a couple of lines in exports.py where
pango_font = pango.FontDescription(self.dad.tree_font)
It's just some legacy and doing nothing?

[giuspen]

You're right @txe it seems a dead leaf, I saw that in two places and nowhere pango_font is then being used, I will remove those lines if you don't before me

[txe]

At the very end of selection_to_clipboard function:

if not self.force_plain_text:
    targets_vector = [TARGET_CTD_PLAIN_TEXT, TARGETS_HTML[0], TARGETS_HTML[1]]
else:
    targets_vector = [TARGET_CTD_PLAIN_TEXT]
self.clipboard.set_with_data([(t, 0, 0) for t in targets_vector],
                                         self.get_func,
                                         self.clear_func,
                                         (plain_text, None, html_text, pixbuf_target))

pixbuf_target is passed into clipboard, but TARGETS_IMAGES is missing in targets_vector variable.
so do I need to add

if pixbuf_target:
    targets_vector.append(TARGETS_IMAGES[0])

or just to remove pixbuf_target (because the text is plain and images will never be there)?

[giuspen]

pixbuf_target should be replaced with None for clarity. the cases of a single character selected are covered before, including the case of a copy image, this is just copy text either rich or plain.

[txe]

Hi @giuspen, It's releated to CtConst strings, I think for the better unicode support we shouldn't use gchar (which is similar to char). Reasons:

• getting character by index from gchar[] is not good, one gchar cannot contain every unicode character
• text buffer iterators return gunichar (32 bits)
• Glib::ustring::[index] returns gunichar (Glib::ustring is a good wrapper which hides operations over a byte array)
• comparing gunichar and gchar gives warnings (and can it be invalid?)

On the other hand, using gunichar str[] = "some text"; is not reasonable, the size of unicode character can very from 1 byte to 4 bytes and thus this str is not valid.
So, ether we keep using gchar* strings and have to use g_utf8_* (from gtk) and some custom functions like str::compare(gunichar, gchar*) and remember we cannot use gchar strings as ordinary strings or
we can use Glib::ustring and never think about this again.

[giuspen]

Hi @txe, I think it makes sense to use Glib::ustring, let's keep it simple

[giuspen]

Hi @txe, I'm having issues to build since you added #include <filesystem> so I temporary applied the changes suggested at https://stackoverflow.com/questions/39231363/fatal-error-filesystem-no-such-file-or-directory but I would rather use http://manual.freeshell.org/glibmm-2.4/reference/html/group__FileUtils.html - isthere a reason why you don't want to use the glibmm version?

FILE_TEST_IS_REGULAR |  
FILE_TEST_IS_SYMLINK |  
FILE_TEST_IS_DIR |  
FILE_TEST_IS_EXECUTABLE |  
FILE_TEST_EXISTS

[txe]

Hi @giuspen, I just didn't run into Glib::file_test when I had been looking for something similar in gtk and I had to use std::filesystem. So there is no reason to use filesystem now.

[giuspen]

Great, thanks @txe

[txe]

Hi @giuspen , I've got a question:
In the app, for some reason, instead of just changing an anchor widget property, a new widget with needed properties is created on the same place. Is it done for undo/redo functionality?
For example, changing width/height of codebox in codebox_change_width_height is doing that, while I might try something like _scrolledwindow.set_size_request.

[giuspen]

Hi @txe, something was done for simplicity, something just because I keep learning and at that time that was not done right. You are more than welcome to rework an implementation that is inefficient. I also have changed the way the widgets nested into the text buffer are handled for example during the porting to gtkmm3, it's a good chance to make it right. I don't think it was for undo/redo specifically because there is a function to manually call to trigger a new state when no automatic event does it (like buffer modified or deleted)

[txe]

HI @giuspen, how are you?
I've got a few days to contribute to the project, is there something I can do?

[giuspen]

Hi @txe it's great to hear from you! The beginning of 2020 has not been the best one for me but not too bad anyway, just after a good boost during the Christmas holidays I can hardly sit at the pc in these days, hope to get back to more relaxing days soon. I was thinking to take care about the undo/redo machine myself and proposing you the print/export to PDF if ok for you but literally anything still missing is open for business. I hope all is good with you and as I said before I would be happy to buy you at least few dinners out :)

[txe]

I'm fine, thanks! BTW, I've been to Italy, visited Avellino (hi to Tony Soprano:)), ancient Pompeii, Rome and Venice. Never seen so many mountains before, and high speed trains are really something. And, fortunately, I could get out from Venice just minutes before rains delayed all flights from there :)
So, back to the proposal, I'll take a look at print/export PDF and about that payment, I'll send in an email when I figure out how to do it.

[txe]

It looks like Gtk::TextIter::forward_to_tag_toggle doesn't work, it just moves an iterator to the buffer end and, as a result, our parser (converter to html) is not working and outputting a plain text.
It was OK when I was working on the clipboard functionality (which uses html).

I checked on Ubuntu and Manjaro Arch, the same thing. It's easy to see by coping text into clipboard and inserting somewhere.

[giuspen]

Gtk::TextIter::forward_to_tag_toggle is used also to save the cherrytree document rich text nodes to disk and that does work, I have tested recently. I would just try saving a rich text node and re opening if that is ok you can compare with how you are using in the html exporter.
If it is code nodes you are talking about, be careful there's a command to ensure the highlighting of the whole buffer to be run before, don't remember if on the buffer or the view at the moment

[txe]

Yeah, thanks! It was good to know that forward_to_tag_toggle was working. Turns out there were two bugs, one with wrong iter offset, another with clipboard. I've fixed them.

[giuspen]

You're a star @txe you'll have to let us know at least roughly where are you based 😊

[txe]

I'm from Russia :)
BTW, can you give an instruction how to build exe file of pygtk version for windows? Other files, I think, I can take from the portable archive, or it is not that simple?

[giuspen]

I'm studying Russian every day on my phone with Duolingo, my wife is Ukrainian :)
In order to build the cherrytree Pygtk2 exe you should first be able to run it from the python source code first http://giuspen.com/cherrytreemanual/#build-windows then you build with "python setup.py py2exe" and yes you can merge the files with the portable archive. It is not simple to set up all though, I can build any exe version for you if you want or if you still need to be independent I can upload my virtual box virtual machine with a windows xp that I use to build all the versions.

[txe]

Wow, that's cool! We can use some Russian here if there is a desire :)

[giuspen]

😄 it is still very limited but surely I would be able to order something to eat at least 😅

[txe]

Hi @giuspen
I'm doing some nice changes in pdf print, so in case you're going to release a new version, wait a bit :)

[giuspen]

Hi @txe sure thanks I'm not going to issue a new version shortly

[txe]

Hi @giuspen I sent you an email :)
About other things, how about brushing up the github repository? If it is convenient for you, you can add me as a collaborator, while I can:

• sort out issues, add tags and remove outdated ones
• add a pinned issue with the C++ progress
  and other things to make overall appearance polish.
  I understand it is not much essential, but nice looking things always cheer up.

[giuspen]

Hi @txe, sure, collaboration invitation on its way! If you will still have time it would be great if you can also have a look at the porting of the undo/redo machine, that is the latest big missing feature in the Gtkmm3 version, others are less time consuming. If too much for you at the moment don't worry I'll try and start it ASAP but may take again one or two weeks before I can do that.

[txe]

Don't worry, I'll take it right away after finishing txt import.

[txe]

Hi @giuspen , I'll be on vocation until the next week.

[giuspen]

Hi @txe have a good rest ;)

[txe]

Hi @giuspen, I've been loaded with work in the last week, so did a half of undo/redo, I'll keep working on it.

[giuspen]

Hi @txe excellent thanks myself I'm focusing bold and run properly on Windows

[txe]

Hi @giuspen , what else can I do for now? (except tables:))

[giuspen]

Hi @txe, would you like to try and port the code node execution functionality from Pygtk2 to Gtkmm3? Otherwise going through the Gtkmm3 top menu items

[giuspen]

Just a touch mistake. Any top menu item not working, or also trying to restore the dark theme functionality that I had to comment out because had issues.

[txe]

Hi @giuspen

Any top menu item not working,

What do you mean?

Also about icons, you changed icon names in ct_menu.cc (for example, 'gtk-open' to 'open'). Are your going to add 'open.svg' into 'future/icons'? otherwise it will not work.

[giuspen]

Hi @txe I phrased it wrong, I meant not connected, whose functionality behind the menu item click is not yet ported.
I'm surprised open.csv is not there/stock not working, I will double check that later, I remember seeing the icon on the toolbar.

[txe]

I meant not connected, whose functionality behind the menu item click is not yet ported.

It's still confusing, 'Edit'->'Configuration' starts the configuration dialog.

There is no 'open.svg' icon in future/icons and the stock icon is 'gtk-open'. Or maybe my theme has 'gtk-open' instead of 'open'?

[giuspen]

Maybe all menu items are already connected to a functionality, I have to review what's left to be ported, will try and do it soon, it is easier to access the phone than the pc for me these days. If you run on your pc "find /use/share/icons -name open.*" do you get anything? All that goes to gtk stock automatically

[txe]

Hmm, I checked Icon Naming Specification and it says it should be 'document-open'. And there is a script which shows them.

[giuspen]

Oh I see. Note that I renamed the icons mainly to be sure that were available out of the standard stock for Windows/msys2. Important is that we don't have duplicates with different names but I don't mind if you want to rename few

[txe]

Sure, I rename some of them,
find /usr/share/icons -name open.* -> shows nothing
find /usr/share/icons -name *open.* -> shows 'document-open.svg'

[giuspen]

Great, I would like to add credit to you as a developer on the cherrytree homepage and in the help -- about dialog but don't know if you would be happy about that, please let me known

[txe]

Thanks, it would be great :)

[giuspen]

Do you want your name to be visible as well or only the nickname?

[txe]

Sure, let's use the name and maybe instead of email, github url be better? (don't want to get a lot of spam)

[txe]

It won't be new icons, gresource_to_c.py fixed my issue with them (I don't use build.sh)

[giuspen]

Ok got it

[txe]

OMG, it was a good opportunity to make an announce that we are moving from Gtk to Qt.

[giuspen]

For the first of April I hope 🥳

[txe]

Hi @giuspen ,
There are things to do, but maybe you bear something particular in mind which needs attention?

Meanwhile, I think about some code organizing. It doesn't need an immediate action, just to keep you informed:

• Document loading spreads thought all code CtMainWin, CtTreeStoree and others. I think it could be more straightforward and put in one file.
• Some files can be similarly named (codebox, table ->widget_table, widget_codebox) or put in one folder (widgets), or something like that.
• Move from make to cmake. Well, cmake has some benefits.

[giuspen]

Hi @txe I'm happy if you want to drive some rework/renamings, I think now would be a good moment for it.
I also prefer cmake and use that at work but I read there were some issues with the internationalisation support, for this reason I started with the autotools for Gtkmm3. I would keep supporting both rather than replacing, at least at the beginning.
After your reworks, I would like to raise the warning level, adding -Wextra and -Weffc++ that helped my group at work a lot to spot issues.
For now first thing I will setup an installer for Windows 7+ and fix the scripts to create the Debian package so that we will be able to distribute the first alpha to get the first feedback.

[txe]

Well, the next announce is that we can't go from Gtk to Qt after that: Qt, Open Source and Corona

[giuspen]

Well I wouldn't have expected this from Qt!

[txe]

@giuspen
Just to give head-ups, I'm still working on organizing code and it will take a few more days to finish.
There is one thing about versions I'd like to discuss.

There will be new features one day or another which will demand changes in file formats.
Supposedly, these changes will not be backward compatible and using an old version program will destroy some information in those files.
To fix it, we need some version check in the program and a question is. Should we introduce it as soon as possible? Or even in pygtk version just in case it will be still around at that time?

[giuspen]

@txe what I've been doing when I completed the possibility in Gtkmm3 to write all documents types and the config.cfg, was to release a new Pygtk2 version that supports reading those peculiarities that the Gtkmm3 version requires. The current Pygtk2 can read what the current Gtkmm3 version writes. I would like to keep the possibility from a newer Gtkmm3 version to read all the older Gtkmm3 versions (sometimes with an auto conversion) and at the beginning the latest Pygtk2 version to be guaranteed to be able to read that as well. Since we will not want to keep updating the Pygtk2 version though, we may indeed mark the Gtkmm3 documents with a version at a certain point and, from the Pygtk2 version, show a message in a dialogue inviting to switch to the Gtkmm3 version to read that document if no longer supported.
Here it's quite hard for me to find time to sit at the computer in lockdown with the kids, I'm so grateful you joined the project, I will try and make installers for now

[giuspen]

Consider that at the beginning people will try the Gtkmm3 version and maybe use it for few days until they realise something still doesn't work well so raise an issue and switch back to the Pygtk2 version until that bug is fixed (or that functionality is polished) this should be supported, their modifications must still be readable from the Pygtk2 version in the first phase

[txe]

Ah, yeah, I fully agree about first releases and have not intention to do otherwise (format changing, it would be total disaster).
I meant, in one or two years, some poor user can (although it is not really probable) try to open a new file (with changed format) with an old version program and it can be fixed it that old version program already has checking mechanism, that's all.

[metal450]

Not relevant to the actual porting, but just some words of encouragement: SUPER excited at the prospect of an upcoming release! I've been keeping my eye on this project for just over a year in anticipation of the day I can take the plunge & escape OneNote. Sounds like it's getting close!! :)

[txe]

Hi @giuspen,
Turns out html paste is not yet implemented, and I think it would be noticeable.
Because you already once did it, what do you think, it would be better to use third-party html lib or some handmade parser would be enough? I checked a few html copy/paste and the data not always looks like a valid html: no header and body.

[txe]

OK, looks like libxml2 includes html parser and supposedly it can handle 'real world' html.

[metal450]

Actually, I'd like to chime in: one of the only two "roadblock" issues I found with the previous CherryTree (the main reason I was waiting for the port before migrating over) probably relates to html paste (or, formatted text paste). Per #496, font sizes were lost. Even if it's not possible to support "freely sizeable" fonts right out of the gate (ie. 10pt, 12pt, etc, like Word or OneNote), if html pasting could even generally preserve sizes I think it would be a big step forward. For example, by using thresholds, where <8 becomes html tiny, <10 becomes html small, <13 normal, and so on. Hopefully it's something that could be taken into account, if paste is yet to be reimplemented anyway :)

[giuspen]

Hi @txe yes it is definitely better to use a library, if you find that libxml2 covers that please check if the C++ wrapper https://developer.gnome.org/libxml++-tutorial/stable/ that we are already using for XML read/write can be used before proceeding with direct C API

[txe]

@metal450, yeah, it's possible to do, I'll try

[metal450]

Awesome! & again, if at all feasible, I'd lean toward retaining "true" font sizes for the best compatibility with other editors. I was just mentioning the tiny/small/normal groupings as a 2nd best choice, if the former proves too complex i.e. do to storage reasons, etc. 👍

[txe]

Hi @giuspen , did you try to compile tests? for some reason, it stopped working until I added '-lm ' flag.
Maybe, it's just my PC going crazy/

[giuspen]

Ho @txe I built yesterday on 18.04 and found a different issue with your latest changes not building for me, we have different versions of autotools and compiler, you have probably the latest since you are running Arch if I recall correctly. That's good it has to build everywhere ;)

[txe]

Actually, I was using Ubuntu Budgie in vmware just for fun and turns out it didn't work well under heavy load; vm and keyboard freezing and there were a lot of crashes. Now, I'm on arch with xfce.

This weekend, I tried to work with 10000 nodes (100x100) to see if it's possible. There are two choke points from the view of creating/loading 10000 nodes:

• slow applying style to text in TextBuffer. When text is put into a textbuffer, that fires a signal and after all nodes are created, in an idle main event loop, signals begin to apply style to text. And it's slow.
• again, when anchors are inserted in a textbuffer, that fires a signal and after all, textbuffer starts to calculate a widget sizes. Moreover it does a few iterations for each widget and it is absolute nightmare, the program is freezed for many minutes even when nodes are not that much.

Currently, it is possible to create/load > 1000 nodes with text per second. With widgets it is not that possible. Sqlite is not a solution because any Find/Replace cause loading of all nodes with the same issue.

Anyway, I see a really simple solution (even for xml):

1. load all nodes with textbuffer without styles and widgets. This allows to use find/replace easily
2. when node is selected, apply style and widgets.

[txe]

BTW, do we need spellchecker and system tray icon in the first release?

[giuspen]

Interesting test @txe! Disappointing for me but your workaround to load without formatting (and clear the formatting when you leave the node?) May be a solution... We need the embedded widgets as well in the search though but those can also be not into the text buffer.
I'm curious why do you use VMware over virtual box? I've always been using the second for personal free use.
About the spell check and system tray since we're not too far it may be worth to try and include it so the features are really complete (except for the imports).
About the imports I was thinking that at least initially we could use a hybrid c++ and python3 to reuse the existing code and have a quick solution. The python 3 code could generate a cherrytree document that then the C++ reads

[giuspen]

I forgot to mention that the windows standalone package and installer are pretty much there but there is an issue I've spotted (with the file paths, of course, pain in the neck) but I'll sort it out shortly

[txe]

There is no much reason to clear the formatting after leaving the node, the job is already done. And maybe anchor widgets have too much layers of widgets (like image inside frame inside EventBox)

I've been using vmware for the last 10 years, so it's hard to recall, maybe initially vmware was more stable than virtual box (my former coworker says Kompas-3D (CAD), we worked on, crashed in virtual box). And I have a couple of old images, I use from time to time, no much sense to convert them.

The problem with imports is that I don't have files to test these imports. otherwise it's not hard to port the code, it's not much there.

[giuspen]

Well if you think there's no much there in the imports you have no excuse not to port it before the first release 🤣 I reckon it's thousands of lines. We can fix the issues when users will try unsuccessfully the imports and will send over their data to reproduce

[txe]

@giuspen , can you add support for trevis ci? it's super easy, I left the link how to do in #738

[giuspen]

Sure @txe I'll look at that this evening

[txe]

Thank you!

[giuspen]

Thanks to you my friend, should be all done.

[giuspen]

Just a note on the filepaths. If you notice Glib is never using the Glib::ustring for the filepaths. See for example https://developer.gnome.org/glibmm/stable/group__MiscUtils.html - I've been doing the same but in the end you are the best person to verify issues doing that since you do use a non ascii character set. Was the case that you found problems using plain std::string? I'm talking about this because having some issues with filepaths on windows even though I have not yet identified the cause. An example is ct_storage_control.h and Glib::ustring get_file_name() { return _file_path.empty() ? "" : Glib::path_get_basename(_file_path); } while the API is std::string Glib::path_get_basename(const std::string& filename)

[txe]

https://developer.gnome.org/gtkmm-tutorial/stable/sec-basics-ustring.html.en

There is a difference when you have to operate with symbols in a string like find or replace. std::string doesn't know about multibyte unicode symbols and works like it is one byte strings. As long as you just copy and compare data it should be fine. When you give std::string to Glib, it is also fine because they know it's unicode.

I suppose , they return std::string because their C analog also return char*, like raw data.

You may have problem in windows, because they use utf-16 there sometimes.

[txe]

Glib::ustring is just wrapper over std::string and has some functions to operate symbols instead bytes.

[txe]

Looks like a good idea to try it myself on windows.

[giuspen]

No rush for you to look at windows, surely eventually your use case is important, but for now I can start fixing the issues I see with my UK English system.

[txe]

@giuspen thanks for adding travis! it's already run into compilation issues :)
I tried a bit cherrytree on windows, the default theme looks surprisingly good.

The bug with the disabled toolbar affected both OS, I'll fix it later.
I started working on gspell, do we need a compiler option to build cherrytree without gspell or whatever?

[txe]

And I'm not sure, do need some c++ wrapper for gspell (it's c library)? there is no gtkmm for it.

[giuspen]

@txe It was a good idea this of Travis, thanks for pushing for it! 😉
I knew you couldn't resist and try windows 😄 I'm happy that you have a feel of the windows version too though! I also was surprised at how good the native windows theme is!
I would not worry about providing macros to build without gspell, it's not such huge library.
About making a wrapper just see how it is, if you find efficient to add a class that exposes gspell or not.

[txe]

Hi @giuspen , what do you think about possibility to cut corners and release a version for linux?
We can postpone Windows installer and imports for a week or two, and focus on issues that prevent a linux release. I sure that will make users happy and gives us some boost and confidence for finishing other things properly.

[txe]

Or we can make a test version package and ask some users to test it

[giuspen]

Hi @txe I agree in starting to package a Linux version only for testers (no rush for the windows package) but I would not advise this to users that want a stable version, for that there is the snap package and I was considering trying to set up a flatpak myself.

[txe]

Ok, how do we start? So far, I have the following issues to fix in my list:

• changing node type
• autosave
• justification
• 7zip lib (pass params as values, not just a string, plus clean up output in console)
• vacuum
• remove global vars (e.g. Find/Replace)

[giuspen]

I would rather have your list completed, then I would do some testing myself only and report the issues/help fixing and then go with an alpha. My concern is to unnecessarily start to get bombed with known issues. In the meantime I will add packaging scripts. If you really want to start before the end of your list let me at least try and setup the proper packaging for Ubuntu this weekend

[txe]

Good, sounds like a plan. I'm going to finish my list (take no more than 2 days) and will wait for your feedback.

[giuspen]

You're a star!

[txe]

You too! To keep the project without giving up, it is something :)

[SadE54]

Nice to see such a winner team for the upcoming c++ release 👍 I can't wait
to test it! 😊

Le ven. 24 avr. 2020 à 19:20, txe [e-mail] a écrit :

You too! To keep the project without giving up, it is something :)

—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/giuspen/cherrytree/issues/444#issuecomment-619143020,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AAV62MHMBPTK3NNA3XEJLDTROHC63ANCNFSM4GOEKWDQ
.

[txe]

@giuspen , I have fine icons on the toolbar/menu and on the node tree, but strange ones which are loaded by _pGtkIconTheme.
about dialog with app icon
changing highlighting
Do you have the same issue?

[txe]

It looks like Gtk theme has cherrytree icon and it has more priority than cherrytree.svg

[txe]

I remember you renamed some icons, I guess, because of that issue. Maybe we need to use prefix ct_ to easily distinguish them?

[giuspen]

@txe I think it is a good idea to prefix our icon stock, is was indeed a similar issue

[giuspen]

Since I'm looking at the packaging and one part of it is the command line manual, have you already looked at supporting the command line arguments of the Pygtk2 version? If not, do you want me to look at that?

[txe]

Ok, you can take it)

[giuspen]

Hi @txe about our discussion the other day about std::string on the filepath, if you see https://developer.gnome.org/glibmm/unstable/group__glibmmEnums.html#gabd6606988f30170f2e53b2b121f2b219 for the command line argument types you spot among the supported types:

BOOL | bool
STRING | Glib::ustring.
INT | gint32
FILENAME | std::string
STRING_VECTOR | std::vector<Glib::ustring>
FILENAME_VECTOR | std::vector<std::string>
DOUBLE | double
INT64 | gint64

very explicitly it is using the plain std::string for the filepaths. Some years ago, before starting the cherrytree porting, I remember that I've been fighting because the Glib::ustring was trying to do unnecessarily encoding/decoding crashing my code

[txe]

Ok, lets use std::string for file paths, I checked a couple of other projects, they do the same, so , I guess, it's working fine

[txe]

Hi @giuspen ,
I saw the issue with linker on travis, but for some reason I couldn't reproduce it on arch and ubuntu 20, I almost installed ubuntu 18, but found out you fixed it :)

I fixed my list of issues, so am going to wait for your feedback.

BTW, you need to try QtCreator with cmake/ninja!
QtCreator works with cmake right out of the box, no issues, no hustle, And making cmake to build the project by ninja instead of makefiles speeds up building process by times.

[giuspen]

Hi @txe, could you continue the work I started with the command line if you have some time?
If not no worries there's no huge rush the snap package works very well.
I'm not sure I will be able to get back on it before the next weekend while I will try and test the multilanguage if I find some time and start testing your work.

Also visual studio code has cmake support and it works very good on linux. I have some concerns about cmake support for the internationalisation but we'll find out very soon.

I wasn't sure you saw Travis failure :) I would like to add windows/msys2 as well to Travis I read it should be possible

[txe]

Sure, I'll take on the command line.
What do you mean by 'the internationalisation support'? generating pot files?

[giuspen]

No the pot is generated via a shell command, I was thinking about the .mo files but those are accessed in runtime it shouldn't really be an issue, I'll verify soon. Note what I've done is parsing the supported switches
(try cherrytree --help) and only printing if you pass a node name (-n nodename), nothing else unfortunately. Note you will find a callback where I'm returning -1 that strangely is to say all good and keep going, see the Gio::Application reference the Gtk::Application is inheriting from

[txe]

BTW, when I started snap store, cherrytree was on the first row Editor's Picks. Does it mean that the program is that popular, or Canonical is just spying on me? :)

[giuspen]

Morning @txe, while I confirm we can go public without the imports from other note taking apps, I think we should support the first 5 menu items which are:

• cherrytree file
• plain text file + folder
• HTML file + folder
  Just in case you are getting bored 😄

[kettenfett]

BTW, when I started snap store, cherrytree was on the first row Editor's Picks. Does it mean that the program is that popular, or Canonical is just spying on me? :)

CherryTree is right on the store front of Snap's website (https://snapcraft.io/store) right now. I have never visited that page before. I'm a windows user, so there you go...

[txe]

Hi @giuspen ,

I want to add support for reading .css and .js files from /usr/share/cherrytree.
For linux (I'll skip windows for now), there is g_get_system_data_dirs (XDG_DATA_DIRS),

What else directories do you want me to check?

The structure of dirs wil be the same as for pygtk?

[giuspen]

Hi @txe if you see Makefile.am you can find all the directories that are going to be installed and what's inside there. You should not use /use/share though but the macro DATADIR exposed from the autotools because that may vary (still see Makefile.am). You can also look at https://github.com/giuspen/cherrytree/blob/master/future/data/manual-install.txt more human readable.

[txe]

Ok

[giuspen]

Actually I'm realising something is still missing, for sure the export to html stuff currently inside of the glade folder, that is also to be added. Note only two syntaxes are out of the new gtksourceview4

[giuspen]

In fact I misunderstood you in the first place the html stuff was exactly what you were interested in, I'm sorry it was a hard day. Those files are to be duplicated into future/data and Makefile.am modified to install them. Note that we also want to support running from the sources folder with those files local and not on the filesystem same as we were doing in Pygtk2

[giuspen]

I can do this just not sure this evening I can find the time

[txe]

Don't worry, from makefile.am, it's clear how to add files, i do it myself.
Do you need help with disabling cpputest?

[txe]

resulted folder will be /usr/share/cherrytree/data , not glade?

[giuspen]

Correct! but we don't want to copy the whole content of data, only cherry pick few files, others go elsewhere. About disabling cpputest I know in configure.ac it is possible to add a command line switch that can be used to cut out cpputest and then not building the tests in Makefile.am but I still would like the default to build the tests and require cpputest. If you want to have a go please go for it cheers

[txe]

Ok, I got it, so then I take a look

[txe]

what version of gettext do you use? mine is 0.20.2
and now I cannot use ./build.sh because of

** error: gettext infrastructure mismatch: using a Makefile.in.in from gettext version 0.19 but the autoconf macros are from gettext version 0.20

[txe]

ok, i spent two hours just trying to build with ./build.sh
and get

error: gettext infrastructure mismatch: using a Makefile.in.in from gettext version 0.19 but the autoconf macros are from gettext version 0.20

20 minutes later something new: autogen.sh doesn't see '--disable-cpputest' in build.sh point blank, though it's ok in console
and finally after some hacking

error: The pkg-config script could not be found or is too old.

Ok, git clean up, and then again from the very beginning.
I'm afraid, I can do nothing about, some code for you to disable cpputest, by default it is turn on

AC_ARG_ENABLE([cpputest], [AC_HELP_STRING([--disable-cpputest], [disable cpputest])],
   [use_cpputest=${enableval}], [use_cpputest=yes])

AM_CONDITIONAL([INCLUDE_CPPUTEST], [test "x${use_cpputest}" = xyes])

if test "x${use_cpputest}" = xyes; then
      PKG_CHECK_MODULES([cpputest], [cpputest])
fi

PKG_CHECK_MODULES([CHERRYTREE], [gtkmm-3.0 gtksourceviewmm-3.0 glibmm-2.4 pangomm-1.4 libxml++-2.6 sqlite3 gspell-1])
````
and in makefile

if INCLUDE_CPPUTEST

endif
```

[txe]

I thinks I can give autoconf a good third place in my black list of hating :)

[giuspen]

I'm sorry @txe I will give it a go with this command line switch, don't get mad on it :)
I may post a question on stack overflow if cannot find a solution, in the past I've been struggling myself with the autotools.

[txe]

It's not like i can't figure out how it works. I'm just really disappointed that instead spending 15 min to do a task, I had to fight for 2 hours against autoconf for no reason.
Anyway, I have to fix it otherwise it would be hard for us to coordinate work. Any chance to move on cmake?

[giuspen]

We can try and move to Cmake for good, that's fine from my end, I would just keep the autotools synchronised wherever possible for now but I can take care of it

[giuspen]

Maybe we can provide two scripts at the root, build_autotools.sh and build_cmake.sh where both build and run the unit tests. You can then change the Travis CI to point to build_cmake.sh

[txe]

I think, it's better to leave it as is, if it helps get a ready package as soon as possible.

For development, cmake already works fine for me, I don't need it in travis (though it looks ugly that I broke it so much). I just don't like that autotools is a fragile point in building packages.

If it's not too personal, why do you want to have autotools as additional?

[giuspen]

No it's not that I'm particularly fond of it! :) You convinced me let's try and replace totally and let's avoid duplications, don't worry for the packaging I'm almost there anyway!

[txe]

good! such a good news for today!

[txe]

About packaging, does it depend on autoconf? Do I need to add something to cmake, like to support installing and support creating packages?

[giuspen]

Yes if you can port the installation information I added to Makefile.am, then that file can go together with configure.ac and autogen.sh

[giuspen]

Currently packaging depends from autotools but I'll change the debian/rules to point to cmake instead

[txe]

ok

[txe]

Hi @giuspen any progress?
In my case, I may try to make cherrytree C++ working on macos

[giuspen]

Hi @txe not too much from my side except I've been adjusting some errors in the packaging and delivered the flatpak to flathub with the help of the maintainers. I plan to complete the Gtkmm3 Debian packaging adding the internationalisation scripts and I would also like to add the windows/msys2 build to Travis in the next couple of days. It would be definitely great if you can have a look at building for Mac (https://github.com/giuspen/cherrytree/issues/762) I know nothing about Mac and have never owned one but many users have asked for it since long time

[txe]

Hi @giuspen ,
libxml++2.6 cannot be used in macos. Anyway this package looks really old, and I'm going to move from libxml++2.6 to libxml++3. I checked this new package exists on msys2, so it shouldn't be a problem.

[giuspen]

That's fine @txe !

[txe]

I need a little help, I cannot find libxml++ version 3 on ubuntu (though macos, arch, win have it)
Can you find it, so I'll be able to fix readme?

[giuspen]

Doesn't show up on synaptic, let me see if anybody talks about it

[giuspen]

Well it's not a big deal if it works fine I can make a debian package out of it and host in launchpad, I'm just curious why it was not packaged 🤔

[txe]

The first release was on 2 Feb 2016, does anyone use it?
And I wonder if fedora has the same issue?

[txe]

I checked a couple of C++ project like jucipp and inkscape, they don't bother with libxml++ and just use libxml. Maybe we need to do the same?

[giuspen]

I just built 3.2 on Ubuntu flawlessly, depending on libxml and glibmm 2.5

[giuspen]

We could even embed it as we made for fmt and p7zip in case... If the API is not too different I would try and use 3.2 but up to you

[txe]

API is different, but I fixed it.

[txe]

About that launchpad, now everyone can use it? or it needs some time?

[giuspen]

giuspen launchpad is up and running I just have to do some packaging files for libxml++3.2 but then the deployment is totally under my control

[txe]

Well, I see
But then cherrytree cannot become an official package because it will depends on an unofficial one.
Looks like better to embed it.

[giuspen]

Ok let's try and embed it

[giuspen]

When the new cherrytree will make it into Debian and Ubuntu anyway they will package libxml++3.2 unless a strong reason not to do it

[giuspen]

libxml++3.2 is even a gnome package

[txe]

right, but for some reason it lives separately from gnome, somewhere on github.

[giuspen]

http://ftp.gnome.org/pub/GNOME/sources/libxml++/

[txe]

Then maybe ping someone and ask why it's not in debian?

[giuspen]

@hellsworth do you know why libxml++3.2 is out of Ubuntu?

[txe]

I almost wrote a wrapper to switch between 2.6 and 3.0, but at the last moment I figure out how to compile 2.6 on macos, so for now we have working cherrytree on macos.
Changes will be committed later, I need to add app icon and check make install

[giuspen]

Cool! You could try and add Mac OS build to Travis CI then same as we can add Windows Msys2 in future https://docs.travis-ci.com/user/multi-os/ https://docs.travis-ci.com/user/reference/windows/

[txe]

Hi @giuspen ,
I mentioned about installing cherrytree C++ on macos in #762, should we do it also on other macos threads, or we wait a bit?

[giuspen]

Hi @txe I think it's the best moment now to spot issues about Mac installation so the more users are willing try the merrier! 😉 It would be great if you try and add the auto build on Travis as well, I soon want to do the same for Windows

[txe]

BTW, I was going through inkscape source and saw that they changed their version after moving to GTK3.
0.xx is GTK2
1.xx is GTK3
Maybe we need the same thing, so users will be less confused in some cases?

[giuspen]

We could start with 0.99.x as the first stabilisation phase and when we believe it is mature we issue 1.0.0, what do you think?

[txe]

Sounds good for me!

[giuspen]

Hi @txe today I was reading https://distrowatch.com/weekly-mobile.php?issue=20200504#ubuntu and when I reached the Software Manager section, the screenshot made me laugh! (Thanks again @hellsworth 😉)

[txe]

Yeah, also cherrytree was mentioned in comments a couple of time)

[hellsworth]

@hellsworth do you know why libxml++3.2 is out of Ubuntu?

@txe Well libxml++ in ubuntu (20.04) is 2.40 because that's the version it is in debian and ubuntu pulls this from debian.

From a snap perspective, we could build whatever version of libxml++ you'd like though and not use the deb dependencies.

[hellsworth]

Also, in case this could help us circumvent older dependency versions in ubuntu/debian deb archives... I'd like to mention that I've created gnome-3-34 buildtime and runtime snaps that build glibmm v2.62.0 (and then it's dependent pangomm, cairomm, mm-common, atkmm, gtkmm) (this was needed to snap inkscape). so as far as the cherrytree snap is considered, we should be able to use the latest version of these packages that is still for gtk3. I'll see if I can find some time to pick up snapping the futures folder (I looked at this a few months ago but it landed on the backburner).

Maybe a silly question.. idk.. but have you all considered going right to gtk4? I believe it's going to be released soonish (at least that's the feeling i get from the devs).

[txe]

Hi, I thought about gtk4, but there is no gtkmm 4.0 on windows (msys2) yet, so moving to it would be inconvenient.
As for libml++ version, turns out using 2.6 on mac is not an issue, so we keep using it. And because libxml++ is just a wrapper, there is no difference between 2.6 and 3.2 in a sense of functionality.

[hellsworth]

ok so the minimum version you need is libxml++ 2.6?

[txe]

Any 2.xx, I just don't think you can find any older than 2.6.

[hellsworth]

Ah yes.. for some reason I was thinking 2.6 = 2.60 > 2.40 ... my bad :) So what ubuntu/debian has is fine.

[metal450]

I see that the gtkmm3 version is officially on flathub :) Is there a Windows build available to start testing - or that's still pending?

[giuspen]

No it's the Pygtk2 version on flathub, same also on snap. For the Gtkmm3 windows (dev) version I'll try this weekend but beware, expect crashes

[metal450]

Ahh, misunderstanding then - thanks. I've been watching the commits & just assuming that they were all related to the new version :)

Crashes are fine. I'm mainly waiting for the new version because (I think) txe implemented html-paste to handle text sizes, which I need for being able to migrate over from OneNote. Then once my data is in CherryTree's xml or mysql, I should be able to easily jump back & forth between pygtk2 & gtkmm3 to test. It's actually just getting my data over w/o losing text size for which I need gtkmm3 :)

[ForeverRainbow]

conan might be an alternative to embedding the dependencies and rebuilding from source every time. The conan centre has a package for fmt and I _think_ p7zip, it also has libxml2 but I do not know about one for libxml++.

Also as second thing, spdlog might be useful as a logger

[giuspen]

@ForeverRainbow we are to use https://developer.gnome.org/glib/stable/glib-Message-Logging.html for logging, soon I will connect a handler that will write to file.
@metal450 I've been having some issues with the windows build and not enough time to sort it out yet, msys2 has broken the cmake based build recently and I am to find a solution, probably this weekend will be the right one, then I'll release some packages of the gtkmm3 version.

[metal450]

Great, thanks for following up - fingers crossed! :)

[ForeverRainbow]

I have a working windows build for development, I use ninja and the cmake package for msys2 (The readme specifies the mingw version of cmake pacman -S cmake vs pacman -S mingw-w64-x86_64-cmake). In order to install cpputest I skipped the autogen.sh step since it kept complaining about autotools issues. Apart from that I just followed the readme in /future.

One thing I did notice was I _had_ to specify -GNinja to cmake, otherwise it found msvc and tried to generate files for it, that may be specific to my system though

[giuspen]

Didn't know about ninja, thanks @ForeverRainbow for sharing your working setup!

[giuspen]

@metal450 please find:
https://www.giuspen.com/software/cherrytree_0.99.0_win64_portable.7z (look for mingw64/bin/cherrytree.exe)
https://www.giuspen.com/software/cherrytree_0.99.0-1_amd64.deb

[metal450]

Awesome, thank you!!

For reporting issues, should I just follow up here, or create a separate new thread?

Off the bat, it seems like HTML paste isn't quite working yet. Here's a screenshot: https://lnk2.tk/snaps/2020-05-24_10.52.36.png

The source text is on the left, and you can see the result of pasting into the old app, & the new gtkmm3. Pasting into gtkmm3 results in that "header text" at the top, & many extra newlines throughout (including some mid-line splits).

[giuspen]

Thanks for testing @metal450 please create a GitHub issue per bug an we'll sort them!

[metal450]

836 :)

[ForeverRainbow]

Does the Zim parser need to be able to match the PascalCase links between pages (so inter-page links -> inter-node links)? I ask because I think in order to do it I think I will need to introduce regex to the parsing logic

[giuspen]

@ForeverRainbow I'm not sure if the links are case sensitive but probably are, better if you create a simple zim document to use as a test and ideally you verify your work with unit tests

[ForeverRainbow]

@giuspen Yeah I am testing it using a Zim notebook, I will add it as a unit test too - What I mean is that Zim has got inter-page linking automatically when you type a PascalCase word it interprets it as a link to another page (which is created when you click it if it doesnt exist), the obviouse thing would be to transform these links into node links, the problem being that there would have to be some kind of second parse where the links were set properly (since they operate on node id's which are unknown until _after_ everything has been imported)

[ForeverRainbow]

How should the unit tests operate as well? Just make sure it doesn't crash? Spit out XML to be read by humans? Both?

[giuspen]

Oh ok, there is a second step with the links to nodes if you have a look at the Pygtk2 implementation, for most of the imports, should be in Zim import too. I should have implemented also recognition of the camel case to become a link to node if a node with that name was found but not 100% sure. I forgot to mention the other day that import from a cherrytree document is very important, more than others but if you prefer to keep going with zim no problem. The unit tests should ideally generate in memory a (cherry) tree where you can then test various assertions like existence of a node, hierarchy of a node, some content that you expect to be found in a node...

[ForeverRainbow]

@giuspen Importing from a CherryTree document is implemented #825, are there features missing from it?

[ForeverRainbow]

Ok turns out I do not need to match PascalCase (phew), Zim does enclose its internal links in [[ ]] so they just need fixing with the correct IDs

[giuspen]

@ForeverRainbow I lost track, though import from cherrytree is not done yet :) It would be great to add few unit tests there too, there are two sample cherrytree documents already versioned that I've been using to test Pygtk2 Vs Gtkmm3 save for SQLite and XML

[ForeverRainbow]

With regards to unit tests, I am unsure how to implement them since the whole CtActions and importing is quite closely tied to CtMainWin, apart from that as well all the import methods in CtActions which allow you to directly specify a path are private (as they should be)

[txe]

Yeah, it doesn't look easy. Can you at least add test data you used? it will be helpful lately.
I think it will be easier to change dependencies when more different imports are in project.

[ForeverRainbow]

@giuspen Looking at glib message logging it looks like a C logging API, I assume it will be wrapped but I feel like spdlog would be an easier to use and setup choice. It has built-in support for things such as rotating file loggers, multiple outputs (e.g log to the terminal and a file at the same time), thread safe loggers, etc. To send a debug log to the default sink (which you can change) it is simply spdlog::debug("My msg with fmt formatting {}", something) (it also has built-in support for fmt). Its also very fast (take a look at the benchmarks in the readme)

It is also header-only so while I realise its an extra dependency it _can_ just be embedded or it has packages for basically every distro package manager

[giuspen]

@ForeverRainbow what I had in mind was to make the logging configurable in config.cfg and by default off. For this reason rotation is not very important. I would stick with g_log but @txe can have his say as well

[ForeverRainbow]

Fair enough, I am fine using g_log if thats the option, spdlog does have runtime configuration logging levels though, so its a simple spdlog::set_level(spdlog::level::off) to turn off logging (and change log levels with different spdlog::level's) also configurable on a per-sink basis

[txe]

Hi, actually, I like spdlog much better.
g_log looks really low level, and examples I saw always are full macros and writting callbacks hacks.
As I see, spdlog covers issues I usually have with loggers. And rotating log is quite important, there are situations when log is turn on constantly and keeping prev logs is necessary. E.g. on windows, when the program now don't have output in console, so users will have to turn on log for some reasons.

[giuspen]

Fair enough, we can use spdlog then 🙂

[ForeverRainbow]

So I am wondering about using boost in cherrytree, in particular the tokenizer library. conan has a boost package which makes it trivial to use if that helps.

The tokenizer library would make writing parsing code for the importers far easier and should also provide a way to implement a csv parser with very little effort.

[txe]

Yeah, boost supports a lot of things but compilation time beyond good and evil, and the current compilation time is barely bearable.

[ForeverRainbow]

@txe I agree compilation time is pretty awful right now and boost would increase that. It may be reduced by only keeping it in source files and keeping the headers clean but yeah. imo the benefits outweigh the extra dependency and compile times but you have probably have a better idea of than me

Another thing: compiling cherrytree from clean (or just changing something like CtActions or CtConst) uses ~3gb of ram for me which is kinda insane (it fluctuates during compilation obv but goes up to ~3gb). I don't know if this is a gtk thing, an issue with my setup, or because of the monolithic header style but is this an issue you've faced as well?

[ForeverRainbow]

Also if we are going with spdlog should I embed it and try and find and convert all the debug/error messages to spdlog?

[txe]

I had to kick boost out every time I used it in work in spite of its features. I see it will give a lot of frustration here too.

About memory, compiling also eats a lot of it, I suppose, because of ninja running a lot of jobs simultaneously. I don't know how to reduce it or speed up the process ((

About logging, I also looked at easylogginpp, what do you think about it?

[ForeverRainbow]

It looks ok, but it doesn't seem to have support for fmt like spdlog does (which I have found to be invaluable) although it does have prinf support. It also seems pretty heavily macro based. I have used spdlog in the past so obv I am biased but I very much like spdlog's C++ style with extending custom sinks and such. It does seem to have support for things natively like logging boost components and custom types but spldog just seems cleaner and simpler tbh.

[txe]

I agree, I just wonder why easylogginpp is that popular.

How you are going to add spdlog? I thinks it's better to add source in git and use it as compiled static lib (to improve compilation time)

The log will output in console and a file. And there to save log file?

[ForeverRainbow]

Well for now I can just add spdlog into the src and yes compile it. I can set up a logger to log to terminal and file if you want but that can be done at any time after the log calls are in place. I can add something to decide on the path to the log files for *nix and windows and put the log file there?

[txe]

Right, the log file is not necessary now and can be skipped

[ForeverRainbow]

I think trying to implement #878 would greatly increase the usability of cherrytree but it would mean a pretty massive feature gap between the C++ version and the python version and I don't know what the status of trying to keep similar features. I am happy to try and implement it if it is fine to do so.

Currently I am thinking that the best way to implement it would be to add multiple editor window support (with the second one read-only) and have the rendered output from pandoc displayed there (which is how markdown and such is usually done)

[txe]

As I understand, the last feature requests can be broken in four parts:

1. to use markdown in the view and convert them into style in place
2. to add import/export markdown
3. to use pandoc to import/export different formats as markdown, latex and others
4. to add a preview for markdown. So users keep notes in markdown, and the preview shows results.

As I understand first one is done? Second can be skip if third point is implemented.

Fourth one is quite challenging and required a lot of thinking and work because the behavior is somewhat different. For example, users can insert images in ordinary notes, so how to keep them in markdown notes? it will be url to something? It will also requires changing in commands related to styles (inserting tag symbols instead of changing a style). And a lot of others.

So, probably, it would be easier to finish third point because it's understandable and straightforward, and then to think how to do the last one.

[ForeverRainbow]

I agree 4 is going to be complex. I was thinking it could be implemented as the user hits a key (or an option) to open the preview pane and then the text in the editor is run through pandoc and displayed, so more complex markdown (like tables, images, etc) would just be left as markdown and only viewed formatted in the preview pane (There could also be an option to format on save, which copies the preview text to the editor text).

1 is _sort of_ implemented, it supports basic formatting such as italics and bold with __ and ** respectively as well as links. Its support for headers and such is somewhat limited though but that is an issue with how possible markdown text detection and replacement is done currently rather than the parser. * also conflicts with the built-in syntax for a list but its not an issue since its the same in markdown.

2 is 50% done, importing markdown works and it supports most (all?) of the basic markdown, so nothing like tables but links, bold, italic, strikethrough, etc are supported. Exporting to markdown should not be too hard since I already have a symbol table for md -> ct xml.

For 3 I think actually the best bet would be to use html as the kind interface language between cherrytree and pandoc, otherwise a custom writer will be needed which could be done but I ran into some problems with and will likely be a pain.

Images and widgets in the text are going to be one of the main problems with 4 I think, because rebuilding widgets and serialising images every few seconds (or whatever the refresh time) is not really an option :p.

[txe]

@giuspen, do you have any issues with building the project on win?
Before, I couldn't build because build -j4 run out of memory, so I had to use just 'make'. But now even that doesn't help, cc1plus.exe is out of memory every time.

[giuspen]

Hi @txe it builds fine for me, I have 16GB of memory on my dual boot laptop (no VM) with msys2 running on Windows 10

[txe]

Same deal with memory and setup. Eventually, I could build it too, but for some reason it is out of memory from time to time. Thought maybe you use some trick, but ok, never mind.

[giuspen]

Hi @txe no tricks at all, my installation is what documented in the README, just have not tried from scratch after we migrated to Cmake and removed all the autotools packages. I always build with "./build.sh" or "./build.sh Debug" and have a couple of commands in ~/.bashrc documented in the README.

[ForeverRainbow]

Since msys2/mingw is being used for windows, is there any issue with using posix functions such as popen? (i.e is MSVC ever going to be targeted)

[txe]

I don't think it is a big issue, code can be changed.

[txe]

Just to share,
turns out there a way to build Gtk3 by MSVC (link), so maybe one day we can build cherrytree by MSVC. It maybe a good idea because msvc produces much better and faster code than msys2/gcc. On the other hand, there is no ready binaries, so we will have to build them ourselves every time.

[ForeverRainbow]

Well I have got a working import from pandoc (its just for testing rn, no ui bits or anything) but it uses pipe and fork which is going to be an issue for windows (I don't think msys supports fork but I might be wrong, cygwin probably does). It is probably possible to duplicate the functionality of my code on windows but I have no idea how to do that :p.

[txe]

Ok, then at least that code should be disabled on win by #ifndef _WIN32

[ForeverRainbow]

Ok so I am working on the windows version of this now, it actually seems pretty easy (fingers crossed it doesnt do something weird) but I am running into the issue of mingw being _incredibly slow_ compiling window.h, this may be an issue local to my machine. Is there any reason that MSVC couldn't work with current depedencies? Because if not I think I will try and set it up and add it to the readme

[txe]

MSVC cannot link Gtk libraries build by msys2/mingw as far as I know. You will have to built gtk yourself using gvsbuild and MSVC.

[ForeverRainbow]

I really think conan would be a good idea here. cpputest, spdlog, fmt, 7zip and Gtk3+ (though without windows support for now) all have packages and would make cross-platform building a hell of a lot easier (the install instructions for all platforms would basically be:

1. Install conan through your package manager or the website
2. Install Gtk3+ packages if your on mingw
3. mkdir build && cd build && conan install ../future -b missing
4. cmake ../future
5. build it

For packages in the conan center (all except Gtk3+ and cpputest I think) there are prebuild binaries for popular toolchains such as gcc7 (ubuntu 18 toolchain) so fresh build times would also improve

[ForeverRainbow]

@txe Ah I see, I will try and get it working later using that link you posted if mingw keeps being slow

[txe]

if conan uses mingw to build cherrytree, then it will be the same slow building, no difference.

[ForeverRainbow]

But it will make getting a development environment running far easier and quicker. It also will mean that dependencies don't need to be bundled directly in cherrytree's source. Which will make commit history cleaner and allow updates for dependencies to be seperate.

conan is a bit like npm but more hands off

[txe]

Still i don't see how canon has any advantage here:

• it's one more dependency to install without removing any other dependency. Also it means additional time to figure out its own problems.
• I don't think canon that popular. if someone wants to play with code, installing canon can be a drawback. And additional job for build/package maintainers.
• fresh build happens hardly ever and not important. As I see most build time spend on building cherrytree code, not on 7z or others.
• dev env setup with conan will be kind of the same. I don't see how it gets easier and faster. Current dev env setup is straightforward, simple and just copy-paste to the console.
• Making 7z, log and fmt (btw, it can be removed because log already has fmt) outer dependencies doesn't give many advantages and, in current state, these libs already work more than fine and never give any issue. To have a fixed version (by keeping them in our source) is kind of advantage.

[ForeverRainbow]

Fair enough, I was mainly thinking that it would allow adding more dependencies easier really but I can see how the current way just works and it would introduce another potential problem with set up.

[txe]

Hi guys,
I do some refactoring, please don't change any files related to imports next few days, otherwise I'd have a hard time to merge changes.

[giuspen]

Sure @txe

[ForeverRainbow]

Wondering whether I should move CtFileSystem into its own separate header as part of this refactor, it would make the seperation between utils cleaner but also its another header file

[txe]

@giuspen , it's your call

[giuspen]

@ForeverRainbow I'm happy that you make a separate header and source file, please also move the unit tests (if existing, cannot remember) to another file named after filesystem

[txe]

@giuspen
It looks like calling Glib::locale_from_utf8 used in writing in sqlite causes issues on win32 machine when text has non-latin symbols. It works ok on linux because locale is usually utf8 too. Another strange thing is that during reading from sqlite there is no conversion. I think we should keep utf8 in db and remove Glib::locale_from_utf8?

[giuspen]

Hi @txe absolutely, there is a test.ctb document I initially used to test read/write of Cyrillic characters, the best thing would be to write some unit tests using those (that was my plan but I'm really struggling ☹️) so that we can be sure it is working on multiple platforms and not having it stop working in future pull requests since absolutely critical. I was planning to use your interface for command line documents export to load a document without the graphical window in the way an then analyse/unit test the tree using first the tree info that I recently implemented and subsequently implement something more to unit test for instance the characters in a node from offset A to B checking also the markup tags.

[giuspen]

Maybe I will postpone the 0.99.2 release until we have that test harness in place while I'll try this evening to issue a new minor Pygtk2 as for plans.

[txe]

@giuspen , I've fixed bug in html export on win32 I mentioned yesterday
@ForeverRainbow , btw, html export looks really nice now, thanks!

[giuspen]

@txe would you issue 0.99.2 or wait for my test harness?

[txe]

I clicked here and there, tested paths and nodes with Cyrillic symbols on win32. I think it gets better than 0.991 in terms of encoding

[giuspen]

Fair enough, I'll try and release both then as soon as get some time

[ForeverRainbow]

On a possibly unrelated note, I was thinking about the documentation of the code in C++ cherrytree, or rather the lack of it.

Personally, as someone still not completely familiar with the code base, and who can't remember the details of functions I write after a few months, it would be really great if there was some kind of "documentation standard" or advice to document new functions and possibly old ones as well.

Being able to generate a doxygen of the codebase for quick reference for example, would imho be very helpful. Doesn't need paragraphs of explanation but just a quick "this does this, don't do this if you need to change something" for example and a description of what each class is for would be invaluable.

I admit I am guilty of not documenting the code I've contributed all that well and I will add documentation to "my" classes (e.g the markdown parser). I can add basic documentation for most classes I think, some I am not sure entirely what they do (good learning opportunity xD) but adding documentation after the fact isn't really sustainable imho

[giuspen]

@ForeverRainbow so far @txe has been able to do multiple big reworks also because we've not started with all the documentation.
I'm not sure the reworks are done yet, writing all the doxygen overhead is more for a more mature code base... What do you think @txe?
I would go with a more generic document separated from the code where there are notes about the coding standards and the main classes.
BTW I'm happy with the coding standard dictated by @txe except for the non using of curly braces for single statements, IMO we should always use it to wrap blocks in if/else/for/while and so on...

[txe]

Well, having the documentation seems to good, but I don't think it's practical or necessary.

• It's a lot of effort: you have to think what to write to be descriptive (and you don't want just to repeat function name) even for one line documentation per function/class. It looks simple, but can be tiring. You will have to update docs otherwise it will lose its use.
• it reduces density of code, instead of reading code you have to scroll up and down skipping blocks of comments: it takes time and makes hard to focus (at least for me)

I believe a good naming and code encapsulating work much better than documentation. The name gives the purpose, public members - what to use, if code works wrongly then underlying functions are done wrongly and should be redesigned. If there are doubts how to use, then there is search in the code. Sparsely adding doxygen docs only breaks code style and irritates eyes.

For example, we can take fs_filesystem.h. If everyone needs to know whats where, do they go to open doxygen docs? Nope, it's easier to open a file in IDE, by clicking on function somewhere or by fuzzy search. Then again, if every function would have 5 lines of doxygen comments, I would have to scroll 7 or 10 screens just to look through, now it's 2. And readability can be even more improved by grouping functions by purpose: operations over files, operations with paths and etc. I don't know how doxygen can beat it.

And honestly, we won't have many newbies, and they need to know C++, gtk, big picture of elements in project. I have doubts they are going to read doxygen docs.

I wouldn't against the documentation in some situations, but here it's just a waste of time, it never will be finished and nobody will use it.

PS: wrote so much to train my skills :)

[txe]

@giuspen , I don't use curly braces for single statements simply because it adds lines, as a result less code on the screen.
I really need to get 4k monitor :)

[giuspen]

@txe it does indeed add one line at the bottom but it is safer, if you look at the code based on the indentation, you may miss two lines both equally indented after an if without a proper curly bracket and the second will be executed by mistake always. Anyway I'm good for you to use what you prefer, you are very attentive, I will just ask contributors to use the curly braces 😉

[giuspen]

0.99.2 is out packaged for Windows and Ubuntu 20.04, look at https://www.giuspen.com/cherrytree/#downl and scroll down

[ForeverRainbow]

Thats fair enough @txe, thanks for giving such a detailed response. doxygen comments can get very annoying when used on _everything_ I agree but in the longer (and sometimes shorter) term, surely it is better to document what the intended behavior of something is when its fresh in your mind instead of having to reread it later?

A lot of methods actually already have comments above them, they just need a / added to be read by doxygen. Yes maintaining documentation is overhead but so is having to remember what x does or why it exists (and possibly working out what it does if someone else wrote it), also most IDEs can be configured to fold comments by default so even long ones wouldnt take up any more than a line from a comprehension standpoint.

My perspective on this is obv very different but I feel like thats a good thing? As someone trying to learn the codebase - documentation is needed from a maintainability perspective especially with multiple people involved, right now learning what everything does and remembering it is feasible but later it likely won't be.

But if you completely disagree or think now is not the right time I respect that.

[txe]

@ForeverRainbow , can you give, for example, class/file from the project which can benefit from doxygen? (excluding CtMainWin because it's a monster)

[ForeverRainbow]

CtFileSystem/fs could benifit for example, some comments to explain that it does not follow the standard, especially that string() cannot be relied on to return platform-native paths. Some functions also are not obviouse at first glance, for instance prepare_export_folder - it prepares an export folder but for what? Why does it return a path and not void? Or external_filepath_open which is pretty self-explanatory in its name but without looking at its implementation there is no way to tell that it actually opens it in a seperate program for the user to view (rather than changing the working dir for example) or that it uses the CtConfig passed to do so.

[ForeverRainbow]

Another example is CtProcess, which in hindsight I really should have documented better. Without looking at the nitty gritty of run() and understanding how g_subprocess_new works, you wouldn't neccarsarily realise that argments _have_ to be seperate (i.e --to x must be added as --to then x and not --to x at once). Also without the documentation on CtProcessError you wouldn't know that you could catch all exceptions from CtProcess just by catching CtProcessError (which you might guess from the fact that its a member but couldnt be sure of without checking all the code).

[txe]

Ok, as I see, fs::path is just a wrapper over std::string for static type convenience and making a difference between std::string and paths. So fs::path::string just returns the "given" std::string. Paths itself are complicate aria, so if fs::path started doing some magic with paths, and it wasn't asked to do it, it'd be a nightmare.

prepare_export_folder is in wrong place, should be in export class, private.
external_filepath_open, wrong name, open_path_with_app is better.
From time to time we do clean up names due to the very reason.

about passing CtConfig* everywhere, it's bits of the fight with global objects, and passing config as argument shouldn't give any second thoughts why function needs a config. It's just kind of inconvenience, so either pass it or make it global.

Ok, I remember nothing about CtProcess, so I can open the file or write CtProcess in IDE and see arguments with fresh mind. And it's just 3 functions, it's easy understand how to use.
agree, args can be an issue, one line example can fix it. Or search of using in the code.
run doesn't return bool, so it's clear it will throw exception.

And about exceptions, in the project we are trying to avoid exceptions and any exception is a big surprise and inconvinience. The exclusion from the rule is sqlite storage because it's convenient to pass message and stop execution from bottom to top. And I don't think it's right, it can break user data.

I think making code self explanatory is better approach and comments are a part of it. The latter should show why some code is done this way and cannot be done other way, or make blocks of code clearer, etc. So I would like to avoid adding a lot of comments just to make documentation system work and then hide these comments with some critical bits by IDE.

Another approach is to add separate documents. It can be done one way or another. At least, it won't interfere with code. But I cannot imagine that I'm going to write there just about classes and what functions there.

Edit: some link from reddit https://old.reddit.com/r/cpp/comments/8lwmkb/which_tool_do_you_use_to_document_your_c_code/dzkde8e/

[ForeverRainbow]

mmm, separate documents would be hell to maintain and don't really help since you have to pull up something external to find the relevant part of the code (although for the use case of doxygen they would work I guess).

Also I agree with the linked post when it comes to libraries which only provide doxygen or similar rather than complete documentation. But this is different, the people who would use code comments in cherrytree are people contributing too it rather than just using an API (so internal documentation).

I just feel like not documenting things now is going to be a maintenance nightmare later on. But I could very well be wrong so :woman_shrugging:.

On another note. Things like

[...] in the project we are trying to avoid exceptions and any exception is a big surprise and inconvinience.

Would be really helpful to have written down somewhere, along with other guidelines for style and such :p

[txe]

Would be really helpful to have written down somewhere, along with other guidelines for style and such :p

Right, actually, it can be done now, I need only to figure out the structure of the document

[javadevelopment4]

Can i checkout the latest version ? Is the migration has been completed?

[giuspen]

@javadevelopment4 the migration is not complete but the Gtkmm3 version is solid, I'm using it daily without issues, you can use it as well being extra careful to backup your data and reporting issues if you find any. Still missing are mainly some imports from other note taking apps.

[kettenfett]

@ForeverRainbow Is Markdown supposed to work with 0.99 already? Couldn't get it to work with cherrytree_0.99.2_win64_portable.7z after activating it in preferences as per #871 and restarting.

If it's supposed to work, I will open an issue for this.

[ForeverRainbow]

@kettenfett Yes it is supposed to work but I haven't really tested it since #917 although I think it was working then. If you could open an issue for it that would be great, I'll have a look at it.

[kettenfett]

I see you already wrote a fix. I can't check up on it unfortunately, because I don't have the setup to compile for windows. I'm dependent on giuspens binaries.

edit: opened issue #957

[giuspen]

@ForeverRainbow I've been finding crashes while cherrytree attempts to remove a file (while doing a "file -- save as" to existing filepath) so I started to unit test more thoroughly the filesystem but I keep on having segmentation faults which I can hardly explain. Could you please have a look at tests_filesystem.cpp -> TEST(FileSystemGroup, remove) try removing my #if 0 #endif // 0 since you know better fs::path maybe you can help me with this

[txe]

@giuspen ,
It's because of recursion, fs::remove calls g_remove which is #define g_remove remove.

[txe]

Maybe a good idea to include a namespace into a function definition (in this case, fs in .cc file)

[txe]

Well, I think my last suggestion won't work, maybe ::g_remove is better

[giuspen]

Hi @txe good spot! If you have a moment and can sort this out with the unit test working removing the #if 0 it would be great, I'm risking a divorce here cannot access the PC for now :)

[txe]

Done

[giuspen]

Many thanks!

[txe]

About hidden options in config (just copied some stuff from source)

• these names can be changed like last used or something like that. autoSyncHighl is used strangelly and can be redone

_uKeyFile->set_string(_currentGroup, "syntax_highlighting", syntaxHighlighting);
_uKeyFile->set_string(_currentGroup, "auto_syn_highl", autoSynHighl);

• missing stuff

_uKeyFile->set_integer(_currentGroup, "embfile_max_size", embfileMaxSize);
_uKeyFile->set_boolean(_currentGroup, "triple_click_paragraph", tripleClickParagraph);
_populate_map_from_current_group(&m_custom_codexec_ext);

Actually, I have no idea what m_custom_codexec_ext does

• new stuff

- path to the journal. Sometimes user asks how to change the name or dates there.
- list of smart replacements.

[giuspen]

m_custom_codexec_ext is just a list of extensions to give to the source code files when you press F5 to build & run automatically
filepath_src_tmp = os.path.join(cons.TMP_FOLDER, "exec_code."+config.get_code_exec_ext(self, code_type))
I'm happy that you rename to less cryptic names and feel free to add few missing controls from the preferences dialog, I want to add few more unit tests and get rid of some warnings before adding new stuff myself

[giuspen]

I found a bug easily reproducible that unfortunately is not happening in the unit tests that I wrote. I'll have a look myself later but if you already know the cause @txe give a shout!
1) ./build/cherrytree tests/data/test.ctb
2) do not visit any node, click file--save as and choose SQLite, password protected, save it and close cherrytree
3) ./build/cherrytree (will open the .ctx)
4) go to the last node, the one with the embedded widgets, they are all gone

[txe]

I've found the reason of the issue, I'll fix.

[giuspen]

Thanks!

[txe]

btw @ForeverRainbow, Although there is a tray in my OS (Arch/XFCE), the option is always disabled. Looks like is_embedded doesn't work as should.

[ForeverRainbow]

@txe I was afraid of that, since there are ways to get the system tray even on GNOME with plugins. I hoped the is_embedded test would work for those since it is the way recommended by Gtk, I think a hidden config option to force the systray on would be best (since anybody who can edit the config file to set it can edit it to recover from not being able to open cherrytree due to being minimised)

[ForeverRainbow]

Also if/when you write up a style guide I can create a clang-format file and run everything through it

[txe]

For some reason, I don't like the idea to reformat the existing code.

[txe]

Back to systray,
People have been using cherrytree on Gnome for a while and I don't remember that they had turned on systray option and then complained about it.
I think if there is no way to check the systray existence, code should be reverted back. It doesn't make sense to have an always disabled systray option and then have a hidden option to turn it on.

[ForeverRainbow]

@txe The systray existence check does work on non-linux systems afaik, the option isnt disabled there. The problem is that it is very easy to enable the systray option when it isnt supported and it will cause cherrytree to seemingly not open at all (and also run in the background forever). It needs to at least be force disabled by default if not detected on start I think.

The reason I say hidden config option is because if someone force enables it without a systray they have to go into the config file to get cherrytree to start properly, which might be beyond someone who enables it by just clicking a checkbox. It shouldn't be simple to make the mistake of enabling it when unsupported thats what I'm trying to say :p

[txe]

I have a laptop with openbox and vm with xfce, both are perfectly linux, on both the check doesn't work and the systray option is always disabled.

[ForeverRainbow]

Yes, I said on non-linux systems, I accept it is broken on linux systems

[ForeverRainbow]

Also possibly reports of this issue #596 and (maybe) #669. Since it appears that cherrytree simply isnt launching and you can only actually tell its running through looking at processes and/or launching from the terminal - its a bit hard to diagnose

[giuspen]

A little one @txe that I found yesterday, since I would be tempted to rework it a bit to fix the issue, I prefer instead to have your feedback on it. Basically if you open a password protected document and then you change your mind and press cancel on the dialog asking for the password, there is an error dialog saying that there was an error parsing the document, this is incorrect. I looked at the code and I see that the user pressing cancel is managed with an exception, I don't particularly like it because also for example if the last document you opened was a password protected one, then the day after you start cherrytree and it asks you for the password for that document but you want another document now so you press cancel, that is a typical use case, not an exceptional use case.
I would prefer to manage returned values rather than exceptions with an error in cases that are not an error but a normal use case but let me know your point. BTW I'm planning to package a new 0.99.x in the evening with the new super duper export to pdf with links ( thanks again @ForeverRainbow )!

[txe]

Yeah, I remember this issue and I had the same thoughts, only one thing stopped me, it was annoyingly uneasy to pass the fact that user canceled the dialog. Maybe throwing a custom exception for that and then, as a result, not showing any messages, is a bit better. But anyway if you have in mind how to fix it, go ahead.

[giuspen]

I thought as an easy at least temporary workaround to parse the error message but that would be a bit flaky because then if you change the message we're back. I'll have a thought anyway, I may push something later.

[txe]

Links are really cool. Btw, how is your progress with testing? Did you do as much as you wanted?

[giuspen]

I made a good progress with my read and write unit tests but I still have to add the embedded widgets harness, hopefully that will come in the next few days. After that it should be fairly easy also to set up unit tests on the imports from other applications into cherrytree when gradually implementing that.

[ForeverRainbow]

Why not make it return an std::optional instead of throwing, or split it into two functions one which gets the password (which is empty if cancelled) and one which decrypts it (which would imho be cleaner).

I can implement that now actually if it sounds good

[giuspen]

Hi @ForeverRainbow I actually already implemented something, currently in 0.99.4, that solves the issue. I returned an empty extracted filepath in case of user cancel rather than throw an error and up the stack in case of empty error I'm not showing the error dialog.

[giuspen]

Hi @txe at the time of implementing the read/write of the tables, I thought it was the right time to stop moving the header to the end (which I have kept doing for backwards compatibility for the pygtk2 life) so I implemented this additional xml attribute "head_front" that, if found at reading time, would mean that there was no need to move the last line to the first position since the rows were already in the right order. I think that I was already writing this argument to true at the time of writing the table but now I cannot find that and in fact the check bool head_back = xml_element->get_attribute_value("head_front").empty() is always false. This would indeed cause an issue for who jumps between the two versions so I should support it in the pygtk2 version as well before re-enforcing it in Gtkmm3 but before I wanted to get what you think about it, do you think it is the right thing to do or you would rather stick with the current format and drop the "head_front"?

[txe]

Sorry about it, I didn't mention that change. I was freaked out when I opened files in pygtk and all tables were broken. But you see, no complains about tables now :)

I don't know, I don't have strong arguments for both cases.
If it was up to me, I would just play it safe and leave it as is because though it definitely looks odd, it does no harm. And having two slightly different formats, checking it every time and maybe having complaining users are bit messy.

But I would certainly add version checking in pygtk because pygtk can read new format (when some changes will be introduced), lose some information while reading and save it wrongly, especially when it's xml.

[giuspen]

That's fine, the plan was to update the Pygtk2 version to support that attribute before the Gtkmm3 version would go public but I've never looked at that until today when I was looking at unit testing the tables and I remembered I didn't want to see the header at the bottom anymore... but there it was :)
I'll leave it as it is for now, other issues have definitely higher priority

[ForeverRainbow]

I am wondering whether it would be better to split the import parsers into a separate application, run from ct_imports. It would make it easier to unit test and provide a language independent interface for converting to cherrytree XML . I am thinking it might work something like:

Most things in ct_parser are moved into a separate binary which is built along with cherrytree. The things in ct_imports then run the separate binary with formatting args and parse the output, turning it into a tree of ct_imported_node's so the import interface won't change.

The separate binary takes formatting args, reads from stdin and outputs some simple container XML, e.g <file><root><slot><rich_text>...</file><file>... which just retains the hierarchy and the converted rich text.

What can't be moved is the markdown real-time formatter and since it is quite heavily connected to the markdown parser that may be tricky to separate.

I like this idea because it would allow changes and fixes to the parser implementations separate from the main source, and would also allow external scripts to utilise the parsers. It may be not such a good idea though, any thoughts?

[giuspen]

I thought about this when I wanted to reuse the python code to import. If I would not have had your help I probably would have opted for an intermediate conversion to cherrytree file document with python and then importing the cherrytree document into the (cherry) tree. At this point I'm not sure, frankly I haven't looked much at the import part. Unit testing is feasible already on the tree after importing the data as I pointed you in the other issue. @txe what do you think?

[giuspen]

We could also use the python (converted to python3) for the remaining documents to import until and if we decide to convert to C++

[ForeverRainbow]

@giuspen The python importer is quite heavily connected to the rest of cherrytree I think? The C++ version only uses CtConfig and only for knowing what type of bullet points to use. I wonder if it would be easier to convert it to python3 and separate it from the rest or to implement the missing importers in C++. But then python has things like BeautifulSoup

[giuspen]

@ForeverRainbow I have no doubt that ideally we don't want that python, it is just a workaround since the time it takes to do all of them is a lot, considering that the inefficiency of the importer for few proprietary types (and often discontinued) does not have any impact on the normal cherrytree efficiency

[txe]

I don't think that separating import code into a different application is improvement:

• current import classes can be already unit tested, they have a simple interface: input - paths; output - xml. If it is not much, it can be improved.
• an additional app can give troubles in distribution
• even to run and fix code is much simpler then it is one project

[ForeverRainbow]

@txe It would still be in one project, and to "run and fix code" would be simpler since you could test the parsers with custom input data separately from everything else. The importer interface would remain the same, they would just call the binary to do the parsing instead of instancing classes. Troubles in distribution yes but if built as part of the same CMake system then it would be like the other dependencies included into the source.

Your call, just seems like an opportunity to make cherrytree more modular

[txe]

I think we can skip an additional abstraction layer if it doesn't resolve any real-world issue.

[ForeverRainbow]

@giuspen I did want to say, I noticed you use alien to generate the rpm's, I haven't used alien but I am on fedora so I can generate them natively if that helps (I locally build each release rpm and install it manually anyway :p).

[giuspen]

@ForeverRainbow thanks I will keep it in mind, the thing is in fedora there is not an LTS strategy so there would be multiple versions to cover am I right?

[ForeverRainbow]

@giuspen I could only build packages for x86 64bit and whatever the latest version of fedora is (currently 32), but it would probably be better than whatever alien produces. Fedora only provides maintenance for ~13 months so yeah no LTS but also no need to package for systems older than that time frame (since people will not be using fedora unless they are preprepared to update to the new releases).

[giuspen]

@ForeverRainbow thanks I will write you an email when I issue a new version so you can package on fedora for me. I did not plan to use alien for the C++ version anyway because the dependencies are much more important than with the python version so hardly a package for a distro is good for another.

[ForeverRainbow]

Maybe not do this now but these are my thoughts on file logging, it (should) be mostly trivial to implement:

I think there should be an option for this though since writing a few mb of debug logs to every time a large export/import happens is probably not ideal :p. That is quite easy to configure since spdlog has its log level set at runtime and it can be changed dynamically (and for different loggers so terminal could do trace and the file do info, for example, or separate error and general logs, etc).

I tried to find something like QStandardPaths for Gtk but I couldn't, does anyone know if such a thing exists outside Qt? If not I can add wrappers in fs which return the log paths for each system (would prefer not to do that manually though).

[giuspen]

I'm not understanding, why would we write MB of logs for imports/exports other than when debugging it

[ForeverRainbow]

@giuspen That was mostly a joke, it is not really mb but it is a considerable amount, when logging at debug level exporting at few thousand page pdf will print over 10k lines or so. If that accumulates it will start to add up and its an extra overhead which isn't needed 99% of the time