Spyder: Crashes when trying to access Preferences after adding custom color schemes

[loewenm]

Description

What steps will reproduce the problem?

1. Open Spyder application
2. Click on Python > Preferences... (⌘,)

What is the expected output? What do you see instead?

Opens the preferences panel.

Please provide any additional information below

Please note: I saved a few custom color themes and noticed that it was starting to bug-out... That's it. I have a very vanilla Spyder environment.

File "/Users/michaelloewen/anaconda/lib/python3.6/site-packages/spyder/app/mainwindow.py", line 2609, in edit_preferences
widget.initialize()
File "/Users/michaelloewen/anaconda/lib/python3.6/site-packages/spyder/plugins/configdialog.py", line 76, in initialize
self.setup_page()
File "/Users/michaelloewen/anaconda/lib/python3.6/site-packages/spyder/plugins/configdialog.py", line 1175, in setup_page
self.scheme_editor_dialog.add_color_scheme_stack(name, custom=True)
File "/Users/michaelloewen/anaconda/lib/python3.6/site-packages/spyder/plugins/configdialog.py", line 1502, in add_color_scheme_stack
value = self.parent.get_option(option)
File "/Users/michaelloewen/anaconda/lib/python3.6/site-packages/spyder/plugins/configdialog.py", line 55, in get_option
return CONF.get(self.CONF_SECTION, option, default)
File "/Users/michaelloewen/anaconda/lib/python3.6/site-packages/spyder/config/user.py", line 378, in get
raise cp.NoOptionError(option, section)
configparser.NoOptionError: No option 'custom-0/normal' in section: 'color_schemes'

Version and main components

• Spyder Version: 3.2.4
• Python Version: 3.6.3
• Qt Versions: 5.6.2, PyQt5 5.6 on Darwin

Dependencies

pyflakes >=0.6.0 :  1.6.0 (OK)
pycodestyle >=2.3:  2.3.1 (OK)
pygments >=2.0   :  2.2.0 (OK)
pandas >=0.13.1  :  0.20.3 (OK)
numpy >=1.7      :  1.13.3 (OK)
sphinx >=0.6.6   :  1.6.3 (OK)
rope >=0.9.4     :  0.10.5 (OK)
jedi >=0.9.0     :  0.10.2 (OK)
psutil >=0.3     :  5.4.0 (OK)
nbconvert >=4.0  :  5.3.1 (OK)
sympy >=0.7.3    :  1.1.1 (OK)
cython >=0.21    :  0.26.1 (OK)
qtconsole >=4.2.0:  4.3.1 (OK)
IPython >=4.0    :  6.1.0 (OK)
pylint >=0.25    :  1.7.4 (OK)

[CAM-Gerlach]

I am no expert, but it appears almost certain the the problem is related to the color themes you saved...if you look at the trace, you will note the problem occurs as Spyder is trying to add your custom color scheme(s). The smoking gun:

self.scheme_editor_dialog.add_color_scheme_stack(name, custom=True)

Therefore, the problem could either be an issue with (one of) the color schemes you are trying to load itself, or a bug in Spyder's handling of custom color schemes. The latter would likely have to be specific either to the macOS platform, or (one of?) the custom schemes you are trying to load, as I could not reproduce the bug on either my Win 10 Ent x64 or my Win 8.1 Pro x64 Spyder installs, with the exact same Spyder, Python, Qt, PyQt, and dependency versions as you, following the steps you described and trying multiple different saved and new custom schemes.

You could try removing your custom schemes manually, making sure Spyder loads, and then adding them one at a time to isolate if a certain one is causing an issue, or it happens to all of them. If the former, then it is either something wrong with the saved scheme, or a Spyder bug that only emerges in specific cases; posting the saved scheme would probably be helpful here. If the latter, try generating/using a "known good" saved color scheme from somewhere if that's possible, to ensure there isn't any commonality between them that's causing the problem, or something specific to saving on your platform. If none of that works, then it could be something platform specific I can't reproduce...not sure; I'm just a (newer) user, not the developer.

[loewenm]

Seeing how that's the only thing that I changed in the IDE, this makes perfect sense...

Problem is, I used the GUI to make the custom color themes. Do you know where the spyder.ini file is located that I need to edit (i.e. remove my custom themes from)?

(typical installation of Anaconda)

[ccordoba12]

It's under your home directory, in the hidden folder .spyder-py3.

[ccordoba12]

No, those are our default options. But your user options are located in

~/.spyder-py3/spyder.ini

[loewenm]

Thank you ccordoba12. I found it! Used the following to navigate there:

defaults write com.apple.finder AppleShowAllFiles YES

killall Finder

And found it here: /Users/michaelloewen/.spyder-py3/spyder.ini

The color schemes were at the very bottom of the file. Note how custom-2 and custom-0 refer to the same template. There's another one in there called 'Tomorrow' which I was trying to use.

[color_schemes]
custom_names = ['custom-2', 'custom-0']
temp/normal = ('#abb2bf', False, False)
temp/comment = ('#969896', False, True)
temp/string = ('#98c379', False, False)
temp/number = ('#d19a66', False, False)
temp/keyword = ('#c678dd', False, False)
temp/builtin = ('#d19a66', False, False)
temp/definition = ('#61afef', True, False)
temp/instance = ('#8abeb7', False, True)
temp/currentcell = #000000
temp/currentline = #0a99bb
temp/occurrence = #373b41
temp/matched_p = #373b41
temp/unmatched_p = #ff9999
temp/ctrlclick = #56b6c2
temp/background = #282c34
temp/sideareas = #21252b
custom-2/background = #fafafa
custom-2/currentline = #efefef
custom-2/currentcell = #4271ae
custom-2/occurrence = #d6d6d6
custom-2/ctrlclick = #4271ae
custom-2/sideareas = #efefef
custom-2/matched_p = #d6d6d6
custom-2/unmatched_p = #ff9999
custom-2/normal = ('#4d4d4c', False, False)
custom-2/keyword = ('#8959a8', False, False)
custom-2/builtin = ('#c82829', False, False)
custom-2/definition = ('#4d4d4c', True, False)
custom-2/comment = ('#8e908c', False, True)
custom-2/string = ('#718c00', False, False)
custom-2/number = ('#f5871f', False, False)
custom-2/instance = ('#3e999f', False, True)
custom-2/name = Tomorrow
custom-1/name = custom-1
custom-1/currentcell = #2c2c2c
custom-1/currentline = #333333
custom-1/occurrence = #7a738f
custom-1/matched_p = #688060
custom-1/unmatched_p = #bd6e76
custom-1/ctrlclick = #0000ff
custom-1/background = #3f3f3f
custom-1/sideareas = #3f3f3f
custom-1/normal = ('#dcdccc', False, False)
custom-1/comment = ('#c453d4', False, True)
custom-1/string = ('#cc9393', False, False)
custom-1/number = ('#8cd0d3', False, False)
custom-1/keyword = ('#dfaf8f', True, False)
custom-1/builtin = ('#efef8f', False, False)
custom-1/definition = ('#efef8f', False, False)
custom-1/instance = ('#dcdccc', False, True)

[loewenm]

Removing all of the code above from the spyder.ini file resolved my problem. Thank you very much guys. I appreciate it! Hopefully, this helps someone else.

[CAM-Gerlach]

As an aside @loewenm , you can also just type

nano ~/.spyder-py3/spyder.ini

(or cat/less/more/vim/emacs/etc.) at the prompt to view the file directly, or

bpcopy < ~/.spyder-py3/spyder.ini

to copy it directly to the clipboard, in lieu of all that finangling to get it to show up in the GUI, and IIRC you can just type the path directly into the Go>To Path dialog in Finder and you can see it...at least, on 10.6 back in my OS X days you could.

[CAM-Gerlach]

If you did all this from the Spyder GUI, however, then I would think something would have to have gone wrong with Spyder writing to this file. Specifically, the names in the list of custom color tables were set to custom-0 and custom-2 but the actual names saved for the properties were custom-1 and custom-2, so it couldn't find the properties for custom-0 and custom-1 never got read in presumably.

[loewenm]

+1 - yeah, all edits were done through the GUI, so there may be something weird going on there.

I'm editing the .ini file directly from now on...

[CAM-Gerlach]

Right, but you really shouldn't have to go to such lengths. If you start from a fresh .ini file and repeat the steps to add several new custom color tables as you did before, then close and reopen Spyder/preferences, can you reproduce the error? If so, then if you describe the steps you did it probably won't be too difficult to isolate and fix. Thanks!

As an aside, if you want to actually +1 comments, you can do so directly by clicking the + emoji icon, and then the "+1"/thumbsup symbol. Not sure if it actually does anything meaningful, but there you go.

[CAM-Gerlach]

@ccordoba12 Just managed to reproduce the bug and isolate the essence of what's going on, and capture the specific effects on the prefs file. There are actually at least two distinct but issues going on that made it rather confusing to nail down precisely.

First, when a color table is deleted from the preferences, its data as well as its key in custom_names gets immediately deleted from the prefs file on disk, and stays that way if the preferences dialog is canceled or the exit button pressed. However, if the "Apply" or "OK" buttom on the preferences dialog is then clicked, the data gets restored the file, but the key does not get added back to the custom_names list so this has no direct harm other than to clutter the preferences file, as it gets overwritten properly when a new color table with that key gets created. custom-1 in the file above is almost certainly an example of that happening. However, this behavior does seem unintentional; it would make more sense if it was reversed, so that "cancel" and "close" on the prefs window would restore the color table (and, additionally, add its key back to custom_keys so it would actually be usable), while OK and Apply would both confirm the change; naively, it would make even more sense to only actually delete it from the file once OK or Apply was actually pressed but I suppose there must be reasons for that. However, this does not directly impact the current issue so I can create a new one if you think I should.

Second, the more problematic issue is that in certain situations (example below), color table key names (at least custom-0, not sure about the rest) can get duplicated in the custom_names list (visible both in the preferences file, and in the UI drop down menu), but there is only ever one set of keys describing the data for that key name (custom-0/name = foo, etc.), and so if either of those two keys is then deleted, then an error immediately occurs in the background internal console as one of the key names still exists in custom_names, but no data on it (including custom-0/name') exists in the file, and will reoccur in the manner described by the OP when the preferences dialog is closed and reopened (or Spyder is closed and re-opened). That, then, explains the spuriouscustom-0seen in the OP'scustom_names` list, and is the proximate cause of the error. Simply deleting that extra key name manually will solve the immediate issue, until something occurs to duplicate the key.

There is actually a third minor issue I found, but I'm fairly sure it has no direct relationship to the OP's so I'll make a separate issue. In any case, getting on with the matter at hand:

Starting from a clean prefs file, shown below each step (# are my comments), it can be reproduced all in the GUI as follows...

custom_names = []
# No data, of course.

Create two new color schemes, named the default...

custom_names = ['custom-0', 'custom-1']
custom-0/name = custom-0
custom-0/currentcell = #fdfdde
# ETC, SNIP
custom-1/name = custom-1
custom-1/background = #697dff
# ETC, SNIP

Delete the first one...

custom_names = ['custom-1']
custom-1/name = custom-1
custom-1/background = #697dff
# ETC, SNIP

All well and good.

Now, hit "create new scheme" and accept the defaults...

custom_names = ['custom-1', 'custom-0']
custom-1/background = #697dff
# ETC, SNIP
custom-1/name = custom-1
custom-0/name = custom-0
custom-0/currentcell = #fdfdde
# ETC, SNIP

Looks okay.

However, if one repeats the last step exactly...

custom_names = ['custom-1', 'custom-0', 'custom-0']
custom-1/background = #697dff
# ETC, SNIP
custom-1/name = custom-1
custom-0/name = custom-0
custom-0/currentcell = #fdfdde
# ETC, SNIP

Uh oh, the key name is duplicated. At this point the prefs file is corrupted and there is no way to fix it from the GUI, but no error yet occurs and both names can be seen from the selection pop up menu.

However, if one of those two duplicates is ever deleted...

custom_names = ['custom-1', 'custom-0']
custom-1/background = #697dff
# ETC, SNIP
custom-1/name = custom-1
# NO CUSTOM-0 DATA

No error box explictly pops up, but the Spyder Internal Console in the background displays the error

>>> Traceback (most recent call last):
  File "C:\Anaconda3\lib\site-packages\spyder\plugins\configdialog.py", line 1369, in delete_scheme
    self.update_combobox()
  File "C:\Anaconda3\lib\site-packages\spyder\plugins\configdialog.py", line 1219, in update_combobox
    self.scheme_choices_dict[self.get_option('{0}/name'.format(n))] = n
  File "C:\Anaconda3\lib\site-packages\spyder\plugins\configdialog.py", line 55, in get_option
    return CONF.get(self.CONF_SECTION, option, default)
  File "C:\Anaconda3\lib\site-packages\spyder\config\user.py", line 378, in get
    raise cp.NoOptionError(option, section)
configparser.NoOptionError: No option 'custom-0/name' in section: 'color_schemes'

Then, if I close and try to reopen preferences, I get the same error as the OP, and it won't open.

For the time being, the problem can be simply solved solved on the user side by just deleting or renaming the extra name in the custom_names list to whatever matches the key for the values; in the above case changing the name in the list of names to custom-1. However, with the problem isolated the bug can surely be fixed on Spyder's end, hopefully for 3.2.5?

Here's the other, seemingly not directly related issue:

Create a color scheme

custom_names = ['custom-0']
custom-0/name = foo
custom-0/currentcell = #fdfdde
# ETC, SNIP

Click the delete button, and select "OK" on the confirmation dialog

custom_names = []

If one clicks cancel or the window close button, all will remain like this. However, if I click "okay" or "apply" the data but not the key gets added back to the file...

custom_names = []
custom-0/name = foo
custom-0/currentcell = #fdfdde
# ETC, SNIP

However, if I then reopen the prefs and create a new color table that ends up having the same key name (custom-0) that data gets overwritten as it should

custom_names = ["custom-0"]
custom-0/name = bar
custom-0/currentcell = #ffffff

Odd behavior; it would seem what happens for "cancel"/close and "OK"/"Apply" should be reversed, and the key name added back to custom_names if anything to properly restore it, but it doesn't really affect the OP's situation (though it is present in their file) so I can create a new issue if you think I should.

[CAM-Gerlach]

Any updates on tracking this down, @ccordoba12 ? Hopefully what I've found should be enough to reproduce and narrow down the culprit...

[ccordoba12]

Nope. Could you take a look at it @CAM-Gerlach?

[CAM-Gerlach]

Sure, I can give it a shot. I'll be traveling etc. the next few days so it might not be until the weekend.

[SuperZing]

Here's a quick and easy method to fix this bug and not restart

0. Close Anaconda and Spyder
1. In user search for spyder.ini, backup in case and edit the original.
2. Find color_schemes then custom_names, it will look like this "custom_names = ['custom-0', 'custom-1', 'custom-3']", delete the custom causing the error.
3. Still wont work, delete the error custom from custom_names

[CAM-Gerlach]

Thanks @SuperZing !