Parlai: json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)

[chadleong]

Bug description
Did a fresh pull from git, tried to run this command python parlai/chat_service/services/browser_chat/run.py --config-path parlai/chat_service/tasks/chatbot/config.yml and I got this error.

[ downloading: http://parl.ai/downloads/_models/convai2/convai2_self_seq2seq_model.opt to C:\Anaconda3\lib\site-packages\parlai-0.1.20200508-py3.7.egg\data\models\convai2\seq2seq\convai2_self_seq2seq_model.opt ]
Downloading convai2_self_seq2seq_model.opt: 100%|█████████████████████████████████████████████████████████████████████████████████████████████| 1.35k/1.35k [00:01<00:00, 1.04kB/s] 
Traceback (most recent call last):
  File "parlai/chat_service/services/browser_chat/run.py", line 54, in <module>
    run(opt)
  File "parlai/chat_service/services/browser_chat/run.py", line 41, in run
    manager = BrowserManager(opt)
  File "C:\Anaconda3\lib\site-packages\parlai-0.1.20200508-py3.7.egg\parlai\chat_service\services\websocket\websocket_manager.py", line 54, in __init__
    self._complete_setup()
  File "C:\Anaconda3\lib\site-packages\parlai-0.1.20200508-py3.7.egg\parlai\chat_service\services\websocket\websocket_manager.py", line 67, in _complete_setup
    self._load_model()
  File "C:\Anaconda3\lib\site-packages\parlai-0.1.20200508-py3.7.egg\parlai\chat_service\services\websocket\websocket_manager.py", line 77, in _load_model
    model_params[model] = create_agent(model_opt).share()
  File "C:\Anaconda3\lib\site-packages\parlai-0.1.20200508-py3.7.egg\parlai\core\agents.py", line 406, in create_agent
    model = create_agent_from_opt_file(opt)
  File "C:\Anaconda3\lib\site-packages\parlai-0.1.20200508-py3.7.egg\parlai\core\agents.py", line 294, in create_agent_from_opt_file
    new_opt = load_opt_file(optfile)
  File "C:\Anaconda3\lib\site-packages\parlai-0.1.20200508-py3.7.egg\parlai\core\opt.py", line 92, in load_opt_file
    opt = json.load(t_handle)
  File "C:\Anaconda3\lib\json\__init__.py", line 296, in load
    parse_constant=parse_constant, object_pairs_hook=object_pairs_hook, **kw)
  File "C:\Anaconda3\lib\json\__init__.py", line 348, in loads
    return _default_decoder.decode(s)
  File "C:\Anaconda3\lib\json\decoder.py", line 337, in decode
    obj, end = self.raw_decode(s, idx=_w(s, 0).end())
  File "C:\Anaconda3\lib\json\decoder.py", line 355, in raw_decode
    raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)

Any ideas ? Using Win10, anaconda 3.7.2 64bit if that helps.

[stephenroller]

Hmm, looks like maybe a bad download. I'd try deleting the file and trying again.

Officially we don't support windows though.

[chadleong]

I tried deleting them and rebuilding it but doesn't help, same error.

Yea I noticed that about Windows because I had to refactor the import sh because it's not compatible with Windows.

[stephenroller]

Some people have reported needing to set their default encoding to UTF-8.

[chadleong]

@stephenroller Thanks :) Adding the encoding solved the problem.

Note for windows user:

Replace all import sh with:

   import pbs
   class Sh(object):
       def getattr(self, attr):
           return pbs.Command(attr)
   sh = Sh()

and you have to add

with open(optfile, 'r', encoding='utf-8') as t_handle:

at parlaicoreopt.py

[github-actions[bot]]

This issue has not had activity in 30 days. Please feel free to reopen if you have more issues. You may apply the "never-stale" tag to prevent this from happening.

[stephenroller]

We should really just document that all of windows requires UTF-8 mode.