Beets: gmusic: Adapt to gmusicapi change in 12.0.0 that moved OAUTH_FILEPATH

[littlejeem]

Problem

I installed the gmusic plugin using the documentation pip install gmusicapi and enabled in my config

Running this command in verbose (-vv) mode:

$ beet -vv import /home/jlivin25/Music/Uploads/

Led to this problem:

user configuration: /home/jlivin25/.config/beets/config.yaml
data directory: /home/jlivin25/.config/beets
plugin paths: /home/jlivin25/.config/beets/myplugins
Sending event: pluginload
Traceback (most recent call last):
  File "/home/jlivin25/.local/bin/beet", line 11, in <module>
    sys.exit(main())
  File "/home/jlivin25/.local/lib/python2.7/site-packages/beets/ui/__init__.py", line 1266, in main
    _raw_main(args)
  File "/home/jlivin25/.local/lib/python2.7/site-packages/beets/ui/__init__.py", line 1249, in _raw_main
    subcommands, plugins, lib = _setup(options, lib)
  File "/home/jlivin25/.local/lib/python2.7/site-packages/beets/ui/__init__.py", line 1135, in _setup
    plugins = _load_plugins(config)
  File "/home/jlivin25/.local/lib/python2.7/site-packages/beets/ui/__init__.py", line 1121, in _load_plugins
    plugins.send("pluginload")
  File "/home/jlivin25/.local/lib/python2.7/site-packages/beets/plugins.py", line 488, in send
    for handler in event_handlers()[event]:
  File "/home/jlivin25/.local/lib/python2.7/site-packages/beets/plugins.py", line 471, in event_handlers
    for plugin in find_plugins():
  File "/home/jlivin25/.local/lib/python2.7/site-packages/beets/plugins.py", line 307, in find_plugins
    _instances[cls] = cls()
  File "/home/jlivin25/.local/lib/python2.7/site-packages/beetsplug/gmusic.py", line 39, in __init__
    u'oauth_file': gmusicapi.clients.OAUTH_FILEPATH,
AttributeError: 'module' object has no attribute 'OAUTH_FILEPATH'

Here's a link to the music files that trigger the bug (if relevant): N/A

Setup
OS: Ubuntu 16.04.6 LTS
Python version: Python 2.7.12
beets version: 1.4.8
Turning off plugins made problem go away (no): In this instance its a plugin i'm trying to use (gmusic)
My configuration (output of beet config) is:

directory: /home/jlivin25/Music/Library/PlayUploads
library: /home/jlivin25/.config/beets/uploads/musiclibrary.blb

import:
    copy: no
    move: yes
    write: yes
    resume: ask
    quiet_fallback: skip
    timid: no
    log: beetslog.txt
ignore: .AppleDouble ._* *~ .DS_Store
art_filename: folder

plugins: embedart chroma fetchart convert permissions gmusic

pluginpath: /home/jlivin25/.config/beets/myplugins

threaded: yes

color: yes

match:
    strong_rec_thresh: 0.55

paths:
    default: $albumartist/$album/$track - $title - $artist
    singleton: Singletons/$track - $title - $artist
    comp: Various/$album/$track - $title - $artist
    albumtype:soundtrack: Soundtracks/$album/$track - $title - $artist

permissions:
    file: 777
    dir: 777

convert:
    never_convert_lossy_files: no
    copy_album_art: yes
    dest: ~/Music/Uploads
    auto: yes
    pretend: no
    embed: yes
    threads: 8
    album_art_maxwidth: 0
    quiet: no

    format: mp3
    formats:
        flac:
            extension: flac
            command: ffmpeg -i $source -y -vn -acodec flac $dest
        alac:
            extension: m4a
            command: ffmpeg -i $source -y -vn -acodec alac $dest
        mp3:
            extension: mp3
            command: ffmpeg -i $source -y -vn -aq 2 $dest
    max_bitrate: 500
    tmpdir:

gmusic:
    email: [email protected]
    password: mypassword
    auto: yes
    uploader_id:
    device_id:
    oauth_file: ~/.config/beets/oauth.cred

Error: https://pastebin.com/y20KKLxc
config.yaml https://pastebin.com/2u877j5s

[sampsyo]

Here's a Markdown tip: use "fenced code blocks" (see the docs) to preserve the formatting of pastes from the terminal.

Looks like version 12.0.0 of the gmusicapi module, which was released in January, removed something this plugin was relying on. Can you try installing an older version of the module to confirm that this is the problem?

[littlejeem]

..........and thats why I love the community; when you get little learning tips like that! Cheers @sampsyo. I'll remove gmusicapi and install an earlier version and report back.

[littlejeem]

ok....soooo

Removed gmusicapi pip uninstall gmusicapi

installed specific version pip install gmusicapi==11.1.1

ran beet gmusic-upload, this ran and prompted me to follow a link and enter an authorisation code (which I've redacted but it did contain a /).

I copy and pasted the authorsation code and then this then threw and error

/home/jlivin25/.local/lib/python2.7/site-packages/gmusicapi/__init__.py:4: GmusicapiWarning: gmusicapi.clients.OAUTH_FILEPATH is deprecated and will be removed; use Musicmanager.OAUTH_FILEPATH
  from gmusicapi.clients import Webclient, Musicmanager, Mobileclient

Visit the following url:
 https://accounts.google.com/o/oauth2/v2/auth?scope=REDACTED
Follow the prompts, then paste the auth code here and hit enter: REDACTED
Traceback (most recent call last):
  File "/home/jlivin25/.local/bin/beet", line 11, in <module>
    sys.exit(main())
  File "/home/jlivin25/.local/lib/python2.7/site-packages/beets/ui/__init__.py", line 1266, in main
    _raw_main(args)
  File "/home/jlivin25/.local/lib/python2.7/site-packages/beets/ui/__init__.py", line 1253, in _raw_main
    subcommand.func(lib, suboptions, subargs)
  File "/home/jlivin25/.local/lib/python2.7/site-packages/beetsplug/gmusic.py", line 78, in upload
    self.authenticate()
  File "/home/jlivin25/.local/lib/python2.7/site-packages/beetsplug/gmusic.py", line 73, in authenticate
    self.m.perform_oauth(oauth_file)
  File "/home/jlivin25/.local/lib/python2.7/site-packages/gmusicapi/clients/shared.py", line 168, in perform_oauth
    storage.put(credentials)
  File "/home/jlivin25/.local/lib/python2.7/site-packages/oauth2client/client.py", line 421, in put
    self.locked_put(credentials)
  File "/home/jlivin25/.local/lib/python2.7/site-packages/oauth2client/file.py", line 83, in locked_put
    self._create_file_if_needed()
  File "/home/jlivin25/.local/lib/python2.7/site-packages/oauth2client/file.py", line 70, in _create_file_if_needed
    open(self._filename, 'a+b').close()
IOError: [Errno 2] No such file or directory: u'~/.config/beets/oauth.cred'

I'm not sure now if this is still an issue or something I'm doing wrong.

[sampsyo]

OK! Looks like that was indeed the problem, but now there's a second problem… I've changed the title of this issue to reflect the problem with gmusicapi 12.0.0.

I notice that you have a custom oauth_file configured. What happens if you use an absolute path for that instead of using ~/... to indicate your home directory? I think this line:
https://github.com/beetbox/beets/blob/3399154adf30c72ab3d1c22570764c72c120cc89/beetsplug/gmusic.py#L65

needs to use as_filename instead of as_str to make proper paths work.

[littlejeem]

Yep that sorted it!

.......I wasn't necessarily going for a custom location I'm just not hugely competent so followed the example from the plugin page...

Changed the oauth file path to an absolute one and that sorted it.

Cheers!

[sampsyo]

Great! I have pushed what I believe to be a fix for both of these problems (i.e., ~/ should work now in that config option, and gmusicapi 12.0.0 or later should also work). Let me know if that gives you any trouble.

[littlejeem]

Ok so I've removed gmusicapi 11.1.1 and reinstalled version 12.

my gmusic api section in the config reads:

gmusic:
    email: [email protected]
    password: mypassword
    auto: yes
    oauth_file: /home/jlivin25/.config/beets/oauth.cred

if I run beet gmusic-upload

I get the following error

Traceback (most recent call last):
  File "/home/jlivin25/.local/bin/beet", line 11, in <module>
    sys.exit(main())
  File "/home/jlivin25/.local/lib/python2.7/site-packages/beets/ui/__init__.py", line 1266, in main
    _raw_main(args)
  File "/home/jlivin25/.local/lib/python2.7/site-packages/beets/ui/__init__.py", line 1249, in _raw_main
    subcommands, plugins, lib = _setup(options, lib)
  File "/home/jlivin25/.local/lib/python2.7/site-packages/beets/ui/__init__.py", line 1135, in _setup
    plugins = _load_plugins(config)
  File "/home/jlivin25/.local/lib/python2.7/site-packages/beets/ui/__init__.py", line 1121, in _load_plugins
    plugins.send("pluginload")
  File "/home/jlivin25/.local/lib/python2.7/site-packages/beets/plugins.py", line 488, in send
    for handler in event_handlers()[event]:
  File "/home/jlivin25/.local/lib/python2.7/site-packages/beets/plugins.py", line 471, in event_handlers
    for plugin in find_plugins():
  File "/home/jlivin25/.local/lib/python2.7/site-packages/beets/plugins.py", line 307, in find_plugins
    _instances[cls] = cls()
  File "/home/jlivin25/.local/lib/python2.7/site-packages/beetsplug/gmusic.py", line 39, in __init__
    u'oauth_file': gmusicapi.clients.OAUTH_FILEPATH,
AttributeError: 'module' object has no attribute 'OAUTH_FILEPATH'

Uninstalling gmusicapi 12, reinstalling 11.1.1, and re-running beet gmusic-upload prompted for google authorisation as expected.

Interestingly and I don't know if this is relevant, then moving to query gmusic for an uploaded artist

beet gmusic-songs -a Ludovico Einaudi

results in a what appears to be an error:

/home/jlivin25/.local/lib/python2.7/site-packages/gmusicapi/__init__.py:4: GmusicapiWarning: gmusicapi.clients.OAUTH_FILEPATH is deprecated and will be removed; use Musicmanager.OAUTH_FILEPATH
  from gmusicapi.clients import Webclient, Musicmanager, Mobileclient
Traceback (most recent call last):
  File "/home/jlivin25/.local/bin/beet", line 11, in <module>
    sys.exit(main())
  File "/home/jlivin25/.local/lib/python2.7/site-packages/beets/ui/__init__.py", line 1266, in main
    _raw_main(args)
  File "/home/jlivin25/.local/lib/python2.7/site-packages/beets/ui/__init__.py", line 1253, in _raw_main
    subcommand.func(lib, suboptions, subargs)
  File "/home/jlivin25/.local/lib/python2.7/site-packages/beetsplug/gmusic.py", line 108, in search
    or Mobileclient.FROM_MAC_ADDRESS).upper()
AttributeError: 'newobject' object has no attribute 'upper'

[sampsyo]

Hmm; it looks like you may not have gotten the latest changes to beets? You can try running from source (and hence picking up the change I just committed) by following the instructions in the FAQ:
http://docs.beets.io/en/stable/faq.html#run-the-latest-source-version-of-beets

[littlejeem]

ok so I removed beets and gmusicapi using, pip uninstall beets, pip uninstall gmusicapi

installed beets using and gmusicapi 12 using, pip install https://github.com/beetbox/beets/tarball/master
pip install gmusic api

The command beet gmusic-upload works correctly

The command beet gmusic-songs fails giving the following error:

Traceback (most recent call last):
  File "/home/jlivin25/.local/bin/beet", line 11, in <module>
    load_entry_point('beets==1.4.9', 'console_scripts', 'beet')()
  File "/home/jlivin25/.local/lib/python2.7/site-packages/beets/ui/__init__.py", line 1266, in main
    _raw_main(args)
  File "/home/jlivin25/.local/lib/python2.7/site-packages/beets/ui/__init__.py", line 1253, in _raw_main
    subcommand.func(lib, suboptions, subargs)
  File "/home/jlivin25/.local/lib/python2.7/site-packages/beetsplug/gmusic.py", line 115, in search
    or Mobileclient.FROM_MAC_ADDRESS).upper()
AttributeError: 'newobject' object has no attribute 'upper'

Thanks for all the help @sampsyo , appreciate all you do. Beets is a thing of wonder to me and I've barely scratched the surface!

[sampsyo]

Yo, it looks like this is yet a _third_ bug in this plugin! Would you mind opening a new issue for this one?

Roughly speaking, I think this:
https://github.com/beetbox/beets/blob/d77a13eb343a2a41e60c50e754694896d354e633/beetsplug/gmusic.py#L113-L115

Needs to become this:

            new_device_id = ((device_id.as_str()
                              or uploader_id.as_str().replace(':', '')).upper()
                             or Mobileclient.FROM_MAC_ADDRESS)

but we'll need to try it out to work out the details.

[littlejeem]

Hi @sampsyo.

Yep no problems, I'll get it posted this afternoon (UK time)...