Paperless: Cannot set email comsumer when using Docker

[HesselTjeerdsma]

Hi!

How do I correctly setup the imap consumer when using docker? Now, I tried to copy all email related settings from the paperless.conf.exampe to docker-compose.env but then I get the error:

[Errno -8] Unrecognized service
--

Here is a copy of my docker-compose.env where I have removed all sensitive information:

# Environment variables to set for Paperless
# Commented out variables will be replaced by a default within Paperless.

# Passphrase Paperless uses to encrypt and decrypt your documents
PAPERLESS_PASSPHRASE=

# The amount of threads to use for text recognition
PAPERLESS_OCR_THREADS=4

# Additional languages to install for text recognition
PAPERLESS_OCR_LANGUAGES=nld

# You can change the default user and group id to a custom one
# USERMAP_UID=1000
# USERMAP_GID=1000

PAPERLESS_CONSUME_MAIL_HOST="imap.gmail.com"
PAPERLESS_CONSUME_MAIL_PORT="993"
PAPERLESS_CONSUME_MAIL_USER=""
PAPERLESS_CONSUME_MAIL_PASS=""
PAPERLESS_EMAIL_SECRET=""

[erikarvstedt]

Is this a Docker or a Python error? In either case, please post the full backtrace/output.

[danielquinn]

@ahyear just posted a full backtrace in #366 and witha little searching online, I have a guess. Can one of you try changing these lines in src/documents/mail.py from this:

        self._host = os.getenv("PAPERLESS_CONSUME_MAIL_HOST")
        self._port = os.getenv("PAPERLESS_CONSUME_MAIL_PORT")
        self._username = os.getenv("PAPERLESS_CONSUME_MAIL_USER")
        self._password = os.getenv("PAPERLESS_CONSUME_MAIL_PASS")
        self._inbox = os.getenv("PAPERLESS_CONSUME_MAIL_INBOX", "INBOX")

to this?

        self._host = os.getenv("PAPERLESS_CONSUME_MAIL_HOST", "").strip()
        self._port = os.getenv("PAPERLESS_CONSUME_MAIL_PORT", "").strip()
        self._username = os.getenv("PAPERLESS_CONSUME_MAIL_USER", "").strip()
        self._password = os.getenv("PAPERLESS_CONSUME_MAIL_PASS", "").strip()
        self._inbox = os.getenv("PAPERLESS_CONSUME_MAIL_INBOX", "INBOX").strip()

It might be something as simple as the Docker environment carrying a newline character into the environment.

[danielquinn]

You should also try setting your environment variables without quotes, as Docker can be annoying in that way:

PAPERLESS_CONSUME_MAIL_HOST=imap.google.com
PAPERLESS_CONSUME_MAIL_PORT=995
PAPERLESS_CONSUME_MAIL_USER=my_user
PAPERLESS_CONSUME_MAIL_PASS=my_pass
PAPERLESS_EMAIL_SECRET=my_word

[ahyear]

ok i'm building the .strip() now
i let you know

If it's the same i'm remove the .strip and will try the second solution.

Edit : OK I miss read... start again...

[ahyear]

The first solution did not work.
i'm trying the second one (remove quotes)

[ahyear]

not better :

paperless_consumer_1
date,stream,content
2018-06-14 15:57:29,stderr,socket.gaierror: [Errno -2] Name does not resolve

2018-06-14 15:57:29,stderr," for res in _socket.getaddrinfo(host, port, family, type, proto, flags):
"
2018-06-14 15:57:29,stderr," File \"/usr/lib/python3.6/socket.py\", line 745, in getaddrinfo
"
2018-06-14 15:57:29,stderr," for res in getaddrinfo(host, port, 0, SOCK_STREAM):
"
2018-06-14 15:57:29,stderr," File \"/usr/lib/python3.6/socket.py\", line 704, in create_connection
"
2018-06-14 15:57:29,stderr," return socket.create_connection((self.host, self.port))
"
2018-06-14 15:57:29,stderr," File \"/usr/lib/python3.6/imaplib.py\", line 284, in _create_socket
"
2018-06-14 15:57:29,stderr, sock = IMAP4._create_socket(self)

2018-06-14 15:57:29,stderr," File \"/usr/lib/python3.6/imaplib.py\", line 1286, in _create_socket
"
2018-06-14 15:57:29,stderr, self.sock = self._create_socket()

2018-06-14 15:57:29,stderr," File \"/usr/lib/python3.6/imaplib.py\", line 294, in open
"
2018-06-14 15:57:29,stderr," IMAP4.open(self, host, port)
"
2018-06-14 15:57:29,stderr," File \"/usr/lib/python3.6/imaplib.py\", line 1296, in open
"
2018-06-14 15:57:29,stderr," self.open(host, port)
"
2018-06-14 15:57:29,stderr," File \"/usr/lib/python3.6/imaplib.py\", line 197, in __init__
"
2018-06-14 15:57:29,stderr," IMAP4.__init__(self, host, port)
"
2018-06-14 15:57:29,stderr," File \"/usr/lib/python3.6/imaplib.py\", line 1283, in __init__
"
2018-06-14 15:57:29,stderr," self._connection = imaplib.IMAP4_SSL(self._host, self._port)
"
2018-06-14 15:57:29,stderr," File \"/usr/src/paperless/src/documents/mail.py\", line 219, in _connect
"
2018-06-14 15:57:29,stderr, self._connect()

2018-06-14 15:57:29,stderr," File \"/usr/src/paperless/src/documents/mail.py\", line 202, in _get_messages
"
2018-06-14 15:57:29,stderr, for message in self._get_messages():

2018-06-14 15:57:29,stderr," File \"/usr/src/paperless/src/documents/mail.py\", line 185, in pull
"
2018-06-14 15:57:29,stderr, self.mail_fetcher.pull()

2018-06-14 15:57:29,stderr," File \"/usr/src/paperless/src/documents/management/commands/document_consumer.py\", line 121, in loop_step
"
2018-06-14 15:57:29,stderr, self.loop_step(mail_delta)

2018-06-14 15:57:29,stderr," File \"/usr/src/paperless/src/documents/management/commands/document_consumer.py\", line 131, in loop_inotify
"
2018-06-14 15:57:29,stderr, self.loop_inotify(mail_delta)

2018-06-14 15:57:29,stderr," File \"/usr/src/paperless/src/documents/management/commands/document_consumer.py\", line 98, in handle
"
2018-06-14 15:57:29,stderr," output = self.handle(args, *options)
"
2018-06-14 15:57:29,stderr," File \"/usr/lib/python3.6/site-packages/django/core/management/base.py\", line 330, in execute
"
2018-06-14 15:57:29,stderr," self.execute(args, *cmd_options)
"
2018-06-14 15:57:29,stderr," File \"/usr/lib/python3.6/site-packages/django/core/management/base.py\", line 283, in run_from_argv
"
2018-06-14 15:57:29,stderr, self.fetch_command(subcommand).run_from_argv(self.argv)

2018-06-14 15:57:29,stderr," File \"/usr/lib/python3.6/site-packages/django/core/management/__init__.py\", line 356, in execute
"
2018-06-14 15:57:29,stderr, utility.execute()

2018-06-14 15:57:29,stderr," File \"/usr/lib/python3.6/site-packages/django/core/management/__init__.py\", line 364, in execute_from_command_line
"
2018-06-14 15:57:29,stderr, execute_from_command_line(sys.argv)

2018-06-14 15:57:29,stderr," File \"/usr/src/paperless/src/manage.py\", line 18, in
"
2018-06-14 15:57:29,stderr,Traceback (most recent call last):

2018-06-14 15:57:26,stderr,Starting document consumer at /consume with inotify

2018-06-14 15:57:09,stderr,socket.gaierror: [Errno -2] Name does not resolve

2018-06-14 15:57:09,stderr," for res in _socket.getaddrinfo(host, port, family, type, proto, flags):
"
2018-06-14 15:57:09,stderr," File \"/usr/lib/python3.6/socket.py\", line 745, in getaddrinfo
"
2018-06-14 15:57:09,stderr," for res in getaddrinfo(host, port, 0, SOCK_STREAM):
"
2018-06-14 15:57:09,stderr," File \"/usr/lib/python3.6/socket.py\", line 704, in create_connection
"
2018-06-14 15:57:09,stderr," return socket.create_connection((self.host, self.port))
"
2018-06-14 15:57:09,stderr," File \"/usr/lib/python3.6/imaplib.py\", line 284, in _create_socket
"
2018-06-14 15:57:09,stderr, sock = IMAP4._create_socket(self)

2018-06-14 15:57:09,stderr," File \"/usr/lib/python3.6/imaplib.py\", line 1286, in _create_socket
"
2018-06-14 15:57:09,stderr, self.sock = self._create_socket()

2018-06-14 15:57:09,stderr," File \"/usr/lib/python3.6/imaplib.py\", line 294, in open
"
2018-06-14 15:57:09,stderr," IMAP4.open(self, host, port)
"
2018-06-14 15:57:09,stderr," File \"/usr/lib/python3.6/imaplib.py\", line 1296, in open
"
2018-06-14 15:57:09,stderr," self.open(host, port)
"
2018-06-14 15:57:09,stderr," File \"/usr/lib/python3.6/imaplib.py\", line 197, in __init__
"
2018-06-14 15:57:09,stderr," IMAP4.__init__(self, host, port)
"
2018-06-14 15:57:09,stderr," File \"/usr/lib/python3.6/imaplib.py\", line 1283, in __init__
"
2018-06-14 15:57:09,stderr," self._connection = imaplib.IMAP4_SSL(self._host, self._port)
"
2018-06-14 15:57:09,stderr," File \"/usr/src/paperless/src/documents/mail.py\", line 219, in _connect
"
2018-06-14 15:57:09,stderr, self._connect()

2018-06-14 15:57:09,stderr," File \"/usr/src/paperless/src/documents/mail.py\", line 202, in _get_messages
"
2018-06-14 15:57:09,stderr, for message in self._get_messages():

2018-06-14 15:57:09,stderr," File \"/usr/src/paperless/src/documents/mail.py\", line 185, in pull
"
2018-06-14 15:57:09,stderr, self.mail_fetcher.pull()

2018-06-14 15:57:09,stderr," File \"/usr/src/paperless/src/documents/management/commands/document_consumer.py\", line 121, in loop_step
"
2018-06-14 15:57:09,stderr, self.loop_step(mail_delta)

2018-06-14 15:57:09,stderr," File \"/usr/src/paperless/src/documents/management/commands/document_consumer.py\", line 131, in loop_inotify
"
2018-06-14 15:57:09,stderr, self.loop_inotify(mail_delta)

2018-06-14 15:57:09,stderr," File \"/usr/src/paperless/src/documents/management/commands/document_consumer.py\", line 98, in handle
"
2018-06-14 15:57:09,stderr," output = self.handle(args, *options)
"
2018-06-14 15:57:09,stderr," File \"/usr/lib/python3.6/site-packages/django/core/management/base.py\", line 330, in execute
"
2018-06-14 15:57:09,stderr," self.execute(args, *cmd_options)
"
2018-06-14 15:57:09,stderr," File \"/usr/lib/python3.6/site-packages/django/core/management/base.py\", line 283, in run_from_argv
"
2018-06-14 15:57:09,stderr, self.fetch_command(subcommand).run_from_argv(self.argv)

2018-06-14 15:57:09,stderr," File \"/usr/lib/python3.6/site-packages/django/core/management/__init__.py\", line 356, in execute
"
2018-06-14 15:57:09,stderr, utility.execute()

2018-06-14 15:57:09,stderr," File \"/usr/lib/python3.6/site-packages/django/core/management/__init__.py\", line 364, in execute_from_command_line
"
2018-06-14 15:57:09,stderr, execute_from_command_line(sys.argv)

2018-06-14 15:57:09,stderr," File \"/usr/src/paperless/src/manage.py\", line 18, in
"
2018-06-14 15:57:09,stderr,Traceback (most recent call last):

2018-06-14 15:57:07,stderr,Starting document consumer at /consume with inotify

2018-06-14 15:56:46,stdout,OK: 321 MiB in 123 packages

2018-06-14 15:56:41,stdout,(1/1) Installing tesseract-ocr-data-fra (3.05.01-r2)

2018-06-14 15:56:40,stdout,fetch http://dl-cdn.alpinelinux.org/alpine/v3.7/community/x86_64/APKINDEX.tar.gz

2018-06-14 15:56:40,stdout,fetch http://dl-cdn.alpinelinux.org/alpine/v3.7/community/x86_64/APKINDEX.tar.gz

2018-06-14 15:56:40,stdout,fetch http://dl-cdn.alpinelinux.org/alpine/v3.7/main/x86_64/APKINDEX.tar.gz

2018-06-14 15:56:39,stdout,fetch http://dl-cdn.alpinelinux.org/alpine/v3.7/main/x86_64/APKINDEX.tar.gz

[danielquinn]

Alright well it sounds like some good old-fashioned debugging is in order then. You need to find out what's being passed to IMAP4_SSL() on this line so I suggest putting some print() statements in there to see what's coming down the pipe:

print("HOST: '{}'".format(self._host))
print("PORT: '{}'".format(self._port))
self._connection = imaplib.IMAP4_SSL(self._host, self._port)

It might also be good to add some print lines in that class' __init__ where those values are set, or even to do a print(os.environ) to see what's in the environment at this stage. If this code is working in the "bare metal" installation, then it's got to be something around how the environment variables are being handled by Docker I guess.

[ahyear]

the host and port are as set in my docker-compose.env. but I set imap.google.com instead of imap.gmail.com it may be that...
testing now, I let you know

[ahyear]

that's it!

double quote was the issue at first and next il was bad imap address. now it woks like a charm ... at least for me.

Thank you for your help!

EDIT: well almost...
now paperless sees the topic but I got a new error:

'paperless_consumer_1
date,stream,content
2018-06-14 21:19:47,stderr,KeyError: 'content-transfer-encoding'

2018-06-14 21:19:47,stderr, raise KeyError(_name)

2018-06-14 21:19:47,stderr," File \"/usr/lib/python3.6/email/message.py\", line 558, in replace_header
"
2018-06-14 21:19:47,stderr," msg.replace_header('content-transfer-encoding', munge_cte[0])
"
2018-06-14 21:19:47,stderr," File \"/usr/lib/python3.6/email/generator.py\", line 189, in _write
"
2018-06-14 21:19:47,stderr, self._write(msg)

2018-06-14 21:19:47,stderr," File \"/usr/lib/python3.6/email/generator.py\", line 116, in flatten
"
2018-06-14 21:19:47,stderr," g.flatten(self, unixfrom=unixfrom)
"
2018-06-14 21:19:47,stderr," File \"/usr/lib/python3.6/email/message.py\", line 158, in as_string
"
2018-06-14 21:19:47,stderr," return super().as_string(maxheaderlen=maxheaderlen, policy=policy)
"
2018-06-14 21:19:47,stderr," File \"/usr/lib/python3.6/email/message.py\", line 968, in as_string
"
2018-06-14 21:19:47,stderr, return self.as_string(policy=self.policy.clone(utf8=True))

2018-06-14 21:19:47,stderr," File \"/usr/lib/python3.6/email/message.py\", line 971, in __str__
"
2018-06-14 21:19:47,stderr, self.body = str(message.get_body())

2018-06-14 21:19:47,stderr," File \"/usr/src/paperless/src/documents/mail.py\", line 61, in __init__
"
2018-06-14 21:19:47,stderr," message = Message(data[0][1], self.logging_group)
"
2018-06-14 21:19:47,stderr," File \"/usr/src/paperless/src/documents/mail.py\", line 241, in _fetch
"
2018-06-14 21:19:47,stderr, for message in self._fetch():

2018-06-14 21:19:47,stderr," File \"/usr/src/paperless/src/documents/mail.py\", line 205, in _get_messages
"
2018-06-14 21:19:47,stderr, for message in self._get_messages():

2018-06-14 21:19:47,stderr," File \"/usr/src/paperless/src/documents/mail.py\", line 185, in pull
"
2018-06-14 21:19:47,stderr, self.mail_fetcher.pull()

2018-06-14 21:19:47,stderr," File \"/usr/src/paperless/src/documents/management/commands/document_consumer.py\", line 121, in loop_step
"
2018-06-14 21:19:47,stderr, self.loop_step(mail_delta)

2018-06-14 21:19:47,stderr," File \"/usr/src/paperless/src/documents/management/commands/document_consumer.py\", line 131, in loop_inotify
"
2018-06-14 21:19:47,stderr, self.loop_inotify(mail_delta)

2018-06-14 21:19:47,stderr," File \"/usr/src/paperless/src/documents/management/commands/document_consumer.py\", line 98, in handle
"
2018-06-14 21:19:47,stderr," output = self.handle(args, *options)
"
2018-06-14 21:19:47,stderr," File \"/usr/lib/python3.6/site-packages/django/core/management/base.py\", line 330, in execute
"
2018-06-14 21:19:47,stderr," self.execute(args, *cmd_options)
"
2018-06-14 21:19:47,stderr," File \"/usr/lib/python3.6/site-packages/django/core/management/base.py\", line 283, in run_from_argv
"
2018-06-14 21:19:47,stderr, self.fetch_command(subcommand).run_from_argv(self.argv)

2018-06-14 21:19:47,stderr," File \"/usr/lib/python3.6/site-packages/django/core/management/__init__.py\", line 356, in execute
"
2018-06-14 21:19:47,stderr, utility.execute()

2018-06-14 21:19:47,stderr," File \"/usr/lib/python3.6/site-packages/django/core/management/__init__.py\", line 364, in execute_from_command_line
"
2018-06-14 21:19:47,stderr, execute_from_command_line(sys.argv)

2018-06-14 21:19:47,stderr," File \"/usr/src/paperless/src/manage.py\", line 18, in
"
2018-06-14 21:19:47,stderr,Traceback (most recent call last):

2018-06-14 21:19:46,stdout,PORT: '993'

2018-06-14 21:19:46,stdout,HOST: 'imap.gmail.com'

[danielquinn]

Excellent! So just to clarify, do you feel like this was a mistake on your part, or that the documentation wasn't clear regarding how to define the variables in this file?

If it's the latter, I'd appreciate it if you could fix the docs so that other people in your position don't have this happen again. Just send me a pull request and I'll accept :-)

[ahyear]

I submited a docker-compose.env update for that.
thank you

[danielquinn]

Hooray! Thank you :-) :heart:

[murmelbahn]

Sorry this might be a dumb question but how can I get the logs which @ahyear posted? I've the same error and I cant find the location of the log files of my Paperless docker version. :(

[bmsleight]

@murmelbahn try journalctl -u paperless-consumer