Rq: ImportError: No module named project.controllers.leads_fb

[mateocam]

Hello !

I am having issues with the rq worker.

This is the structure of my app:

app/
    run.py
    project/
        __init__.py
        clock.py
        views.py
        worker.py
        controllers/
           __init__.py
           leads_fb.py

clock is executed once I run my app and this is the code:

from apscheduler.schedulers.background import BackgroundScheduler
from rq import Queue, get_current_job
from rq.job import Job 
from worker import conn
from controllers import leads_fb
import logging
import os


q = Queue(connection=conn)
sched = BackgroundScheduler()

def enque_jobs():
    info = '1aooEOTBQz1pAW915-JonL3Jh2mhclR5EZTHGotFBe7s||blah blah blah'
    job = q.enqueue(leads_fb.prueba, info)  
    print 'This job is run every 3 seconds.'

if os.environ.get("WERKZEUG_RUN_MAIN") == "true":
    sched.add_job(enque_jobs, 'interval', seconds=3)
    sched.start()

log = logging.getLogger('apscheduler.executors.default')
# log.setLevel(logging.INFO)  # DEBUG

# fmt = logging.Formatter('%(levelname)s:%(name)s:%(message)s')
h = logging.StreamHandler()
# h.setFormatter(fmt)
log.addHandler(h)

then clock enqueues jobs with the rq worker which is worker.py:

import os

import redis
from rq import Worker, Queue, Connection

listen = ['high', 'default', 'low']

redis_url = os.getenv('REDISTOGO_URL', 'redis://localhost:6379')

conn = redis.from_url(redis_url)

if __name__ == '__main__':
    with Connection(conn):
        worker = Worker(map(Queue, listen))
        worker.work()

and the function that the queued job is execution is controllers/leads_fb.py:

from wksh import init_wksh, get_last_row, write_in_cell

def prueba(info):
    print 'aaaaa'
    info = info.split('||')
    id_client = info[0]
    str_prueba = info[1]
    wksh = init_wksh(id_client)
    print wksh
    row = get_last_row(wksh)

    write_in_cell(wksh,'A'+str(row),str_prueba)
    print 'funciona  ' + str(row)

when I run my app with the run.py file in the root directory it gives me the error: ImportError: No module named project.controllers.leads_fb

my run.py file has this code:

from project import app
from project import worker
from project import clock

if __name__ == "__main__":
    app.run(debug=True,host='127.0.0.1', port=1234)

and my app/project/__init__.py file has this

from flask import Flask

app = Flask(__name__)
app.config.from_object('config')

from . import views
from . import worker
from . import clock

Please help!! I've been stuck with this for a week and can't manage to fix this. I have read other thread issues and have come to understand it has to do something with the PYTHONPATH that points my module directory to my rq worker but I have no idea how to implement this. @selwin

[mateocam]

how can I add the import path for my worker.py file? I have tested and if I start the worker in the root directory with the command line 'rqworker' it works fine. My app thinks that the worker.py file is in the root directory, but it's actually in project/ inside the root directory. How and where do I specify this so it works locally and also deployed in Heroku?

[mateocam]

how can I add the import path for my worker.py file? I have tested and if I start the worker in the root directory with the command line 'rqworker' it works fine. My app thinks that the worker.py file is in the root directory, but it's actually in project/ inside the root directory. How and where do I specify this so it works locally and also deployed in Heroku? @nvie I have searched rq's documentation and I can't find it! please help.

[selwin]

You can call the rq worker command with --path argument.

[mateocam]

@selwin I am calling the worker this way: rq worker --path project/
and it still doesn't work. I am calling the worker in the root directory of my project. what am I missing?

[rizplate]

@mateocam it is just regular python sys path module import stuff. you just have to run it from correct directory or install your package to system libraries.
Please put your example code with a single bash script to start your worker, somewhere on github, so that someone can clone it and get it working for you