The digest emails are currently not being sent.
ArgumentError: wrong number of arguments (given 0, expected 1) is shown on Sidekiq dashboard (https://publiclab.org/sidekiq/retries).
ViditChitkara
All 33 comments
I dug up the log for this on the server too!
2018-09-23T17:01:50.908Z 1 TID-gogckya6x DigestMailJob JID-09573ed86ee039fe4c77e804 INFO: fail: 0.017 sec
2018-09-23T17:01:50.909Z 1 TID-gogckya6x WARN: {"context":"Job raised exception","job":{"class":"DigestMailJob","args":[],"retry":true,"queue":"default","jid":"09573ed86ee039fe4c77e804","created_at":1537644781.5130486,"enqueued_at":1537722110.8390274,"error_message":"wrong number of arguments (given 0, expected 1)","error_class":"ArgumentError","failed_at":1537644781.8256888,"retry_count":12,"retried_at":1537707061.948535},"jobstr":"{\"class\":\"DigestMailJob\",\"args\":[],\"retry\":true,\"queue\":\"default\",\"jid\":\"09573ed86ee039fe4c77e804\",\"created_at\":1537644781.5130486,\"enqueued_at\":1537722110.8390274,\"error_message\":\"wrong number of arguments (given 0, expected 1)\",\"error_class\":\"ArgumentError\",\"failed_at\":1537644781.8256888,\"retry_count\":12,\"retried_at\":1537707061.948535}"}
2018-09-23T17:01:50.910Z 1 TID-gogckya6x WARN: ArgumentError: wrong number of arguments (given 0, expected 1)
2018-09-23T17:01:50.910Z 1 TID-gogckya6x WARN: /app/app/jobs/digest_mail_job.rb:4:in `perform'
/usr/local/bundle/gems/sidekiq-5.2.1/lib/sidekiq/processor.rb:185:in `execute_job'
/usr/local/bundle/gems/sidekiq-5.2.1/lib/sidekiq/processor.rb:167:in `block (2 levels) in process'
/usr/local/bundle/gems/sidekiq-5.2.1/lib/sidekiq/middleware/chain.rb:128:in `block in invoke'
/usr/local/bundle/gems/sidekiq-5.2.1/lib/sidekiq/middleware/chain.rb:133:in `invoke'
/usr/local/bundle/gems/sidekiq-5.2.1/lib/sidekiq/processor.rb:166:in `block in process'
/usr/local/bundle/gems/sidekiq-5.2.1/lib/sidekiq/processor.rb:137:in `block (6 levels) in dispatch'
/usr/local/bundle/gems/sidekiq-5.2.1/lib/sidekiq/job_retry.rb:98:in `local'
/usr/local/bundle/gems/sidekiq-5.2.1/lib/sidekiq/processor.rb:136:in `block (5 levels) in dispatch'
/usr/local/bundle/gems/sidekiq-5.2.1/lib/sidekiq/rails.rb:42:in `block in call'
/usr/local/bundle/gems/activesupport-5.2.0/lib/active_support/execution_wrapper.rb:87:in `wrap'
/usr/local/bundle/gems/activesupport-5.2.0/lib/active_support/reloader.rb:73:in `block in wrap'
/usr/local/bundle/gems/activesupport-5.2.0/lib/active_support/execution_wrapper.rb:87:in `wrap'
/usr/local/bundle/gems/activesupport-5.2.0/lib/active_support/reloader.rb:72:in `wrap'
/usr/local/bundle/gems/sidekiq-5.2.1/lib/sidekiq/rails.rb:41:in `call'
/usr/local/bundle/gems/sidekiq-5.2.1/lib/sidekiq/processor.rb:132:in `block (4 levels) in dispatch'
/usr/local/bundle/gems/sidekiq-5.2.1/lib/sidekiq/processor.rb:217:in `stats'
/usr/local/bundle/gems/sidekiq-5.2.1/lib/sidekiq/processor.rb:127:in `block (3 levels) in dispatch'
/usr/local/bundle/gems/sidekiq-5.2.1/lib/sidekiq/job_logger.rb:8:in `call'
/usr/local/bundle/gems/sidekiq-5.2.1/lib/sidekiq/processor.rb:126:in `block (2 levels) in dispatch'
/usr/local/bundle/gems/sidekiq-5.2.1/lib/sidekiq/job_retry.rb:73:in `global'
/usr/local/bundle/gems/sidekiq-5.2.1/lib/sidekiq/processor.rb:125:in `block in dispatch'
/usr/local/bundle/gems/sidekiq-5.2.1/lib/sidekiq/logging.rb:48:in `with_context'
/usr/local/bundle/gems/sidekiq-5.2.1/lib/sidekiq/logging.rb:42:in `with_job_hash_context'
/usr/local/bundle/gems/sidekiq-5.2.1/lib/sidekiq/processor.rb:124:in `dispatch'
/usr/local/bundle/gems/sidekiq-5.2.1/lib/sidekiq/processor.rb:165:in `process'
/usr/local/bundle/gems/sidekiq-5.2.1/lib/sidekiq/processor.rb:83:in `process_one'
/usr/local/bundle/gems/sidekiq-5.2.1/lib/sidekiq/processor.rb:71:in `run'
/usr/local/bundle/gems/sidekiq-5.2.1/lib/sidekiq/util.rb:16:in `watchdog'
/usr/local/bundle/gems/sidekiq-5.2.1/lib/sidekiq/util.rb:25:in `block in safe_thread'
Key section:
2018-09-23T17:01:50.910Z 1 TID-gogckya6x WARN: ArgumentError: wrong number of arguments (given 0, expected 1)
2018-09-23T17:01:50.910Z 1 TID-gogckya6x WARN: /app/app/jobs/digest_mail_job.rb:4:in `perform'
It looks like we are asking for a parameter in peform(frequency) but when it's called it's not getting the parameter
Vidit, do you know where perform() is called from and why it doesn't have a frequency param set?
jywarren
on 23 Sep 2018
The perform_async (does same work as perform) function is called in schedule.rb, I guess maybe some sort of syntactical error
https://github.com/publiclab/plots2/blob/2522459bfdc21068df2e0e808a8addc6726808e0/config/schedule.rb#L31-L37
ViditChitkara
on 23 Sep 2018
Digging out on this. Can we make this a fto issue?
ViditChitkara
on 23 Sep 2018
If you can identify the change we should make to fix it, and it's only a
few lines, then yeah, I think it'd make a great FTO!
On Sun, Sep 23, 2018, 1:23 PM Vidit notifications@github.com wrote:
Digging out on this. Can we make this a fto issue?
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/publiclab/plots2/issues/3404#issuecomment-423832510,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AABfJ4vZUdNp1ymRSClv7Qq0vZ8sJkvWks5ud8NvgaJpZM4W1uzK
.
jywarren
on 23 Sep 2018
I guess, there isn't any syntactical error. But the cronjobs are not updating according to schedule.rb.
@icarito do we need to specify environment in makefile when we deploy the latest code?
I did whenever --update-crontab --set environment=development to make the it work. I believe the same command needs to be done in the makefile with production instead.
https://github.com/publiclab/plots2/blob/2522459bfdc21068df2e0e808a8addc6726808e0/Makefile#L28
ViditChitkara
on 23 Sep 2018
Hmm, were you able to get past the "0 parameters where expected 1" error?
On Sun, Sep 23, 2018, 2:59 PM Vidit notifications@github.com wrote:
I guess, there isn't any syntactical error. But the cronjobs are not
updating according to schedule.rb.
@icarito https://github.com/icarito do we need to specify environment
in makefile when we deploy the latest code?I did whenever --update-crontab --set environment=development to make the
it work. I believe the same command needs to be done in the makefile with
production instead.https://github.com/publiclab/plots2/blob/2522459bfdc21068df2e0e808a8addc6726808e0/Makefile#L28
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/publiclab/plots2/issues/3404#issuecomment-423839176,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AABfJzpC5nFopzc1FpxllMO8B7h4i7hzks5ud9nigaJpZM4W1uzK
.
jywarren
on 23 Sep 2018
yeah I tested locally with
every 1.minutes do
runner "DigestMailJob.perform_async"
end
and
every 1.minutes do
runner "DigestMailJob.perform_async(0)"
end
The same error occurred for the first one and worked perfectly for the latter one.
ViditChitkara
on 23 Sep 2018
Ok, so do you know where we can insert the right 0 parameter when process()
is called?
On Sun, Sep 23, 2018, 3:16 PM Vidit notifications@github.com wrote:
yeah I tested locally with
every 1.minutes do runner "DigestMailJob.perform_async" end
and
every 1.minutes do runner "DigestMailJob.perform_async(0)" endThe same error occurred for the first one and worked perfectly for the
latter one.—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/publiclab/plots2/issues/3404#issuecomment-423840502,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AABfJ-1ooB2OI1ybg-rm0H1mofrGjv9bks5ud94EgaJpZM4W1uzK
.
jywarren
on 23 Sep 2018
Its actually there in the code, right here:-
https://github.com/publiclab/plots2/blob/2522459bfdc21068df2e0e808a8addc6726808e0/config/schedule.rb#L31-L37
but the problem I guess is that, the crontab isn't updated after this code was deployed. It still assumes that no argument is passed with the function. Does this make sense?
ViditChitkara
on 23 Sep 2018
Ah, ok! Cool, do we track the crontab settings in git or is there a
reference to compare the production settings against?
Super it looks like you've got this going. Thanks!
On Sun, Sep 23, 2018, 3:30 PM Vidit notifications@github.com wrote:
Its actually there in the code, right here:-
https://github.com/publiclab/plots2/blob/2522459bfdc21068df2e0e808a8addc6726808e0/config/schedule.rb#L31-L37
but the problem I guess is that, the crontab isn't updated after this code
was deployed. It still assumes that no argument is passed with the
function. Does this make sense?—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/publiclab/plots2/issues/3404#issuecomment-423841549,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AABfJ6rlxY85c1oUVgWgHuZbM4pQRZH3ks5ud-E1gaJpZM4W1uzK
.
jywarren
on 23 Sep 2018
I guess we can't directly view the cronjobs which are scheduled. It can be viewed by logging on the server only. All we need to do is make the update-crontab command run on production. Needs some more digging though. Will look into it.
ViditChitkara
on 24 Sep 2018
I can log in and look - but what should i be looking for? @icarito would
you mind trying to coordinate with Vidit on this one? Thanks!
On Mon, Sep 24, 2018 at 12:10 AM Vidit notifications@github.com wrote:
I guess we can't directly view the cronjobs which are scheduled. It can be
viewed by logging on the server only. All we need to do is make the
update-crontab command run on production. Needs some more digging though.
Will look into it.—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/publiclab/plots2/issues/3404#issuecomment-423873363,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AABfJ_NKu3qnRESYG1BkP75Vif6RGDQ5ks5ueFr2gaJpZM4W1uzK
.
jywarren
on 24 Sep 2018
Hi @ViditChitkara - I had missed this one! When does update-crontab need to run? currently we run it on each redeployment: https://github.com/publiclab/plots2/blob/master/Makefile#L17
icarito
on 15 Oct 2018
Re reading I see you've been thru the Makefile already. The Makefile is run with environment variables including RAILS_ENV=production.
I'll list the crontab of web container in production...
icarito
on 15 Oct 2018
$ RAILS_ENV=production docker-compose -f containers/docker-compose-production.yml exec web crontab -l
# Begin Whenever generated tasks for: /app/config/schedule.rb at: 2018-10-04 21:39:15 +0000
* * * * * /bin/bash -l -c 'date -u >> /app/public/cron_log.log 2>&1'
0 0 * * * /bin/bash -l -c 'cd /app && bundle exec bin/rails runner -e production '\''DigestMailJob.perform_async(0)'\'' >> /app/public/cron_log.log 2>&1'
0 0 1,8,15,22 * * /bin/bash -l -c 'cd /app && bundle exec bin/rails runner -e production '\''DigestMailJob.perform_async(1)'\'' >> /app/public/cron_log.log 2>&1'
# End Whenever generated tasks for: /app/config/schedule.rb at: 2018-10-04 21:39:15 +0000
$ RAILS_ENV=production docker-compose -f containers/docker-compose-production.yml exec web service cron status
[ ok ] cron is running.
Ah ok so can we figure out how the crontab was originally set and then
update it to include the new parameter format?
On Mon, Oct 15, 2018, 12:29 PM Sebastian Silva notifications@github.com
wrote:
$ RAILS_ENV=production docker-compose -f containers/docker-compose-production.yml exec web service cron status
[ ok ] cron is running.—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/publiclab/plots2/issues/3404#issuecomment-429705119,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AABfJwOP1BBdklOCZ4f8v_uZDfO9o7_Cks5ulA8AgaJpZM4W1uzK
.
jywarren
on 17 Oct 2018
OK, i changed the test digest code to:
DigestMailJob.perform_async(0) where it had lacked the parameter. Now we have to figure out why the daily/weekly ones aren't running, which should be a different issue.
I tried running DigestMailJob.perform_async(0) on the rails console on the production machine, in production mode. But didn't get an email. Looking at logs now.
jywarren
on 14 Dec 2018
OK, i ran user.send_digest_email
Rendered subscription_mailer/send_digest.html.erb (508.9ms)
SubscriptionMailer#send_digest: processed outbound mail in 557.7ms
ActionView::Template::Error: undefined local variable or method `node' for #<#<Class:0x000055b3ad5f1598>:0x000055b3ad5af238>
from app/views/subscription_mailer/send_digest.html.erb:33:in `block in _app_views_subscription_mailer_send_digest_html_erb___3263606767765639606_47115098102720'
from app/views/subscription_mailer/send_digest.html.erb:10:in `_app_views_subscription_mailer_send_digest_html_erb___3263606767765639606_47115098102720'
from app/mailers/subscription_mailer.rb:61:in `send_digest'
from app/models/user.rb:404:in `send_digest_email'
from (irb):5
We should be testing send_digest_email!
jywarren
on 14 Dec 2018
Also I wonder if there's a version mismatch on the ActiveJob we're working with... it says there is no method perform_later although the latest Rails Guides say that should work... i likewise don't see docs for perform_async in the current rails guids: https://guides.rubyonrails.org/active_job_basics.html
jywarren
on 14 Dec 2018
OK i think i solved the template error above in 7ab9f68
jywarren
on 14 Dec 2018
Made a test for user.send_digest_email in https://github.com/publiclab/plots2/pull/4305 !
jywarren
on 14 Dec 2018
Digest emails now send when we press the test button on profile pages!
Later: fix formatting/images/bio profile pics of the digest!
Scheduled sends don't work - for some reason, the setting at https://publiclab.org/settings keeps getting switched off! It stays on for a little while but then gets disabled the next day... not sure how this happens!
jywarren
on 3 Jan 2019
OK, attempted fix in https://github.com/publiclab/plots2/pull/4523 -- it looks like the code was deleting ALL digest user tags for all users, rather than just for one user! Eek!
Hopefully this fixes it!
jywarren
on 5 Jan 2019
Digests still aren't sending, oddly. Need to look at logs more. The setting itself is all good now, i think. We just need to know why it's not triggering each day.
jywarren
on 10 Jan 2019
It looks like the whenever runner is not finding the gems, because cron gives it a clean environment ( without some environment variables we need ). I've attempted a fix with https://github.com/publiclab/plots2/pull/4619/commits . How do you test it?
icarito
on 15 Jan 2019
Maybe some clues here?
https://github.com/javan/whenever/search?utf8=%E2%9C%93&q=environment&type=
Seems there are some ways to set it...
On Mon, Jan 14, 2019 at 11:34 PM Sebastian Silva notifications@github.com
wrote:
It looks like the whenever runner is not finding the gems, because cron
gives it a clean environment ( without some environment variables we need
). I've attempted a fix with
https://github.com/publiclab/plots2/pull/4619/commits . How do you test
it?—
You are receiving this because you modified the open/close state.
Reply to this email directly, view it on GitHub
https://github.com/publiclab/plots2/issues/3404#issuecomment-454264308,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AABfJ9biQtEfO_qLd3ULXKMpndDJKnXiks5vDVpTgaJpZM4W1uzK
.
jywarren
on 17 Jan 2019
ah! what about:
https://github.com/javan/whenever/issues/732#issuecomment-404576122
On Wed, Jan 16, 2019 at 7:05 PM Jeffrey Warren jeff@unterbahn.com wrote:
Maybe some clues here?
https://github.com/javan/whenever/search?utf8=%E2%9C%93&q=environment&type=Seems there are some ways to set it...
On Mon, Jan 14, 2019 at 11:34 PM Sebastian Silva notifications@github.com
wrote:It looks like the whenever runner is not finding the gems, because cron
gives it a clean environment ( without some environment variables we need
). I've attempted a fix with
https://github.com/publiclab/plots2/pull/4619/commits . How do you test
it?—
You are receiving this because you modified the open/close state.
Reply to this email directly, view it on GitHub
https://github.com/publiclab/plots2/issues/3404#issuecomment-454264308,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AABfJ9biQtEfO_qLd3ULXKMpndDJKnXiks5vDVpTgaJpZM4W1uzK
.
jywarren
on 17 Jan 2019
Trying it now at https://github.com/publiclab/plots2/pull/4619, pushed to unstable
icarito
on 17 Jan 2019
I tried a couple of alternatives https://github.com/publiclab/plots2/pull/4619/commits/4288bf797b3b249d97a51e0ca4135f2fab1e11a7 adding to config/schedule.rb but didn't succeed yet. Looking for ideas!
icarito
on 17 Jan 2019
Hi!,
I've pinpointed the issue, child jobs don't inherit the app environment so we have to pass variables like this:
https://github.com/publiclab/plots2/pull/4619/files
With these variables (PATH, GEM_HOME, SECRET_KEY_BASE, and REDIS_URL), I don't get a trace in unstable, but no email attempt either. Perhaps more environment variables are needed?
We're so close! Please merge #4619 and let's see!
icarito
on 22 Jan 2019
2019-01-26 00:00:04 1gnBOG-0004uP-2Z <= [email protected] H=(tycho2) [162.242.250.44] P=esmtp S=1635 id=E1gnBOF-0001Ue-UZ@tycho2 2019-01-26 00:00:04 1gnBOG-0004uP-2Z ** [email protected] R=dnslookup T=remote_smtp H=mail.mapknitter.org [216.70.64.217] X=TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256 CV=no DN="OU=Domain Control Validated,CN=*.accessdomain.com": SMTP error from remote mail server after RCPT TO:<[email protected]>: 550 Unrouteable address
2019-01-26 00:00:04 1gnBOG-0004uS-H9 <= <> R=1gnBOG-0004uP-2Z U=Debian-exim P=local S=3050
2019-01-26 00:00:04 1gnBOG-0004uP-2Z Completed
2019-01-26 00:00:04 1gnBOG-0004uS-H9 ** [email protected] R=dnslookup T=remote_smtp H=mail.mapknitter.org [216.70.64.217] X=TL
S1.2:ECDHE_RSA_AES_256_GCM_SHA384:256 CV=no DN="OU=Domain Control Validated,CN=*.accessdomain.com": SMTP error from remote mail se
rver after RCPT TO:<[email protected]>: 550 Unrouteable address 2019-01-26 00:00:04 1gnBOG-0004uS-H9 Frozen (delivery error message) 2019-01-26 00:00:21 1gnBOX-0004v7-1J <= [email protected] H=(localhost.localdomain) [172.27.0.5] P=esmtp S=2818 [email protected]
2019-01-26 00:00:22 1gnBOY-0004vE-3m <= [email protected] H=(localhost.localdomain) [172.27.0.5] P=esmtp S=10122 id=5c4b
[email protected] 2019-01-26 00:00:22 1gnBOY-0004vI-CC <= [email protected] H=(localhost.localdomain) [172.27.0.5] P=esmtp S=2572 [email protected] 2019-01-26 00:00:22 1gnBOY-0004vE-3m => [email protected] R=dnslookup T=remote_smtp H=ASPMX.L.GOOGLE.COM [172.217.197.27] X=TLS1.2:ECDHE_RSA_CHACHA20_POLY1305:256 CV=yes DN="C=US,ST=California,L=Mountain View,O=Google LLC,CN=mx.google.com" C="250 2.0.0 OK 154
8460822 d22si526751qvf.130 - gsmtp" 2019-01-26 00:00:22 1gnBOY-0004vE-3m Completed
Was this it? :blush:
icarito
on 26 Jan 2019
Yes!!!!!!
On Fri, Jan 25, 2019, 10:28 PM Sebastian Silva <[email protected]
wrote:
2019-01-26 00:00:04 1gnBOG-0004uP-2Z <= [email protected] H=(tycho2) [162.242.250.44] P=esmtp S=1635 id=E1gnBOF-0001Ue-UZ@tycho2 2019-01-26 00:00:04 1gnBOG-0004uP-2Z * [email protected] R=dnslookup T=remote_smtp H=mail.mapknitter.org [216.70.64.217] X=TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256 CV=no DN="OU=Domain Control Validated,CN=.accessdomain.com": SMTP error from remote mail server after RCPT TO:warren@mapknitter.org: 550 Unrouteable address
2019-01-26 00:00:04 1gnBOG-0004uS-H9 <= <> R=1gnBOG-0004uP-2Z U=Debian-exim P=local S=3050
2019-01-26 00:00:04 1gnBOG-0004uP-2Z Completed
2019-01-26 00:00:04 1gnBOG-0004uS-H9 ** [email protected] R=dnslookup T=remote_smtp H=mail.mapknitter.org [216.70.64.217] X=TL
S1.2:ECDHE_RSA_AES_256_GCM_SHA384:256 CV=no DN="OU=Domain Control Validated,CN=*.accessdomain.com": SMTP error from remote mail se
rver after RCPT TO:warren@mapknitter.org: 550 Unrouteable address 2019-01-26 00:00:04 1gnBOG-0004uS-H9 Frozen (delivery error message) 2019-01-26 00:00:21 1gnBOX-0004v7-1J <= [email protected] H=(localhost.localdomain) [172.27.0.5] P=esmtp S=2818 [email protected]
2019-01-26 00:00:22 1gnBOY-0004vE-3m <= [email protected] H=(localhost.localdomain) [172.27.0.5] P=esmtp S=10122 id=5c4b
[email protected] 2019-01-26 00:00:22 1gnBOY-0004vI-CC <= [email protected] H=(localhost.localdomain) [172.27.0.5] P=esmtp S=2572 [email protected] 2019-01-26 00:00:22 1gnBOY-0004vE-3m => [email protected] R=dnslookup T=remote_smtp H=ASPMX.L.GOOGLE.COM [172.217.197.27] X=TLS1.2:ECDHE_RSA_CHACHA20_POLY1305:256 CV=yes DN="C=US,ST=California,L=Mountain View,O=Google LLC,CN=mx.google.com" C="250 2.0.0 OK 154
8460822 d22si526751qvf.130 - gsmtp" 2019-01-26 00:00:22 1gnBOY-0004vE-3m Completed
Was this it? 😊
—
You are receiving this because you modified the open/close state.
Reply to this email directly, view it on GitHub
https://github.com/publiclab/plots2/issues/3404#issuecomment-457788220,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AABfJ8hJbREDavOh6yWxOgIEWA32qCaBks5vG68OgaJpZM4W1uzK
.
jywarren
on 26 Jan 2019
Related issues
grvsachdeva
·
3Comments
keshavsethi
·
3Comments
![first-timers[bot] picture](https://avatars.githubusercontent.com/in/4832?v=4&s=40)
first-timers[bot]
·
3Comments
grvsachdeva
·
3Comments
first-timers[bot]
·
3Comments