Parse-server: 2.6.5 not sending pushes, went back to 2.4.2, pushes are ok

Created on 3 Nov 2017  ·  87Comments  ·  Source: parse-community/parse-server

Issue Description

Updated to 2.6.5, pushes not working, i'm having problems with pushes:

Went back to 2.4.2:

npm install [email protected] --save

everything is working well

I've seen #4256, and I've seen this comment, which might be related to my problems:

hey @flovilmart are you saying this will be updated in 2.6.4?
@jeffreyjackson yes!

Expected Results

I'd like to have most recent version, cause I feel like lack of performance issues updates in last versions.

Environment Setup

  • Server

    • parse-server version: 2.6.5

    • Operating System: ubuntu16.04-x86_64

Logs/Trace

0|parse-wr | verbose: _PushStatus n59yItvDR0: sending push to 1 installations
0|parse-wr | verbose: Sending push to 1
0|parse-wr | verbose: _PushStatus n59yItvDR0: sent push! 0 success, 1 failures
0|parse-wr | verbose: _PushStatus n59yItvDR0: needs cleanup devicesToRemove=[]
stale

All 87 comments

Can you provide your push configuration please?

Over on SO there is lots of chatter about APNS arbitrarily throttling delivery of data pushes and a new process DuetActivitySchedulerDaemon that can intercept a push notification that has been successfully delivered to your device and prevent it from being delivered to your appDelegate based on things like frequency of the silent notifications and battery state.

I spent two days trying to figure out why my server's silent push notifications would mysteriously stop being received by my test devices until I restarted them. I haven't 100% verified that this is the cause but if you are using silent notifications to trigger activity you need to handle the use case that they could be throttled by APNS or blocked at the device which means your appDelegate may never see them.

What's bugging me is that the push adapter is not reporting/logging the error when sending the push. As you're running with VERBOSE=1, and the failure was related to APNS not being able to transmit, it should be logged. There isn't any other logs related to the parse-server-push-adapter? Can you try the same running the sever locally and targeting the push with your test device? (also, provide your push configuration please).

I'm running with VERBOSE=1

My push configuration is:

push: {
ios: {
pfx: "/path/to/certs/file.p12",
passphrase: "",
bundleId: "com.company.AppName",
production: true
}
},

I'll try running it locally, and report my results. Thank you in advance!

Small point not directly related bundleId is deprecated and replaced with topic.

brianyyz, thanks for reply, yeah, I have had deprecation warning, but I already tried using topic instead of bundleId, same story...

@dongalor do you have some more logs for us?

@dongalor would you be able to share the code that is creating the push payload as well ?

Here's my code, which I use to test my pushes:
https://gist.github.com/dongalor/96d13ee1b0344d222f4a4caf899bd8ed

this exact code is working well on [email protected]

If you run the query on itself does it work? ie: does it return installations?

I have the same problem here @dongalor. What is the database you are using?

My problem is because Im using Postgres. The 2.6.5 version support "expiration_interval" now. When we update the version, the column in _PushStatus is missing then, the push notification stops. I added the column manually and works now.

When schema change, we need a better option to solve problems like this.

The log trace:

2017-11-10T18:53:32.413657765Z error: _PushStatus undefined: error while sending push error: column "expiration_interval" of relation "_PushStatus" does not exist
2017-11-10T18:53:32.413692926Z     at Connection.parseE (/parse-server/node_modules/pg/lib/connection.js:546:11)
2017-11-10T18:53:32.413700050Z     at Connection.parseMessage (/parse-server/node_modules/pg/lib/connection.js:371:19)
2017-11-10T18:53:32.413705701Z     at TLSSocket.<anonymous> (/parse-server/node_modules/pg/lib/connection.js:114:22)
2017-11-10T18:53:32.413711165Z     at emitOne (events.js:96:13)
2017-11-10T18:53:32.413715592Z     at TLSSocket.emit (events.js:188:7)
2017-11-10T18:53:32.413720755Z     at readableAddChunk (_stream_readable.js:176:18)
2017-11-10T18:53:32.413725436Z     at TLSSocket.Readable.push (_stream_readable.js:134:10)
2017-11-10T18:53:32.413733407Z     at TLSWrap.onread (net.js:547:20)

@paulovitin If that's the case using 2.6.3 would work, as the expiration_interval was introduced in 2.6.5 (2.6.4 has a CLI bug in it but you could use that too). I would think that the schema would have been auto-updated on start though.

@montymxb we have a docker auto-redeploy, and the auto-updated for schema doesnt happened.

I'll take a look and see if I can replicate this upgrade issue regarding expiration_interval.

@paulovitin I can confirm that there is a Schema upgrade issue, but only in postgres. Mongodb looks to handle it properly. I'm going to open up a separate issue with further details on that specifically, including reproducability.

@dongalor if you're using postgres as your database it's possible that you're experiencing the same issue post-upgrade.

::EDIT:: Postgres upgrade issue opened under #4338

@flovilmart > If you run the query on itself does it work? ie: does it return installations?
yes

@montymxb, @paulovitin
no, I'm using mongodb :( that's sad

at this moment i'm still using 2.4.2, i wish I could come back to 2.6.5 :)

Ok, I may have an idea, to be clear, your users hold the deviceTokens as well as the installation right?

installationId? yes!

@dongalor the gist you provided don’t mention installationId at all, just matching on device token.

Also, just a note on architecture, usually a user can have many installation, so I would recommend you set the user on the installation objects, instead of storing the installationId’s on the users.

@flovilmart sorry, now I understand the question, do my users table hold installationId? No, but I have system table Installation, which is being filled automatically.

So, my users table doesn't hold installationId, but my users table hold deviceToken field.

Ok so the gist is correct, I'll have a look.

So far, I can pin point the issue in the push adapter, as everything in parse-server is going smooth. It is an issue with all your push’s, or just some?

@flovilmart actually i've provided my gist (https://gist.github.com/dongalor/96d13ee1b0344d222f4a4caf899bd8ed), all other pushes use the same Push class, so none of them work (

also when I downgrade to 2.4.2, everything works fine..

Did you try sending a push through the dashboard?

@flovilmart yeah, same result :( is there anything i can show you in my configuration to check?

Ok, the issue seem to lie in the push adapter itself, I need more time to investigate the issue.

if i go back to 2.4.2 , do i have to downgrade parse-dashboard as well to work well sending pushes throught it ?

npm install [email protected] --save

@hopye actually this is the only command I used to downgrade parse without downgrading dashboard, I guess there's no need in additional actions, also I think there are dependencies between dashboard and parse-server

We are also having problems with push for 2.6.5. There are constant error messages from parse-push-adapter and push are sometime delayed rather significantly (a few hours). However, our users can still receive push notifications, both iOS and Android devices.

The errosr we get look like:

node-pre-gyp ERR! parse-server-push-adapter APNS VError: apn write failed: Not receiving Ping response after 150000 ms
node-pre-gyp ERR! parse-server-push-adapter APNS     at Stream.stream.on.err (/app/node_modules/parse-server/node_modules/apn/lib/client.js:105:21)
node-pre-gyp ERR! parse-server-push-adapter APNS     at emitOne (events.js:96:13)
node-pre-gyp ERR! parse-server-push-adapter APNS     at Stream.emit (events.js:188:7)
node-pre-gyp ERR! parse-server-push-adapter APNS     at _connection._streamIds.forEach (/app/node_modules/parse-server/node_modules/apn/lib/protocol/endpoint.js:155:16)
node-pre-gyp ERR! parse-server-push-adapter APNS     at Array.forEach (native)
node-pre-gyp ERR! parse-server-push-adapter APNS     at EventEmitter.close [as _close] (/app/node_modules/parse-server/node_modules/apn/lib/protocol/endpoint.js:139:33)
node-pre-gyp ERR! parse-server-push-adapter APNS     at emitOne (events.js:101:20)
node-pre-gyp ERR! parse-server-push-adapter APNS     at TLSSocket.emit (events.js:188:7)
node-pre-gyp ERR! parse-server-push-adapter APNS     at _handle.close (net.js:497:12)
node-pre-gyp ERR! parse-server-push-adapter APNS     at TCP.done [as _onclose] (_tls_wrap.js:332:7)

and

node-pre-gyp ERR! parse-server-push-adapter APNS  APNS error transmitting to device %s with error %s the_device_token { VError: apn write failed: Not receiving Ping response after 150000 ms
node-pre-gyp ERR! parse-server-push-adapter APNS     at Stream.stream.on.err (/app/node_modules/parse-server/node_modules/apn/lib/client.js:105:21)
node-pre-gyp ERR! parse-server-push-adapter APNS     at emitOne (events.js:96:13)
node-pre-gyp ERR! parse-server-push-adapter APNS     at Stream.emit (events.js:188:7)
node-pre-gyp ERR! parse-server-push-adapter APNS     at _connection._streamIds.forEach (/app/node_modules/parse-server/node_modules/apn/lib/protocol/endpoint.js:155:16)
node-pre-gyp ERR! parse-server-push-adapter APNS     at Array.forEach (native)
node-pre-gyp ERR! parse-server-push-adapter APNS     at EventEmitter.close [as _close] (/app/node_modules/parse-server/node_modules/apn/lib/protocol/endpoint.js:139:33)
node-pre-gyp ERR! parse-server-push-adapter APNS     at emitOne (events.js:101:20)
node-pre-gyp ERR! parse-server-push-adapter APNS     at TLSSocket.emit (events.js:188:7)
node-pre-gyp ERR! parse-server-push-adapter APNS     at _handle.close (net.js:497:12)
node-pre-gyp ERR! parse-server-push-adapter APNS     at TCP.done [as _onclose] (_tls_wrap.js:332:7)

@promisenxu I also see this issue. Looks like a bug in node-apn: https://github.com/node-apn/node-apn/issues/583

@promisenxu so those are probably false positives, as the error occur but comes from the ping, not the delivery itself.

I am tinkering with a workaround for that issue.

@dongalor @flovilmart
you could check Installation.appIdentifier.
https://github.com/parse-community/parse-server-push-adapter/blob/8cfc7fea2a405743f35c2b0b15c641f325ddda80/src/APNS.js#L237

 _chooseProviders(appIdentifier) {
    // If the device we need to send to does not have appIdentifier, any provider could be a qualified provider
    /*if (!appIdentifier || appIdentifier === '') {
        return this.providers.map((provider) => provider.index);
    }*/

    // Otherwise we try to match the appIdentifier with topic on provider
    let qualifiedProviders = this.providers.filter((provider) => appIdentifier === provider.topic);

    if (qualifiedProviders.length > 0) {
      return qualifiedProviders;
    }

    // If qualifiedProviders empty, add all providers without topic
    return this.providers
      .filter((provider) => !provider.topic || provider.topic === '');
  }

I think it's related here. if Installation.appIdentifier is empty,this method will choose providers without topic.apprently,notification can't send without topic successfuly.

so, parse looks like updated to 2.7.1, is it possible, that this issue is included in this update? Or this is only parse-push-adapter related?

bump :) any news about pushes on new version?

tried upgrading to 2.7.4 and still got no pushes :( what am i doing wrong?
2.4.2 is working like a charm, but i need compound queries related to #4399 which are not available on 2.4.2

Logs:

verbose: _PushStatus 9H6Rd7TWUE: sending push to installations with 1 batches
verbose: Sending push to 1
verbose: _PushStatus 9H6Rd7TWUE: sent push! 0 success, 1 failures
verbose: _PushStatus 9H6Rd7TWUE: needs cleanup devicesToRemove=[]

how are your push configured?

bump, sorry for bothering
i lack compound queries by objectId soooo much )

I have no idea what can be wrong. Are the queries properly returning the installations?

FWIW I'm 2.6.5 in production and no problems with silent and user visible push notifications. Happy to share code or config if it helps.

We’re also not seeing any issue on the latest version

Could this be a prod versus dev APNS config ?

Hey brian,

What version are you using ? Please share config code anyways...

Lg

Sent from my iPhone 6 Plus

On Apr 30, 2018, at 8:26 AM, Brian notifications@github.com wrote:

Could this be a prod versus dev APNS config ?


You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub, or mute the thread.

Here you go

// Example express application adding the parse-server module to expose Parse
// compatible API routes.

var express = require('express');
var ParseServer = require('parse-server').ParseServer;
var path = require('path');

var databaseUri = process.env.DATABASE_URI || process.env.MONGODB_URI;

if (!databaseUri) {
  console.log('DATABASE_URI not specified, falling back to localhost.');
} 

var api = new ParseServer({
  databaseURI: databaseUri                        || 'mongodb://localhost:27017/dev',
  appId: process.env.APP_ID                       || 'myAppId',
  masterKey: process.env.MASTER_KEY               || '', 
  clientKey: process.env.CLIENT_KEY               || '', 
  restAPIKey: process.env.REST_API_KEY            || '', 
  cloud: process.env.CLOUD_CODE_MAIN              || __dirname + '/cloud/main.js',
  serverURL: process.env.SERVER_URL               || 'http://localhost:1337/parse',  
  publicServerURL: process.env.PUBLIC_SERVER_URL  || "http://localhost:1337/parse",
  appName: process.env.APP_NAME                   || "myAppName",
  userSensitiveFields: ["email"],
  allowClientClassCreation: process.env.ALLOW_CLIENT_CLASS_CREATION                 || false,
  verifyUserEmails: process.env.VERIFY_USER_EMAILS                                  || true,
  preventLoginWithUnverifiedEmail: process.env.PREVENT_LOGIN_WITH_UNVERIFIED_EMAIL  || false,
  verbose: process.env.VERBOSE                                                      || false,
  push: {
      ios: [
        {
          pfx: './certs/APNSProductionCertificate.p12',
          topic: 'edu.self.brianmacdonald.xxxxx',
          production: true
        },
        {
          pfx: './certs/APNSDevelopmentCertificate.p12',
          topic: 'edu.self.brianmacdonald.xxxxx',
          production: false
        }
      ]
    },
   // The email adapter
  emailAdapter: {
    module: 'parse-server-simple-mailgun-adapter',
    options: {
      // The address that your emails come from
      fromAddress: '[email protected]',
      // Your domain from mailgun.com
      domain: 'mg.xxxxxxxx.ca',
      // Your API key from mailgun.com
      apiKey: process.env.MAILGUN_PRIVATE_API_KEY     || 'myMailgunAPIKey'
    }
  }
});

var app = express();

// Serve the Parse API on the /parse URL prefix
var mountPath = process.env.PARSE_MOUNT || '/parse';
app.use(mountPath, api);

var port = process.env.PORT || 1337;
var httpServer = require('http').createServer(app);
httpServer.listen(port, function() {
    console.log('parse-server running on port ' + port + '.');
});

This could very well be a prod vs dev thing indeed. Do you have anything working at all? Dev certs to dev devices / prod cert to prod devices ?

I think it could also be a provisioning profile thing on iOS. If you're not signing the app with a provisioning profile that matches the certs, I don't think you get push notifications. I've found that lately Xcode makes me sign in again every 3 or so days, used to last a month+ before I switched to High Sierra.

@brianyyz thanks for sharing config, the only difference with my config is empty passphrase in my config : (

@flovilmart thanks for reply, I don't have dev push configuration, actually I'm deploying this production push configuration code to production

@JacobJT thank you for your information, I've forwarded it to my IOS developers, signing might be an issue :)

maybe i can show you my config, skipped sensitive information index.js:

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

Actually, I'm still unable to move to newer version :(

Did you try v3.0? At this point besides a bad configuration? We have no idea what’s wrong

@flovilmart thank you, I'll try 3.0.
Also, I've shown my configuration push cut:
https://gist.github.com/dongalor/c1ed9443d37430dc729d6fdaab7513fa

Maybe a long shot but looking at your config you have an absolute path specified for your certificate. On you hosted server is this a valid path? When ever I've used push on hosted servers e.g. Heroku I've used relative path like './certs/12345.p12' and I've never included passphrase.

@dongalor Did you resolve this? I have the same issue

As mentioned above pushes are working for me on 3.0 both locally via ngrok and on remote Heroku server.

I also see no issues with push notifications running the latest parse server release.

@brianyyz yeah, i've tried using relative path, same story :(
@antonfire still working on it, i need to solve this pretty soon though

I'm setting up 3.x.x branch, so I'll report if it's working there )

Ok, I did 3.1.1 setup with bare configuration

here is index.js, which I launch to start my parse 3.1.1:
https://gist.github.com/dongalor/39ce1918bcf6c1eef3d5a137e4f092e9

here is my cloud function, I use to test pushes:
https://gist.github.com/dongalor/6dda14517778fecf3127f9d29fbe2a0c

here's log:

https://gist.github.com/dongalor/aa7f1d4807a575647fd7f25f88f2a856

Absolutely the same problem, so sad :(

@dongalor how are your installation objects created? Through the iOS SDK or manually?

Because all _Installation objects require a proper appIdentifier set so the push adapter can select the proper certificate / configuration to use to deliver the push.

As mentioned here: https://github.com/parse-community/parse-server/issues/4313#issuecomment-352136108

@flovimart Damn it🥇, Bingo! Now I got it, we don't have appIdentifier in our Installation table, so, with parse 2.4.2 (which is sending pushes right now) we don't need apIdentifier, I guess, right? :)

Is it possible to assign apIdentifier manually without IOS SDK?

@dongalor Sure you can just fire up mongo cli and modify _Installation documents in your database to manually set appIdentifier.

I'm deliberately not copy pasting any command to do it as this will be a massive (harmless) update, so I'd like you to understand what are you doing...

@mman First of all, thank you so much for the support in solving my problem!

I have a development server, where I can do any dangerous stuff :)

So, I've made a small experiment, I've entered my appId from my parse's index.js file to every installation record and tried doing the same test push sending as here, what I've got in logs is:

verbose: _PushStatus UgmAPHkeyG: sending push to installations with 1 batches verbose: Sending push to 1 verbose: _PushStatus UgmAPHkeyG: sent push! 0 success, 1 failures verbose: _PushStatus UgmAPHkeyG: needs cleanup devicesToRemove=[]

By the way, am I getting right this value, which is able to be in appIdentifier field in Installation table?

@dongalor Please try once again with VERBOSE_PARSE_SERVER_PUSH_ADAPTER=1 environment variable set, you should get more detailed diagnostics in the log.

@mman so, I run my parse 3.1.1 instance this way, without pm2 for test purposes:

VERBOSE=1 VERBOSE_PARSE_SERVER_PUSH_ADAPTER=1 node index.js

and I get this in my log:

verbose: RESPONSE from [POST] /parse/functions/testPush: {
  "response": {
    "result": true
  }
} result=true
verbose: REQUEST for [POST] /parse/push: {
  "where": {
    "deviceToken": {
      "$select": {
        "key": "deviceToken",
        "query": {
          "where": {
            "objectId": "3ePnw29d9C"
          },
          "className": "_User"
        }
      }
    }
  },
  "data": {
    "data": {
      "subscriberId": "xku6EHzzwD",
      "subscriberName": "Test Subscriber",
      "code": "3"
    },
    "alert": "Free hotdogs at the Parse concession stand!"
  }
} method=POST, url=/parse/push, user-agent=node-XMLHttpRequest, Parse/js2.1.0 (NodeJS 11.2.0), accept=*/*, content-type=text/plain, host=localhost:1339, content-length=404, connection=close, key=deviceToken, objectId=3ePnw29d9C, className=_User, subscriberId=xku6EHzzwD, subscriberName=Test Subscriber, code=3, alert=Free hotdogs at the Parse concession stand!
verbose: RESPONSE from [POST] /parse/push: {
  "headers": {
    "X-Parse-Push-Status-Id": "GsEETYJSAd"
  },
  "response": {
    "result": true
  }
} X-Parse-Push-Status-Id=GsEETYJSAd, result=true
verbose: _PushStatus GsEETYJSAd: sending push to installations with 1 batches
verbose: Sending push to 1
verbose: _PushStatus GsEETYJSAd: sent push! 0 success, 1 failures
verbose: _PushStatus GsEETYJSAd: needs cleanup devicesToRemove=[]

Add a column in your installation class with the name appIdentifier and the type string. Then populate all your existing installation objects with your app id eg com.blahblah. Then try sending again

Will need to ensure new installation objects are populated with this field on push registration client side

One could as well make a fix in the adapter.

I would suggest removing the appIdentifier as a requirement in the installation class as this is specified in the push adaptor settings in index.js. I can have a look at a pull request for this if that would help?

That certainly would help

Ok I’ll get on it

@antonfire are you thinking that this would be an either / or approach (look for appIdentifier first and if absent use push adapter settings from index.js) or just ignore the appIdentifier in Installation and rely solely on config from index.js?

That’s not that simple, if the installation does not specify an app identifier and multiple are configured, the adapter need to retry. In any case you should also fix your clients so they set a proper appIdentifier.

Not setting an appIdentifier is not conform to the specification. And for all that I care, I don’t have to merge or provide a fix. All the official SDKs set this value.

@flovilmart that is where I was headed. This starts to feel like it would be breaking away from the way that the sdks behave.

Initially, the fact that we didn’t take into account the appIdentifier was a bug.

Not everyone uses the sdk to register push, a waterfall approach would be best to prevent breaking changes for users that haven’t upgraded or installed since the breaking change

@antonfire how do we know how many users of parse are not using an sdk? I think this would be an important metric to evaluate whether a breaking change is justified. At a minimum it would inform how to approach the change.

@antonfire I am not sure where you’re getting at. The spec in the rest documentation has always been to pass an appIdentifier. That the server before 2.6 was permissive was a bug on the server.

If you feel any of this is wrong, this is an opensource project, so you know what to do.

@antonfire Oh My God! Now, when I'm using topic/bundleId value in my test user Installation record, I have an error! I think we are very close to beat this problem of my. Thank you so much in advance, guys!

Here's what log says:

verbose: _PushStatus 7GzcpE2zDK: sending push to installations with 1 batches
verbose: Sending push to 1
ERR! parse-server-push-adapter APNS APNS error transmitting to device 5083c519e51624bd6e1ff3dca9c1dac8afce4433190319d71e2015876e0bb922 with status 400 and reason BadDeviceToken
verbose: _PushStatus 7GzcpE2zDK: sent push! 0 success, 1 failures
verbose: _PushStatus 7GzcpE2zDK: needs cleanup devicesToRemove=[5083c519e51624bd6e1ff3dca9c1dac8afce4433190319d71e2015876e0bb922]

That’s likely because the token is invalid

@dongalor Error 400 happens when you try to send push from parse server using a production certificate or token to a debug build of your app, IOW the push token of your app is a one that is valid for sandbox. servers.

@dongalor Set up like this so works on production and development:

push: {
ios: [
{
pfx: 'PushDevCert.p12',
topic: 'com.xxxxxx.xxxxx',
production: false
},
{
pfx: 'PushDistCert.p12',
topic: 'com.xxxxxx.xxxxx',
production: true
}]

}

@flovilmart @brianyyz I was just thinking of a fallback to the bundle id where appIdentifier is undefined

@dongalor Ensure your certificates are current and valid. Ensure you are sending to current valid device tokens.

@dongalor (going all the way back to the beginning) In your AppDelegate are you saving the device token that returns in

func application(application: UIApplication, didRegisterForRemoteNotificationsWithDeviceToken deviceToken: NSData) {
  // Store the deviceToken in the current Installation and save it to Parse
  let installation = PFInstallation.currentInstallation()
  installation.setDeviceTokenFromData(deviceToken)
  installation.saveInBackground()
}

before sending a push to the device? The last updated time stamp on the Installation should get updated if you are. Just want to make sure we’re covering all the bases.

@flovilmart @mman @antonfire I @brianyyz My IOS developer registered a new user on 3.1.1 parse and it's working! :) So, I guess the problem now was with deviceToken.

Thank you so much for the support, now I can move all backend to 3.1.1!

P.S. We can remove wontfix label now, I guess!

Glad to hear. Also the issue is on the push adapter, so it won’t be fixed in this version of parse-server

Excellent news!

@flovilmart got it, right :)
@brianyyz 👍

Was this page helpful?
0 / 5 - 0 ratings

Related issues

shardul08 picture shardul08  ·  3Comments

kilabyte picture kilabyte  ·  4Comments

mohmagdy picture mohmagdy  ·  3Comments

jiawenzhang picture jiawenzhang  ·  4Comments

pulse1989 picture pulse1989  ·  3Comments