Paypal-checkout-components: PayPal Subscriptions incorrect start_time and next_billing_time dates on Live and Sandbox

[dgpro]

In short:
Subscription webhooks (like BILLING.SUBSCRIPTION.ACTIVATED) and subscription details API endpoint return incorrect start_time which is always previous date at 23:00 and incorrect next_billing_time close to create_time which is in the past, regardless of subscription duration period on both Live and Sandbox.

In details:
We are using PayPal JS SDK (paypal.com/sdk/js) to create a subscription payment button, webhooks for notifications and subscriptions REST API V1 to retrieve details.
We've setup monthly recurring plan with 7 days trial with following billing details:

billing cycle: Every 1 month (pays at beginning of each billing cycle)
Number of billing cycles: As long as plan is offered
Trial period price: $0.00 USD for 7 days (no tax)
Stop collecting payments after: 1 missed billing cycles

When subscription is created using payment button the app receives subscriptionID and we immediately retrieve its details using API V1 (https://developer.paypal.com/docs/api/subscriptions/v1/#subscriptions_get) which returns a response with incorrect both start_time (the previous day) and next_billing_time (in the past, but
should be in 7 days)
(irrelevant details are omitted for shorter output)

{
    "status": "ACTIVE",
    "status_update_time": "2020-06-12T09:47:21Z",
    "id": "I-VG4D3KBL67JT",
    "plan_id": "P-2EC848750R2281112L3QLJPI",
    "start_time": "2020-06-11T23:00:00Z",
    "quantity": "1",
    "shipping_amount":{},
    "subscriber":{},
    "billing_info":
    {
        "outstanding_balance":{},
        "cycle_executions": [],
        "next_billing_time": "2020-06-12T09:47:21Z",
        "failed_payments_count": 0
    },
    "create_time": "2020-06-12T09:46:49Z",
    "update_time": "2020-06-12T09:47:21Z",
    "transactions": null
}

Then comes in the BILLING.SUBSCRIPTION.ACTIVATED webhook with the same incorrect dates

{
    "id": "WH-8H756832S2001654V-1X369026W1516043G",
    "event_version": "1.0",
    "create_time": "2020-06-12T09:47:37.837Z",
    "resource_type": "subscription",
    "resource_version": "2.0",
    "event_type": "BILLING.SUBSCRIPTION.ACTIVATED",
    "summary": "Subscription activated",
    "resource":
    {
        "shipping_amount":{},
        "start_time": "2020-06-11T23:00:00Z",
        "update_time": "2020-06-12T09:47:21Z",
        "quantity": "1",
        "subscriber":{},
        "billing_info":
        {
            "outstanding_balance":{},
            "cycle_executions": [],
            "next_billing_time": "2020-06-12T09:47:21Z",
            "failed_payments_count": 0
        },
        "create_time": "2020-06-12T09:46:49Z",
        "links": [],
        "id": "I-VG4D3KBL67JT",
        "plan_id": "P-2EC848750R2281112L3QLJPI",
        "status": "ACTIVE",
        "status_update_time": "2020-06-12T09:47:21Z"
    },
    "links": []
}

BUT! If I manually request subscription details using API V1 later on using the same endpoint it returns correct next_billing_time (in 7 days from start time)

{
    "status": "ACTIVE",
    "status_update_time": "2020-06-12T09:50:26Z",
    "id": "I-VG4D3KBL67JT",
    "plan_id": "P-2EC848750R2281112L3QLJPI",
    "start_time": "2020-06-11T23:00:00Z",
    "quantity": "1",
    "shipping_amount": {},
    "subscriber": {},
    "billing_info": {
        "outstanding_balance": {},
        "cycle_executions": [],
        "next_billing_time": "2020-06-19T10:00:00Z",
        "failed_payments_count": 0
    },
    "create_time": "2020-06-12T09:46:49Z",
    "update_time": "2020-06-12T09:50:26Z",
}

This behaviour is the same on Sandbox and Live even for plans that don't have free trial period like 6 months plan and subscription details api response has the same incorrect dates

{
    "status": "ACTIVE",
    "status_update_time": "2020-06-12T09:50:39Z",
    "id": "I-VAETFF0V9BAP",
    "plan_id": "P-8FP363974T0998009L3QLJ6Q",
    "start_time": "2020-06-11T23:00:00Z",
    "quantity": "1",
    "shipping_amount":{},
    "subscriber":{},
    "billing_info":
    {
        "outstanding_balance":{},
        "cycle_executions": [],
        "next_billing_time": "2020-06-12T09:50:37Z",
        "failed_payments_count": 0
    },
    "create_time": "2020-06-12T09:49:57Z",
    "update_time": "2020-06-12T09:50:39Z",
    "transactions": null
}

The same dates are in BILLING.SUBSCRIPTION.ACTIVATED webhook event.

But later on the same request returns correct next_billing_time

{
    "status": "ACTIVE",
    "status_update_time": "2020-06-12T09:55:27Z",
    "id": "I-VAETFF0V9BAP",
    "plan_id": "P-8FP363974T0998009L3QLJ6Q",
    "start_time": "2020-06-11T23:00:00Z",
    "quantity": "1",
    "shipping_amount": {},
    "subscriber": {},
    "billing_info": {
        "outstanding_balance": {},
        "cycle_executions": [],
        "last_payment": {
            "amount": {},
            "time": "2020-06-12T09:50:38Z"
        },
        "next_billing_time": "2020-12-12T10:00:00Z",
        "failed_payments_count": 0
    },
    "create_time": "2020-06-12T09:49:57Z",
    "update_time": "2020-06-12T09:55:27Z",
    "links": []
}

This is a big issue as our system determines subscription duration based on payment provider details as it could be any period (free 7 trial or paid 6 month etc.) but PayPal API returns inconsistent and unreliable results especially critical is next_billing_time which is in the past, but after some time (not sure how long the delay is) the same endpoint returns correct next_billing_time.

P.S. All the response details are from Sandbox, but I can provide Live details as well privately.

[bluepnume]

@ravishekhar can you take a look? Thanks!

[ravishekhar]

Looking into it.

[ravibpatel]

+1 Facing the same issue.

[smartbitz]

I have the same issue on Sandbox

[ravishekhar]

We are actively working on finding the root cause of this issue. I will update the thread when its done.

[jkk]

This is blocking our team from launching PayPal subscriptions. Any updates?

[sinedoOo]

Any update on this? I have the same issue on sandbox, and I'm not sure if we can deploy PayPal Subscriptions to production.

[ravishekhar]

We are working on the fix. The fix will be available by 8/31. Will update the thread once changes are rolled out.

[jagpreetuppal-kiwi]

@ravishekhar As mentioned in your last comment, fixes will be available by 8/31, Do you have any update on fixes of PayPal Subscriptions incorrect start_time and next_billing_time dates on Live and Sandbox?

[ravibpatel]

@ravishekhar Any updates on fixes for this issue?

[dloker]

We're having similar issues. We're calling the paypal API on our server and using webhooks to grab information and the next_billing_time is consistently wrong even after payment has went through. There is a delay, seemingly arbitrary, that if you call the subscription endpoint the next_billing_time will get updated. Unfortunately, if the user cancels the sub before that point, they get charged and next_billing_time will never be correct and we don't know what the expiration date of the subscription is. Because we allow for a large variety of subscriptions, it is quite difficult to deal with this stale information.

If we can't use next_billing_time, what other mechanism can we use for calculating the expiration date?

[HeuHax]

This is still an issue. Any ETA?

[CaptainBeOS]

I read elsewhere that PayPal have no intention of changing this behaviour, which led me to understand that we all must be missing something. After much digging and experimenting I can share the solution to this problem with you all and hope nobody else wastes so much time.

It seems that when the initial BILLING.SUBSCRIPTION.ACTIVATED event is triggered it is done so BEFORE the payment has actually cleared. The fact that the subscription is marked as ACTIVE is in my opinion misleading. It should be APPROVED at this stage. The dates you see on the JSON object reflect the fact that the next payment is due now. According to the API documentation next_billing_time is always 10:00am GMT on the day payment is due, which may well be in the past as we have all observed.

It turns out that the dates get updated, not 20 minutes later, but as soon as the payment is cleared. Wait for the PAYMENT.SALE.COMPLETED webhook and then call the Get subscription details API. Yes, the only way out of this pickle is to make this API call.

curl -v -X GET https://api.sandbox.paypal.com/v1/billing/subscriptions/I-BW452GLLEP1G \
-H "Content-Type: application/json" \
-H "Authorization: Bearer Access-Token"

The resulting object will be correct.

This makes more sense if you start playing with delayed subscription start_times as I did. For delayed start times you will get the BILLING.SUBSCRIPTION.ACTIVATED as soon as the subscription is created and the start_time in that object will be the start of the day indicated by the start_time you passed in the API or in the Javascript SDK and next_billing_time will be 10:00am GMT on that date . Your customer is not billed until start_time passes and then when the sale completes you get PAYMENT.SALE.COMPLETED and that is your chance to activate the subscription with the condition that start_time has passed and status = 'ACTIVE'.

This also translates to renewals. I spent an age trying to find any indication what happens when there is a successful renewal. The webhook to use for renewals is also PAYMENT.SALE.COMPLETED. Query the subscription details API again and the resulting object will contain the new next_billing_time.

I've not got far enough in my investigations to work out the conclusion to the following hypothisis: However I suspect that there is very little need to store next_billing_time as you all seem to be aiming for. Paypal alleges that they will send BILLING.SUBSCRIPTION.SUSPENDED webhook events if payment fails. That being the case responding to PAYMENT.SALE.COMPLETED and BILLING.SUBSCRIPTION.SUSPENDED alone (possibly also BILLING.SUBSCRIPTION.CANCELLED) by querying the subscription details API to get the "status" and "start_time", should, when coupled with the condition that start_time has passed and status = 'ACTIVE', be sufficient to determine that a subscription should continue. We'll see...

Hope this helps. TTFN

[ravibpatel]

@CaptainBeOS I am also using the same technique for some time now but problem with this is that next_billing_time is not honored when PayPal charges the user for renewal. It should be 10:00am GMT but it charges the user's card at random time. I tried it on sandbox using per day subscription.

[CaptainBeOS]

@ravibpatel I think this is just the way it is: I'm guessing that if PayPal start processing all renewals due that day at 10:00am it probably takes quite some time to complete the batch process. There must be hundreds of thousands for them to get through. So PAYMENT.SALE.COMPLETED could he hours later. Plus there might be problems settling, e.g. customer's card details have expired, insufficient funds, etc. So I presume this random time you have observed, is after 10:00am GMT but usually on the same day?

I can see this being a problem if you were operating day long subscription cycles and wanted to terminate the subscription at the exact same time of day that they started the subscription. But my understanding is that PayPal don't intend their subscription model to be used this way. They seem to be working to the principle that if a subscription is taken out on date A, the "expiry" will be end of the day "interval_unit" later. So next_billing_time is set to 10:00am on that expiry date giving PayPal 14 hours for the transaction to complete, which seems perfectly reasonable. The true expiry time is therefore next_billing_time rounded up to the end of the day. For day long subscriptions you might just have to concede that some lucky customers are going to get up to 14 hours free, but by the same token others are going to get billed up to 14 hours early. Does it really matter if they carry on for several cycles and the effect gets diluted?

[mariusz-schimke-iteo]

@CaptainBeOS, a similar issue brought me here. What we already do is request subscription details on handling the PAYMENT.SALE.COMPLETED webhook event. Today we noticed that at that moment the request still returns out-of-date results. When we repeat it after a while, the next_billing_time is up to date. I'm lost.

[CaptainBeOS]

@mariusz-schimke-iteo I was going to post about this because as soon as I took things out of dev into test it also failed to work.

It turns out waiting for PAYMENT.SALE.COMPLETED is met with a marginal success rate and it seems to depends on how fast paypal updates the database their end and how quick your software responds to the web hook. After much experimentation I've found that the incorrect details can persist for between 1ms and 7-8 minutes after the "create_time" of the PAYMENT.SALE.COMPLETED. Quite some window to work with. So in a dev environment using something like local tunnel, which will add a bit of latency time to the web hook, it's easy to reach a false conclusion when paypal is running quick. My bad.

The good news is that once the details are updated there is a very way reliable way of detecting it. But, you have to periodically poll the Paypal API to achieve it. Which unfortunately means a background process. Not another Daemon!

I've taken to storing the PAYMENT.SALE.COMPLETED messages in a queue and then every 5 minutes using a background process to poll the API for each item in the queue. After a day items are flushed from the queue. I figure at that point it's a job for the support desk.

When Paypal finally update the subscription object I've found that the "update_time" is always after the "create_time" of the PAYMENT.SALE.COMPLETE. Though a niggly feeling told me to not to trust this condition. This is because it can literally be just a couple of milliseconds difference. So it would only take a slight change in the way Paypal run their DB triggers for the objects "update_time" to be before the "create_time". Instead I've taken to a condition with a much bigger margin of error:

"create_time" of the PAYMENT.SALE.COMPLETE web hook < The subscription objects "billing_info"."next_billing_time"

I.e. I'm just looking for the next billing time moving on and using the payment time as the reference point.

This works for future start_times as well as "now" start_times, since in both cases the initial next_billing_time is rounded down to midnight (UTC) from the given start_time and auto payment is taken at 10:00am UTC. Hence it gives a bigger margin of error.

So in summary the flow is:

1) Store the PAYMENT.SALE.COMPLETED webhook data (or at least the create_time) in some sort of "event" in a queue, e.g. a database table.
2) Periodically loop though oustanding "events" and poll the API for each of them
3) Once "create_time" of the PAYMENT.SALE.COMPLETE web hook < The subscription objects "billing_info"."next_billing_time" then you are good to use the informaton and delete the "event" from your queue.

It's not as elegant as the principle of webhooks would have it. In fact it almost negates the need for webhooks completely. But it works and we have gone live with it.

GOOD LUCK!

[HeuHax]

@CaptainBeOS It really should not be that hard xD xD
This is something that has been working at Stripe since forever....

[mariusz-schimke-iteo]

@CaptainBeOS, thanks a lot for your valuable input!

[HeuHax]

Just wanted to point out that even the email notification is wrong ......

[malikwas]

Seems like this problem still isn't solved? Our workaround for this is comparing the plan_id for the subscription object that we get from the billing_agreement_id in the PAYMENT.SALE.COMPLETED webhook to the two plan IDs we keep in memory (monthly and yearly plan IDs). Based on the equality, we update the subscription by one month or one year.

[optimistiks]

The problem isn't solved. When subscription is first created, and then activated, if it has a trial billing cycle (with 0$ cost), the trial billing cycle is first marked as "remaining": 1 and "completed": 0, and the next_billing_date is today, in the webhook event payload (BILLING.SUBSCRIPTION.ACTIVATED). Somewhere behind the scenes in Paypal's backend the trial billing cycle then changes to "remaining":0 and "completed":1, and next_billing_date changes to a correct one (can be checked via Postman), however there is no webhook event that is sent at that moment. The trial period is free, which means there is no PAYMENT.SALE.COMPLETED event. The last event you get is BILLING.SUBSCRIPTION.ACTIVATED, and even if you re-request the subscription at the moment of the event, you won't have correct next_billing_date.

[AO-AO]

+1 Facing the same issue.

[Paradoxor]

+1 Having the exact same issue
"last_payment": { "amount": { "currency_code": "EUR", "value": "14.9" }, "time": "2020-12-22T10:53:26Z" }, "next_billing_time": "2020-12-22T10:53:24Z", "failed_payments_count": 0

Our last payment is 2 seconds after the next billing time.

Can anyone confirm that the future "billing.activate" webhook calls than have the correct next_billing_time?

Thanks in advance

[Paradoxor]

Update, we just checked. There are no more webhooks send when the next_billing_time is in the past.
This means with that bug inplace we can't use Paypal subscription right now.

[atomic-tang]

Any updates on this? This is still a problem for me

[Paradoxor]

@atomic-tang we solved it by listening on the PAYMENT.SALE.COMPLETE webhook and than we do another query to the subscription endpoint. That is than returning the correct next_biling_time. I hope this helps.
We also opend a bug ticket on the paypal support page.
BTW the PAYMENT.SALE.COMPLETE is also triggered on the renewal but does not contain the the next_billing_time

[atomic-tang]

@mariusz-schimke-iteo I was going to post about this because as soon as I took things out of dev into test it also failed to work.

It turns out waiting for PAYMENT.SALE.COMPLETED is met with a marginal success rate and it seems to depends on how fast paypal updates the database their end and how quick your software responds to the web hook. After much experimentation I've found that the incorrect details can persist for between 1ms and 7-8 minutes after the "create_time" of the PAYMENT.SALE.COMPLETED. Quite some window to work with. So in a dev environment using something like local tunnel, which will add a bit of latency time to the web hook, it's easy to reach a false conclusion when paypal is running quick. My bad.

@Paradoxor It's not guaranteed that the next_billing_time will be updated in time when fetching the subscription immediately after receiving the PAYMENT.SALE.COMPLETED webhook unless it was fixed recently? I wasn't able to find anything to confirm this. Did I miss something and can you confidently rely on this implementation to always give you the correct next billing time?

[CaptainBeOS]

@Paradoxer That was the method we used initially, but you will find on occasions that next_billing_time won't be updated by the time you query the subscription end point. The only reliable option is to take PAYMENT.SALE.COMPLETED as notification to start querying the end point periodically. This is certainly true in the case of renewals.

Just on an aside here, I read a foot note somewhere that Paypal had no intention of changing this as they don't view it as a bug. Don't quote me on that as it was somewhere in the vast number of posts I sifted through trying to work out how to beat this problem. This stance is all very well, but I hope that they don't mind the great number of extra queries to their subscription end point that will happen as a result.

But then, thinking about it, maybe repeated calls to their API is exactly what they want. In the same way that a third party Authentication service such as Azure Active directory service requires us to make a background call to their authorization server to complete the authorization process, perhaps Paypal expects us to make a call to the PayPal subscription end point every time a subscriber authenticates as part of the authorization process...

[atomic-tang]

@CaptainBeOS I was thinking of returning a non 200 Http status to the webhook so that PayPal will resend the webhook again at a later time where the next_billing_time will hopefully be updated correctly. This way we don't have to poll or set up a background process to periodically call the subscription end point. The problem with this approach though is that it will take a couple of hours for the webhook to be sent again as I had read somewhere that PayPal resends webhook events at most 25 times over a period of 3 days. This solution may not be acceptable for certain use cases if timing is critical.

[atomic-tang]

The only reliable option is to take PAYMENT.SALE.COMPLETED as notification to start querying the end point periodically. This is certainly true in the case of renewals.

@CaptainBeOS Also to clarify, when you say that it is true in the case of renewals, did you mean that calling the subscription endpoint immediately after the PAYMENT.SALE.COMPLETED webhook event will always return an incorrect next_billing_time or is the next_billing_time always updated?

[CaptainBeOS]

@atomic-tang That's a great idea for avoiding a background process. Well done.

I suspect you will find that the payload in the webhook event will not have updated at a later date. Paypal are probably using a message queue to deliver the webhook, so the payload will be static. In which case you will still have to query the subscription end point.

As far as the certainly true statement I was referring @Paradoxor ' s observation that next_billing_time is omitted in the PAYMENT.SALE.COMPLETED webhook for renewals, which means querying the subscription end point is absolutely necessary.

[atomic-tang]

@CaptainBeOS Thanks for the clarification. After calling the subscription endpoint when a subscription is renewed with the PAYMENT.SALE.COMPLETED webhook event did you still occasionally experience the same issue with the next_billing_time or does the issue only occur for the first payment?

[eqcb]

To make it even more confusing they just added a BILLING.SUBSCRIPTION.RENEWED web-hook but there's no mention of it on the actual web-hook list documentation for subscriptions
What will be send upon a renewal? PAYMENT.SALE.COMPLETED, BILLING.SUBSCRIPTION.RENEWED or both?

[agaertner]

We got the same issue.

[Dreemsuncho]

I found that if we suspend the subscription and activate it again, the next_billing_time is correct. Stupid technique but at least it works.
Also for the yearly subscription, it seems the time is correct always for some reason.

[do-web]

I find it unbelievable that such a stock exchange company's things are so poorly implemented.