Device-os: Clarification about pending Particle.publish() delivery on cloud disconnections

[ScruffR]

In connection with this thread I had a short discussion with Mat (@m-mcgowan) whether an introduced fix (maybe #469 or PRs #909/#918) which addressed the issue of events not being delivered without a 5sec delay between publish and sleep, also applies to WiFi.off() (and also Particle.disconnect() or other commands that would cut the connection).
The outcome of that discussion wasn't really clear, so I decied to file this issue for safe keeping ;-)

  Particle.connect();
  waitFor(Particle.connected, 10000);
  Particle.publish("event", "1", PRIVATE);

  // this used to be required, but isn't anymore for 0.5.3+for sleep()
  //for (uint32_t ms=millis(); millis()-ms < 5000; Particle.process());

  // this definetly works on Electron and probably on Photon too
  //System.sleep(SLEEP_MODE_DEEP, 60);

  // but how about these?
  WiFi.off();
  // WiFi.disconnect();
  // WiFi.listen(); // ???
  // Cellular.off();
  // Cellular.disconnect();
  // Particle.disconnect();
  // and whatever I've forgotten

It would be good if a pending publish would first get delivered (maybe even blocking, or with an additional parameter boolean force = false to give the user contol) before this action cuts the connection.

Maybe in the same context keeping these and sleep() from cutting the connection during an OTA update might save useres some headache why their code just won't stick. There are multiple threads about this issue in a way or another. That even (or especially) seems to be a problem for Console users who rely on pending updates to be delivered correctly without too much fuss in their code.
(respective issue #1166)

---

Completeness:

• [X] Minimum test case added
• [X] Device, system and user firmware versions stated

[ScruffR]

Two months and no Particle confirmation???

This problem still seems to exist in v0.6.1-rc.1 on Photons even for System.sleep()

When entering Stop Mode Sleep immediately after a Particle.publish() the event never reaches the console and on next wake the device doesn't seem to connect properly either.
With a delay(5000) between publish and sleep the code works for a Photon.
But for Electrons the very same code works now even without the delay.

---

This did cost me several hours of backtracking after taking working Electron code to a Photon assuming that this fix would also apply to the WiFi devices (but since it only applies to UDP and wasn't extended to TCP for some obscure reason - that's probably my on stupidity)

[technobly]

Sorry I don't have time to dig into this fully right now, but I think the new NO_ACK flag for Particle.publish() may help here. https://prerelease-docs.particle.io/reference/firmware/photon/#particle-publish-

Also there will be Promises coming in 0.7.0-rc.1, see examples here:
https://github.com/spark/firmware/pull/1144

Please let me know what is missing after that.

[ScruffR]

Thanks Brett, for the info and also for promising promises 👍

At first this didn't make sense, since I do want exactly the opposite

I think the new NO_ACK flag for Particle.publish() may help here

I want to have the event delivered, not fire-and-forget (which NO_ACK does).
But after following the link I saw that you might actually mean WITH_ACK 😉
So I'll try that. But it's still puzzling that the same code behaves that much different between two alegedly widely "code compatible" devices.
Even if there are workarounds for each platform, it would still be desirable to have both systems behave as similar as possible (and UDP vs. TCP is something hidden from the average user, so that doesn't count as an excuse 😎)

IMHO there is no good reason why the Photon doesn't flush the pending event publishes before going to sleep while the Electron does.

[ScruffR]

@technobly, I have tried WITH_ACK but it didn't change anything about my problem

The annoying thing about the issue is that the unfinished publishes even seem to clog up the sockets so that the Photon will stop reconnecting to WiFi after a few wake cycles (with or without WITCH_ACK). It just keeps rapid blinking green forever.
But with delay(5000) the device behaves as it should.

[sergeuz]

It's been a while since I examined our networking code for the last time, but, in general, I agree that the firmware should close its TCP connection gracefully, regardless of WITH_ACK flag, i.e. flush all pending events, then half-close the socket and wait until the cloud closes the connection completely.

[technobly]

Related to https://github.com/spark/firmware/issues/1176 when Cellular.off() / Cellular.disconnect() is called

[joergseiler]

It has been suggested that I post my findings here

Related issues might be issue #1034

I've posted a simple program that shows the unreliability or complete non-functionality of the WITH_ACK feature on the particle forum;

https://community.particle.io/t/particle-publish-with-ack-still-not-working-properly/41922/2

Maybe someone with more experience than I can have a look at this again. My issue is with the photon.

[blackTay]

I have the same problem: On the Photon, when falling asleep (not deep sleep) directly after publishing, the message never reaches the console. No matter if the ACK flag is set or not.

[ScruffR]

@blackTay, can you also state what system version(s) you tested this with?

@technobly, I got the feeling this issue was closed prematurely, or we have a regression - can we have it opened again?

[ScruffR]

Still present in 1.1.0 and 1.2.1-rc.2
@technobly, seeing WITH_ACK also not working as expected with Mesh devices - i.e. Xenon not publishing with a undelayed Mesh.off() call - I'm still feeling this issue should not have been closed.

Having the issue solved for Cellular devices only doesn't seem like actually solving it, does it?
Especially since the original issue was explicitly refering to WiFi (no extending it to Mesh).

Related topic
https://community.particle.io/t/particle-publish-with-with-ack-not-working-properly/30582/6?u=scruffr