Signal-android: Signal server down

[ryannathans]

I have:

• [ x] searched open and closed issues for duplicates
• [ x] read https://github.com/WhisperSystems/Signal-Android/wiki/Submitting-useful-bug-reports

---

Bug description

Nobody I speak to can send messages on signal, all Android users in Australia.

Steps to reproduce

Try to send message, throws an error.

Screenshots

Device info

Multiple different Android phones, different Android versions.

Link to debug log

https://hastebin.com/umakuzuyan

[porjo]

I can confirm this. I'm in Australia using Android app and cannot send any messages currently

[GAS85]

🇩🇪Germany - the same. No messages could be send

[duck-rh]

Same here in Japan. Tried desktop and Android apps, on two different providers.

[Daigz]

Confirmed in Sweden as well.

[jharveyb]

Confirmed in USA (West Coast) - no connectivity from desktop stable app or mobile beta.

[ryannathans]

For some reason logging issues is disabled on the signal-server repo

[eth0up]

Confirmed in mid-USA, I'm unable to send any messages from the mobile client

[soy-capitan]

Confirmed from iOS and OS X as well.

[roykhoury]

I can't send any messages either, from Canada

[obesechess]

Down for me as well. Portland OR. Started around 11:30pm PST, currently 12:15am PST

[andy-ta]

@roykhoury GIETMAMAW

[yankee14]

@moxie0 Down for me, 0815Z, central Oklahoma, US.

EDIT: Obligatory link to reddit discussion

[ryannathans]

Is there just one server, on one IP address for an application of this scale?

[healler3108]

Confirmed in Russia. Both Android and iOS clients.

[ionut-ardelean]

Confirmed in Romania on both Android and Mac clients

[gileshinchcliff]

Down in London, UK also. Android client.

[xiroV]

Down in both Finland and Denmark as well (Android).

[rabsonic]

Confirmed in Poland 8:30 LT (Android)

[EErikas]

Down in Lithuania, Android macOS and Windows clients

[natrius]

Down in austria.
Come on, at least put some informations out on the homepage and on twitter and whatever.
THATS exactly the reason people would like a federated network. And bad information management does not help :/

[gandhiano]

Federate signal! ✊

[dusatvoj]

Confirmed in Czech Republic and tested on Japan ProtonVPN server

[ruicruzpt]

Also down in Portugal.

It's a general issue.

[supremeleaderNK]

Down in Pyongyang, North Korea. All clients.

[Adocentyn]

Down in Paris France.

[dankawka]

Down in Poland. Doesn't matter if mobile or desktop.

[gandhiano]

And back (and again gone).
Still thinking that federation is a must for an application of such global importance for mankind.

[ryannathans]

@gandhiano DoS attack perhaps if there are reports of intermittent service restoration?

[Cerberus0]

Please respect the project's contributing guidelines:

Not a discussion forum

Please do not use this issue tracker as a discussion forum. Discussion related to the bug in question should of course go to the issue itself. However other discussion should take place at the community forum. You can use that forum to discuss any Signal related topics or to just hang out with your fellow users.

Don't bump issues

Every time someone comments on an issue, GitHub sends email to everyone who is watching the repository (currently around 500 people). Thus bumping issues with :+1:s, _me toos_ or asking for updates generates a lot of unnecessary email notifications. Moreover bumping an issue does not help solve it. Please be respectful of everyone's time and only comment if you have relevant new information to add.

I'm sure Signal's developers will make an official announcement on Twitter when this issue has been resolved: https://twitter.com/signalapp

Edit: In other words, be patient and do not comment or ask for updates in this thread.

[base-jump-org]

Down in France and Switzerland too.
Look like down worldwide ...

[ryannathans]

I feel like we need a decentralised network where anyone can spin up their own instance and add it to the cluster similar to how crypto currencies don't have a single point of failure.

Or at bare minimum something DoS resistant.. Anything that's not a single point of failure.

[MartinX3]

Mirrored Server would be fine.

[lister-wrt]

Intermittent here in Australia. Some messages that say "failed to send" on my phone are actually arriving.

[ryannathans]

@MartinX3 maybe something simple like anycast dns with a couple servers globally?

[NFH26]

Down in Saudi as well.

On Mon, 5 Mar 2018 at 12:23 PM Ryan Shaw [e-mail] wrote:

@MartinX3 https://github.com/martinx3 maybe something simple like
anycast dns with a couple servers globally?

—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/signalapp/Signal-Android/issues/7460#issuecomment-370357727,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AFoVdAxxnKbpWUSwPG7jB2rc85DfOk0Mks5tbQQPgaJpZM4Sb2Nv
.

>

~ Nayef Alharbi
Email: Alharbi.[email protected]
Mobile# [phone]

[MartinX3]

@ryannathans
Yeahr, I thinked the same way ! :D
Google does the same.
Mirrored server in several locations and the fastest server will answer.
Pretty nice to load balance, too.

[Sipeht]

Down in Italy as well

[versable]

Looks like a DDOS attack, since the DNS entry has been changed to 127.0.0.1, which is the "NO U" defense of a DDOS attack.

~  dig textsecure-service.whispersystems.org

; <<>> DiG 9.10.3-P4-Debian <<>> textsecure-service.whispersystems.org
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 51101
;; flags: qr rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 4096
;; QUESTION SECTION:
;textsecure-service.whispersystems.org. IN A

;; ANSWER SECTION:
textsecure-service.whispersystems.org. 228 IN A 127.0.0.1

;; Query time: 80 msec
;; SERVER: 192.168.1.1#53(192.168.1.1)
;; WHEN: Mon Mar 05 10:26:50 CET 2018
;; MSG SIZE  rcvd: 82

[ryannathans]

@MartinX3 it's a good idea but we'd need a lot of servers to protect against (D)DoS attacks by well funded bad actors (or botnets).

The community run decentralised approach handles that attack but it's a bit harder and actually required code changes to implement!

[garcia88]

Down in Syria too.

[MartinX3]

@ryannathans Signal could sell pre configured raspberry pi 3b which only need to be connected to the own internet and act as support server.
Like the Tor Network

The raspi 3b is a strong low cost hardware with a large community.

[ryannathans]

@MartinX3 or just cut the hardware cost and allow people to install a background service on their servers and PCs

[e6udlpdc1b]

Is it a coincidence that today is day 1 of the first session of China's 13th National People's Congress? The very Congress meeting that is going to remove Xi's 2 term limit.

Past few days VPN in China has been very erratic and hard to connect to, it wouldn't be surprising if China is initiating the DDOS attack to shutdown Signal...

[versable]

DNS is back, service is reachable again:

;; ANSWER SECTION:
textsecure-service.whispersystems.org. 53 IN A  52.20.12.194
textsecure-service.whispersystems.org. 53 IN A  52.70.150.128
textsecure-service.whispersystems.org. 53 IN A  52.45.240.105
textsecure-service.whispersystems.org. 53 IN A  52.4.185.129
textsecure-service.whispersystems.org. 53 IN A  52.2.243.87
textsecure-service.whispersystems.org. 53 IN A  52.70.14.137
textsecure-service.whispersystems.org. 53 IN A  52.2.222.12
textsecure-service.whispersystems.org. 53 IN A  52.54.15.42

[supremeleaderNK]

The users should seize the means of ~production~ communication and run signal servers at home. I am willing to do so myself. After all it is not rocket science.

[NFH26]

It’s up now in Saudi

On Mon, 5 Mar 2018 at 12:47 PM versable [e-mail] wrote:

DNS is back, service is reachable again:

;; ANSWER SECTION:textsecure-service.whispersystems.org. 53 IN A 52.20.12.194textsecure-service.whispersystems.org. 53 IN A 52.70.150.128textsecure-service.whispersystems.org. 53 IN A 52.45.240.105textsecure-service.whispersystems.org. 53 IN A 52.4.185.129textsecure-service.whispersystems.org. 53 IN A 52.2.243.87textsecure-service.whispersystems.org. 53 IN A 52.70.14.137textsecure-service.whispersystems.org. 53 IN A 52.2.222.12textsecure-service.whispersystems.org. 53 IN A 52.54.15.42

—
You are receiving this because you commented.

Reply to this email directly, view it on GitHub
https://github.com/signalapp/Signal-Android/issues/7460#issuecomment-370364464,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AFoVdPOtXY9eM3SC9B-WoAilxazoVIoOks5tbQmZgaJpZM4Sb2Nv
.

>

~ Nayef Alharbi
Email: Alharbi.[email protected]
Mobile# [phone]

[MartinX3]

I would prefer a dedicated Hardware with "plug and play".
The energy costs would be too high, if i run my Computer 24/7.
Als I don't want to trash it with severall services.
It could cut my performance also.

Signal.org could also sell the raspi's to get a " donation" from every sold unit.

Am 5. März 2018 10:45:02 MEZ schrieb Ryan Shaw [e-mail]:

@MartinX3 or just cut the hardware cost and allow people to install a
background service on their servers and PCs

--
You are receiving this because you were mentioned.
Reply to this email directly or view it on GitHub:
https://github.com/signalapp/Signal-Android/issues/7460#issuecomment-370363832

--
Mit freundlichen Grüßen

Martin Dünkelmann

[ryannathans]

We know it's on, please don't spam saying it's on.

@versable intriguing, we already have multiple servers judging by those dns records. We clearly need decentralised systems if we're to resist such attacks in the future.

@MartinX3 you can buy your own raspberry and put the service on that then if you wish :p

[hismayilov]

Down in Azerbaijan too.

[MartinX3]

I would prefer to buy a pre configured raspi to donate to signal.org :)
I also don't have the time to develop a service, which would do the task.
It would be better, if the signal developers develop it.
They know the infrastructure far better.

They could offer a preconfigured linux distro delievered by PINN or the other raspi bootloader. :)

Am 5. März 2018 10:52:26 MEZ schrieb Ryan Shaw [e-mail]:

We know it's on, please don't spam saying it's on.

@versable intriguing, we already have multiple servers judging by those
dns records. We clearly need decentralised systems if we're to resist
such attacks in the future.

@MartinX3 you can buy your own raspberry and put the service on that
then if you wish :p

--
You are receiving this because you were mentioned.
Reply to this email directly or view it on GitHub:
https://github.com/signalapp/Signal-Android/issues/7460#issuecomment-370365945

--
Mit freundlichen Grüßen

Martin Dünkelmann

[MartinX3]

Up in germany, thanks tomthe developers. :)

[3rg1s]

They are just a small team, shouldn’t take more to bring the service up!

Edit: Up in my Country :)

[ghost]

Doesn't they use AWS servers to manage everything? Keeping it on rPI would be silly and completely dependent on the poor bandwidth(no offence but not everyone has 1GB / s) of the people who host it.

[petcap]

You can add this to your hosts file as a temporary fix:
52.22.204.209 textsecure-service.whispersystems.org

[MartinX3]

Scale vertical instead of horizontal would be the magic keyword here.
Combined with a smart load balancer like any DNS
The more raspis are connected, the stronger is the network against failures.

Am 5. März 2018 11:01:17 MEZ schrieb angaunba [e-mail]:

Doesn't they use AWS servers to manage everything? Keeping it on rPI
would be silly and completely dependent on the poor bandwidth(no
offence but not everyone has 1GB / s) of the people who host it.

--
You are receiving this because you were mentioned.
Reply to this email directly or view it on GitHub:
https://github.com/signalapp/Signal-Android/issues/7460#issuecomment-370368410

--
Mit freundlichen Grüßen

Martin Dünkelmann

[KarolTrzeszczkowski]

Up in Poland :)

[ar329]

Up in Canada. Yay!!

[GAS85]

@Cerberus0 Still no info...

[537]

Up in Bahrain!

[versable]

@NayefAlharbi, you might want to redact your phone number

[MartinX3]

We need an info page on the homepage about the service status.
https://github.com/signalapp/Signal-Android/issues/3835
https://github.com/signalapp/Signal-Android/issues/7461

[vanitasvitae]

https://signal.org/blog/the-ecosystem-is-moving/
:smile:

[ghost]

So no information about what really happened? Something more than _"brief outage that has been resolved"_ https://github.com/signalapp/Signal-Desktop/issues/2106#issuecomment-370502588

[mawed]

Just for documentation:

https://mobile.twitter.com/signalapp/status/970751021445144576

[ghost]

Service was disrupted for few hours and I just don't believe it was by "huge user traffic". If you had a bug or someone fucked up something then no problem no one is perfect but you should explain. As a foundation you should be one hundred percent transparent with your users and this explanation is just "complete and utter garbage" and "bullshit" as would Linus say.

[ryannathans]

Looks like someone got DoSed and doesn't want to admit it ;)

Servers can probably be scaled to anyone who wishes to volunteer a node without "federating". Server version doesn't match? Don't let the node into the DNS load balancing pool. Throw in auto updating and make the binaries public too for new nodes and authenticity sake. Heck, you could just use some generic containerised approach.

Either that or find another way to circumvent DoS attacks..

[sigenc]

yeah i too don't believe this. It looks like ddos, we need to have the ability to run nodes that have remote attestation

[rainerzufall]

Let's all stop discussing this here on github. Let's do the discussion in the community forum: https://community.signalusers.org

Additionally Signal just answered on twitter:

The outage was not caused by a DDoS or any other type of attack. (https://twitter.com/signalapp/status/970781128331214849?s=20)