Arduino: SecureWifiClient::verify always returning false after MQTT connected

[jeffcharles]

Basic Info

Hardware

Hardware: ESP-12
Core Version: 2.3.0-rc1

Description

When I attempt to verify WifiClientSecure connection, after connecting to an MQTT server using https://github.com/CanTireInnovations/pubsubclient, I always get a false result, even when the fingerprint provided matches what's on the certificate. Using 2.2.0 instead of 2.3.0-rc1 results in the correct behaviour.

The server I'm connecting to in the sketch below is iotmqtt.cantireinnovations.com on port 8883. The commands I ran to fetch the fingerprint were:

$ openssl s_client -servername iotmqtt.cantireinnovations.com -connect iotmqtt.cantireinnovations.com:8883 < /dev/null 2>/dev/null | openssl x509 -fingerprint -noout -in /dev/stdin
SHA1 Fingerprint=D1:D0:43:2A:2E:DD:19:84:19:D4:AF:FF:F9:60:EC:88:03:D7:82:7E

$ dig +short iotmqtt.cantireinnovations.com
52.70.42.71
52.6.63.156

$ openssl s_client -servername iotmqtt.cantireinnovations.com -connect 52.70.42.71:8883 < /dev/null 2>/dev/null | openssl x509 -fingerprint -noout -in /dev/stdin
SHA1 Fingerprint=D1:D0:43:2A:2E:DD:19:84:19:D4:AF:FF:F9:60:EC:88:03:D7:82:7E

$ openssl s_client -servername iotmqtt.cantireinnovations.com -connect 52.6.63.156:8883 < /dev/null 2>/dev/null | openssl x509 -fingerprint -noout -in /dev/stdin
SHA1 Fingerprint=D1:D0:43:2A:2E:DD:19:84:19:D4:AF:FF:F9:60:EC:88:03:D7:82:7E

Settings in IDE

Module: Adafruit Huzzah ESP8266
Flash Size: 4MB
CPU Frequency: 80Mhz
Upload Using: SERIAL

Sketch

bool connectMqtt() {

  if( mqttClient.connect( config.userDeviceId, config.userDeviceId, config.mqttPassword ) ) {
    if ( !wifiClientSecure.verify( "D1:D0:43:2A:2E:DD:19:84:19:D4:AF:FF:F9:60:EC:88:03:D7:82:7E", "iotmqtt.cantireinnovations.com" ) ) {
      logger.println( "Connected to broker but failed to verify MQTT certificate" );
      mqttClient.disconnect();
      return false;
    }

    logger.connectivity().println( "Connected to MQTT broker" );

    onMqttConnected();

    return true;

  } else {
    logger.connectivity().printf( "MQTT connection failed: %s\n", mqttStateStr() ); 

    return false;
  }
}

Debug Messages

ssl/tls1.c:549 malloc 6864, left 21488
please start sntp first !
State:  sending Client Hello (1)
State:  receiving Server Hello (2)
State:  receiving Certificate (11)
crypto/bigint.c:1072 realloc 1032, left 17032
crypto/bigint.c:1072 realloc 1032, left 13472
State:  receiving Server Hello Done (14)
crypto/bigint.c:1072 realloc 1024, left 10392
State:  sending Client Key Exchange (16)
State:  sending Finished (16)
State:  receiving Finished (16)
:wcs ra 4fingerprint doesn't match
Connected to broker but failed to verify MQTT certificate

Want to back this issue? Post a bounty on it! We accept bounties via Bountysource.

[igrr]

Can you please post the part of the sketch where you are passing host name to mqttClient?

[jeffcharles]

WiFiClientSecure wifiClientSecure;
WiFiClient wifiClient;
PubSubClient mqttClient;

void setupMqtt() {

  mqttClient = PubSubClient(
        // config.mqttHost,
        // "192.168.2.1",
        "iotmqtt.cantireinnovations.com",
        // config.mqttPort,
        8883,
        mqttCallback,
        // wifiClient
        wifiClientSecure
    );

  logger.print( "Connecting to " );
  logger.print( config.mqttHost );
  logger.print( ":" );
  logger.print( config.mqttPort );
  logger.print( " as " );
  logger.println( config.userDeviceId );
}

[igrr]

Okay, the issue comes from the fact that you can only call WiFiClientSecure::verify _after_ WiFiClientSecure::connect has finished and _before_ any read/write calls have been done. It appears that PubSubClient::connect does some reading and writing, so you can not call verify afterwards.
This may be resolved in two ways:

• when #1851 is implemented, we can add a function to load trusted certificates into WiFiClientSecure before WiFiClientSecure::connect is called. verify will not be necessary because connect will do verification internally and bail out if verification fails.
• add a hook (callback) to PubSubClient which would allow to call verify between client.connect and read/write operations

[igrr]

Actually, given the fact that MQTT handshake is quite short, we can work around this issue. Instead of purging X.509 certificates on read/write operations, we can purge them lazily, if more memory is needed for fragment buffer. I have checked that this fixes verification issue.

Edit: should be fixed in 0f0386e.