Iotedge: Error creating a device-to-cloud connection

[yenseaman]

Expected Behavior

Current Behavior

Steps to Reproduce

1.
2.
3.
4.

Context (Environment)

Output of iotedge check

Click here

~#  iotedge check --verbose
    Configuration checks
    --------------------
    √ config.yaml is well-formed
    √ config.yaml has well-formed connection string
    √ container engine is installed and functional
    √ config.yaml has correct hostname
    × config.yaml has correct URIs for daemon mgmt endpoint
        Unable to find image 'mcr.microsoft.com/azureiotedge-diagnostics:1.0.7' locally
        1.0.7: Pulling from azureiotedge-diagnostics
        docker: no matching manifest for linux/arm64 in the manifest list entries.
        See 'docker run --help'.
            caused by: docker returned exit code: 125, stderr = Unable to find image 'mcr.microsoft.com/azureiotedge-diagnostics:1.0.7' locally
                       1.0.7: Pulling from azureiotedge-diagnostics
                       docker: no matching manifest for linux/arm64 in the manifest list entries.
                       See 'docker run --help'.
    ‼ latest security daemon
        Installed IoT Edge daemon has version 1.0.7 but version 1.0.8 is available.
        Please see https://aka.ms/iotedge-update-runtime for update instructions.
    √ host time is close to real time
    × container time is close to host time
        Could not query local time inside container
            caused by: docker returned exit code: 125, stderr = Unable to find image 'mcr.microsoft.com/azureiotedge-diagnostics:1.0.7' locally
                       1.0.7: Pulling from azureiotedge-diagnostics
                       docker: no matching manifest for linux/arm64 in the manifest list entries.
                       See 'docker run --help'.
    √ DNS server
    ‼ production readiness: certificates
        Device is using self-signed, automatically generated certs.
        Please see https://aka.ms/iotedge-prod-checklist-certs for best practices.
    √ production readiness: certificates expiry
    ‼ production readiness: container engine
        Device is not using a production-supported container engine (moby-engine).
        Please see https://aka.ms/iotedge-prod-checklist-moby for details.
    ‼ production readiness: logs policy
        Container engine is not configured to rotate module logs which may cause it run out of disk space.
        Please see https://aka.ms/iotedge-prod-checklist-logs for best practices.
        You can ignore this warning if you are setting log policy per module in the Edge deployment.

    Connectivity checks
    -------------------
    √ host can connect to and perform TLS handshake with IoT Hub AMQP port
    √ host can connect to and perform TLS handshake with IoT Hub HTTPS port
    √ host can connect to and perform TLS handshake with IoT Hub MQTT port
    × container on the default network can connect to IoT Hub AMQP port
        Container on the default network could not connect to AcompanyHub.azure-devices.net:5671
            caused by: docker returned exit code: 125, stderr = Unable to find image 'mcr.microsoft.com/azureiotedge-diagnostics:1.0.7' locally
                       1.0.7: Pulling from azureiotedge-diagnostics
                       docker: no matching manifest for linux/arm64 in the manifest list entries.
                       See 'docker run --help'.
    × container on the default network can connect to IoT Hub HTTPS port
        Container on the default network could not connect to AcompanyHub.azure-devices.net:443
            caused by: docker returned exit code: 125, stderr = Unable to find image 'mcr.microsoft.com/azureiotedge-diagnostics:1.0.7' locally
                       1.0.7: Pulling from azureiotedge-diagnostics
                       docker: no matching manifest for linux/arm64 in the manifest list entries.
                       See 'docker run --help'.
    × container on the default network can connect to IoT Hub MQTT port
        Container on the default network could not connect to AcompanyHub.azure-devices.net:8883
            caused by: docker returned exit code: 125, stderr = Unable to find image 'mcr.microsoft.com/azureiotedge-diagnostics:1.0.7' locally
                       1.0.7: Pulling from azureiotedge-diagnostics
                       docker: no matching manifest for linux/arm64 in the manifest list entries.
                       See 'docker run --help'.
    × container on the IoT Edge module network can connect to IoT Hub AMQP port
        Container on the azure-iot-edge network could not connect to AcompanyHub.azure-devices.net:5671
            caused by: docker returned exit code: 125, stderr = Unable to find image 'mcr.microsoft.com/azureiotedge-diagnostics:1.0.7' locally
                       1.0.7: Pulling from azureiotedge-diagnostics
                       docker: no matching manifest for linux/arm64 in the manifest list entries.
                       See 'docker run --help'.
    × container on the IoT Edge module network can connect to IoT Hub HTTPS port
        Container on the azure-iot-edge network could not connect to AcompanyHub.azure-devices.net:443
            caused by: docker returned exit code: 125, stderr = Unable to find image 'mcr.microsoft.com/azureiotedge-diagnostics:1.0.7' locally
                       1.0.7: Pulling from azureiotedge-diagnostics
                       docker: no matching manifest for linux/arm64 in the manifest list entries.
                       See 'docker run --help'.
    × container on the IoT Edge module network can connect to IoT Hub MQTT port
        Container on the azure-iot-edge network could not connect to AcompanyHub.azure-devices.net:8883
            caused by: docker returned exit code: 125, stderr = Unable to find image 'mcr.microsoft.com/azureiotedge-diagnostics:1.0.7' locally
                       1.0.7: Pulling from azureiotedge-diagnostics
                       docker: no matching manifest for linux/arm64 in the manifest list entries.
                       See 'docker run --help'.
    × Edge Hub can bind to ports on host
        Could not check current state of Edge Hub container
            caused by: docker returned exit code: 1, stderr = Error response from daemon: no such image: edgeHub: invalid reference format: repository name must be lowercase

    One or more checks raised errors.

Device (Host) Operating System

~# uname -a
Linux 4.4.124t #1 SMP PREEMPT Thu Aug 29 01:03:14 UTC 2019 aarch64 aarch64 aarch64 GNU/Linux

Architecture

Container Operating System

Runtime Versions

iotedged

Edge Agent

Edge Hub

Docker

    ~# docker --version
    Docker version 18.03.0, build 0f1bb35

Logs

nmap

root@b9b6b991e942:/app# nmap -p5671 seamanhub.azure-devices.net

Starting Nmap 7.60 ( https://nmap.org ) at 2019-09-25 03:36 UTC
Nmap scan report for seamanhub.azure-devices.net (13.89.231.149)
Host is up (0.21s latency).

PORT     STATE SERVICE
5671/tcp open  amqps

Nmap done: 1 IP address (1 host up) scanned in 6.33 seconds
root@b9b6b991e942:/app# nmap -p8883 seamanhub.azure-devices.net

Starting Nmap 7.60 ( https://nmap.org ) at 2019-09-25 03:39 UTC
Nmap scan report for seamanhub.azure-devices.net (13.89.231.149)
Host is up (0.21s latency).

PORT     STATE SERVICE
8883/tcp open  secure-mqtt

Nmap done: 1 IP address (1 host up) scanned in 6.17 seconds
root@b9b6b991e942:/app# nmap -p443 seamanhub.azure-devices.net

Starting Nmap 7.60 ( https://nmap.org ) at 2019-09-25 03:40 UTC
Nmap scan report for seamanhub.azure-devices.net (13.89.231.149)
Host is up (0.21s latency).

PORT    STATE SERVICE
443/tcp open  https

Nmap done: 1 IP address (1 host up) scanned in 6.14 seconds

edge-agent logs

    # docker logs edgeAgent

    2019-09-23 01:35:15 +00:00 Starting Edge Agent
    2019-09-23 01:35:16.601 +00:00 Edge Agent Main()
    <6> 2019-09-23 01:35:18.203 +00:00 [INF] [Microsoft.Azure.Devices.Edge.Agent.Service.Program] - Initializing Edge Agent.
    <6> 2019-09-23 01:35:19.267 +00:00 [INF] [Microsoft.Azure.Devices.Edge.Agent.Service.Program] - Version - 1.0.8.1.24554792 (8a9cfedfc6c7455c080dcee6ee21978069024d2f)
    <6> 2019-09-23 01:35:19.269 +00:00 [INF] [Microsoft.Azure.Devices.Edge.Agent.Service.Program] -
    <LOGO skipped>
    <6> 2019-09-23 01:35:19.650 +00:00 [INF] [Microsoft.Azure.Devices.Edge.Agent.Service.Program] - Experimental features configuration: {"Enabled":false,"DisableCloudSubscriptions":false,"EnableUploadLogs":false}
    <7> 2019-09-23 01:35:20.100 +00:00 [DBG] [Microsoft.Azure.Devices.Edge.Agent.Service.Program] - Waiting on shutdown handler to trigger
    <6> 2019-09-23 01:35:22.015 +00:00 [INF] [Microsoft.Azure.Devices.Edge.Agent.IoTHub.EdgeAgentConnection] - Started operation refresh twin config
    <7> 2019-09-23 01:35:22.051 +00:00 [DBG] [Microsoft.Azure.Devices.Edge.Agent.IoTHub.EdgeAgentConnection] - Begin refreshing twin from upstream...
    <7> 2019-09-23 01:35:22.094 +00:00 [DBG] [Microsoft.Azure.Devices.Edge.Agent.IoTHub.EdgeAgentConnection] - Getting module client to refresh the twin
    <7> 2019-09-23 01:35:22.166 +00:00 [DBG] [Microsoft.Azure.Devices.Edge.Agent.IoTHub.EdgeAgentConnection] - Initialize twin refresh with interval '01:00:00'
    <6> 2019-09-23 01:35:22.190 +00:00 [INF] [Microsoft.Azure.Devices.Edge.Agent.IoTHub.ModuleClientProvider] - Edge agent attempting to connect to IoT Hub via Amqp_Tcp_Only...
    <7> 2019-09-23 01:35:22.321 +00:00 [DBG] [Microsoft.Azure.Devices.Edge.Agent.IoTHub.ConfigSources.TwinConfigSource] - TwinConfigSource Created
    <7> 2019-09-23 01:35:22.330 +00:00 [DBG] [Microsoft.Azure.Devices.Edge.Agent.Core.ConfigSources.FileBackupConfigSource] - Edge agent config backup created here - backup.json
    <6> 2019-09-23 01:35:24.929 +00:00 [INF] [Microsoft.Azure.Devices.Edge.Agent.Service.Modules.AgentModule] - Created persistent store at /tmp/edgeAgent
    <7> 2019-09-23 01:35:25.028 +00:00 [DBG] [Microsoft.Azure.Devices.Edge.Agent.Edgelet.ModuleManagementHttpClient] - Making a Http call to unix:///var/run/iotedge/mgmt.sock to Getting System Info
    <7> 2019-09-23 01:35:25.168 +00:00 [DBG] [Microsoft.Azure.Devices.Edge.Util.Uds.HttpUdsMessageHandler] - Connecting socket /var/run/iotedge/mgmt.sock
    <7> 2019-09-23 01:35:25.193 +00:00 [DBG] [Microsoft.Azure.Devices.Edge.Util.Uds.HttpUdsMessageHandler] - Connected socket /var/run/iotedge/mgmt.sock
    <7> 2019-09-23 01:35:25.229 +00:00 [DBG] [Microsoft.Azure.Devices.Edge.Util.Uds.HttpUdsMessageHandler] - Sending request http://mgmt.sock/systeminfo?api-version=2019-01-30
    <7> 2019-09-23 01:35:25.440 +00:00 [DBG] [Microsoft.Azure.Devices.Edge.Util.Uds.HttpUdsMessageHandler] - Response received OK
    <7> 2019-09-23 01:35:25.587 +00:00 [DBG] [Microsoft.Azure.Devices.Edge.Agent.Edgelet.ModuleManagementHttpClient] - Received a valid Http response from unix:///var/run/iotedge/mgmt.sock for Getting System Info
    <7> 2019-09-23 01:35:25.867 +00:00 [DBG] [Microsoft.Azure.Devices.Edge.Agent.Core.Agent] - Edge agent created.
    <7> 2019-09-23 01:35:25.921 +00:00 [DBG] [Microsoft.Azure.Devices.Edge.Agent.Core.Agent] - Starting reconcile operation
    <7> 2019-09-23 01:35:26.024 +00:00 [DBG] [Microsoft.Azure.Devices.Edge.Agent.Edgelet.ModuleManagementHttpClient] - Making a Http call to unix:///var/run/iotedge/mgmt.sock to List modules
    <7> 2019-09-23 01:35:26.066 +00:00 [DBG] [Microsoft.Azure.Devices.Edge.Util.Uds.HttpUdsMessageHandler] - Connecting socket /var/run/iotedge/mgmt.sock
    <7> 2019-09-23 01:35:26.068 +00:00 [DBG] [Microsoft.Azure.Devices.Edge.Util.Uds.HttpUdsMessageHandler] - Connected socket /var/run/iotedge/mgmt.sock
    <7> 2019-09-23 01:35:26.069 +00:00 [DBG] [Microsoft.Azure.Devices.Edge.Util.Uds.HttpUdsMessageHandler] - Sending request http://mgmt.sock/modules?api-version=2019-01-30
    <7> 2019-09-23 01:35:26.123 +00:00 [DBG] [Microsoft.Azure.Devices.Edge.Util.Uds.HttpUdsMessageHandler] - Response received OK
    <7> 2019-09-23 01:35:26.193 +00:00 [DBG] [Microsoft.Azure.Devices.Edge.Agent.Core.Agent] - Getting edge agent config...
    <7> 2019-09-23 01:35:26.441 +00:00 [DBG] [Microsoft.Azure.Devices.Edge.Agent.Edgelet.ModuleManagementHttpClient] - Received a valid Http response from unix:///var/run/iotedge/mgmt.sock for List modules
    <4> 2019-09-23 01:35:31.271 +00:00 [WRN] [Microsoft.Azure.Devices.Edge.Agent.Core.ConfigSources.FileBackupConfigSource] - Empty edge agent config was received. Attempting to read config from backup file (backup.json) instead
    <6> 2019-09-23 01:35:31.302 +00:00 [INF] [Microsoft.Azure.Devices.Edge.Agent.Core.ConfigSources.FileBackupConfigSource] - Edge agent config backup file does not exist - backup.json
    <7> 2019-09-23 01:35:31.384 +00:00 [DBG] [Microsoft.Azure.Devices.Edge.Agent.Core.Agent] - Finished reconcile operation
    <7> 2019-09-23 01:35:33.479 +00:00 [DBG] [Microsoft.Azure.Devices.Edge.Agent.IoTHub.EdgeAgentConnection] - Connection status changed to Disconnected with reason Retry_Expired
    <7> 2019-09-23 01:35:36.398 +00:00 [DBG] [Microsoft.Azure.Devices.Edge.Agent.Core.Agent] - Starting reconcile operation
    <7> 2019-09-23 01:35:36.400 +00:00 [DBG] [Microsoft.Azure.Devices.Edge.Agent.Edgelet.ModuleManagementHttpClient] - Making a Http call to unix:///var/run/iotedge/mgmt.sock to List modules
    <7> 2019-09-23 01:35:36.403 +00:00 [DBG] [Microsoft.Azure.Devices.Edge.Util.Uds.HttpUdsMessageHandler] - Connecting socket /var/run/iotedge/mgmt.sock
    <7> 2019-09-23 01:35:36.406 +00:00 [DBG] [Microsoft.Azure.Devices.Edge.Util.Uds.HttpUdsMessageHandler] - Connected socket /var/run/iotedge/mgmt.sock
    <7> 2019-09-23 01:35:36.410 +00:00 [DBG] [Microsoft.Azure.Devices.Edge.Util.Uds.HttpUdsMessageHandler] - Sending request http://mgmt.sock/modules?api-version=2019-01-30
    <7> 2019-09-23 01:35:36.426 +00:00 [DBG] [Microsoft.Azure.Devices.Edge.Agent.Core.Agent] - Getting edge agent config...
    <7> 2019-09-23 01:35:36.436 +00:00 [DBG] [Microsoft.Azure.Devices.Edge.Util.Uds.HttpUdsMessageHandler] - Response received OK
    <7> 2019-09-23 01:35:36.440 +00:00 [DBG] [Microsoft.Azure.Devices.Edge.Agent.Edgelet.ModuleManagementHttpClient] - Received a valid Http response from unix:///var/run/iotedge/mgmt.sock for List modules
    <4> 2019-09-23 01:35:41.436 +00:00 [WRN] [Microsoft.Azure.Devices.Edge.Agent.Core.ConfigSources.FileBackupConfigSource] - Empty edge agent config was received. Attempting to read config from backup file (backup.json) instead
    <6> 2019-09-23 01:35:41.437 +00:00 [INF] [Microsoft.Azure.Devices.Edge.Agent.Core.ConfigSources.FileBackupConfigSource] - Edge agent config backup file does not exist - backup.json
    <7> 2019-09-23 01:35:41.439 +00:00 [DBG] [Microsoft.Azure.Devices.Edge.Agent.Core.Agent] - Finished reconcile operation
    <7> 2019-09-23 01:35:46.441 +00:00 [DBG] [Microsoft.Azure.Devices.Edge.Agent.Core.Agent] - Starting reconcile operation
    <7> 2019-09-23 01:35:46.442 +00:00 [DBG] [Microsoft.Azure.Devices.Edge.Agent.Edgelet.ModuleManagementHttpClient] - Making a Http call to unix:///var/run/iotedge/mgmt.sock to List modules
    <7> 2019-09-23 01:35:46.443 +00:00 [DBG] [Microsoft.Azure.Devices.Edge.Util.Uds.HttpUdsMessageHandler] - Connecting socket /var/run/iotedge/mgmt.sock
    <7> 2019-09-23 01:35:46.447 +00:00 [DBG] [Microsoft.Azure.Devices.Edge.Util.Uds.HttpUdsMessageHandler] - Connected socket /var/run/iotedge/mgmt.sock
    <7> 2019-09-23 01:35:46.450 +00:00 [DBG] [Microsoft.Azure.Devices.Edge.Util.Uds.HttpUdsMessageHandler] - Sending request http://mgmt.sock/modules?api-version=2019-01-30
    <7> 2019-09-23 01:35:46.467 +00:00 [DBG] [Microsoft.Azure.Devices.Edge.Util.Uds.HttpUdsMessageHandler] - Response received OK
    <7> 2019-09-23 01:35:46.470 +00:00 [DBG] [Microsoft.Azure.Devices.Edge.Agent.Core.Agent] - Getting edge agent config...
    <7> 2019-09-23 01:35:46.476 +00:00 [DBG] [Microsoft.Azure.Devices.Edge.Agent.Edgelet.ModuleManagementHttpClient] - Received a valid Http response from unix:///var/run/iotedge/mgmt.sock for List modules
    <4> 2019-09-23 01:35:51.473 +00:00 [WRN] [Microsoft.Azure.Devices.Edge.Agent.Core.ConfigSources.FileBackupConfigSource] - Empty edge agent config was received. Attempting to read config from backup file (backup.json) instead
    <6> 2019-09-23 01:35:51.474 +00:00 [INF] [Microsoft.Azure.Devices.Edge.Agent.Core.ConfigSources.FileBackupConfigSource] - Edge agent config backup file does not exist - backup.json
    <7> 2019-09-23 01:35:51.475 +00:00 [DBG] [Microsoft.Azure.Devices.Edge.Agent.Core.Agent] - Finished reconcile operation
    <6> 2019-09-23 01:35:53.876 +00:00 [INF] [Microsoft.Azure.Devices.Edge.Agent.IoTHub.ModuleClientProvider] - Edge agent attempting to connect to IoT Hub via Amqp_WebSocket_Only...
    <7> 2019-09-23 01:35:56.479 +00:00 [DBG] [Microsoft.Azure.Devices.Edge.Agent.Core.Agent] - Starting reconcile operation
    <7> 2019-09-23 01:35:56.480 +00:00 [DBG] [Microsoft.Azure.Devices.Edge.Agent.Edgelet.ModuleManagementHttpClient] - Making a Http call to unix:///var/run/iotedge/mgmt.sock to List modules
    <7> 2019-09-23 01:35:56.482 +00:00 [DBG] [Microsoft.Azure.Devices.Edge.Util.Uds.HttpUdsMessageHandler] - Connecting socket /var/run/iotedge/mgmt.sock
    <7> 2019-09-23 01:35:56.484 +00:00 [DBG] [Microsoft.Azure.Devices.Edge.Util.Uds.HttpUdsMessageHandler] - Connected socket /var/run/iotedge/mgmt.sock
    <7> 2019-09-23 01:35:56.491 +00:00 [DBG] [Microsoft.Azure.Devices.Edge.Util.Uds.HttpUdsMessageHandler] - Sending request http://mgmt.sock/modules?api-version=2019-01-30
    <7> 2019-09-23 01:35:56.504 +00:00 [DBG] [Microsoft.Azure.Devices.Edge.Agent.Core.Agent] - Getting edge agent config...
    <7> 2019-09-23 01:35:56.509 +00:00 [DBG] [Microsoft.Azure.Devices.Edge.Util.Uds.HttpUdsMessageHandler] - Response received OK
    <7> 2019-09-23 01:35:56.511 +00:00 [DBG] [Microsoft.Azure.Devices.Edge.Agent.Edgelet.ModuleManagementHttpClient] - Received a valid Http response from unix:///var/run/iotedge/mgmt.sock for List modules
    <4> 2019-09-23 01:36:01.507 +00:00 [WRN] [Microsoft.Azure.Devices.Edge.Agent.Core.ConfigSources.FileBackupConfigSource] - Empty edge agent config was received. Attempting to read config from backup file (backup.json) instead
    <6> 2019-09-23 01:36:01.508 +00:00 [INF] [Microsoft.Azure.Devices.Edge.Agent.Core.ConfigSources.FileBackupConfigSource] - Edge agent config backup file does not exist - backup.json
    <7> 2019-09-23 01:36:01.509 +00:00 [DBG] [Microsoft.Azure.Devices.Edge.Agent.Core.Agent] - Finished reconcile operation
    <7> 2019-09-23 01:36:04.254 +00:00 [DBG] [Microsoft.Azure.Devices.Edge.Agent.IoTHub.EdgeAgentConnection] - Connection status changed to Disconnected with reason Retry_Expired
    <7> 2019-09-23 01:36:06.514 +00:00 [DBG] [Microsoft.Azure.Devices.Edge.Agent.Core.Agent] - Starting reconcile operation
    <7> 2019-09-23 01:36:06.514 +00:00 [DBG] [Microsoft.Azure.Devices.Edge.Agent.Edgelet.ModuleManagementHttpClient] - Making a Http call to unix:///var/run/iotedge/mgmt.sock to List modules
    <7> 2019-09-23 01:36:06.515 +00:00 [DBG] [Microsoft.Azure.Devices.Edge.Util.Uds.HttpUdsMessageHandler] - Connecting socket /var/run/iotedge/mgmt.sock
    <7> 2019-09-23 01:36:06.516 +00:00 [DBG] [Microsoft.Azure.Devices.Edge.Util.Uds.HttpUdsMessageHandler] - Connected socket /var/run/iotedge/mgmt.sock
    <7> 2019-09-23 01:36:06.517 +00:00 [DBG] [Microsoft.Azure.Devices.Edge.Util.Uds.HttpUdsMessageHandler] - Sending request http://mgmt.sock/modules?api-version=2019-01-30
    <7> 2019-09-23 01:36:06.530 +00:00 [DBG] [Microsoft.Azure.Devices.Edge.Agent.Core.Agent] - Getting edge agent config...
    <7> 2019-09-23 01:36:06.540 +00:00 [DBG] [Microsoft.Azure.Devices.Edge.Util.Uds.HttpUdsMessageHandler] - Response received OK
    <7> 2019-09-23 01:36:06.545 +00:00 [DBG] [Microsoft.Azure.Devices.Edge.Agent.Edgelet.ModuleManagementHttpClient] - Received a valid Http response from unix:///var/run/iotedge/mgmt.sock for List modules
    <4> 2019-09-23 01:36:11.534 +00:00 [WRN] [Microsoft.Azure.Devices.Edge.Agent.Core.ConfigSources.FileBackupConfigSource] - Empty edge agent config was received. Attempting to read config from backup file (backup.json) instead
    <6> 2019-09-23 01:36:11.534 +00:00 [INF] [Microsoft.Azure.Devices.Edge.Agent.Core.ConfigSources.FileBackupConfigSource] - Edge agent config backup file does not exist - backup.json
    <7> 2019-09-23 01:36:11.535 +00:00 [DBG] [Microsoft.Azure.Devices.Edge.Agent.Core.Agent] - Finished reconcile operation
    <7> 2019-09-23 01:36:16.549 +00:00 [DBG] [Microsoft.Azure.Devices.Edge.Agent.Core.Agent] - Starting reconcile operation
    <7> 2019-09-23 01:36:16.549 +00:00 [DBG] [Microsoft.Azure.Devices.Edge.Agent.Edgelet.ModuleManagementHttpClient] - Making a Http call to unix:///var/run/iotedge/mgmt.sock to List modules
    <7> 2019-09-23 01:36:16.550 +00:00 [DBG] [Microsoft.Azure.Devices.Edge.Util.Uds.HttpUdsMessageHandler] - Connecting socket /var/run/iotedge/mgmt.sock
    <7> 2019-09-23 01:36:16.550 +00:00 [DBG] [Microsoft.Azure.Devices.Edge.Util.Uds.HttpUdsMessageHandler] - Connected socket /var/run/iotedge/mgmt.sock
    <7> 2019-09-23 01:36:16.551 +00:00 [DBG] [Microsoft.Azure.Devices.Edge.Util.Uds.HttpUdsMessageHandler] - Sending request http://mgmt.sock/modules?api-version=2019-01-30
    <7> 2019-09-23 01:36:16.553 +00:00 [DBG] [Microsoft.Azure.Devices.Edge.Agent.Core.Agent] - Getting edge agent config...
    <7> 2019-09-23 01:36:16.567 +00:00 [DBG] [Microsoft.Azure.Devices.Edge.Util.Uds.HttpUdsMessageHandler] - Response received OK
    <7> 2019-09-23 01:36:16.570 +00:00 [DBG] [Microsoft.Azure.Devices.Edge.Agent.Edgelet.ModuleManagementHttpClient] - Received a valid Http response from unix:///var/run/iotedge/mgmt.sock for List modules
    <4> 2019-09-23 01:36:21.555 +00:00 [WRN] [Microsoft.Azure.Devices.Edge.Agent.Core.ConfigSources.FileBackupConfigSource] - Empty edge agent config was received. Attempting to read config from backup file (backup.json) instead
    <6> 2019-09-23 01:36:21.555 +00:00 [INF] [Microsoft.Azure.Devices.Edge.Agent.Core.ConfigSources.FileBackupConfigSource] - Edge agent config backup file does not exist - backup.json
    <7> 2019-09-23 01:36:21.556 +00:00 [DBG] [Microsoft.Azure.Devices.Edge.Agent.Core.Agent] - Finished reconcile operation
    <4> 2019-09-23 01:36:24.618 +00:00 [WRN] [Microsoft.Azure.Devices.Edge.Agent.IoTHub.ModuleClientProvider] - Error creating a device-to-cloud connection
    System.AggregateException: One or more errors occurred. (Transient network error occurred, please retry.) (Transient network error occurred, please retry.) ---> Microsoft.Azure.Devices.Client.Exceptions.IotHubCommunicationException: Transient network error occurred, please retry. ---> System.Net.Sockets.SocketException: Resource temporarily unavailable
       at Microsoft.Azure.Devices.Client.Transport.Amqp.AmqpConnector.InitializeTransport(TimeSpan timeout)
       at Microsoft.Azure.Devices.Client.Transport.Amqp.AmqpConnector.OpenConnectionAsync(TimeSpan timeout)
       at Microsoft.Azure.Devices.Client.Transport.Amqp.AmqpConnectionHolder.EnsureConnection(TimeSpan timeout)
       at Microsoft.Azure.Devices.Client.Transport.Amqp.AmqpConnectionHolder.AmqpSessionCreator(DeviceIdentity deviceIdentity, ILinkFactory linkFactory, AmqpSessionSettings amqpSessionSettings, TimeSpan timeout)
       at Microsoft.Azure.Devices.Client.Transport.Amqp.AmqpUnit.OpenAsync(TimeSpan timeout)
       at Microsoft.Azure.Devices.Client.Transport.Amqp.AmqpTransportHandler.OpenAsync(CancellationToken cancellationToken)
       at Microsoft.Azure.Devices.Client.Transport.ProtocolRoutingDelegatingHandler.OpenAsync(CancellationToken cancellationToken)
       at Microsoft.Azure.Devices.Client.Transport.ErrorDelegatingHandler.<>c__DisplayClass22_0.<<ExecuteWithErrorHandlingAsync>b__0>d.MoveNext()
    --- End of stack trace from previous location where exception was thrown ---
       at Microsoft.Azure.Devices.Client.Transport.ErrorDelegatingHandler.ExecuteWithErrorHandlingAsync[T](Func`1 asyncOperation)
       --- End of inner exception stack trace ---
       at Microsoft.Azure.Devices.Client.Transport.ErrorDelegatingHandler.ExecuteWithErrorHandlingAsync[T](Func`1 asyncOperation)
       at Microsoft.Azure.Devices.Client.Transport.RetryDelegatingHandler.<>c__DisplayClass32_0.<<OpenAsyncInternal>b__0>d.MoveNext()
    --- End of stack trace from previous location where exception was thrown ---
       at Microsoft.Azure.Devices.Client.Transport.RetryDelegatingHandler.EnsureOpenedAsync(CancellationToken cancellationToken)
       at Microsoft.Azure.Devices.Client.InternalClient.OpenAsync()
       at Microsoft.Azure.Devices.Edge.Agent.IoTHub.ModuleClientProvider.CreateAndOpenSdkModuleClient(UpstreamProtocol upstreamProtocol, ConnectionStatusChangesHandler statusChangedHandler) in /home/vsts/work/1/s/edge-agent/src/Microsoft.Azure.Devices.Edge.Agent.IoTHub/ModuleClientProvider.cs:line 184
       at Microsoft.Azure.Devices.Edge.Util.Fallback.ExecuteAsync[T](Func`1[] options)
       --- End of inner exception stack trace ---
    ---> (Inner Exception #0) Microsoft.Azure.Devices.Client.Exceptions.IotHubCommunicationException: Transient network error occurred, please retry. ---> System.Net.Sockets.SocketException: Resource temporarily unavailable
       at Microsoft.Azure.Devices.Client.Transport.Amqp.AmqpConnector.InitializeTransport(TimeSpan timeout)
       at Microsoft.Azure.Devices.Client.Transport.Amqp.AmqpConnector.OpenConnectionAsync(TimeSpan timeout)
       at Microsoft.Azure.Devices.Client.Transport.Amqp.AmqpConnectionHolder.EnsureConnection(TimeSpan timeout)
       at Microsoft.Azure.Devices.Client.Transport.Amqp.AmqpConnectionHolder.AmqpSessionCreator(DeviceIdentity deviceIdentity, ILinkFactory linkFactory, AmqpSessionSettings amqpSessionSettings, TimeSpan timeout)
       at Microsoft.Azure.Devices.Client.Transport.Amqp.AmqpUnit.OpenAsync(TimeSpan timeout)
       at Microsoft.Azure.Devices.Client.Transport.Amqp.AmqpTransportHandler.OpenAsync(CancellationToken cancellationToken)
       at Microsoft.Azure.Devices.Client.Transport.ProtocolRoutingDelegatingHandler.OpenAsync(CancellationToken cancellationToken)
       at Microsoft.Azure.Devices.Client.Transport.ErrorDelegatingHandler.<>c__DisplayClass22_0.<<ExecuteWithErrorHandlingAsync>b__0>d.MoveNext()
    --- End of stack trace from previous location where exception was thrown ---
       at Microsoft.Azure.Devices.Client.Transport.ErrorDelegatingHandler.ExecuteWithErrorHandlingAsync[T](Func`1 asyncOperation)
       --- End of inner exception stack trace ---
       at Microsoft.Azure.Devices.Client.Transport.ErrorDelegatingHandler.ExecuteWithErrorHandlingAsync[T](Func`1 asyncOperation)
       at Microsoft.Azure.Devices.Client.Transport.RetryDelegatingHandler.<>c__DisplayClass32_0.<<OpenAsyncInternal>b__0>d.MoveNext()
    --- End of stack trace from previous location where exception was thrown ---
       at Microsoft.Azure.Devices.Client.Transport.RetryDelegatingHandler.EnsureOpenedAsync(CancellationToken cancellationToken)
       at Microsoft.Azure.Devices.Client.InternalClient.OpenAsync()
       at Microsoft.Azure.Devices.Edge.Agent.IoTHub.ModuleClientProvider.CreateAndOpenSdkModuleClient(UpstreamProtocol upstreamProtocol, ConnectionStatusChangesHandler statusChangedHandler) in /home/vsts/work/1/s/edge-agent/src/Microsoft.Azure.Devices.Edge.Agent.IoTHub/ModuleClientProvider.cs:line 184
       at Microsoft.Azure.Devices.Edge.Util.Fallback.ExecuteAsync[T](Func`1[] options)<---

    ---> (Inner Exception #1) Microsoft.Azure.Devices.Client.Exceptions.IotHubCommunicationException: Transient network error occurred, please retry. ---> System.Net.WebSockets.WebSocketException: Unable to connect to the remote server ---> System.Net.Http.HttpRequestException: Resource temporarily unavailable ---> System.Net.Sockets.SocketException: Resource temporarily unavailable
       at System.Net.Http.ConnectHelper.ConnectAsync(String host, Int32 port, CancellationToken cancellationToken)
       --- End of inner exception stack trace ---
       at System.Net.Http.ConnectHelper.ConnectAsync(String host, Int32 port, CancellationToken cancellationToken)
       at System.Threading.Tasks.ValueTask`1.get_Result()
       at System.Net.Http.HttpConnectionPool.ConnectAsync(HttpRequestMessage request, Boolean allowHttp2, CancellationToken cancellationToken)
       at System.Threading.Tasks.ValueTask`1.get_Result()
       at System.Net.Http.HttpConnectionPool.CreateHttp11ConnectionAsync(HttpRequestMessage request, CancellationToken cancellationToken)
       at System.Threading.Tasks.ValueTask`1.get_Result()
       at System.Net.Http.HttpConnectionPool.GetHttpConnectionAsync(HttpRequestMessage request, CancellationToken cancellationToken)
       at System.Threading.Tasks.ValueTask`1.get_Result()
       at System.Net.Http.HttpConnectionPool.SendWithRetryAsync(HttpRequestMessage request, Boolean doRequestAuth, CancellationToken cancellationToken)
       at System.Net.Http.RedirectHandler.SendAsync(HttpRequestMessage request, CancellationToken cancellationToken)
       at System.Net.WebSockets.WebSocketHandle.ConnectAsyncCore(Uri uri, CancellationToken cancellationToken, ClientWebSocketOptions options)
       --- End of inner exception stack trace ---
       at System.Net.WebSockets.WebSocketHandle.ConnectAsyncCore(Uri uri, CancellationToken cancellationToken, ClientWebSocketOptions options)
       at System.Net.WebSockets.ClientWebSocket.ConnectAsyncCore(Uri uri, CancellationToken cancellationToken)
       at Microsoft.Azure.Devices.Client.Transport.Amqp.AmqpConnector.CreateClientWebSocketAsync(Uri websocketUri, TimeSpan timeout)
       at Microsoft.Azure.Devices.Client.Transport.Amqp.AmqpConnector.CreateClientWebSocketTransportAsync(TimeSpan timeout)
       at Microsoft.Azure.Devices.Client.Transport.Amqp.AmqpConnector.InitializeTransport(TimeSpan timeout)
       at Microsoft.Azure.Devices.Client.Transport.Amqp.AmqpConnector.OpenConnectionAsync(TimeSpan timeout)
       at Microsoft.Azure.Devices.Client.Transport.Amqp.AmqpConnectionHolder.EnsureConnection(TimeSpan timeout)
       at Microsoft.Azure.Devices.Client.Transport.Amqp.AmqpConnectionHolder.AmqpSessionCreator(DeviceIdentity deviceIdentity, ILinkFactory linkFactory, AmqpSessionSettings amqpSessionSettings, TimeSpan timeout)
       at Microsoft.Azure.Devices.Client.Transport.Amqp.AmqpUnit.OpenAsync(TimeSpan timeout)
       at Microsoft.Azure.Devices.Client.Transport.Amqp.AmqpTransportHandler.OpenAsync(CancellationToken cancellationToken)
       at Microsoft.Azure.Devices.Client.Transport.ProtocolRoutingDelegatingHandler.OpenAsync(CancellationToken cancellationToken)
       at Microsoft.Azure.Devices.Client.Transport.ErrorDelegatingHandler.<>c__DisplayClass22_0.<<ExecuteWithErrorHandlingAsync>b__0>d.MoveNext()
    --- End of stack trace from previous location where exception was thrown ---
       at Microsoft.Azure.Devices.Client.Transport.ErrorDelegatingHandler.ExecuteWithErrorHandlingAsync[T](Func`1 asyncOperation)
       --- End of inner exception stack trace ---
       at Microsoft.Azure.Devices.Client.Transport.ErrorDelegatingHandler.ExecuteWithErrorHandlingAsync[T](Func`1 asyncOperation)
       at Microsoft.Azure.Devices.Client.Transport.RetryDelegatingHandler.<>c__DisplayClass32_0.<<OpenAsyncInternal>b__0>d.MoveNext()
    --- End of stack trace from previous location where exception was thrown ---
       at Microsoft.Azure.Devices.Client.Transport.RetryDelegatingHandler.EnsureOpenedAsync(CancellationToken cancellationToken)
       at Microsoft.Azure.Devices.Client.InternalClient.OpenAsync()
       at Microsoft.Azure.Devices.Edge.Agent.IoTHub.ModuleClientProvider.CreateAndOpenSdkModuleClient(UpstreamProtocol upstreamProtocol, ConnectionStatusChangesHandler statusChangedHandler) in /home/vsts/work/1/s/edge-agent/src/Microsoft.Azure.Devices.Edge.Agent.IoTHub/ModuleClientProvider.cs:line 184
       at Microsoft.Azure.Devices.Edge.Util.Fallback.ExecuteAsync[T](Func`1[] options)<---
    <7> 2019-09-23 01:36:24.806 +00:00 [DBG] [Microsoft.Azure.Devices.Edge.Agent.IoTHub.ModuleClientProvider] - Retrying connection to IoT Hub. Current retry count 1.
    <6> 2019-09-23 01:36:24.806 +00:00 [INF] [Microsoft.Azure.Devices.Edge.Agent.IoTHub.ModuleClientProvider] - Edge agent attempting to connect to IoT Hub via Amqp_Tcp_Only...
    <7> 2019-09-23 01:36:26.558 +00:00 [DBG] [Microsoft.Azure.Devices.Edge.Agent.Core.Agent] - Starting reconcile operation
    <7> 2019-09-23 01:36:26.559 +00:00 [DBG] [Microsoft.Azure.Devices.Edge.Agent.Edgelet.ModuleManagementHttpClient] - Making a Http call to unix:///var/run/iotedge/mgmt.sock to List modules
    <7> 2019-09-23 01:36:26.560 +00:00 [DBG] [Microsoft.Azure.Devices.Edge.Util.Uds.HttpUdsMessageHandler] - Connecting socket /var/run/iotedge/mgmt.sock
    <7> 2019-09-23 01:36:26.561 +00:00 [DBG] [Microsoft.Azure.Devices.Edge.Util.Uds.HttpUdsMessageHandler] - Connected socket /var/run/iotedge/mgmt.sock
    <7> 2019-09-23 01:36:26.562 +00:00 [DBG] [Microsoft.Azure.Devices.Edge.Util.Uds.HttpUdsMessageHandler] - Sending request http://mgmt.sock/modules?api-version=2019-01-30
    <7> 2019-09-23 01:36:26.567 +00:00 [DBG] [Microsoft.Azure.Devices.Edge.Agent.Core.Agent] - Getting edge agent config...
    <7> 2019-09-23 01:36:26.585 +00:00 [DBG] [Microsoft.Azure.Devices.Edge.Util.Uds.HttpUdsMessageHandler] - Response received OK
    <7> 2019-09-23 01:36:26.588 +00:00 [DBG] [Microsoft.Azure.Devices.Edge.Agent.Edgelet.ModuleManagementHttpClient] - Received a valid Http response from unix:///var/run/iotedge/mgmt.sock for List modules
    <4> 2019-09-23 01:36:31.568 +00:00 [WRN] [Microsoft.Azure.Devices.Edge.Agent.Core.ConfigSources.FileBackupConfigSource] - Empty edge agent config was received. Attempting to read config from backup file (backup.json) instead
    <6> 2019-09-23 01:36:31.568 +00:00 [INF] [Microsoft.Azure.Devices.Edge.Agent.Core.ConfigSources.FileBackupConfigSource] - Edge agent config backup file does not exist - backup.json
    <7> 2019-09-23 01:36:31.568 +00:00 [DBG] [Microsoft.Azure.Devices.Edge.Agent.Core.Agent] - Finished reconcile operation
    <7> 2019-09-23 01:36:34.820 +00:00 [DBG] [Microsoft.Azure.Devices.Edge.Agent.IoTHub.EdgeAgentConnection] - Connection status changed to Disconnected with reason Retry_Expired
    <7> 2019-09-23 01:36:36.569 +00:00 [DBG] [Microsoft.Azure.Devices.Edge.Agent.Core.Agent] - Starting reconcile operation
    <7> 2019-09-23 01:36:36.569 +00:00 [DBG] [Microsoft.Azure.Devices.Edge.Agent.Edgelet.ModuleManagementHttpClient] - Making a Http call to unix:///var/run/iotedge/mgmt.sock to List modules
    <7> 2019-09-23 01:36:36.570 +00:00 [DBG] [Microsoft.Azure.Devices.Edge.Util.Uds.HttpUdsMessageHandler] - Connecting socket /var/run/iotedge/mgmt.sock
    <7> 2019-09-23 01:36:36.572 +00:00 [DBG] [Microsoft.Azure.Devices.Edge.Util.Uds.HttpUdsMessageHandler] - Connected socket /var/run/iotedge/mgmt.sock
    <7> 2019-09-23 01:36:36.573 +00:00 [DBG] [Microsoft.Azure.Devices.Edge.Util.Uds.HttpUdsMessageHandler] - Sending request http://mgmt.sock/modules?api-version=2019-01-30
    <7> 2019-09-23 01:36:36.574 +00:00 [DBG] [Microsoft.Azure.Devices.Edge.Agent.Core.Agent] - Getting edge agent config...
    <7> 2019-09-23 01:36:36.587 +00:00 [DBG] [Microsoft.Azure.Devices.Edge.Util.Uds.HttpUdsMessageHandler] - Response received OK
    <7> 2019-09-23 01:36:36.590 +00:00 [DBG] [Microsoft.Azure.Devices.Edge.Agent.Edgelet.ModuleManagementHttpClient] - Received a valid Http response from unix:///var/run/iotedge/mgmt.sock for List modules
    <4> 2019-09-23 01:36:41.578 +00:00 [WRN] [Microsoft.Azure.Devices.Edge.Agent.Core.ConfigSources.FileBackupConfigSource] - Empty edge agent config was received. Attempting to read config from backup file (backup.json) instead
    <6> 2019-09-23 01:36:41.578 +00:00 [INF] [Microsoft.Azure.Devices.Edge.Agent.Core.ConfigSources.FileBackupConfigSource] - Edge agent config backup file does not exist - backup.json
    <7> 2019-09-23 01:36:41.579 +00:00 [DBG] [Microsoft.Azure.Devices.Edge.Agent.Core.Agent] - Finished reconcile operation
    <7> 2019-09-23 01:36:46.581 +00:00 [DBG] [Microsoft.Azure.Devices.Edge.Agent.Core.Agent] - Starting reconcile operation
    <7> 2019-09-23 01:36:46.581 +00:00 [DBG] [Microsoft.Azure.Devices.Edge.Agent.Edgelet.ModuleManagementHttpClient] - Making a Http call to unix:///var/run/iotedge/mgmt.sock to List modules
    <7> 2019-09-23 01:36:46.582 +00:00 [DBG] [Microsoft.Azure.Devices.Edge.Util.Uds.HttpUdsMessageHandler] - Connecting socket /var/run/iotedge/mgmt.sock
    <7> 2019-09-23 01:36:46.586 +00:00 [DBG] [Microsoft.Azure.Devices.Edge.Util.Uds.HttpUdsMessageHandler] - Connected socket /var/run/iotedge/mgmt.sock
    <7> 2019-09-23 01:36:46.588 +00:00 [DBG] [Microsoft.Azure.Devices.Edge.Util.Uds.HttpUdsMessageHandler] - Sending request http://mgmt.sock/modules?api-version=2019-01-30
    <7> 2019-09-23 01:36:46.610 +00:00 [DBG] [Microsoft.Azure.Devices.Edge.Agent.Core.Agent] - Getting edge agent config...
    <7> 2019-09-23 01:36:46.613 +00:00 [DBG] [Microsoft.Azure.Devices.Edge.Util.Uds.HttpUdsMessageHandler] - Response received OK
    <7> 2019-09-23 01:36:46.617 +00:00 [DBG] [Microsoft.Azure.Devices.Edge.Agent.Edgelet.ModuleManagementHttpClient] - Received a valid Http response from unix:///var/run/iotedge/mgmt.sock for List modules
    <4> 2019-09-23 01:36:51.614 +00:00 [WRN] [Microsoft.Azure.Devices.Edge.Agent.Core.ConfigSources.FileBackupConfigSource] - Empty edge agent config was received. Attempting to read config from backup file (backup.json) instead
    <6> 2019-09-23 01:36:51.614 +00:00 [INF] [Microsoft.Azure.Devices.Edge.Agent.Core.ConfigSources.FileBackupConfigSource] - Edge agent config backup file does not exist - backup.json
    <7> 2019-09-23 01:36:51.615 +00:00 [DBG] [Microsoft.Azure.Devices.Edge.Agent.Core.Agent] - Finished reconcile operation
    <6> 2019-09-23 01:36:55.147 +00:00 [INF] [Microsoft.Azure.Devices.Edge.Agent.IoTHub.ModuleClientProvider] - Edge agent attempting to connect to IoT Hub via Amqp_WebSocket_Only...
    <7> 2019-09-23 01:36:56.615 +00:00 [DBG] [Microsoft.Azure.Devices.Edge.Agent.Core.Agent] - Starting reconcile operation
    <7> 2019-09-23 01:36:56.616 +00:00 [DBG] [Microsoft.Azure.Devices.Edge.Agent.Edgelet.ModuleManagementHttpClient] - Making a Http call to unix:///var/run/iotedge/mgmt.sock to List modules
    <7> 2019-09-23 01:36:56.616 +00:00 [DBG] [Microsoft.Azure.Devices.Edge.Util.Uds.HttpUdsMessageHandler] - Connecting socket /var/run/iotedge/mgmt.sock
    <7> 2019-09-23 01:36:56.617 +00:00 [DBG] [Microsoft.Azure.Devices.Edge.Util.Uds.HttpUdsMessageHandler] - Connected socket /var/run/iotedge/mgmt.sock
    <7> 2019-09-23 01:36:56.618 +00:00 [DBG] [Microsoft.Azure.Devices.Edge.Util.Uds.HttpUdsMessageHandler] - Sending request http://mgmt.sock/modules?api-version=2019-01-30
    <7> 2019-09-23 01:36:56.621 +00:00 [DBG] [Microsoft.Azure.Devices.Edge.Agent.Core.Agent] - Getting edge agent config...
    <7> 2019-09-23 01:36:56.641 +00:00 [DBG] [Microsoft.Azure.Devices.Edge.Util.Uds.HttpUdsMessageHandler] - Response received OK
    <7> 2019-09-23 01:36:56.649 +00:00 [DBG] [Microsoft.Azure.Devices.Edge.Agent.Edgelet.ModuleManagementHttpClient] - Received a valid Http response from unix:///var/run/iotedge/mgmt.sock for List modules
    <4> 2019-09-23 01:37:01.626 +00:00 [WRN] [Microsoft.Azure.Devices.Edge.Agent.Core.ConfigSources.FileBackupConfigSource] - Empty edge agent config was received. Attempting to read config from backup file (backup.json) instead
    <6> 2019-09-23 01:37:01.627 +00:00 [INF] [Microsoft.Azure.Devices.Edge.Agent.Core.ConfigSources.FileBackupConfigSource] - Edge agent config backup file does not exist - backup.json
    <7> 2019-09-23 01:37:01.627 +00:00 [DBG] [Microsoft.Azure.Devices.Edge.Agent.Core.Agent] - Finished reconcile operation
    <7> 2019-09-23 01:37:05.167 +00:00 [DBG] [Microsoft.Azure.Devices.Edge.Agent.IoTHub.EdgeAgentConnection] - Connection status changed to Disconnected with reason Retry_Expired
    <7> 2019-09-23 01:37:06.625 +00:00 [DBG] [Microsoft.Azure.Devices.Edge.Agent.Core.Agent] - Starting reconcile operation
    <7> 2019-09-23 01:37:06.625 +00:00 [DBG] [Microsoft.Azure.Devices.Edge.Agent.Edgelet.ModuleManagementHttpClient] - Making a Http call to unix:///var/run/iotedge/mgmt.sock to List modules
    <7> 2019-09-23 01:37:06.626 +00:00 [DBG] [Microsoft.Azure.Devices.Edge.Util.Uds.HttpUdsMessageHandler] - Connecting socket /var/run/iotedge/mgmt.sock
    <7> 2019-09-23 01:37:06.626 +00:00 [DBG] [Microsoft.Azure.Devices.Edge.Util.Uds.HttpUdsMessageHandler] - Connected socket /var/run/iotedge/mgmt.sock
    <7> 2019-09-23 01:37:06.627 +00:00 [DBG] [Microsoft.Azure.Devices.Edge.Util.Uds.HttpUdsMessageHandler] - Sending request http://mgmt.sock/modules?api-version=2019-01-30
    <7> 2019-09-23 01:37:06.631 +00:00 [DBG] [Microsoft.Azure.Devices.Edge.Agent.Core.Agent] - Getting edge agent config...
    <7> 2019-09-23 01:37:06.641 +00:00 [DBG] [Microsoft.Azure.Devices.Edge.Util.Uds.HttpUdsMessageHandler] - Response received OK
    <7> 2019-09-23 01:37:06.643 +00:00 [DBG] [Microsoft.Azure.Devices.Edge.Agent.Edgelet.ModuleManagementHttpClient] - Received a valid Http response from unix:///var/run/iotedge/mgmt.sock for List modules
    <4> 2019-09-23 01:37:11.634 +00:00 [WRN] [Microsoft.Azure.Devices.Edge.Agent.Core.ConfigSources.FileBackupConfigSource] - Empty edge agent config was received. Attempting to read config from backup file (backup.json) instead
    <6> 2019-09-23 01:37:11.635 +00:00 [INF] [Microsoft.Azure.Devices.Edge.Agent.Core.ConfigSources.FileBackupConfigSource] - Edge agent config backup file does not exist - backup.json
    <7> 2019-09-23 01:37:11.635 +00:00 [DBG] [Microsoft.Azure.Devices.Edge.Agent.Core.Agent] - Finished reconcile operation
    <7> 2019-09-23 01:37:16.635 +00:00 [DBG] [Microsoft.Azure.Devices.Edge.Agent.Core.Agent] - Starting reconcile operation
    <7> 2019-09-23 01:37:16.635 +00:00 [DBG] [Microsoft.Azure.Devices.Edge.Agent.Edgelet.ModuleManagementHttpClient] - Making a Http call to unix:///var/run/iotedge/mgmt.sock to List modules
    <7> 2019-09-23 01:37:16.636 +00:00 [DBG] [Microsoft.Azure.Devices.Edge.Util.Uds.HttpUdsMessageHandler] - Connecting socket /var/run/iotedge/mgmt.sock
    <7> 2019-09-23 01:37:16.637 +00:00 [DBG] [Microsoft.Azure.Devices.Edge.Util.Uds.HttpUdsMessageHandler] - Connected socket /var/run/iotedge/mgmt.sock
    <7> 2019-09-23 01:37:16.637 +00:00 [DBG] [Microsoft.Azure.Devices.Edge.Util.Uds.HttpUdsMessageHandler] - Sending request http://mgmt.sock/modules?api-version=2019-01-30
    <7> 2019-09-23 01:37:16.639 +00:00 [DBG] [Microsoft.Azure.Devices.Edge.Agent.Core.Agent] - Getting edge agent config...
    <7> 2019-09-23 01:37:16.670 +00:00 [DBG] [Microsoft.Azure.Devices.Edge.Util.Uds.HttpUdsMessageHandler] - Response received OK
    <7> 2019-09-23 01:37:16.672 +00:00 [DBG] [Microsoft.Azure.Devices.Edge.Agent.Edgelet.ModuleManagementHttpClient] - Received a valid Http response from unix:///var/run/iotedge/mgmt.sock for List modules
    <4> 2019-09-23 01:37:21.637 +00:00 [WRN] [Microsoft.Azure.Devices.Edge.Agent.Core.ConfigSources.FileBackupConfigSource] - Empty edge agent config was received. Attempting to read config from backup file (backup.json) instead
    <6> 2019-09-23 01:37:21.637 +00:00 [INF] [Microsoft.Azure.Devices.Edge.Agent.Core.ConfigSources.FileBackupConfigSource] - Edge agent config backup file does not exist - backup.json
    <7> 2019-09-23 01:37:21.637 +00:00 [DBG] [Microsoft.Azure.Devices.Edge.Agent.Core.Agent] - Finished reconcile operation

config check

 ~# curl -sSL https://raw.githubusercontent.com/moby/moby/master/contrib/check-config.sh -o check-config.sh
    ~# chmod +x check-config.sh
    ~# ./check-config.sh
    info: reading kernel config from /proc/config.gz ...

    Generally Necessary:
    - cgroup hierarchy: properly mounted [/sys/fs/cgroup]
    - CONFIG_NAMESPACES: enabled
    - CONFIG_NET_NS: enabled
    - CONFIG_PID_NS: enabled
    - CONFIG_IPC_NS: enabled
    - CONFIG_UTS_NS: enabled
    - CONFIG_CGROUPS: enabled
    - CONFIG_CGROUP_CPUACCT: enabled
    - CONFIG_CGROUP_DEVICE: enabled
    - CONFIG_CGROUP_FREEZER: enabled
    - CONFIG_CGROUP_SCHED: enabled
    - CONFIG_CPUSETS: enabled
    - CONFIG_MEMCG: enabled
    - CONFIG_KEYS: enabled
    - CONFIG_VETH: enabled
    - CONFIG_BRIDGE: enabled
    - CONFIG_BRIDGE_NETFILTER: enabled
    - CONFIG_NF_NAT_IPV4: enabled
    - CONFIG_IP_NF_FILTER: enabled
    - CONFIG_IP_NF_TARGET_MASQUERADE: enabled
    - CONFIG_NETFILTER_XT_MATCH_ADDRTYPE: enabled
    - CONFIG_NETFILTER_XT_MATCH_CONNTRACK: enabled
    - CONFIG_NETFILTER_XT_MATCH_IPVS: enabled
    - CONFIG_IP_NF_NAT: enabled
    - CONFIG_NF_NAT: enabled
    - CONFIG_NF_NAT_NEEDED: enabled
    - CONFIG_POSIX_MQUEUE: enabled
    - CONFIG_DEVPTS_MULTIPLE_INSTANCES: enabled

    Optional Features:
    - CONFIG_USER_NS: enabled
    - CONFIG_SECCOMP: enabled
    - CONFIG_CGROUP_PIDS: enabled
    - CONFIG_MEMCG_SWAP: missing
    - CONFIG_MEMCG_SWAP_ENABLED: missing
    - CONFIG_MEMCG_KMEM: enabled
    - CONFIG_BLK_CGROUP: enabled
    - CONFIG_BLK_DEV_THROTTLING: missing
    - CONFIG_IOSCHED_CFQ: enabled
    - CONFIG_CFQ_GROUP_IOSCHED: missing
    - CONFIG_CGROUP_PERF: enabled
    - CONFIG_CGROUP_HUGETLB: missing
    - CONFIG_NET_CLS_CGROUP: missing
    - CONFIG_CGROUP_NET_PRIO: missing
    - CONFIG_CFS_BANDWIDTH: enabled
    - CONFIG_FAIR_GROUP_SCHED: enabled
    - CONFIG_RT_GROUP_SCHED: enabled
    - CONFIG_IP_NF_TARGET_REDIRECT: enabled
    - CONFIG_IP_VS: enabled
    - CONFIG_IP_VS_NFCT: missing
    - CONFIG_IP_VS_PROTO_TCP: missing
    - CONFIG_IP_VS_PROTO_UDP: missing
    - CONFIG_IP_VS_RR: missing
    - CONFIG_EXT4_FS: enabled
    - CONFIG_EXT4_FS_POSIX_ACL: enabled
    - CONFIG_EXT4_FS_SECURITY: enabled
    - Network Drivers:
      - "overlay":
        - CONFIG_VXLAN: enabled
        - CONFIG_BRIDGE_VLAN_FILTERING: enabled
          Optional (for encrypted networks):
          - CONFIG_CRYPTO: enabled
          - CONFIG_CRYPTO_AEAD: enabled
          - CONFIG_CRYPTO_GCM: enabled
          - CONFIG_CRYPTO_SEQIV: enabled
          - CONFIG_CRYPTO_GHASH: enabled
          - CONFIG_XFRM: enabled
          - CONFIG_XFRM_USER: enabled
          - CONFIG_XFRM_ALGO: enabled
          - CONFIG_INET_ESP: enabled
          - CONFIG_INET_XFRM_MODE_TRANSPORT: enabled
      - "ipvlan":
        - CONFIG_IPVLAN: enabled
      - "macvlan":
        - CONFIG_MACVLAN: enabled
        - CONFIG_DUMMY: enabled
      - "ftp,tftp client in container":
        - CONFIG_NF_NAT_FTP: enabled (as module)
        - CONFIG_NF_CONNTRACK_FTP: enabled (as module)
        - CONFIG_NF_NAT_TFTP: enabled
        - CONFIG_NF_CONNTRACK_TFTP: enabled
    - Storage Drivers:
      - "aufs":
        - CONFIG_AUFS_FS: missing
      - "btrfs":
        - CONFIG_BTRFS_FS: missing
        - CONFIG_BTRFS_FS_POSIX_ACL: missing
      - "devicemapper":
        - CONFIG_BLK_DEV_DM: missing
        - CONFIG_DM_THIN_PROVISIONING: missing
      - "overlay":
        - CONFIG_OVERLAY_FS: enabled
      - "zfs":
        - /dev/zfs: missing
        - zfs command: missing
        - zpool command: missing

    Limits:
    - /proc/sys/kernel/keys/root_maxkeys: 1000000

Additional Information

DNS set to 1.1.1.1

Have good connection with DN inside container.

~# curl ifconfig.co
60.249.211.103

root@54b34caf4452:/app# curl ifconfig.co
60.249.211.103

[mohsin076]

hy yenseaman, i am having a same issue did You resolved your issue. all the modules are connected and running but the data from device to azure blob is not syncing.

[yenseaman]

I purchased azure developer support, and have had a great support. Then got a refund from MS... in the meanwhile exclude it form project.

[mohsin076]

Can you share how they helped u and resolved the problem . i don't have a developer support. it will be really helpful. Thanks

[yenseaman]

Actually they didn't help at all and asked me to purchase higher support plans. That's why I am not using it now.

[mohsin076]

well thanks bro but i actually solved it by myself.

On Tue, 7 Jan 2020 at 8:00 AM, yenseaman [e-mail] wrote:

Actually they didn't help at all and ask me to purchase higher support
plans. That's why I am not using it now.

—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/Azure/iotedge/issues/1752?email_source=notifications&email_token=AKHXZLHYMC2XJ4LEEHFSKB3Q4PV5BA5CNFSM4I2HMD42YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEIHQ2EQ#issuecomment-571411730,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AKHXZLGAVHBM4LG7BLG7DL3Q4PV5BANCNFSM4I2HMD4Q
.