Mbed-os: [greentea] RTL8195AM netsocket test failures
Description
Environment: Windows 10
Mbed CLI version: 1.8.0
Python version: 2.7.14
IAR version: 7.80.1.11864
ARMC5 version: 5060750
Mbed OS tag: mbed-os-5.10.0-rc2
Summary of netsocket test failures with ARMC5 and the RTL8195AM board:
mbedgt: test suite report:
+-----------------------+-------------------+-----------------------------+---------+--------------------+-------------+
| target | platform_name | test suite | result | elapsed_time (sec) | copy_method |
+-----------------------+-------------------+-----------------------------+---------+--------------------+-------------+
| REALTEK_RTL8195AM-ARM | REALTEK_RTL8195AM | mbed-os-tests-netsocket-dns | OK | 102.49 | default |
| REALTEK_RTL8195AM-ARM | REALTEK_RTL8195AM | mbed-os-tests-netsocket-tcp | TIMEOUT | 550.15 | default |
| REALTEK_RTL8195AM-ARM | REALTEK_RTL8195AM | mbed-os-tests-netsocket-udp | FAIL | 523.8 | default |
| REALTEK_RTL8195AM-ARM | REALTEK_RTL8195AM | mbed-os-tests-network-wifi | FAIL | 85.2 | default |
+-----------------------+-------------------+-----------------------------+---------+--------------------+-------------+
mbedgt: test suite results: 2 FAIL / 1 OK / 1 TIMEOUT
mbedgt: test case report:
+-----------------------+-------------------+-----------------------------+---------------------------------------+--------+--------+---------+--------------------+
| target | platform_name | test suite | test case | passed | failed | result | elapsed_time (sec) |
+-----------------------+-------------------+-----------------------------+---------------------------------------+--------+--------+---------+--------------------+
| REALTEK_RTL8195AM-ARM | REALTEK_RTL8195AM | mbed-os-tests-netsocket-dns | ASYNCHRONOUS_DNS | 1 | 0 | OK | 0.12 |
| REALTEK_RTL8195AM-ARM | REALTEK_RTL8195AM | mbed-os-tests-netsocket-dns | ASYNCHRONOUS_DNS_CACHE | 1 | 0 | OK | 0.21 |
| REALTEK_RTL8195AM-ARM | REALTEK_RTL8195AM | mbed-os-tests-netsocket-dns | ASYNCHRONOUS_DNS_CANCEL | 1 | 0 | OK | 5.1 |
| REALTEK_RTL8195AM-ARM | REALTEK_RTL8195AM | mbed-os-tests-netsocket-dns | ASYNCHRONOUS_DNS_EXTERNAL_EVENT_QUEUE | 1 | 0 | OK | 2.32 |
| REALTEK_RTL8195AM-ARM | REALTEK_RTL8195AM | mbed-os-tests-netsocket-dns | ASYNCHRONOUS_DNS_INVALID_HOST | 1 | 0 | OK | 0.54 |
| REALTEK_RTL8195AM-ARM | REALTEK_RTL8195AM | mbed-os-tests-netsocket-dns | ASYNCHRONOUS_DNS_NON_ASYNC_AND_ASYNC | 1 | 0 | OK | 0.48 |
| REALTEK_RTL8195AM-ARM | REALTEK_RTL8195AM | mbed-os-tests-netsocket-dns | ASYNCHRONOUS_DNS_SIMULTANEOUS | 1 | 0 | OK | 0.57 |
| REALTEK_RTL8195AM-ARM | REALTEK_RTL8195AM | mbed-os-tests-netsocket-dns | ASYNCHRONOUS_DNS_SIMULTANEOUS_CACHE | 1 | 0 | OK | 0.28 |
| REALTEK_RTL8195AM-ARM | REALTEK_RTL8195AM | mbed-os-tests-netsocket-dns | ASYNCHRONOUS_DNS_SIMULTANEOUS_REPEAT | 1 | 0 | OK | 12.5 |
| REALTEK_RTL8195AM-ARM | REALTEK_RTL8195AM | mbed-os-tests-netsocket-dns | ASYNCHRONOUS_DNS_TIMEOUTS | 1 | 0 | OK | 2.29 |
| REALTEK_RTL8195AM-ARM | REALTEK_RTL8195AM | mbed-os-tests-netsocket-dns | SYNCHRONOUS_DNS | 1 | 0 | OK | 0.03 |
| REALTEK_RTL8195AM-ARM | REALTEK_RTL8195AM | mbed-os-tests-netsocket-dns | SYNCHRONOUS_DNS_CACHE | 1 | 0 | OK | 0.25 |
| REALTEK_RTL8195AM-ARM | REALTEK_RTL8195AM | mbed-os-tests-netsocket-dns | SYNCHRONOUS_DNS_INVALID | 1 | 0 | OK | 0.24 |
| REALTEK_RTL8195AM-ARM | REALTEK_RTL8195AM | mbed-os-tests-netsocket-dns | SYNCHRONOUS_DNS_MULTIPLE | 1 | 0 | OK | 0.14 |
| REALTEK_RTL8195AM-ARM | REALTEK_RTL8195AM | mbed-os-tests-netsocket-tcp | TCPSOCKET_CONNECT_INVALID | 1 | 0 | OK | 0.18 |
| REALTEK_RTL8195AM-ARM | REALTEK_RTL8195AM | mbed-os-tests-netsocket-tcp | TCPSOCKET_ECHOTEST | 1 | 0 | OK | 7.24 |
| REALTEK_RTL8195AM-ARM | REALTEK_RTL8195AM | mbed-os-tests-netsocket-tcp | TCPSOCKET_ECHOTEST_BURST | 1 | 0 | OK | 136.53 |
| REALTEK_RTL8195AM-ARM | REALTEK_RTL8195AM | mbed-os-tests-netsocket-tcp | TCPSOCKET_ECHOTEST_BURST_NONBLOCK | 1 | 0 | OK | 156.99 |
| REALTEK_RTL8195AM-ARM | REALTEK_RTL8195AM | mbed-os-tests-netsocket-tcp | TCPSOCKET_ECHOTEST_NONBLOCK | 1 | 0 | OK | 31.72 |
| REALTEK_RTL8195AM-ARM | REALTEK_RTL8195AM | mbed-os-tests-netsocket-tcp | TCPSOCKET_ENDPOINT_CLOSE | 0 | 0 | SKIPPED | 0.0 |
| REALTEK_RTL8195AM-ARM | REALTEK_RTL8195AM | mbed-os-tests-netsocket-tcp | TCPSOCKET_OPEN_CLOSE_REPEAT | 1 | 0 | OK | 0.01 |
| REALTEK_RTL8195AM-ARM | REALTEK_RTL8195AM | mbed-os-tests-netsocket-tcp | TCPSOCKET_OPEN_LIMIT | 1 | 0 | OK | 0.07 |
| REALTEK_RTL8195AM-ARM | REALTEK_RTL8195AM | mbed-os-tests-netsocket-tcp | TCPSOCKET_RECV_100K | 0 | 0 | ERROR | 0.0 |
| REALTEK_RTL8195AM-ARM | REALTEK_RTL8195AM | mbed-os-tests-netsocket-tcp | TCPSOCKET_RECV_100K_NONBLOCK | 0 | 0 | SKIPPED | 0.0 |
| REALTEK_RTL8195AM-ARM | REALTEK_RTL8195AM | mbed-os-tests-netsocket-tcp | TCPSOCKET_RECV_TIMEOUT | 0 | 0 | SKIPPED | 0.0 |
| REALTEK_RTL8195AM-ARM | REALTEK_RTL8195AM | mbed-os-tests-netsocket-tcp | TCPSOCKET_SEND_REPEAT | 0 | 0 | SKIPPED | 0.0 |
| REALTEK_RTL8195AM-ARM | REALTEK_RTL8195AM | mbed-os-tests-netsocket-tcp | TCPSOCKET_SEND_TIMEOUT | 0 | 0 | SKIPPED | 0.0 |
| REALTEK_RTL8195AM-ARM | REALTEK_RTL8195AM | mbed-os-tests-netsocket-tcp | TCPSOCKET_THREAD_PER_SOCKET_SAFETY | 1 | 0 | OK | 5.15 |
| REALTEK_RTL8195AM-ARM | REALTEK_RTL8195AM | mbed-os-tests-netsocket-udp | UDPSOCKET_ECHOTEST | 1 | 0 | OK | 3.93 |
| REALTEK_RTL8195AM-ARM | REALTEK_RTL8195AM | mbed-os-tests-netsocket-udp | UDPSOCKET_ECHOTEST_BURST | 0 | 1 | FAIL | 437.52 |
| REALTEK_RTL8195AM-ARM | REALTEK_RTL8195AM | mbed-os-tests-netsocket-udp | UDPSOCKET_ECHOTEST_BURST_NONBLOCK | 0 | 0 | SKIPPED | 0.0 |
| REALTEK_RTL8195AM-ARM | REALTEK_RTL8195AM | mbed-os-tests-netsocket-udp | UDPSOCKET_ECHOTEST_NONBLOCK | 1 | 0 | OK | 5.98 |
| REALTEK_RTL8195AM-ARM | REALTEK_RTL8195AM | mbed-os-tests-netsocket-udp | UDPSOCKET_OPEN_CLOSE_REPEAT | 1 | 0 | OK | 0.02 |
| REALTEK_RTL8195AM-ARM | REALTEK_RTL8195AM | mbed-os-tests-netsocket-udp | UDPSOCKET_OPEN_LIMIT | 1 | 0 | OK | 0.06 |
| REALTEK_RTL8195AM-ARM | REALTEK_RTL8195AM | mbed-os-tests-netsocket-udp | UDPSOCKET_SENDTO_INVALID | 1 | 0 | OK | 0.03 |
| REALTEK_RTL8195AM-ARM | REALTEK_RTL8195AM | mbed-os-tests-netsocket-udp | UDPSOCKET_SENDTO_REPEAT | 0 | 0 | SKIPPED | 0.0 |
| REALTEK_RTL8195AM-ARM | REALTEK_RTL8195AM | mbed-os-tests-netsocket-udp | UDPSOCKET_SENDTO_TIMEOUT | 1 | 0 | OK | 0.16 |
| REALTEK_RTL8195AM-ARM | REALTEK_RTL8195AM | mbed-os-tests-network-wifi | WIFI-CONNECT | 0 | 0 | SKIPPED | 0.0 |
| REALTEK_RTL8195AM-ARM | REALTEK_RTL8195AM | mbed-os-tests-network-wifi | WIFI-CONNECT-DISCONNECT-REPEAT | 0 | 0 | SKIPPED | 0.0 |
| REALTEK_RTL8195AM-ARM | REALTEK_RTL8195AM | mbed-os-tests-network-wifi | WIFI-CONNECT-NOCREDENTIALS | 0 | 1 | FAIL | 15.03 |
| REALTEK_RTL8195AM-ARM | REALTEK_RTL8195AM | mbed-os-tests-network-wifi | WIFI-CONNECT-PARAMS-CHANNEL | 0 | 0 | SKIPPED | 0.0 |
| REALTEK_RTL8195AM-ARM | REALTEK_RTL8195AM | mbed-os-tests-network-wifi | WIFI-CONNECT-PARAMS-CHANNEL-FAIL | 0 | 0 | SKIPPED | 0.0 |
| REALTEK_RTL8195AM-ARM | REALTEK_RTL8195AM | mbed-os-tests-network-wifi | WIFI-CONNECT-PARAMS-NULL | 0 | 0 | SKIPPED | 0.0 |
| REALTEK_RTL8195AM-ARM | REALTEK_RTL8195AM | mbed-os-tests-network-wifi | WIFI-CONNECT-PARAMS-VALID-SECURE | 0 | 0 | SKIPPED | 0.0 |
| REALTEK_RTL8195AM-ARM | REALTEK_RTL8195AM | mbed-os-tests-network-wifi | WIFI-CONNECT-PARAMS-VALID-UNSECURE | 0 | 0 | SKIPPED | 0.0 |
| REALTEK_RTL8195AM-ARM | REALTEK_RTL8195AM | mbed-os-tests-network-wifi | WIFI-CONNECT-SECURE | 0 | 0 | SKIPPED | 0.0 |
| REALTEK_RTL8195AM-ARM | REALTEK_RTL8195AM | mbed-os-tests-network-wifi | WIFI-CONNECT-SECURE-FAIL | 0 | 0 | SKIPPED | 0.0 |
| REALTEK_RTL8195AM-ARM | REALTEK_RTL8195AM | mbed-os-tests-network-wifi | WIFI-CONSTRUCTOR | 1 | 0 | OK | 2.01 |
| REALTEK_RTL8195AM-ARM | REALTEK_RTL8195AM | mbed-os-tests-network-wifi | WIFI-GET-RSSI | 0 | 0 | SKIPPED | 0.0 |
| REALTEK_RTL8195AM-ARM | REALTEK_RTL8195AM | mbed-os-tests-network-wifi | WIFI-SCAN | 0 | 0 | SKIPPED | 0.0 |
| REALTEK_RTL8195AM-ARM | REALTEK_RTL8195AM | mbed-os-tests-network-wifi | WIFI-SCAN-NULL | 0 | 0 | SKIPPED | 0.0 |
| REALTEK_RTL8195AM-ARM | REALTEK_RTL8195AM | mbed-os-tests-network-wifi | WIFI-SET-CHANNEL | 0 | 0 | SKIPPED | 0.0 |
| REALTEK_RTL8195AM-ARM | REALTEK_RTL8195AM | mbed-os-tests-network-wifi | WIFI-SET-CREDENTIAL | 0 | 0 | SKIPPED | 0.0 |
+-----------------------+-------------------+-----------------------------+---------------------------------------+--------+--------+---------+--------------------+
mbedgt: test case results: 2 FAIL / 21 SKIPPED / 29 OK / 1 ERROR
mbedgt: completed in 1269.42 sec
mbedgt: exited with code 3
[mbed] ERROR: "mbedgt" returned error.
Code: 3
Path: "C:\Users\jenplu01\Repos\oob-5.10\network_test"
Command: "mbedgt --test-spec .\BUILD\tests\REALTEK_RTL8195AM\ARM\test_spec.json -n mbed-os-tests-network-*,mbed-os-tests-netsocket*"
Tip: You could retry the last command with "-v" flag for verbose output
---
Issue request type
[ ] Question
[ ] Enhancement
[X] Bug
yennster
All 73 comments
IAR failures:
mbedgt: test suite report:
+-----------------------+-------------------+-----------------------------+---------+--------------------+-------------+
| target | platform_name | test suite | result | elapsed_time (sec) | copy_method |
+-----------------------+-------------------+-----------------------------+---------+--------------------+-------------+
| REALTEK_RTL8195AM-IAR | REALTEK_RTL8195AM | mbed-os-tests-netsocket-dns | FAIL | 89.51 | default |
| REALTEK_RTL8195AM-IAR | REALTEK_RTL8195AM | mbed-os-tests-netsocket-tcp | TIMEOUT | 550.23 | default |
| REALTEK_RTL8195AM-IAR | REALTEK_RTL8195AM | mbed-os-tests-netsocket-udp | FAIL | 294.08 | default |
| REALTEK_RTL8195AM-IAR | REALTEK_RTL8195AM | mbed-os-tests-network-wifi | FAIL | 87.61 | default |
+-----------------------+-------------------+-----------------------------+---------+--------------------+-------------+
mbedgt: test suite results: 3 FAIL / 1 TIMEOUT
mbedgt: test case report:
+-----------------------+-------------------+-----------------------------+---------------------------------------+--------+--------+---------+--------------------+
| target | platform_name | test suite | test case | passed | failed | result | elapsed_time (sec) |
+-----------------------+-------------------+-----------------------------+---------------------------------------+--------+--------+---------+--------------------+
| REALTEK_RTL8195AM-IAR | REALTEK_RTL8195AM | mbed-os-tests-netsocket-dns | ASYNCHRONOUS_DNS | 1 | 0 | OK | 0.12 |
| REALTEK_RTL8195AM-IAR | REALTEK_RTL8195AM | mbed-os-tests-netsocket-dns | ASYNCHRONOUS_DNS_CACHE | 0 | 1 | FAIL | 2.6 |
| REALTEK_RTL8195AM-IAR | REALTEK_RTL8195AM | mbed-os-tests-netsocket-dns | ASYNCHRONOUS_DNS_CANCEL | 0 | 0 | SKIPPED | 0.0 |
| REALTEK_RTL8195AM-IAR | REALTEK_RTL8195AM | mbed-os-tests-netsocket-dns | ASYNCHRONOUS_DNS_EXTERNAL_EVENT_QUEUE | 0 | 0 | SKIPPED | 0.0 |
| REALTEK_RTL8195AM-IAR | REALTEK_RTL8195AM | mbed-os-tests-netsocket-dns | ASYNCHRONOUS_DNS_INVALID_HOST | 0 | 0 | SKIPPED | 0.0 |
| REALTEK_RTL8195AM-IAR | REALTEK_RTL8195AM | mbed-os-tests-netsocket-dns | ASYNCHRONOUS_DNS_NON_ASYNC_AND_ASYNC | 0 | 0 | SKIPPED | 0.0 |
| REALTEK_RTL8195AM-IAR | REALTEK_RTL8195AM | mbed-os-tests-netsocket-dns | ASYNCHRONOUS_DNS_SIMULTANEOUS | 1 | 0 | OK | 0.62 |
| REALTEK_RTL8195AM-IAR | REALTEK_RTL8195AM | mbed-os-tests-netsocket-dns | ASYNCHRONOUS_DNS_SIMULTANEOUS_CACHE | 1 | 0 | OK | 0.23 |
| REALTEK_RTL8195AM-IAR | REALTEK_RTL8195AM | mbed-os-tests-netsocket-dns | ASYNCHRONOUS_DNS_SIMULTANEOUS_REPEAT | 0 | 0 | SKIPPED | 0.0 |
| REALTEK_RTL8195AM-IAR | REALTEK_RTL8195AM | mbed-os-tests-netsocket-dns | ASYNCHRONOUS_DNS_TIMEOUTS | 0 | 0 | SKIPPED | 0.0 |
| REALTEK_RTL8195AM-IAR | REALTEK_RTL8195AM | mbed-os-tests-netsocket-dns | SYNCHRONOUS_DNS | 0 | 0 | SKIPPED | 0.0 |
| REALTEK_RTL8195AM-IAR | REALTEK_RTL8195AM | mbed-os-tests-netsocket-dns | SYNCHRONOUS_DNS_CACHE | 0 | 0 | SKIPPED | 0.0 |
| REALTEK_RTL8195AM-IAR | REALTEK_RTL8195AM | mbed-os-tests-netsocket-dns | SYNCHRONOUS_DNS_INVALID | 0 | 0 | SKIPPED | 0.0 |
| REALTEK_RTL8195AM-IAR | REALTEK_RTL8195AM | mbed-os-tests-netsocket-dns | SYNCHRONOUS_DNS_MULTIPLE | 0 | 0 | SKIPPED | 0.0 |
| REALTEK_RTL8195AM-IAR | REALTEK_RTL8195AM | mbed-os-tests-netsocket-tcp | TCPSOCKET_CONNECT_INVALID | 1 | 0 | OK | 0.18 |
| REALTEK_RTL8195AM-IAR | REALTEK_RTL8195AM | mbed-os-tests-netsocket-tcp | TCPSOCKET_ECHOTEST | 1 | 0 | OK | 7.85 |
| REALTEK_RTL8195AM-IAR | REALTEK_RTL8195AM | mbed-os-tests-netsocket-tcp | TCPSOCKET_ECHOTEST_BURST | 0 | 0 | ERROR | 0.0 |
| REALTEK_RTL8195AM-IAR | REALTEK_RTL8195AM | mbed-os-tests-netsocket-tcp | TCPSOCKET_ECHOTEST_BURST_NONBLOCK | 0 | 0 | SKIPPED | 0.0 |
| REALTEK_RTL8195AM-IAR | REALTEK_RTL8195AM | mbed-os-tests-netsocket-tcp | TCPSOCKET_ECHOTEST_NONBLOCK | 1 | 0 | OK | 26.75 |
| REALTEK_RTL8195AM-IAR | REALTEK_RTL8195AM | mbed-os-tests-netsocket-tcp | TCPSOCKET_ENDPOINT_CLOSE | 0 | 0 | SKIPPED | 0.0 |
| REALTEK_RTL8195AM-IAR | REALTEK_RTL8195AM | mbed-os-tests-netsocket-tcp | TCPSOCKET_OPEN_CLOSE_REPEAT | 1 | 0 | OK | 0.01 |
| REALTEK_RTL8195AM-IAR | REALTEK_RTL8195AM | mbed-os-tests-netsocket-tcp | TCPSOCKET_OPEN_LIMIT | 1 | 0 | OK | 0.09 |
| REALTEK_RTL8195AM-IAR | REALTEK_RTL8195AM | mbed-os-tests-netsocket-tcp | TCPSOCKET_RECV_100K | 0 | 0 | SKIPPED | 0.0 |
| REALTEK_RTL8195AM-IAR | REALTEK_RTL8195AM | mbed-os-tests-netsocket-tcp | TCPSOCKET_RECV_100K_NONBLOCK | 0 | 0 | SKIPPED | 0.0 |
| REALTEK_RTL8195AM-IAR | REALTEK_RTL8195AM | mbed-os-tests-netsocket-tcp | TCPSOCKET_RECV_TIMEOUT | 0 | 0 | SKIPPED | 0.0 |
| REALTEK_RTL8195AM-IAR | REALTEK_RTL8195AM | mbed-os-tests-netsocket-tcp | TCPSOCKET_SEND_REPEAT | 0 | 0 | SKIPPED | 0.0 |
| REALTEK_RTL8195AM-IAR | REALTEK_RTL8195AM | mbed-os-tests-netsocket-tcp | TCPSOCKET_SEND_TIMEOUT | 0 | 0 | SKIPPED | 0.0 |
| REALTEK_RTL8195AM-IAR | REALTEK_RTL8195AM | mbed-os-tests-netsocket-tcp | TCPSOCKET_THREAD_PER_SOCKET_SAFETY | 1 | 0 | OK | 5.57 |
| REALTEK_RTL8195AM-IAR | REALTEK_RTL8195AM | mbed-os-tests-netsocket-udp | UDPSOCKET_ECHOTEST | 1 | 0 | OK | 4.68 |
| REALTEK_RTL8195AM-IAR | REALTEK_RTL8195AM | mbed-os-tests-netsocket-udp | UDPSOCKET_ECHOTEST_BURST | 0 | 1 | FAIL | 196.47 |
| REALTEK_RTL8195AM-IAR | REALTEK_RTL8195AM | mbed-os-tests-netsocket-udp | UDPSOCKET_ECHOTEST_BURST_NONBLOCK | 0 | 0 | SKIPPED | 0.0 |
| REALTEK_RTL8195AM-IAR | REALTEK_RTL8195AM | mbed-os-tests-netsocket-udp | UDPSOCKET_ECHOTEST_NONBLOCK | 1 | 0 | OK | 6.18 |
| REALTEK_RTL8195AM-IAR | REALTEK_RTL8195AM | mbed-os-tests-netsocket-udp | UDPSOCKET_OPEN_CLOSE_REPEAT | 1 | 0 | OK | 0.01 |
| REALTEK_RTL8195AM-IAR | REALTEK_RTL8195AM | mbed-os-tests-netsocket-udp | UDPSOCKET_OPEN_LIMIT | 1 | 0 | OK | 0.08 |
| REALTEK_RTL8195AM-IAR | REALTEK_RTL8195AM | mbed-os-tests-netsocket-udp | UDPSOCKET_SENDTO_INVALID | 1 | 0 | OK | 0.04 |
| REALTEK_RTL8195AM-IAR | REALTEK_RTL8195AM | mbed-os-tests-netsocket-udp | UDPSOCKET_SENDTO_REPEAT | 0 | 0 | SKIPPED | 0.0 |
| REALTEK_RTL8195AM-IAR | REALTEK_RTL8195AM | mbed-os-tests-netsocket-udp | UDPSOCKET_SENDTO_TIMEOUT | 1 | 0 | OK | 0.04 |
| REALTEK_RTL8195AM-IAR | REALTEK_RTL8195AM | mbed-os-tests-network-wifi | WIFI-CONNECT | 0 | 0 | SKIPPED | 0.0 |
| REALTEK_RTL8195AM-IAR | REALTEK_RTL8195AM | mbed-os-tests-network-wifi | WIFI-CONNECT-DISCONNECT-REPEAT | 0 | 0 | SKIPPED | 0.0 |
| REALTEK_RTL8195AM-IAR | REALTEK_RTL8195AM | mbed-os-tests-network-wifi | WIFI-CONNECT-NOCREDENTIALS | 0 | 1 | FAIL | 15.03 |
| REALTEK_RTL8195AM-IAR | REALTEK_RTL8195AM | mbed-os-tests-network-wifi | WIFI-CONNECT-PARAMS-CHANNEL | 0 | 0 | SKIPPED | 0.0 |
| REALTEK_RTL8195AM-IAR | REALTEK_RTL8195AM | mbed-os-tests-network-wifi | WIFI-CONNECT-PARAMS-CHANNEL-FAIL | 0 | 0 | SKIPPED | 0.0 |
| REALTEK_RTL8195AM-IAR | REALTEK_RTL8195AM | mbed-os-tests-network-wifi | WIFI-CONNECT-PARAMS-NULL | 0 | 0 | SKIPPED | 0.0 |
| REALTEK_RTL8195AM-IAR | REALTEK_RTL8195AM | mbed-os-tests-network-wifi | WIFI-CONNECT-PARAMS-VALID-SECURE | 0 | 0 | SKIPPED | 0.0 |
| REALTEK_RTL8195AM-IAR | REALTEK_RTL8195AM | mbed-os-tests-network-wifi | WIFI-CONNECT-PARAMS-VALID-UNSECURE | 0 | 0 | SKIPPED | 0.0 |
| REALTEK_RTL8195AM-IAR | REALTEK_RTL8195AM | mbed-os-tests-network-wifi | WIFI-CONNECT-SECURE | 0 | 0 | SKIPPED | 0.0 |
| REALTEK_RTL8195AM-IAR | REALTEK_RTL8195AM | mbed-os-tests-network-wifi | WIFI-CONNECT-SECURE-FAIL | 0 | 0 | SKIPPED | 0.0 |
| REALTEK_RTL8195AM-IAR | REALTEK_RTL8195AM | mbed-os-tests-network-wifi | WIFI-CONSTRUCTOR | 1 | 0 | OK | 2.17 |
| REALTEK_RTL8195AM-IAR | REALTEK_RTL8195AM | mbed-os-tests-network-wifi | WIFI-GET-RSSI | 0 | 0 | SKIPPED | 0.0 |
| REALTEK_RTL8195AM-IAR | REALTEK_RTL8195AM | mbed-os-tests-network-wifi | WIFI-SCAN | 0 | 0 | SKIPPED | 0.0 |
| REALTEK_RTL8195AM-IAR | REALTEK_RTL8195AM | mbed-os-tests-network-wifi | WIFI-SCAN-NULL | 0 | 0 | SKIPPED | 0.0 |
| REALTEK_RTL8195AM-IAR | REALTEK_RTL8195AM | mbed-os-tests-network-wifi | WIFI-SET-CHANNEL | 0 | 0 | SKIPPED | 0.0 |
| REALTEK_RTL8195AM-IAR | REALTEK_RTL8195AM | mbed-os-tests-network-wifi | WIFI-SET-CREDENTIAL | 0 | 0 | SKIPPED | 0.0 |
+-----------------------+-------------------+-----------------------------+---------------------------------------+--------+--------+---------+--------------------+
mbedgt: test case results: 3 FAIL / 33 SKIPPED / 16 OK / 1 ERROR
mbedgt: completed in 1028.94 sec
mbedgt: exited with code 4
[mbed] ERROR: "mbedgt" returned error.
Code: 4
Path: "C:\Users\jenplu01\Repos\oob-5.10\network_test"
Command: "mbedgt --test-spec .\BUILD\tests\REALTEK_RTL8195AM\IAR\test_spec.json -n mbed-os-tests-network-*,mbed-os-tests-netsocket*"
Tip: You could retry the last command with "-v" flag for verbose output
---
@SeppoTakalo FYI..
ashok-rao
on 14 Sep 2018
@yennster Would you mind reconfirming that this is still an issue today? The Austin site had some network issues for most of the day yesterday, which created false failures with the netsocket test in CI.
The issues seem to have resolved themselves, so I'm curious if this is still an issue as well.
cmonr
on 14 Sep 2018
It looks something for @samchuarm to investigate and work with Realtek.
MarceloSalazar
on 17 Sep 2018
Hi @ARMmbed/team-realtek @M-ichae-l , please help look into these GT netsocket test failures for RTL8195AM and address them as soon as you can. Thanks.
samchuarm
on 18 Sep 2018
By the way, do we confirm the failures are not caused by the network issue happened in Austin site as @cmonr mentioned?
samchuarm
on 18 Sep 2018
This is the result from our CI
04:22:48 +---------------------------+-------------------+-----------------------------+---------+--------------------+-------------+
04:22:48 | target | platform_name | test suite | result | elapsed_time (sec) | copy_method |
04:22:48 +---------------------------+-------------------+-----------------------------+---------+--------------------+-------------+
04:22:48 | REALTEK_RTL8195AM-GCC_ARM | REALTEK_RTL8195AM | mbed-os-tests-netsocket-dns | TIMEOUT | 199.91 | default |
04:22:48 | REALTEK_RTL8195AM-GCC_ARM | REALTEK_RTL8195AM | mbed-os-tests-netsocket-tcp | TIMEOUT | 560.05 | default |
04:22:48 | REALTEK_RTL8195AM-GCC_ARM | REALTEK_RTL8195AM | mbed-os-tests-netsocket-udp | FAIL | 510.15 | default |
04:22:48 | REALTEK_RTL8195AM-GCC_ARM | REALTEK_RTL8195AM | mbed-os-tests-network-wifi | FAIL | 169.83 | default |
04:22:48 +---------------------------+-------------------+-----------------------------+---------+--------------------+-------------+
04:22:48 mbedgt: test suite results: 2 FAIL / 2 TIMEOUT
04:22:48 mbedgt: test case report:
04:22:48 +---------------------------+-------------------+-----------------------------+------------------------------------+--------+--------+---------+--------------------+
04:22:48 | target | platform_name | test suite | test case | passed | failed | result | elapsed_time (sec) |
04:22:48 +---------------------------+-------------------+-----------------------------+------------------------------------+--------+--------+---------+--------------------+
04:22:48 | REALTEK_RTL8195AM-GCC_ARM | REALTEK_RTL8195AM | mbed-os-tests-netsocket-dns | mbed-os-tests-netsocket-dns | 0 | 1 | ERROR | 199.91 |
04:22:48 | REALTEK_RTL8195AM-GCC_ARM | REALTEK_RTL8195AM | mbed-os-tests-netsocket-tcp | TCPSOCKET_CONNECT_INVALID | 1 | 0 | OK | 18.31 |
04:22:48 | REALTEK_RTL8195AM-GCC_ARM | REALTEK_RTL8195AM | mbed-os-tests-netsocket-tcp | TCPSOCKET_ECHOTEST | 1 | 0 | OK | 13.33 |
04:22:48 | REALTEK_RTL8195AM-GCC_ARM | REALTEK_RTL8195AM | mbed-os-tests-netsocket-tcp | TCPSOCKET_ECHOTEST_BURST | 0 | 0 | ERROR | 0.0 |
04:22:48 | REALTEK_RTL8195AM-GCC_ARM | REALTEK_RTL8195AM | mbed-os-tests-netsocket-tcp | TCPSOCKET_ECHOTEST_BURST_NONBLOCK | 0 | 0 | SKIPPED | 0.0 |
04:22:48 | REALTEK_RTL8195AM-GCC_ARM | REALTEK_RTL8195AM | mbed-os-tests-netsocket-tcp | TCPSOCKET_ECHOTEST_NONBLOCK | 1 | 0 | OK | 30.07 |
04:22:48 | REALTEK_RTL8195AM-GCC_ARM | REALTEK_RTL8195AM | mbed-os-tests-netsocket-tcp | TCPSOCKET_ENDPOINT_CLOSE | 0 | 0 | SKIPPED | 0.0 |
04:22:48 | REALTEK_RTL8195AM-GCC_ARM | REALTEK_RTL8195AM | mbed-os-tests-netsocket-tcp | TCPSOCKET_OPEN_CLOSE_REPEAT | 1 | 0 | OK | 0.11 |
04:22:48 | REALTEK_RTL8195AM-GCC_ARM | REALTEK_RTL8195AM | mbed-os-tests-netsocket-tcp | TCPSOCKET_OPEN_LIMIT | 1 | 0 | OK | 0.11 |
04:22:48 | REALTEK_RTL8195AM-GCC_ARM | REALTEK_RTL8195AM | mbed-os-tests-netsocket-tcp | TCPSOCKET_RECV_100K | 0 | 0 | SKIPPED | 0.0 |
04:22:48 | REALTEK_RTL8195AM-GCC_ARM | REALTEK_RTL8195AM | mbed-os-tests-netsocket-tcp | TCPSOCKET_RECV_100K_NONBLOCK | 0 | 0 | SKIPPED | 0.0 |
04:22:48 | REALTEK_RTL8195AM-GCC_ARM | REALTEK_RTL8195AM | mbed-os-tests-netsocket-tcp | TCPSOCKET_RECV_TIMEOUT | 0 | 0 | SKIPPED | 0.0 |
04:22:48 | REALTEK_RTL8195AM-GCC_ARM | REALTEK_RTL8195AM | mbed-os-tests-netsocket-tcp | TCPSOCKET_SEND_REPEAT | 0 | 0 | SKIPPED | 0.0 |
04:22:48 | REALTEK_RTL8195AM-GCC_ARM | REALTEK_RTL8195AM | mbed-os-tests-netsocket-tcp | TCPSOCKET_SEND_TIMEOUT | 0 | 0 | SKIPPED | 0.0 |
04:22:48 | REALTEK_RTL8195AM-GCC_ARM | REALTEK_RTL8195AM | mbed-os-tests-netsocket-tcp | TCPSOCKET_THREAD_PER_SOCKET_SAFETY | 1 | 0 | OK | 8.76 |
04:22:48 | REALTEK_RTL8195AM-GCC_ARM | REALTEK_RTL8195AM | mbed-os-tests-netsocket-udp | UDPSOCKET_ECHOTEST | 1 | 0 | OK | 2.18 |
04:22:48 | REALTEK_RTL8195AM-GCC_ARM | REALTEK_RTL8195AM | mbed-os-tests-netsocket-udp | UDPSOCKET_ECHOTEST_BURST | 0 | 1 | FAIL | 210.94 |
04:22:48 | REALTEK_RTL8195AM-GCC_ARM | REALTEK_RTL8195AM | mbed-os-tests-netsocket-udp | UDPSOCKET_ECHOTEST_BURST_NONBLOCK | 1 | 0 | OK | 195.44 |
04:22:48 | REALTEK_RTL8195AM-GCC_ARM | REALTEK_RTL8195AM | mbed-os-tests-netsocket-udp | UDPSOCKET_ECHOTEST_NONBLOCK | 1 | 0 | OK | 9.71 |
04:22:48 | REALTEK_RTL8195AM-GCC_ARM | REALTEK_RTL8195AM | mbed-os-tests-netsocket-udp | UDPSOCKET_OPEN_CLOSE_REPEAT | 1 | 0 | OK | 0.0 |
04:22:48 | REALTEK_RTL8195AM-GCC_ARM | REALTEK_RTL8195AM | mbed-os-tests-netsocket-udp | UDPSOCKET_OPEN_LIMIT | 1 | 0 | OK | 0.11 |
04:22:48 | REALTEK_RTL8195AM-GCC_ARM | REALTEK_RTL8195AM | mbed-os-tests-netsocket-udp | UDPSOCKET_SENDTO_INVALID | 1 | 0 | OK | 0.0 |
04:22:48 | REALTEK_RTL8195AM-GCC_ARM | REALTEK_RTL8195AM | mbed-os-tests-netsocket-udp | UDPSOCKET_SENDTO_REPEAT | 1 | 0 | OK | 0.21 |
04:22:48 | REALTEK_RTL8195AM-GCC_ARM | REALTEK_RTL8195AM | mbed-os-tests-netsocket-udp | UDPSOCKET_SENDTO_TIMEOUT | 1 | 0 | OK | 0.11 |
04:22:48 | REALTEK_RTL8195AM-GCC_ARM | REALTEK_RTL8195AM | mbed-os-tests-network-wifi | WIFI-CONNECT | 1 | 0 | OK | 2.53 |
04:22:48 | REALTEK_RTL8195AM-GCC_ARM | REALTEK_RTL8195AM | mbed-os-tests-network-wifi | WIFI-CONNECT-DISCONNECT-REPEAT | 1 | 0 | OK | 22.8 |
04:22:48 | REALTEK_RTL8195AM-GCC_ARM | REALTEK_RTL8195AM | mbed-os-tests-network-wifi | WIFI-CONNECT-NOCREDENTIALS | 0 | 1 | FAIL | 15.07 |
04:22:48 | REALTEK_RTL8195AM-GCC_ARM | REALTEK_RTL8195AM | mbed-os-tests-network-wifi | WIFI-CONNECT-PARAMS-CHANNEL | 0 | 1 | FAIL | 1.73 |
04:22:48 | REALTEK_RTL8195AM-GCC_ARM | REALTEK_RTL8195AM | mbed-os-tests-network-wifi | WIFI-CONNECT-PARAMS-CHANNEL-FAIL | 0 | 1 | FAIL | 1.22 |
04:22:48 | REALTEK_RTL8195AM-GCC_ARM | REALTEK_RTL8195AM | mbed-os-tests-network-wifi | WIFI-CONNECT-PARAMS-NULL | 0 | 2 | FAIL | 17.54 |
04:22:48 | REALTEK_RTL8195AM-GCC_ARM | REALTEK_RTL8195AM | mbed-os-tests-network-wifi | WIFI-CONNECT-PARAMS-VALID-SECURE | 1 | 0 | OK | 5.13 |
04:22:48 | REALTEK_RTL8195AM-GCC_ARM | REALTEK_RTL8195AM | mbed-os-tests-network-wifi | WIFI-CONNECT-PARAMS-VALID-UNSECURE | 0 | 0 | ERROR | 0.0 |
04:22:48 | REALTEK_RTL8195AM-GCC_ARM | REALTEK_RTL8195AM | mbed-os-tests-network-wifi | WIFI-CONNECT-SECURE | 1 | 0 | OK | 2.49 |
04:22:48 | REALTEK_RTL8195AM-GCC_ARM | REALTEK_RTL8195AM | mbed-os-tests-network-wifi | WIFI-CONNECT-SECURE-FAIL | 1 | 0 | OK | 6.77 |
04:22:48 | REALTEK_RTL8195AM-GCC_ARM | REALTEK_RTL8195AM | mbed-os-tests-network-wifi | WIFI-CONSTRUCTOR | 1 | 0 | OK | 2.29 |
04:22:48 | REALTEK_RTL8195AM-GCC_ARM | REALTEK_RTL8195AM | mbed-os-tests-network-wifi | WIFI-GET-RSSI | 1 | 0 | OK | 4.14 |
04:22:48 | REALTEK_RTL8195AM-GCC_ARM | REALTEK_RTL8195AM | mbed-os-tests-network-wifi | WIFI-SCAN | 1 | 0 | OK | 1.31 |
04:22:48 | REALTEK_RTL8195AM-GCC_ARM | REALTEK_RTL8195AM | mbed-os-tests-network-wifi | WIFI-SCAN-NULL | 1 | 0 | OK | 1.3 |
04:22:48 | REALTEK_RTL8195AM-GCC_ARM | REALTEK_RTL8195AM | mbed-os-tests-network-wifi | WIFI-SET-CHANNEL | 0 | 3 | FAIL | 0.0 |
04:22:48 | REALTEK_RTL8195AM-GCC_ARM | REALTEK_RTL8195AM | mbed-os-tests-network-wifi | WIFI-SET-CREDENTIAL | 0 | 1 | FAIL | 0.11 |
04:22:48 +---------------------------+-------------------+-----------------------------+------------------------------------+--------+--------+---------+--------------------+
04:22:48 mbedgt: test case results: 7 FAIL / 7 SKIPPED / 23 OK / 3 ERROR
04:22:48 mbedgt: completed in 1440.12 sec
SeppoTakalo
on 18 Sep 2018
Thanks @SeppoTakalo . In that case, Hi @M-ichae-l , please help look into these failures. Thanks.
samchuarm
on 18 Sep 2018
PR ARMmbed/mbed-os#6292 is missing from new EMAC interface
VeijoPesonen
on 18 Sep 2018
Since the netsocket tests were fully passed when Realtek delivered the EMAC interface, it seems these failures are due to the 5.10 updates.
samchuarm
on 20 Sep 2018
Hi,
Sorry for the late reply. I just finished testing the issue with 5.9 and 5.10-rc2. The tests are all passed from my side. As @samchuarm said the reason may be the 5.10 updates or setting error.
In addition, I have found that "RealtekInterface.json" from "~mbed-os\tools\test_configs" is missing after 5.9.
M-ichae-l
on 20 Sep 2018
@samchuarm What makes you assume that all tests have passed by that time?
During the EMAC work, only EMAC tests have been ran.
I cannot see any evidence that these test have been ran before when this driver has been developed:
SeppoTakalo
on 20 Sep 2018
@M-ichae-l Hi, can you provide the log from the test run?
Also, are you running the full, extented, test set as instructed here: https://github.com/ARMmbed/mbed-os/blob/master/TESTS/netsocket/README.md
For some of the testcases, we might have outdated configurations as there are now maintenance work being done in the test infra. Those will be resolved soon.
This is the result from last night:
04:23:32 mbedgt: checking for GCOV data...
04:23:32 mbedgt: mbed-host-test-runner: stopped and returned 'FAIL'
04:23:32 mbedgt: test on hardware with target id: DUMMY
04:23:32 mbedgt: test suite 'mbed-os-tests-netsocket-udp' ..................................................... FAIL in 507.84 sec
04:23:32 test case: 'UDPSOCKET_ECHOTEST' .............................................................. OK in 3.43 sec
04:23:32 test case: 'UDPSOCKET_ECHOTEST_BURST' ........................................................ FAIL in 197.24 sec
04:23:32 test case: 'UDPSOCKET_ECHOTEST_BURST_NONBLOCK' ............................................... OK in 215.07 sec
04:23:32 test case: 'UDPSOCKET_ECHOTEST_NONBLOCK' ..................................................... OK in 3.27 sec
04:23:32 test case: 'UDPSOCKET_OPEN_CLOSE_REPEAT' ..................................................... OK in 0.00 sec
04:23:32 test case: 'UDPSOCKET_OPEN_LIMIT' ............................................................ OK in 0.00 sec
04:23:32 test case: 'UDPSOCKET_SENDTO_INVALID' ........................................................ OK in 0.15 sec
04:23:32 test case: 'UDPSOCKET_SENDTO_REPEAT' ......................................................... OK in 0.32 sec
04:23:32 test case: 'UDPSOCKET_SENDTO_TIMEOUT' ........................................................ OK in 0.00 sec
04:23:32 mbedgt: test case summary: 8 passes, 1 failure
04:23:32 mbedgt: all tests finished!
04:23:32 mbedgt: shuffle seed: 0.5112512543
04:23:32 mbedgt: exporting to JUNIT file 'junit.xml'...
04:23:32 mbedgt: exporting to TEXT 'test_results.txt'...
04:23:32 mbedgt: exporting to HTML file 'test_results.html'...
04:23:32 mbedgt: test suite report:
04:23:32 +---------------------------+-------------------+-----------------------------+---------+--------------------+-------------+
04:23:32 | target | platform_name | test suite | result | elapsed_time (sec) | copy_method |
04:23:32 +---------------------------+-------------------+-----------------------------+---------+--------------------+-------------+
04:23:32 | REALTEK_RTL8195AM-GCC_ARM | REALTEK_RTL8195AM | mbed-os-tests-netsocket-dns | TIMEOUT | 200.18 | default |
04:23:32 | REALTEK_RTL8195AM-GCC_ARM | REALTEK_RTL8195AM | mbed-os-tests-netsocket-tcp | TIMEOUT | 560.21 | default |
04:23:32 | REALTEK_RTL8195AM-GCC_ARM | REALTEK_RTL8195AM | mbed-os-tests-netsocket-udp | FAIL | 507.84 | default |
04:23:32 | REALTEK_RTL8195AM-GCC_ARM | REALTEK_RTL8195AM | mbed-os-tests-network-wifi | FAIL | 170.09 | default |
04:23:32 +---------------------------+-------------------+-----------------------------+---------+--------------------+-------------+
04:23:32 mbedgt: test suite results: 2 FAIL / 2 TIMEOUT
04:23:32 mbedgt: test case report:
04:23:32 +---------------------------+-------------------+-----------------------------+------------------------------------+--------+--------+---------+--------------------+
04:23:32 | target | platform_name | test suite | test case | passed | failed | result | elapsed_time (sec) |
04:23:32 +---------------------------+-------------------+-----------------------------+------------------------------------+--------+--------+---------+--------------------+
04:23:32 | REALTEK_RTL8195AM-GCC_ARM | REALTEK_RTL8195AM | mbed-os-tests-netsocket-dns | mbed-os-tests-netsocket-dns | 0 | 1 | ERROR | 200.18 |
04:23:32 | REALTEK_RTL8195AM-GCC_ARM | REALTEK_RTL8195AM | mbed-os-tests-netsocket-tcp | TCPSOCKET_CONNECT_INVALID | 1 | 0 | OK | 18.23 |
04:23:32 | REALTEK_RTL8195AM-GCC_ARM | REALTEK_RTL8195AM | mbed-os-tests-netsocket-tcp | TCPSOCKET_ECHOTEST | 1 | 0 | OK | 5.93 |
04:23:32 | REALTEK_RTL8195AM-GCC_ARM | REALTEK_RTL8195AM | mbed-os-tests-netsocket-tcp | TCPSOCKET_ECHOTEST_BURST | 0 | 0 | ERROR | 0.0 |
04:23:32 | REALTEK_RTL8195AM-GCC_ARM | REALTEK_RTL8195AM | mbed-os-tests-netsocket-tcp | TCPSOCKET_ECHOTEST_BURST_NONBLOCK | 0 | 0 | SKIPPED | 0.0 |
04:23:32 | REALTEK_RTL8195AM-GCC_ARM | REALTEK_RTL8195AM | mbed-os-tests-netsocket-tcp | TCPSOCKET_ECHOTEST_NONBLOCK | 1 | 0 | OK | 30.15 |
04:23:32 | REALTEK_RTL8195AM-GCC_ARM | REALTEK_RTL8195AM | mbed-os-tests-netsocket-tcp | TCPSOCKET_ENDPOINT_CLOSE | 0 | 0 | SKIPPED | 0.0 |
04:23:32 | REALTEK_RTL8195AM-GCC_ARM | REALTEK_RTL8195AM | mbed-os-tests-netsocket-tcp | TCPSOCKET_OPEN_CLOSE_REPEAT | 1 | 0 | OK | 0.0 |
04:23:32 | REALTEK_RTL8195AM-GCC_ARM | REALTEK_RTL8195AM | mbed-os-tests-netsocket-tcp | TCPSOCKET_OPEN_LIMIT | 1 | 0 | OK | 0.0 |
04:23:32 | REALTEK_RTL8195AM-GCC_ARM | REALTEK_RTL8195AM | mbed-os-tests-netsocket-tcp | TCPSOCKET_RECV_100K | 0 | 0 | SKIPPED | 0.0 |
04:23:32 | REALTEK_RTL8195AM-GCC_ARM | REALTEK_RTL8195AM | mbed-os-tests-netsocket-tcp | TCPSOCKET_RECV_100K_NONBLOCK | 0 | 0 | SKIPPED | 0.0 |
04:23:32 | REALTEK_RTL8195AM-GCC_ARM | REALTEK_RTL8195AM | mbed-os-tests-netsocket-tcp | TCPSOCKET_RECV_TIMEOUT | 0 | 0 | SKIPPED | 0.0 |
04:23:32 | REALTEK_RTL8195AM-GCC_ARM | REALTEK_RTL8195AM | mbed-os-tests-netsocket-tcp | TCPSOCKET_SEND_REPEAT | 0 | 0 | SKIPPED | 0.0 |
04:23:32 | REALTEK_RTL8195AM-GCC_ARM | REALTEK_RTL8195AM | mbed-os-tests-netsocket-tcp | TCPSOCKET_SEND_TIMEOUT | 0 | 0 | SKIPPED | 0.0 |
04:23:32 | REALTEK_RTL8195AM-GCC_ARM | REALTEK_RTL8195AM | mbed-os-tests-netsocket-tcp | TCPSOCKET_THREAD_PER_SOCKET_SAFETY | 1 | 0 | OK | 5.73 |
04:23:32 | REALTEK_RTL8195AM-GCC_ARM | REALTEK_RTL8195AM | mbed-os-tests-netsocket-udp | UDPSOCKET_ECHOTEST | 1 | 0 | OK | 3.43 |
04:23:32 | REALTEK_RTL8195AM-GCC_ARM | REALTEK_RTL8195AM | mbed-os-tests-netsocket-udp | UDPSOCKET_ECHOTEST_BURST | 0 | 1 | FAIL | 197.24 |
04:23:32 | REALTEK_RTL8195AM-GCC_ARM | REALTEK_RTL8195AM | mbed-os-tests-netsocket-udp | UDPSOCKET_ECHOTEST_BURST_NONBLOCK | 1 | 0 | OK | 215.07 |
04:23:32 | REALTEK_RTL8195AM-GCC_ARM | REALTEK_RTL8195AM | mbed-os-tests-netsocket-udp | UDPSOCKET_ECHOTEST_NONBLOCK | 1 | 0 | OK | 3.27 |
04:23:32 | REALTEK_RTL8195AM-GCC_ARM | REALTEK_RTL8195AM | mbed-os-tests-netsocket-udp | UDPSOCKET_OPEN_CLOSE_REPEAT | 1 | 0 | OK | 0.0 |
04:23:32 | REALTEK_RTL8195AM-GCC_ARM | REALTEK_RTL8195AM | mbed-os-tests-netsocket-udp | UDPSOCKET_OPEN_LIMIT | 1 | 0 | OK | 0.0 |
04:23:32 | REALTEK_RTL8195AM-GCC_ARM | REALTEK_RTL8195AM | mbed-os-tests-netsocket-udp | UDPSOCKET_SENDTO_INVALID | 1 | 0 | OK | 0.15 |
04:23:32 | REALTEK_RTL8195AM-GCC_ARM | REALTEK_RTL8195AM | mbed-os-tests-netsocket-udp | UDPSOCKET_SENDTO_REPEAT | 1 | 0 | OK | 0.32 |
04:23:32 | REALTEK_RTL8195AM-GCC_ARM | REALTEK_RTL8195AM | mbed-os-tests-netsocket-udp | UDPSOCKET_SENDTO_TIMEOUT | 1 | 0 | OK | 0.0 |
04:23:32 | REALTEK_RTL8195AM-GCC_ARM | REALTEK_RTL8195AM | mbed-os-tests-network-wifi | WIFI-CONNECT | 1 | 0 | OK | 2.41 |
04:23:32 | REALTEK_RTL8195AM-GCC_ARM | REALTEK_RTL8195AM | mbed-os-tests-network-wifi | WIFI-CONNECT-DISCONNECT-REPEAT | 1 | 0 | OK | 24.13 |
04:23:32 | REALTEK_RTL8195AM-GCC_ARM | REALTEK_RTL8195AM | mbed-os-tests-network-wifi | WIFI-CONNECT-NOCREDENTIALS | 0 | 1 | FAIL | 15.09 |
04:23:32 | REALTEK_RTL8195AM-GCC_ARM | REALTEK_RTL8195AM | mbed-os-tests-network-wifi | WIFI-CONNECT-PARAMS-CHANNEL | 1 | 0 | OK | 0.98 |
04:23:32 | REALTEK_RTL8195AM-GCC_ARM | REALTEK_RTL8195AM | mbed-os-tests-network-wifi | WIFI-CONNECT-PARAMS-CHANNEL-FAIL | 0 | 1 | FAIL | 0.88 |
04:23:32 | REALTEK_RTL8195AM-GCC_ARM | REALTEK_RTL8195AM | mbed-os-tests-network-wifi | WIFI-CONNECT-PARAMS-NULL | 0 | 2 | FAIL | 17.43 |
04:23:32 | REALTEK_RTL8195AM-GCC_ARM | REALTEK_RTL8195AM | mbed-os-tests-network-wifi | WIFI-CONNECT-PARAMS-VALID-SECURE | 1 | 0 | OK | 4.82 |
04:23:32 | REALTEK_RTL8195AM-GCC_ARM | REALTEK_RTL8195AM | mbed-os-tests-network-wifi | WIFI-CONNECT-PARAMS-VALID-UNSECURE | 1 | 0 | OK | 4.94 |
04:23:32 | REALTEK_RTL8195AM-GCC_ARM | REALTEK_RTL8195AM | mbed-os-tests-network-wifi | WIFI-CONNECT-SECURE | 1 | 0 | OK | 2.41 |
04:23:32 | REALTEK_RTL8195AM-GCC_ARM | REALTEK_RTL8195AM | mbed-os-tests-network-wifi | WIFI-CONNECT-SECURE-FAIL | 1 | 0 | OK | 6.91 |
04:23:32 | REALTEK_RTL8195AM-GCC_ARM | REALTEK_RTL8195AM | mbed-os-tests-network-wifi | WIFI-CONSTRUCTOR | 1 | 0 | OK | 2.29 |
04:23:32 | REALTEK_RTL8195AM-GCC_ARM | REALTEK_RTL8195AM | mbed-os-tests-network-wifi | WIFI-GET-RSSI | 1 | 0 | OK | 4.06 |
04:23:32 | REALTEK_RTL8195AM-GCC_ARM | REALTEK_RTL8195AM | mbed-os-tests-network-wifi | WIFI-SCAN | 1 | 0 | OK | 1.32 |
04:23:32 | REALTEK_RTL8195AM-GCC_ARM | REALTEK_RTL8195AM | mbed-os-tests-network-wifi | WIFI-SCAN-NULL | 1 | 0 | OK | 1.27 |
04:23:32 | REALTEK_RTL8195AM-GCC_ARM | REALTEK_RTL8195AM | mbed-os-tests-network-wifi | WIFI-SET-CHANNEL | 0 | 3 | FAIL | 0.0 |
04:23:32 | REALTEK_RTL8195AM-GCC_ARM | REALTEK_RTL8195AM | mbed-os-tests-network-wifi | WIFI-SET-CREDENTIAL | 0 | 1 | FAIL | 0.1 |
04:23:32 +---------------------------+-------------------+-----------------------------+------------------------------------+--------+--------+---------+--------------------+
04:23:32 mbedgt: test case results: 6 FAIL / 7 SKIPPED / 25 OK / 2 ERROR
04:23:32 mbedgt: completed in 1438.51 sec
Note: DNS_CACHE cases are known to fail, fix already submitted in https://github.com/ARMmbed/mbed-os/pull/8171
SeppoTakalo
on 20 Sep 2018
@SeppoTakalo , I see your point. It was my misunderstanding.
samchuarm
on 20 Sep 2018
@M-ichae-l , Hi can you please address Seppo's question?
samchuarm
on 21 Sep 2018
@SeppoTakalo
I am running mbed test -n mbed-os-tests-netsocket*. But I did not get all testcases. So I am going to test again. One more thing, does this issue only happens on windows 10 ?
M-ichae-l
on 25 Sep 2018
It should not be host OS specific.
SeppoTakalo
on 25 Sep 2018
Hi @M-ichae-l , do you have update for running the tests?
samchuarm
on 2 Oct 2018
Hi @samchuarm, @AdamZhang0124 is currently taking over this issue. We have tested that it is not hosted OS specific.
M-ichae-l
on 2 Oct 2018
Thanks @M-ichae-l . @AdamZhang0124 , are you able to reproduce the issues now?
samchuarm
on 2 Oct 2018
@M-ichae-l, @samchuarm
Hi Sam, nice to meet you! I have tried to run the aforementioned network socket test plan under the following settings:
Environment: Windows 10
Mbed CLI version: 1.8.0
Python version: 2.7.14
Mbed OS: mbed-os-5.10.0-rc2
However, it seems like some test cases were not generated while I was testing.
I am still working on it and will let you know for further updates, thanks!
AdamZhang0124
on 3 Oct 2018
Hi @AdamZhang0124
Do you see testcases in "skipped" state after a build?
The reason for this is that if the testcases do not find configuration values they are expecting, they will skip the build.
For example, if the device does not provide default interface, or if some values missing.
See example test configuration from: https://github.com/ARMmbed/mbed-os/blob/master/tools/test_configs/EthernetInterface.json
Then examples, how test cases skip the build if values are not provided:
https://github.com/ARMmbed/mbed-os/blob/master/TESTS/netsocket/tcp/main.cpp#L19
https://github.com/ARMmbed/mbed-os/blob/master/TESTS/network/wifi/main.cpp#L19
https://github.com/ARMmbed/mbed-os/blob/master/TESTS/network/wifi/main.cpp#L38
SeppoTakalo
on 3 Oct 2018
Hi @AdamZhang0124 , thanks for the update. Is there a log that you can share so Seppo could help pinpoint where you might configure the tests wrong?
samchuarm
on 3 Oct 2018
Hi @samchuarm and @SeppoTakalo,
These are the test logs generated from the tests I just conducted for your information:
dns_log.txt
tcp_log.txt
udp_log.txt
wifi_log.txt
For the wifi test, there may have some setting errors that I am troubleshooting now.
Also, some of the test cases were not being included after a build like the test log that you posted earlier.
AdamZhang0124
on 3 Oct 2018
I still see that all the extended tests are missing.
https://github.com/ARMmbed/mbed-os/blob/master/TESTS/netsocket/udp/main.cpp#L98..L103
https://github.com/ARMmbed/mbed-os/blob/master/TESTS/netsocket/tcp/main.cpp#L141..L150
I instructed earlier to run the full set: https://github.com/ARMmbed/mbed-os/issues/8124#issuecomment-423097427
SeppoTakalo
on 3 Oct 2018
And what comest to WiFi tests, there is a reason why tests are skipped:
+-----------------------+-------------------+--------------------+------------------------------------+--------+--------+---------+--------------------+
| target | platform_name | test suite | test case | passed | failed | result | elapsed_time (sec) |
+-----------------------+-------------------+--------------------+------------------------------------+--------+--------+---------+--------------------+
| REALTEK_RTL8195AM-IAR | REALTEK_RTL8195AM | tests-network-wifi | WIFI-CONNECT | 0 | 0 | SKIPPED | 0.0 |
| REALTEK_RTL8195AM-IAR | REALTEK_RTL8195AM | tests-network-wifi | WIFI-CONNECT-DISCONNECT-REPEAT | 0 | 0 | SKIPPED | 0.0 |
| REALTEK_RTL8195AM-IAR | REALTEK_RTL8195AM | tests-network-wifi | WIFI-CONNECT-NOCREDENTIALS | 0 | 1 | FAIL | 15.04 |
| REALTEK_RTL8195AM-IAR | REALTEK_RTL8195AM | tests-network-wifi | WIFI-CONNECT-PARAMS-CHANNEL | 0 | 0 | SKIPPED | 0.0 |
| REALTEK_RTL8195AM-IAR | REALTEK_RTL8195AM | tests-network-wifi | WIFI-CONNECT-PARAMS-CHANNEL-FAIL | 0 | 0 | SKIPPED | 0.0 |
Greentea by default runs tests until one fails, then rest are skipped, because device is assumed to be unknown state.
SeppoTakalo
on 3 Oct 2018
And what comest to WiFi tests, there is a reason why tests are skipped:
+-----------------------+-------------------+--------------------+------------------------------------+--------+--------+---------+--------------------+ | target | platform_name | test suite | test case | passed | failed | result | elapsed_time (sec) | +-----------------------+-------------------+--------------------+------------------------------------+--------+--------+---------+--------------------+ | REALTEK_RTL8195AM-IAR | REALTEK_RTL8195AM | tests-network-wifi | WIFI-CONNECT | 0 | 0 | SKIPPED | 0.0 | | REALTEK_RTL8195AM-IAR | REALTEK_RTL8195AM | tests-network-wifi | WIFI-CONNECT-DISCONNECT-REPEAT | 0 | 0 | SKIPPED | 0.0 | | REALTEK_RTL8195AM-IAR | REALTEK_RTL8195AM | tests-network-wifi | WIFI-CONNECT-NOCREDENTIALS | 0 | 1 | FAIL | 15.04 | | REALTEK_RTL8195AM-IAR | REALTEK_RTL8195AM | tests-network-wifi | WIFI-CONNECT-PARAMS-CHANNEL | 0 | 0 | SKIPPED | 0.0 | | REALTEK_RTL8195AM-IAR | REALTEK_RTL8195AM | tests-network-wifi | WIFI-CONNECT-PARAMS-CHANNEL-FAIL | 0 | 0 | SKIPPED | 0.0 |Greentea by default runs tests until one fails, then rest are skipped, because device is assumed to be unknown state.
Oh I see. Is there a way to test all the rest cases even if one fail in one time rather than //comment that fail case in the test code and run the rest?
AdamZhang0124
on 3 Oct 2018
I still see that all the extended tests are missing.
https://github.com/ARMmbed/mbed-os/blob/master/TESTS/netsocket/udp/main.cpp#L98..L103
https://github.com/ARMmbed/mbed-os/blob/master/TESTS/netsocket/tcp/main.cpp#L141..L150I instructed earlier to run the full set: #8124 (comment)
I see the problem, thanks. I will try to build the extended test cases and let you know the results.
AdamZhang0124
on 3 Oct 2018
I believe, but not totally sure that continuing the testcase execution can be achieved by modifying the failure-handler as documented here:
https://github.com/ARMmbed/mbed-os/tree/master/features/frameworks/utest#default-handlers
If you have the setup ready, try modifying this line:
https://github.com/ARMmbed/mbed-os/blob/master/TESTS/network/wifi/main.cpp#L93
to
Specification specification(test_setup, cases, greentea_continue_handlers);
Internal Jira reference: https://jira.arm.com/browse/IOTPART-6550
adbridge
on 4 Oct 2018
Hi @AdamZhang0124 , do you have update on the test results?
samchuarm
on 7 Oct 2018
Hi @samchuarm, I have run the extended tests and some of the test cases are failed or have errors. I will working on that once I have finished my primary task and I will let you know the progress as soon as possible, thanks!
AdamZhang0124
on 8 Oct 2018
Hi @AdamZhang0124 do you have update on this? Thanks.
samchuarm
on 25 Oct 2018
Hi @samchuarm , I am working on troubleshooting the udp.
The udp test result is currently inconsistent. Sometimes the test is ok but sometimes fail.
The log which has all test cases passed is attached for your reference:
udp_test.txt
Thanks
AdamZhang0124
on 26 Oct 2018
Hi @AdamZhang0124 , are you running the full extended test set as instructed here https://github.com/ARMmbed/mbed-os/blob/master/TESTS/netsocket/README.md ?
Hi @SeppoTakalo , does it seem a valid log to you? Thanks.
samchuarm
on 26 Oct 2018
Yes, that is the full set of UDP tests.
SeppoTakalo
on 29 Oct 2018
Thanks @SeppoTakalo , can you suggest what could cause the inconsistent result?
Hi @AdamZhang0124 , can you share what currently fails from the UDP tests? Also what about other socket tests?
There is a PR being blocked by this issue. It'd be appreciated if you can prioritize it. Thanks.
samchuarm
on 29 Oct 2018
Hi @samchuarm , currently the UDP test will fail for 'UDPSOCKET_ECHOTEST_BURST' and 'UDPSOCKET_ECHOTEST_BURST_NONBLOCK' test cases. These failures are caused by high loss ratio. In the udp test, the loss ratio tolerance is 30%. But in practice, 50%-60% of packets can be received in most of time and sometimes >70% can be received. I am now reasoning the high loss ratio and seeking a way to reduce that.
AdamZhang0124
on 30 Oct 2018
Same failures happen on TCP tests as well. For TCP tests, 'TCPSOCKET_ECHOTEST_BURST', 'TCPSOCKET_ECHOTEST_BURST_NONBLOCK', 'TCPSOCKET_RECV_100K', 'TCPSOCKET_RECV_100K_NONBLOCK', 'TCPSOCKET_RECV_TIMEOUT' test cases have problem.
AdamZhang0124
on 30 Oct 2018
Hi @AdamZhang0124 , any update on these failures? Have you been able to fix them on RTL8195AM?
samchuarm
on 5 Nov 2018
Hi @SeppoTakalo , per Realtek's update, they are able to lower the UDP ‘ECHOTEST_BURST’ loss ratio to 10%, that is about more than 450 over 500 packets can be received. Do you have any other suggestion for further improving the loss ratio in order to meet the test requirement? Thanks.
samchuarm
on 6 Nov 2018
Hi @samchuarm and @SeppoTakalo , I have found that increasing 'PBUF_POOL_SIZE' in lwipstack.h to 20 can reduce the loss ratio to the acceptable range. But I am not sure if it is allowed to edit.
AdamZhang0124
on 7 Nov 2018
Hi @SeppoTakalo , is increasing 'PBUF_POOL_SIZE' allowed for partners platform? Can this be an acceptable solution? Thx.
samchuarm
on 7 Nov 2018
Some platforms do change that value through Mbed OS configuration system https://github.com/ARMmbed/mbed-os/blob/master/features/lwipstack/mbed_lib.json#L116
SeppoTakalo
on 7 Nov 2018
Thanks @SeppoTakalo , if that value is allowed to change by partners, do we accept changing the value as the solution to this issue? Any concern? Thx
samchuarm
on 8 Nov 2018
Thanks @SeppoTakalo , if that value is allowed to change by partners, do we accept changing the value as the solution to this issue? Any concern? Thx
Renesas target already defines this config (line 158 in the config), should be fine to be set by the target.
0xc0170
on 8 Nov 2018
Thanks @0xc0170 .
Hi @AdamZhang0124 , please look into adding this config for RTL8195AM once you verify increasing 'PBUF_POOL_SIZE' can fully resolve netsocket failures. Thx.
samchuarm
on 8 Nov 2018
Hi, @samchuarm and @SeppoTakalo, I have modified the buffer size to 20 and optimized memory location by editing the linker file.
The package loss ratio of the udp burst tests has now been reduced to the acceptable range.
Please also help to verify the performance from your side.
Files need to be modified:
- mbed-os\features\lwipstack\ lwipopts.h
define PBUF_POOL_SIZE 20
- mbed-os\targets\TARGET_Realtek\TARGET_AMEBA\TARGET_MCU_RTL8195Adevice\TOOLCHAIN_ARM_STD\ rtl8195a.sct
rtl8195a.sct.txt
(PS: Please modify file name extension from .sct.txt to .sct before replacement)
My test result is attached for your information:
udp19112018.txt
AdamZhang0124
on 19 Nov 2018
Also, I always see the following ERROR when I am running the TCP burst test:
As the reason of this ERROR is uncertain, it will be much helpful if you have any idea or solution on that.
Many thanks!
The detailed test log is attached for your information:
tcp19112018.txt
AdamZhang0124
on 19 Nov 2018
Hi @AdamZhang0124 and @ARMmbed/team-realtek , any update on adding the config 'PBUF_POOL_SIZE' for RTL8195AM and ensuring it could resolve the netsocket failures? Thanks.
samchuarm
on 19 Nov 2018
You should not modify the lwipopts.h file.
Instead add target override that changes the MBED_CONF_LWIP_PBUF_POOL_SIZE as other platforms do.
See: https://github.com/ARMmbed/mbed-os/blob/master/features/lwipstack/mbed_lib.json#L158
SeppoTakalo
on 19 Nov 2018
Hi @AdamZhang0124 , please follow @SeppoTakalo 's instruction to add the target override for MBED_CONF_LWIP_PBUF_POOL_SIZE.
Hi @SeppoTakalo , does the test results for the UDP tests look ok to you? They seem all passed.
And could you please help give some pointers for Realtek about the TCP burst test errors that Adam is seeing? Thanks.
samchuarm
on 20 Nov 2018
Hi @samchuarm and @SeppoTakalo, I have identified the reason which causes udp test and tcp test failure. It is related to the thread stack size and packet buffer size. I need to do more tests to find suitable values.
So far, the dns, udp, tcp tests can all pass.
fullrun20_11_2018.txt
Other than that, the Network Wifi test still has problems, I will keep on working on that.
AdamZhang0124
on 20 Nov 2018
+-----------------------+-------------------+-----------------------------+--------+--------------------+-------------+
| target | platform_name | test suite | result | elapsed_time (sec) | copy_method |
+-----------------------+-------------------+-----------------------------+--------+--------------------+-------------+
| REALTEK_RTL8195AM-ARM | REALTEK_RTL8195AM | mbed-os-tests-netsocket-dns | OK | 117.4 | default |
| REALTEK_RTL8195AM-ARM | REALTEK_RTL8195AM | mbed-os-tests-netsocket-tcp | OK | 170.99 | default |
| REALTEK_RTL8195AM-ARM | REALTEK_RTL8195AM | mbed-os-tests-netsocket-udp | OK | 147.79 | default |
+-----------------------+-------------------+-----------------------------+--------+--------------------+-------------+
mbedgt: test suite results: 3 OK
This looks good now.
SeppoTakalo
on 20 Nov 2018
@SeppoTakalo , thanks for the review.
Hi @AdamZhang0124 , thanks for the work to fix these. Can you show us how you implement the fix for preview? Can you share your netsocket WiFi test results? Does it fail the same way as reported? Thanks.
samchuarm
on 21 Nov 2018
Hi @samchuarm , the failure of UDP test and TCP test is due to the small heap size. The default heap size is 1600. Only by changing the 'mem-size' value in mbed_lib.json to override the default value could fix this issue.
The network WIFI test log is attached for your information:
wifi_test_log.txt
AdamZhang0124
on 21 Nov 2018
Thanks @AdamZhang0124 , that's quite many failures in the WiFi tests. Anything do you suspect that could cause these tests fail?
Hi @SeppoTakalo and @0xc0170 , is it okay for Realtek to override the 'mem-size' value in mbed_lib.json for passing UDP/TCP netsocket tests ?
samchuarm
on 22 Nov 2018
Hi @samchuarm, I have found the problem. Some failures are caused because the unsecured AP was not set properly. This is the new test result and I am debugging the reminding failures. Thanks!
AdamZhang0124
on 22 Nov 2018
@samchuarm and @AdamZhang0124 Yes. It is OK to override the mem-size for this particular board if it is backed up with a test result.
Many platform already override the value.
SeppoTakalo
on 22 Nov 2018
Hi @SeppoTakalo, I forgot to attach the full test log.
fullrun_22112018.txt
The udp and tcp tests are now able to pass after increasing the heap value. The packet loss ratio now can be reduced to less than 0.01.
AdamZhang0124
on 22 Nov 2018
Hi @AdamZhang0124 , any update on the WiFi tests? Thanks.
samchuarm
on 28 Nov 2018
Hi @samchuarm , so far I have fixed the WIFI-SCAN test case failure. This failure is due to the wrong expression of the 'wifi-secure-protocol' value in mbed_app.json file.
As in 'mbed-os/TESTS/netsocket/README.md', the 'wifi-secure-protocol' valid values are suggested to be 'WEP, WPA, WPA2, WPA_WPA2'.
However, for 'WPA_WPA2', the correct expression should be 'WPA/WPA2' as indicated in Nsapi_types.h.
There is a small typo that need to be noticed. After I change to WPA/WPA2 based on my wifi settings, this WIFI-SCAN test case is able to pass.
AdamZhang0124
on 30 Nov 2018
Hi @SeppoTakalo , can you please review Adam's comments? Is this something needs a PR to address? Thanks.
Hi @AdamZhang0124 , what about other WiFi test failures? Are you able to address them now? Thanks.
samchuarm
on 3 Dec 2018
@AdamZhang0124
I don't see the problem.
What mbed_app.json file are you referring to?
The WPA_WPA2 is the correct form, as indicated in valid values are ... description.
This is used in NetworkInterfaceDefaults.cpp with C preprocessor to append NSAPI_SECURITY_ prefix into the string. So the final string would be NSAPI_SECURITY_WPA_WPA2 which is the enum on the line you have highlighted.
If you would use WPA/WPA2 the string would be replaced with NSAPI_SECURITY_WPA/WPA2 where WPA2 is not known, which should then lead to build failure.
SeppoTakalo
on 3 Dec 2018
Hi @SeppoTakalo,
In mbed_app.json, the _"wifi-secure-protocol"_ refers to _MBED_CONF_APP_WIFI_SECURE_PROTOCOL_.
In Wifi_scan.cpp, the program checks the value of _get_security()_.
In Get_security.cpp, the program compares the _MBED_CONF_APP_WIFI_SECURE_PROTOCOL_ value with the preset char and then returns _NSAPI_SECURITY_WPA_WPA2_
Therefore, I believe the value of _MBED_CONF_APP_WIFI_SECURE_PROTOCOL_ should follow the preset char naming which uses WPA/WPA2 to represent WPA_WPA2.
Please correct me if I am wrong, thanks.
AdamZhang0124
on 4 Dec 2018
Ah.. true.. you are correct.
Seems that our documentation and example configuration file are out of sync.
The WPA/WPA2 form is used in the get_security()
The WPA_WPA2 form is used in nsapi.default-wifi-security and in the README.md you pointed.
Looks like we need to unify those and use only the WPA_WPA2 form. Thank you for pointing this out.
SeppoTakalo
on 4 Dec 2018
Hi @SeppoTakalo , thanks for the review. Then I guess WIFI-SCAN test failures are not Realtek platform specific.
Hi @AdamZhang0124 , What are the remaining WiFi failures you're still working on? Any progress?
I'd suggest you to start submitting PRs to check in the fixes for the dns/tcp/udp test failures since PR review/CI/approval takes some time. Thanks.
samchuarm
on 5 Dec 2018
Hi @samchuarm ,
I have solved the remaining test case failures. But for 'WIFI-CONNECT-DISCONNECT-REPEAT', the test result is sometimes failed. So I need more time to verify the problem and make it stable. Thanks.
AdamZhang0124
on 6 Dec 2018
Hi @AdamZhang0124 , this looks a lot better now. Thank you.
While you're still working on the test case of 'WIFI-CONNECT-DISCONNECT-REPEAT', can you start submitting PR to check in fixes for the failures you've addressed?
samchuarm
on 7 Dec 2018
Hi @samchuarm , I have fixed all the test case failures and start final checking the codes before submitting PR.
Btw, I have seen the new test case extension #9001 , should I test them also before I submit a PR?
Thanks.
10122018FullRun.txt
AdamZhang0124
on 10 Dec 2018
Hi @AdamZhang0124 , that's great to know. Thanks for the work. Please submit PR to check in the fixes that you have now as soon as you could. Please associate the PR with this ticket.
In case of failing the newly added tests, you can address it in a separate PR.
samchuarm
on 11 Dec 2018
Hi @samchuarm and @SeppoTakalo ,
I have included the newly added test cases and the test is able to pass.
Also, I have submitted the pull request #9084 , please help review.
Btw, in the tcp test case under 'mbed-os\TESTS\netsocket\tcp', there is a missing comma in line 166 in main.cpp file which causes compile error.
AdamZhang0124
on 12 Dec 2018
Hi @AdamZhang0124 , thanks for the new test results update and PR.
Hi @SeppoTakalo , please help review the test results and PR changes. Thanks.
samchuarm
on 12 Dec 2018
Related issues
bcostm
·
4Comments
MarceloSalazar
·
3Comments
sarahmarshy
·
4Comments
0xc0170
·
3Comments
neilt6
·
4Comments