Mbed-os: UBLOX_EVK_ODIN_W2 does not pass Greentea testcase WIFI-CONNECT-PARAMS-CHANNEL and WIFI-CONNECT-PARAMS-CHANNEL-FAIL

[VeijoPesonen]

Description

• Type: Bug
• Priority: Major

---

Bug

Target
UBLOX_EVK_ODIN_W2

Toolchain:
GCC_ARM

Toolchain version:
arm-none-eabi-g++.exe (GNU Tools for ARM Embedded Processors 6-2017-q2-update) 6.3.1 20170620 (release) [ARM/embedded-6-branch revision 249437]
mbed-cli version:
1.2.2

mbed-os sha:
4e222952d

DAPLink version:
0221

Expected behavior
WIFI-CONNECT-PARAMS-CHANNEL succeeds
WIFI-CONNECT-PARAMS-CHANNEL-FAIL succeeds

Actual behavior
WIFI-CONNECT-PARAMS-CHANNEL
Expected NSAPI_ERROR_OK Was NSAPI_ERROR_DEVICE_UNSUPPORTED

[1511534267.07][CONN][INF] found KV pair in stream: {{__testcase_name;WIFI-CONNECT-PARAMS-CHANNEL}}, queued...
[1511534267.12][CONN][RXD] >>> Running case #1: 'WIFI-CONNECT-PARAMS-CHANNEL'...
[1511534267.18][CONN][INF] found KV pair in stream: {{__testcase_start;WIFI-CONNECT-PARAMS-CHANNEL}}, queued...
[1511534268.28][CONN][RXD] :36::FAIL: Expected 0 Was -3002
[1511534268.34][CONN][INF] found KV pair in stream: {{__testcase_finish;WIFI-CONNECT-PARAMS-CHANNEL;0;1}}, queued...
[1511534268.43][CONN][RXD] >>> 'WIFI-CONNECT-PARAMS-CHANNEL': 0 passed, 1 failed with reason 'Assertion Failed'

WIFI-CONNECT-PARAMS-CHANNEL-FAIL
Expected NSAPI_ERROR_CONNECTION_TIMEOUT or NSAPI_ERROR_NO_CONNECTION, no success

[1511533689.77][CONN][INF] found KV pair in stream: {{__testcase_name;WIFI-CONNECT-PARAMS-CHANNEL-FAIL}}, queued...
[1511533689.83][CONN][RXD] >>> Running case #1: 'WIFI-CONNECT-PARAMS-CHANNEL-FAIL'...
[1511533689.89][CONN][INF] found KV pair in stream: {{__testcase_start;WIFI-CONNECT-PARAMS-CHANNEL-FAIL}}, queued...
[1511533691.00][CONN][RXD] :36::FAIL: Expression Evaluated To FALSE
[1511533691.07][CONN][INF] found KV pair in stream: {{__testcase_finish;WIFI-CONNECT-PARAMS-CHANNEL-FAIL;0;1}}, queued...
[1511533691.16][CONN][RXD] >>> 'WIFI-CONNECT-PARAMS-CHANNEL-FAIL': 0 passed, 1 failed with reason 'Assertion Failed'

Steps to reproduce
Compile and run the test cases separately - in other words comment out other testcases from main() and run one at a time. You also need to work around the issue #5546 .

mbed test --compile -c -t GCC_ARM -m UBLOX_EVK_ODIN_W2 --app-config .\TESTS\network\wifi\template_mbed_app.txt -n tests-network-wifi
mbedhtrun -f .\BUILD\tests\UBLOX_EVK_ODIN_W2\GCC_ARM\TESTS\network\wifi\wifi.bin -d X: -p COMY:9600

[VeijoPesonen]

@SeppoTakalo, @kjbracey-arm, @juhaylinen, @jarlamsa , @KariHaapalehto for your information

[0xc0170]

@ARMmbed/team-ublox

[andreaslarssonublox]

Hi, I have looked at the test cases and this is what I found:

1. The test cases do not seem to be complete, e.g. wifi_connect does not contain any disconnect. Therefore the next test case will most likely fail because we're already connected.
2. Old test cases used NSAPI_SECURITY_WPA_WPA2 security mode but current use NSAPI_SECURITY_WPA2 which is not supported by the ODIN driver right now.
3. The wrong error code is returned in wifi_connect_params_channel_fail.cpp by the ODIN driver. This will be fixed in an upcoming release.

[VeijoPesonen]

@andreaslarssonublox , thanks for the information.

The test cases do not seem to be complete, e.g. wifi_connect does not contain any disconnect. Therefore the next test case will most likely fail because we're already connected.

For me it looks like the test cases make an assumption that connect() will implicitly reset everything and then connection setup is started with a clean slate. It is debatable is this a right approach.

Old test cases used NSAPI_SECURITY_WPA_WPA2 security mode but current use NSAPI_SECURITY_WPA2 which is not supported by the ODIN driver right now.

May I take it that NSAPI_SECURITY_WPA2 is to be supported in the (near) future?

[andreaslarssonublox]

The ODIN driver will need to be disconnected before making a new connection so if this is a new requirement we need to make changes in the driver. I guess all NetworkInterfaces should behave the same so it should be the same for Cellular and Ethernet as well. Please indicate if this should be changed for the ODIN driver. I think the documentation needs to be updated as well in that case NetworkInterface.h and WiFiInterface.h.

Some Wi-Fi drivers, including ODIN and esp8266, does not allow to set a specific security requirement from the station side when using WPA/WPA2. They will use whatever the AP has been configured for.
Therefore it might be better to write the test cases to suite most drivers.
We don't see a real benefit in supporting the NSAPI_SECURITY_WPA and NSAPI_SECURITY_WPA2 for now. An application should use NSAPI_SECURITY_WPA_WPA2. Please indicate if you see any use cases where NSAPI_SECURITY_WPA or NSAPI_SECURITY_WPA2 must be used.

[kjbracey-arm]

I personally would not be expecting a connect() to reset any existing connection. We might need it to be do nothing to fit in with some of the event stuff that's happening (returning either "OK" or more likely "already connected").

[SeppoTakalo]

@andreaslarssonublox Is it not a valid case for client to require only WPA2 when connecting?

If the driver supports both WPA and WPA2, I don't see any reason why application is not allowed to force WPA2 when connecting.

What comes to disconnecting, that is a valid point.
In every testcase, the driver is constructed from the scratch. https://github.com/ARMmbed/mbed-os/blob/master/TESTS/network/wifi/get_interface.cpp#L54
I'm assuming that destructor of the driver should take care of disconnecting and all cleanups.

[andreaslarssonublox]

Usually the AP decides the security level but if the client has stricter requirements it might require only WPA2. However the driver itself only supports WPA_WPA2 mode on the client side therefore we have not added support for only WPA2. It might be possible to add WPA2 only mode in the driver but we have no plans right now to do it.

Maybe I have misunderstood the API but this is the way I interpret the security parameters for the Wi-Fi client:

• NSAPI_SECURITY_WPA - connect only using WPA. Connection will be setup if either WPA or WPA_WPA2 is used on AP.
• NSAPI_SECURITY_WPA2 - connect only using WPA2. Connection will be setup if either WPA2 or WPA_WPA2 is used on AP.
• NSAPI_SECURITY_WPA_WPA2 - connect using WPA or WPA2. Connection will be setup if either WPA, WPA2 or WPA_WPA2 is used on AP.

When looking at the esp8266 code there seems to be no distinction between WPA, WPA2 and WPA_WPA2 see https://github.com/ARMmbed/esp8266-driver/blob/master/ESP8266/ESP8266.cpp which then is behaving erroneous if the interpretation above is correct.

The ODIN destructor does not do a disconnect but we can try to fix it in the next release of the driver.

[SeppoTakalo]

@andreaslarssonublox Thanks for the info.

I think your assumption about security levels are correct. I only thought that drivers supporting WPA2 would be able to select the WPA2-only mode. If this is not the case, as you told, I need to rewrite some testcases that I have defined. I think this is understandable limitation that we just need to accept. Drivers will always have limitations so returning NSAPI_ERROR_UNSUPPORTED is the correct behaviour, as Ublox driver does. So no changes required there.

And yes, you are correct. ESP8266 driver does not handle security parameter at all in set_credentials() call. This is a bug.

What comes to destructor, I think I have incorrectly been assuming a proper cleanup. In reality, we don't have possibility of removing interfaces from stacks in Mbed OS. And quite frankly, we probably never will. These are limited embedded devices so plug&play is not a feature we are working towards. So the whole destructor of interface does not make sense. So what we can and should do is to call ::disconnect() at the end of each testcase. All cleanup should be in disconnection phase.

[TuomoHautamaki]

Any estimate when this would be fixed?

[ciarmcom]

ARM Internal Ref: MBOTRIAGE-122

[cmonr]

ARMmbed/team-ublox, any updates?

[juhaylinen]

Seems to be fixed in https://github.com/ARMmbed/mbed-os/pull/7579