Stf: Introducing a powerful system for device booking & partitioning

[denis99999]

Hi,

The related pull request #1056 named "Group feature" is made of:

• the Group concept on which rely a booking system and a partitioning system, involving new tabs in the GUI both for setting and monitoring actions on groups, devices and users
• an administrator level in addition of the native user one, with increased rights on users, devices and groups
• a Contact Support button allowing for instance to contact the administrator by email from the login page in order to get an account (e.g. LDAP)
• a Logout button allowing to close properly the web session with the STF server
• 50 new APIs for management of devices, users and groups (e.g. bookings)

This Group feature has been added into STF taking care to maintain the backward compatibility with third party client software using STF APIs.

Moreover, although several screens and tabs have been added to the native STF UI to serve the considered new feature, the backward compatibility is also maintained at UI level so that users can continue to use STF as before if they have no need to use the booking & partitioning systems.

Nevertheless, due to the adding of new built-in objects into the database impacting existing tables, it is necessary to rebuild the database by erasing it first (i.e. only access tokens may be easily saved).

Here is a full documentation about this new feature including also some implementation details: GroupFeature-1.0-STF-OrangeSA.pdf

Don't hesitate to test it from here: https://github.com/denis99999/stf/tree/group-feature

[issue-label-bot[bot]]

Issue-Label Bot is automatically applying the label feature_request to this issue, with a confidence of 0.89. Please mark this comment with :thumbsup: or :thumbsdown: to give our bot feedback!

Links: app homepage, dashboard and code for this bot.

[sorccu]

I’ve only been able to briefly scan through the PDF - which is great, by the way - so far, but this seems like an excellent addition. Unfortunately I myself have very limited time to test it out, so it would be very helpful if anyone interested in device booking could provide their feedback. I know there’s a bunch of you out there :)

One thing I picked up was that the PDF mentioned a possible need to nuke RethinkDB once. Is that a hard requirement?

[denis99999]

Yes, the Rethinkdb database has to be rebuilt from scratch otherwise it will does not work! But it is easy to save in a secure way the existing DB before to make the test (ex: # mv /opt/rethinkdb /opt/rethinkdb.org)

[thinkhy]

@denis99999 I've went through the design doc, which shows a comprehensive solution for user/group/device management.
I will try to test the feature and feedback.

[denis99999]

Hi @thinkhy,
Thank you very much for your contribution, don't hesitate to contact me for any question!

[pawbednarz]

Hello @denis99999, can you please give me some advices how to install your feature?

[denis99999]

Hi @pawbednarz,

You have simply to clone https://github.com/denis99999/stf.git, then switching to the group-feature branch and follow the STF installation instructions.

Once it is done, you have to read the PDF documentation joined in this PR to well understand how device reservation works!

And finally enjoy :-)

Don't hesitate to contact for any other question!

[pawbednarz]

Thanks @denis99999, i just forgot to switch the branch. Very good job with this improvement! Greetings!

[skumar1122]

Great work @denis99999

Do you know if there is any ability of using stf with admin rights. For example,
Scenario-

• I have connected multiple devices.
• I am running Automated Tests on Few devices.
• I want to block others to use devices in which my automated tests are running.

Thanks
Shiva

[denis99999]

Hi @skumar1122 ,

Yes of course, I developed the booking/partitioning system for this kind of purpose, using it you have just to reserve for you the considered few devices for the time you need!
For detailed information, take a look to the PDF documentation I joined to this PR.
Regards.
Denis.

[skumar1122]

Hey @denis99999
Thanks for the reply. I am checking above document and I couldn't see STF start instructions with your feature branch. Could you point me please ?

It runs(stf local) same like before and I am seeing new changes like you mentioned in your document.

Thanks
Shiv

[denis99999]

See sections 9 & 10, setting correctly NTP server and rethink database, all is explained in the PDF with a complete example, you have just to read !
Regards.
Denis.

[skumar1122]

Sorry,I did see this section and I logged in with admin details and I am still seeing old GUI.
Do I need to start STF in different way and I am using Mac OS.

9 System configuration
Obviously, it is required to get time synchronization with accurate time servers (e.g. using NTP on Linux) on the STF machines, in order the booking & partitioning systems work fine.

10 Database migration
The Group feature introduced into STF require some built-in objects in the database to work fine:

• the administrator user which has the following default credentials: o name:
  o email:
• the root standard group to which the users and devices belong, which has a default name equal to
  Common
  Moreover, some tables like ‘users’ and ‘devices’ ones which reference the previous build-in objects have been modified.
  So, to enjoy these new feature, it is really required to rebuild the database by erasing it first (i.e. only access tokens may be easily saved)
  Note you can of course override the default values of previous built-in objects by setting the following environment variables (e.g. in /etc/environment file on Linux) while rebuilding the database by launching the stf-migrate.service service file (cf. DEPLOYMENT.md):
• STF_ROOT_GROUP_NAME => root standard group name
• STF_ADMIN_NAME => administrator user name
• STF_ADMIN_EMAIL => administrator user email

[denis99999]

It seems you did not switch to the group-feature branch, isn't it ? remind also you have to rebuild the database (i.e. remove then create)

[jupe]

Hi,

I've played a while with your fork and I'm impressed :) I still faced some issues:

First: It looks a bit like DB schema is not compatible and causing following kind of errors:

Exception

2019-12-10T05:36:42.351Z ERR/processor 21538 [proc002] Unable to save the initial state of Device "H8WDU15B02000760" TypeError: Cannot read property 'email' of undefined
    at /home/tester/stf/lib/units/processor/index.js:106:40
    at runCallback (timers.js:794:20)
    at tryOnImmediate (timers.js:752:5)
    at processImmediate [as _immediateCallback] (timers.js:729:5)
From previous event:
    at Router.<anonymous> (/home/tester/stf/lib/units/processor/index.js:86:10)
    at Router.emit (/home/tester/stf/node_modules/eventemitter3/index.js:118:35)
    at Router.<anonymous> (/home/tester/stf/lib/wire/router.js:36:12)
    at emitTwo (events.js:126:13)
    at exports.Socket.emit (events.js:214:7)
    at exports.Socket.Socket._flushRead (/home/tester/stf/node_modules/zmq/lib/index.js:638:15)
    at exports.Socket.Socket._flushReads (/home/tester/stf/node_modules/zmq/lib/index.js:676:23)
    at Object._zmq.onReadReady (/home/tester/stf/node_modules/zmq/lib/index.js:297:10)

I'm not sure if following errors related to same:

Exception

2019-12-10T05:34:33.956Z INF/device:plugins:mute 21406 [FA542YJ13460] Will not mute master volume
ReqlDriverCompileError: Cannot wrap undefined with r.expr().
    at ReqlDriverCompileError.ReqlError [as constructor] (/home/tester/stf/node_modules/rethinkdb/errors.js:23:13)
    at ReqlDriverCompileError.ReqlCompileError [as constructor] (/home/tester/stf/node_modules/rethinkdb/errors.js:35:51)
    at new ReqlDriverCompileError (/home/tester/stf/node_modules/rethinkdb/errors.js:46:57)
    at Function.<anonymous> (/home/tester/stf/node_modules/rethinkdb/ast.js:4031:11)
    at Function.expr (/home/tester/stf/node_modules/rethinkdb/util.js:43:16)
    at /home/tester/stf/lib/db/api.js:1218:14
    at new Func (/home/tester/stf/node_modules/rethinkdb/ast.js:3450:17)
    at Function.<anonymous> (/home/tester/stf/node_modules/rethinkdb/ast.js:4041:12)
    at Function.expr (/home/tester/stf/node_modules/rethinkdb/util.js:43:16)
    at funcWrap (/home/tester/stf/node_modules/rethinkdb/ast.js:34:19)
    at Function.<anonymous> (/home/tester/stf/node_modules/rethinkdb/ast.js:597:35)
    at Function.filter (/home/tester/stf/node_modules/rethinkdb/util.js:64:16)
    at dbapi.loadDevices (/home/tester/stf/lib/db/api.js:1217:36)
    at getGenericDevices (/home/tester/stf/lib/units/api/controllers/devices.js:31:3)
    at getDevices (/home/tester/stf/lib/units/api/controllers/devices.js:213:7)
    at swagger_router (/home/tester/stf/node_modules/swagger-node-runner/fittings/swagger_router.js:104:13)

2019-12-10T05:35:37.787Z FTL/device 21495 [8BQX1E4Y5] Setup had an error Error: Failed more than 3 times in 10000ms
    at TouchConsumer._failLimitExceeded (/home/tester/stf/lib/units/device/plugins/touch/index.js:230:26)
    at FailCounter.emit (/home/tester/stf/node_modules/eventemitter3/index.js:117:35)
    at FailCounter.inc (/home/tester/stf/lib/util/failcounter.js:29:10)
    at TouchConsumer._outputEnded (/home/tester/stf/lib/units/device/plugins/touch/index.js:223:24)
    at RiskyStream.emit (/home/tester/stf/node_modules/eventemitter3/index.js:115:35)
    at RiskyStream.<anonymous> (/home/tester/stf/lib/util/riskystream.js:14:12)
    at emitNone (events.js:111:20)
    at Socket.emit (events.js:208:7)
    at endReadableNT (_stream_readable.js:1064:12)
    at _combinedTickCallback (internal/process/next_tick.js:138:11)
From previous event:
    at startConsumer (/home/tester/stf/lib/units/device/plugins/touch/index.js:511:14)
    at /home/tester/stf/lib/units/device/plugins/touch/index.js:526:12
    at SerialSyrup.ParallelSyrup.invoke (/home/tester/stf/node_modules/stf-syrup/lib/parallel.js:54:24)
    at /home/tester/stf/node_modules/stf-syrup/lib/serial.js:43:33
    at _combinedTickCallback (internal/process/next_tick.js:131:7)
    at process._tickCallback (internal/process/next_tick.js:180:9)
2019-12-10T05:35:37.788Z FTL/util:lifecycle 21495 [8BQX1E4Y5] Shutti

Unhandled rejection TypeError: Cannot read property 'repetitions' of undefined
    at /home/tester/stf/lib/units/api/controllers/groups.js:126:43
    at bound (domain.js:301:14)
    at runBound (domain.js:314:12)
    at runCallback (timers.js:794:20)
    at tryOnImmediate (timers.js:752:5)
    at processImmediate [as _immediateCallback] (timers.js:729:5)

But propose to check those out.

NOTE: After I swipe whole DB and restart stf again all above exceptions go away and server started smoothly. Also my android 10 device appears, great job! I quickly played with it - installed apk and started it - so far everything works as expected from android 10 point of view !

One minor UX issues I faced:

renaming group could go a bit smoother - approval should close edit mode, see screen capture below:

My environment:

• x86-64 ubuntu19 server
• chrome -browser
• running stf in local mode

[denis99999]

Hi @jupe ,

Yes, as explained in the PDF documentation about Group feature I joined, STF database has to be rebuilt, see also the discussion #1057 about that.

Concerning the UI, thank you for your proposal but the logic is to click on the "blue" buttons to open/close the corresponding windows, while a validation button as for "Name" or "Schedule" has the semantic to commit the current edition, but not to stop the edition, and is active only in case of change, so I feel it would be not consistent to use such button to shortcut the closing of the window.