Kimai2: Time tracking with RFID

[AliGeenius]

Hi/Hallo/Servus everyone!
First of all thanks for this geenius project ;).
I found Kimai 2 days ago so I'm completly new to this.

My idea is to connect kimai to RFID readers:

(0. Kimai already installed)

1. Buy a cheap ESP8266 microcontroller
2. Flash firmware
3. Connect cheap RFID modul to it
4. Login on the ESP8266 WiFi interface and connect it to the running kimai
5. Link RFID IDs to user and or activities
6. User presence: start tracking / stop tracking of presence (and last activity)

I tried out some API stuff and also read the dev docu but I can't find a clear entry point to start a plugin development.

Actually I have these ideas to do it:

1. Connect directly to database via subserver thats receiving the ID's and managing trackings
2. Write a plugin to manage RFID IDs and try to use the API (would be the best)
3. Use a subserver and use the API
4. Just add a new RFID field to users and use the API (but maybe without good error handling)

Things I can:
Electronics, C, C++, C#, Java, Python, SQL

Things I need:

1. Link RFID IDs to user
2. Requesting with an RFID ID Kimai
3. Kimai does the handling of the tracking
4. Kimai responses with a state: start/stop(maybe with duration)/fail(maybe reason)

I would push the source code and shematics of the ESP8266 stuff on GitHub and when I get into the developing of a plugin that stuff too.

But first I want to ask you for suggestions to this or help how to get into the development and what I'll need to build. Because I really start at 0 in PHP coding :D

Greetings Alex

[kevinpapst]

Hi Alex,

thats something which I have on my roadmap as well. I already bought some hardware for a PI setup, but didn't have the time to work on it yet.
My first issue will be a barcode login, which is simpler to setup, as they work like a keyboard.

RFID is a bit more complicated, as it needs hardware and an extra app that listens for the RFID events and then talks to Kimai via API.
Don't use the database directly, but write a small script in Python that communicates with the API.
There are life-cycle events in place which perform calculations on database updates. They are depending on system configurations, so replicating that interal logic is a "no no" ;-)

Building a plugin is simple ... the easiest solution is to copy an existing plugin. Check the marketplace on the website, the "Custom CSS extension" could be a starting point. But I guess that will become rather complex, if you don't know PHP and Symfony.

The plugin needs to ship:

• a new user management screen for connecting RFID tags with users
• an API controller is needed for starting / stopping records with RFID tokens
• a configuration screen to set eg. the customer/project/activity which will be used for the time records

It's all there in my head ... but how to proceed?

[AliGeenius]

Hello,

Thanks for your suggestions!
I guess the barcode managing could also be used as an extension for the RFIDs or vice versa that would be nice :D

I tried to import the whole Kimai repo in PHPStorm after installing a lot of additional stuff but PHPStorm got some problems with webpack (wrong format and bla bla a lot of people already got this problem).
Without the knowledge of PHP I'm not able to start developing in vim or notepad and I need some help of an IDE to faster achieve an overview.

So I decided to code a quick and dirty Kimai bridge thats connected to an MQTT broker and the Kimai API (not finished yet).
The ESP chips only publish the RFID and the bridge will react on that ID by searching in a little patch file for the user ID.
With the user ID the bridge requests timesheets for starting and stoping the tracking.
For each state the bridge receives from the API it will publish on the topic of the RFID device.
That will reduce WiFi traffic a lot and should be a good clear structure.
The only problem is the patching file but for testing and getting things started it's a KISS solution :D

When I'm more motivated I'll retry to get PHPStorm working or I'll try another IDE.

[hmr-it-jr]

@kevinpapst I have a QR-Code variant 90 % percent ready to use.

[kevinpapst]

Wow, thats super cool! Do you plan to release/sell that?

[rarvin]

This being used a login to office and log off from office plug-in module

would be really good.

Regards,
Aravind

[AliGeenius]

Hello I'm back again!
My mqtt bridge, the hardware part and api requests work great (~1 1/2 months daily use), but the bridge is not that easy to manage and I like to do develop a plugin.

Currently I'm just publishing the ID and my bridge subscribes the channel of the WiFi reader then it generates the requests to the kimai api. The bridge publishes a state like unknown, starting or stopping a record to get feedback on the WiFi reader.

You suggested to build an API Controller. So I have to extend the BaseApiController but how can I build the API controller into a plugin? Or shall I directly integrate it into kimai?
I currently try to understand kimai and its structures but for the API dev you didn't post any docs.

My first steps would be hard coded so that the API controller directly converts a known ID to a username that I have a base for testing with the WiFi reader. Than I try to develop the "GUI" stuff with additional ID's. Something like a error screen also would be nice.

Greetings Alex

[kevinpapst]

Hi @AliGeenius ,
do it step by step:

• build an API controller inside the Kimai/src/API directory
• build a new GUI Controller inside Kimai/src/Controller with the templates inside Kimai/templates/

Once all of this works, I can help you to convert it into a plugin ... or I had the time to create a demo bundle with API support.

[AliGeenius]

Hey @kevinpapst

I'm not able to get a route to my ../api/customID. On the api/doc/ page theres only the model CustomIDEntity available.
Things I did:

• editing nelmio_api_doc.yaml
• creating Entity.CustomID.yml
• creating CustomIDController.php (in the API directory)
• creating CustomID.php CustomIDRepository object

They are based on the user stuff. Just the Repository is loading the data (id,customID,userID) from a xml file. I spent lots of hours to find some route I maybe forgot but its truely not easy for me to handle that project as a php newbie :D

I hope you can help me with this information.

[kevinpapst]

@AliGeenius please have a look at the demo bundle: https://github.com/Keleo/DemoBundle
It includes an example for a new API method. But you actually don't need an API method, you can also simply create a controller action and return json from it.

[AliGeenius]

Jeeeesus,
that will speed up things a lot :D. Thanks!
I'll give it a new try.

[AliGeenius]

Hi @kevinpapst
Yeah me the PHP noob is back..
This week I started a new try and It was really hard to get into the symfony, doctrine and twig stuff if you are a newbie.
And after studying a lot of the code I have to call out a big respect for this kimai project!

But now I'm ready to take off with the main controller.
Actually you get the plugin widget in the system setting section can search for entrys, add new delete edit etc..
Data is stored in the database under remote_plugin:
A remote entry has its unique id, a remote id, a relation to a user and a last login datetime.

So now I'm on the api controller and I've a question.
Is there a simple way to communicate from my api controller to other api controllers?
For starting/stopping/toggle timesheets I need a request/post.

Greetings Alex

[kevinpapst]

Why would you do that if you can use the PHP classes (services in symfony) directly in your controller?

[AliGeenius]

Thats exactly what I'm trying to do maybe my question wasn't that precise.
In my first case I simply want the api controller to start/stop recent timesheets. I get the user from my remote repo and now I'm looking for a good solution to proof for a existing timesheet for that relating user, starting/stoping them or search one by other criteria. I don't know all the mechanics of Symfony.
Later I'll maybe build in a config option for handling the behavior of starting timesheets like generating a default one, restarting the latest of the user, duplicating the latest from another user or just throwing an exception/error.

In the Symfoni docs they would use the services simple like
public function stop(string $rem, TimeSheetController $controller) { // ... }
Or by adding them to the controller with the constructor. That auto routing is really dark magic for me.
A completly different way I found was forwarding.
$response = $this->forward('App\Controller\TimeSheetController::find...'

What method would you suggest to use that classes/services?
And what would be a good way to get the timesheets for a user? Using the repo or controller or is there a better solution?
Starting stopping seems for me to be made with the TimeSheetService.

[AliGeenius]

In the Symfoni docs they would use the services simple like
public function stop(string $rem, TimeSheetController $controller) { // ... }
Or by adding them to the controller with the constructor. That auto routing is really dark magic for me.

Okay both solutions work but it's still scary for me :D

[kevinpapst]

Always use the parameter auto-binding / dependency injection container.
Don't use internal request forwarding - that is the real dark magic 😁

It might feel weird in the beginning to only declare dependencies without caring how it got there,
but its super helpful and you won't work without in the future.

[AliGeenius]

I did it!, my first and I guess last php/symfony exercise is a kimai plugin 😄

0 % code coverage but that should be enough for it^^

Just have to do the pagination for the view and my search doesn't search for user names only for the remote ids.

I only use one permission for everything is that ok for a plugin?

[kevinpapst]

Great 👍 But I don't see an active repo in your profile?!?

You decide how your plugin works.
I can have a look for possible improvements once you shared the code.

[AliGeenius]

I invited you because I'm thinking about to do a Donationware or selling it for a few bugs that my coffe got payed :D

[AliGeenius]

So the hardware part is also finished only missing the HTTPS client..
I hacked a Sonoff Touch switch for it because it has a good case and a build in power supply. Building it with modules would cost ~10€ and I guess you dont need a soldering iron.

The RFID module also fits in it very well like it was made for it.

Soldering is easy just 5 cables. But the crystal of the reader must be reattached because of its height.

I added a captive portal for doing the settings on the really first start.

So when everything works the controller parses the user id and the duration of a timesheet. See the second try there I registred the ID on a user. That will be maybe used for a small info OLED display.

And at least the website entry for a not registred card

The RFID reader has a interrupt out maybe I'll do a battery RFID reader version where the chip uses deep sleep to spent some energy so you could attach it everywhere.

Any suggestions?

[kevinpapst]

Wow, super cool project!
Do you have any plans to write that down? I'd really appreciate if we could share that as blog entry.

[AliGeenius]

Thanke you 😄
Yes I'll make a detailed guide with pictures this weekend.
But for the plug and play solution I'm missing another RFID module so it's gonna be a temporarly not prooved version.

[kevinpapst]

After seeing this I searched for a bit through the internet and found out, that there are RFID readers available which act as keyboard (same like most usb barcode readers).
I didn't knew that these kinds exist.
So I ordered one (12 €) to play with it, lets see what we can achieve with it...

But this self built solution is so much cooler 😁 and its much better, as it triggers the API and not some input form in a website.
I'd really like to know the code behind that magic.
Is that a pre-built raspy image for raid usage, that allows this wifi selection with the input of the URL parameters?

[AliGeenius]

I'm not using a Raspberry that would be a bit of overkill for this project 😆
The controller in the Sonoff switch is a ESP8265 @ 80 MHz and 1 MB flash.
Theres a similar chips like the ESP8266 out or the faster ESP32 with 240MHz (dual core).
But you could also connect the reader that I'm using with a raspberry and trigger the API with a small script.
Actually the code is about ~200 lines with debug.
But I'm not that happy with the perfomance of a request it's pending between 1-3 seconds.

I'll try to speed things up:

1. by building a light version of the toggle API route that just responses with the timesheet duration to reduce the transfering and parsing time
2. build a new optimized solution without the Arduino framework (wouldn't be that userfriendly for others)
3. run the controller @ 160 MHz

But the guide is growing so stay tuned 😄

[kevinpapst]

Well the raspberry could be used as server for Kimai, so it would be a self-contained solution, which would probably solve the performance issue as well.

But I don't know much about hardware or that kind of machine-near programming, so I a guess I am not a big help here^^

[AliGeenius]

That's true! A preconfigured PI image would also be a handy solution 😄

I found the probleme..
First I thought it would be a probleme because I'm using HTTP1.0 (need it for streaming the data).
Without streaming and HTTP1.1 and a persistent keep-alive connection -> no performance increase.
Then I made the small toggle API solution with user and duration -> no performance increase.
I also set doubled the controller frequency to 160 MHz -> no performance increase.

The solution was to leave my test XAMPP environment and install Kimai on my local Debian server with Nginx.
A response takes now 290-370 ms (without keep-alive) and the full parsing only 1-2 additional ms and I think that should be fast enough for a terminal.
So I really killed my time with the wrong debugging but dont know why my XAMPP environment is that slow 😄

Requesting: 69e4ae8d 341 ms User id: Anna Smith Duration: 0 min 342 ms

[kevinpapst]

Is your local environment running in APP_ENV=dev? The debug toolbar and env environment in general is collecting many logs/ profiling data ... so each requests costs much more than in APP_ENV=prod

[AliGeenius]

After finishing the API plugin I went to the production environment for a realistic debugging.

[AliGeenius]

Sorry that the docu isn't finished yet. I'm playing a bit with a display 😄

[kevinpapst]

Wow, that is so cool! Now 3d-print a all-in-one box and you can easily sell that!

[AliGeenius]

So now it's finally done.
I made it modular so it's useable with or without display.
Additionally you can select some debug options.
Little preview
Just have to finish the docu :'D

[kevinpapst]

Freaking awesome 🥇 Really, really like it 👍

[AliGeenius]

Thank you 😄
I pushed the code and it's docu now.
https://github.com/AliGeenius/Kimai-RFID-ESP
Just have to decide what to do with the API plugin^^

[Batman313v]

@AliGeenius Have you published the API Plugin anywhere? I also had a question about the RFID. Is it only for a single user or can I have 3 cards registered to 3 different people? I am very new to Kimai, I just installed it yesterday 😅 and our old time clock only had an rfid option so everyone is already used to doing that and has tags/ cards. I was also wondering if I could have multiple readers at different locations. I love that you've done this and I love the Sonoff hack 😂 I am a 3D modeler and plan to create and print a model for a NodeMCU and the OLED display to fit inside that can be mounted on the wall if this will work for our setup. If you're interested I can send you the model once its done for the GitHub page

[AliGeenius]

@Batman313v Sorry there wasn't a notification for you post.
I didn't publish it yet, but you can have it for a little coffee donation :D.

The RFIDs are linked to the Kimai users, so you can use different RFIDs for different (single) users. But you could also link multiple RFIDs to the same user. Only the RFID must be unique.

The ESP is just sending a HTTP post as a client over the API, so you are not limited by just one device.
You also could connect to a server out of your LAN.
Only thing to keep in mind is that I didn't implement the HTTPS client yet.

A 3D model would be great! Thank you dude :D
I was also thinking about the battery version with interrupts but there havn't been any requests for it so we stay wired^^

[kangelos]

How do you take your coffee ?

[AliGeenius]

@kangelos Espresso black without sugar, caffee americano or lungo with some milk :smile:

buymeacoff.ee/aligeenius