Hi,
Some of you might have seen my port for the Infinity Ergodox for the TMK firmware here. While the port works, and I don't have any major issues at the moment, I'm not very happy working with the TMK firmware. Therefore I have been thinking of the option to move to the QMK firmware instead.
I have already described some of the major problems I have with the TMK development here. But perhaps the biggest one is that I can't seem to get my pull requests accepted, at least not within a reasonable time. I have had six pull request open for almost one month now, and two more for the tmk_keyboard repository. And none of them has even got a single comment by @tmk.
Most of those are small ones, like warning fixes, which should have no reason to not being accepted. But the big thing is that the tmk_core doesn't support the customization point that I need. So I first suggested a very lightweight approach, which he didn't like, so I did some bigger refacoring.
I'm not the only one with problems @eltang for example, has a perfectly valid pull request Update functions used to write to EEPROM, for which I see no reason for having to wait.
I also don't like that keyboards not made by @tmk himself are classified as second-class citizen. It started even before the tmk_core split, by no longer accepting pull requests for new keyboards. But after the tmk_core split it has become even worse, since all new features goes into the tmk_keyboard repository first, and then perhaps after several months, tmk_core gets them. And only then the other keyboards can use it.
I understand @tmk's desire to control everything, and keep everything up to his standards. And he has done a really good job with getting the firmware to the state it is. But for me as a contributor, it's an almost impossible situation, as I see so much potential, but I'm unable to do anything about it.
The QMK firmware on the other hand is moving forward at a much faster pace, and it has has some very attractive features for me, compared to the TMK firmware. It also seems to be much more community driven.
I also like that it in addition to bringing all the supported together in one repository, also brings in the user keymaps, to keep the whole community together. Additionally I don't see why the Ergodox EZ and Infinity Ergodox keymaps could not be shared, even if they have different underlying hardware.
The biggest challenge moving the Infinity Ergodox support to the QMK firmware is that there's no support for ChibiOS at the moment. So support for that has to be ported.
But before doing that, I would like to refactor the existing code a bit. At the moment the tmk core contains quite a bit of duplicated code for different controllers, and in many places the required hardware abstraction layer is missing. So I would like to refactor that first. For example by making the drivers more generic, like this pull request. This would of course be a huge change, which would require careful testing, for which I would need help from the community, since I don't own any of the supported keyboards myself.
The above refactoring would make it much easier to add support for new hardware in the future. I would actually like to add support for PC at some point quite shortly afterwards. This would allow for adding unit tests, and also making an emulator on the PC side.
After that ChibiOS support could be added, together with adding support for the Infinity Ergodox, so it can be tested.
The Infinity Ergodox comes with two additional libraries.
So I would like that these two libraries becomes part of the QMK too, rather as something external. Of course keyboards don't need to use them, if they are not needed. But especially the visualization one could be useful for a lot of keyboards.
Furthermore I have added an visualization emulator for the Infinity Ergodox that can be run from Windows. Currently it's only emulating the visualization stuff, by rendering the LCD and LEDs, on a simple 2D keyboard. The long term goal for this, is to have this emulator communicate with the real keyboard in a way that let's you change keymaps on the emulator side, without having to flash the hardware. This would be mostly useful for debugging, and users building their keymaps.That's why I would like to add PC support for the tmk_core.
In the ideal world I would like to see that QMK and TMK are joined together. So instead of adding ChibiOS support for QMK, effort would be made to make the two into one. But this seems like mission impossible. I hope not though, and I highly welcome @tmk to comment, in addition to the maintainers of this repository.
So before starting to work on this, I'm asking what do you think? Is this a good idea? Would you accept something like this? Are there any parts that you feel shouldn't be done? Do you want some more detailed refactoring plans, before sending the first pull requests?
Yeah! I'd be happy to merge in ChibiOS support. I don't think TMK and QMK coming to together really makes sense though - our repos have pretty different goals. They share a lot of code obviously, but our recent diverge from forkship just shows how differently they're growing. Any sort of combined repo would just be frustrating for both @tmk and the collaborators here (or maybe just me).
I like the refactoring changes in tmk_core#19 and would merge that, but I'd like to reserve the right to make some other big refactoring changes myself in the future :) exactly what that entails, I'm not sure at this time.
On the Ergodoxes being able to share keymaps, I like the idea of that, but I'm not sure how that would be done without some symlinking - I (and I'm sure @ezuk) would like to keep the keymaps contained in the ergodox_ez folder. Allowing the user to only have to venture into the keyboard project folder makes a lot of sense.e
PC emulation sounds awesome - testing would be a dream :) The serial and visualiser sound fun to mess with as well!
Thanks,
I will start to work on it immediately, but without promising any time table, as I'm probably not very likely to get much done before my summer holidays start in a couple of weeks.
I'll try to make as independent changes as possible, with several smaller pull requests rather than one big bang of everything. I will also try, whenever possible, to keep the original commits from TMK, probably by cherry picking, in order to give credit to the original authors, mostly @flabbergast and @tmk.
As you requested, I will keep the refactoring quite minimal, and only restrict it to things that makes it easier to add ChibiOS support.
One option for the keymaps would be to change the folder structure slightly to have
ergodox
keymaps
ez
infinity
But that would of course be a bit of a hassle for the users, and maybe even cause merge conflicts, since the location changes. But maybe not too much, since the main change, that affects the keymaps would just be a rename from ergodox_ez to ergodox.
The keymaps would also not always be automatically compatible, since some of them includes some ergodox_ez specific headers. But that's mostly for the custom led setting calls in the matrix scan. So that could be moved out to it's own file, or kept in the same file with ifdefs. The Infinity version would probably use it's own visualizer though.
Another alternative at the cost of a slightly harder to navigate structure, is this.
ergodox_common
keymaps (ported keymaps that works with both keyboards)
ergodox_ez
keymaps (ez specific)
ergodox_infinity
keymaps (infinity specific)
The ported and moved keymaps could have a readme file in the old location, which tells that they can now be found in the ergodox_common folder.
Finally there's this options
ergodox_ez -- stays mostly the same as before
keymaps
ergodox_infinity
keymaps
The Infinity keymaps, would just use normal c includes, or modify the make files to include the relevant parts of the ergodox_ez keymaps.
I can think of a few more option, or variations of these. But since we have plenty of time to think about that, I won't go through them now.
@fredizzimo I'm not sure if you're aware, but those folders don't contain just keymaps. They also contain Makefiles and source files which might not be compatible with both keyboards.
@eltang
Yes, I'm aware of that, and as I mentioned above the keymaps won't be automatically compatible. But the differences are rather minor, and could most likely be resolved with small refactoring. For example using #ifdef or using two separate files for those things.
It's however clear that the keymaps won't be compatible by default though, some sort of porting would be needed.
Awesome! I'd want to run this by @ezuk of course, but I like the this structure:
ergodox
keymaps
ez
infinity
I'd be open to allowing a command line make argument to specify which ergodox variety gets built - this could be useful for other keyboards as well, to compile different variations (revisions) of the same board (maybe this would be useful to @skullydazed as well?).
The easiest would definitely be this though:
ergodox_ez -- stays mostly the same as before
keymaps
ergodox_infinity
keymaps
Maybe we can go that route for now, and work on combining them in the future. I imagine it'll be pretty easy to just copy keymaps from one folder to the other if folks are interested in porting them.
@jackhumbert,
How do you prefer to include the ChibiOS library itself?
The mbed library is currently located in the tmk_core\tool\mbed\mbed-sdk directory. I suppose as a either a direct copy or a git subtree copy.
So the logical place for the ChibiOS would be along that, as a git subtree. Or should we pull out the external dependencies to another location that makes more sense(for example external in the root of the repository)?
I also noticed that the mbed library is currently unused, since the keyboards which were using it are now all deleted, so maybe I should start of by deleting all mbed functionality completely?
Either that, or we have to add back the onekey test keyboard for it. If that path is chosen, I would start by moving the mbed sdk to the chosen location. I was also planning to get rid of the mbed specific common.mk file, as it's almost the same as the common.mk file in the tmk_core directory, and otherwise ChibiOS would have yet another variant.
There's also a question of adding ChibiOS as a git subtree or submodule. A subtree would be easier to work with for users of the firmware, since they don't have to do any git submodule related commands. However a sub module would keep the repository smaller. I definitely don't want to do as the TMK does by default, having the users copy the library into the right location manually.
I'd be in favor of a /lib folder for all of that - the lufa stuff could be put there as well. Doing a subtree sounds good - I'm not too worried about repo size.
I'm actually not too familiar with mbed, and until now, QMK has been exclusively AVR (which is why ARM projects were pruned). Is the mbed stuff necessary for ARM support? My understanding of how ChibiOS and mbed work into the development chain is pretty limited.
Ok, I will use the lib folder.
I'm not familiar with mbed either. But it seems like it was very experimental in the TMK firmware too, with only the onekey and the Infinity 60% supported.
The Infinity Ergodox doesn't need it, and I'm pretty sure that the Infinity 60% could be implemented by ChibiOS instead if mbed, if someone wanted to add support for it.
So based on that, I would say it's safe to remove it.
@jackhumbert @fredizzimo Do you think it would be a good idea to update LUFA in the process? QMK's copy hasn't been updated in quite a while.
At least it would make sense to move that to the lib folder as well.
There are also at least the following other libraries used
Hi folks, let me know if you need any help with ChibiOS-related things (but I think Fred knows things better than me by now). Two comments about the above things: Infinity 60% is already done also with Chibios.
Not that I have any say in this, but I'd personally prefer ChibiOS as a submodule - the main repo is 130MB (out of which 47MB is .git), and given that majority of people won't actually use it... {of course you can prune it quite a lot, but then it's going to be more difficult to pull the upstream changes}.
Just a bit more explanation for the people who don't have much experience with ARMs: that world is much more diverse than AVR - while the ARM core is guaranteed to be the same, all the 'peripheral' things (e.g GPIO, power, USB, ...) differ (a lot!) from manufacturer to manufacturer, and even within a manufacturer (as they develop new MCUs). So instead of writing to MCU registers directly, programmers for ARM very often use a software layer (called HAL) between their code and the MCU hardware. Now, there is a choice: manufacturers of course provide some HAL for their MCUs, but they are absolutely incompatible between different manufacturers. So, if someone wants to at least somewhat support different manufacturers, there are basically 2 choices these days: ChibiOS and mbed.
About mbed: this is purely a HAL, supports a lot of MCUs, but uses C++ and it's geared towards the usage in their online IDE. Not really my cup of tea, but it's probably the way to go if you ever want to use LPC chips.
About chibios: it really has 2 different (and more-less usable independently) parts: a HAL (which is the more important aspect for TMK/QMK) and a real-time operating system (RTOS). The RTOS kernel doesn't have to be actually used for TMK/QMK, but it offers some nice features IMO (so TMK uses it the way it's written now).
Finally: it's of course possible to do away with accessing the registers directly, and there are 2 notable successful efforts {both only for one particular family of ARM chips, namely Freescale/NXP Kinetis - this covers Teensy 3.x, LC and Input Club's keyboards}, namely PJRC's arduino core for Teensies and Jacob's (a.k.a. kiibohd) keyboard code.
When I originally setup support for Clueboard, I debated having a #define vs separate directories. I settled on two directories mainly because it seemed more foolproof when my fork was separate from the main qmk branch. Reevaluating that decision now, I have a couple thoughts.
I think it makes a lot of sense that keymaps for boards with the same layout but different underlying architectures can be shared. There's little reason that a keymap for clueboard1 couldn't be used on clueboard2. I think other keyboards will have the same need to share keymaps between different boards, EG if/when the planck is released with a different mcu (like the at90usb1286.)
I agree with @jackhumbert that this is the structure to use:
<layout_name> # EG, ergodox
keymaps
<architecture_1>
<architecture_2>
...
<architecture_N>
This is a lot to process on a Sunday, but I wanted to chime in and ask that we used something like this:
ergodox_ez -- stays mostly the same as before
keymaps
ergodox_infinity
keymaps
I.e, it needs to be immediately clear wheter it's the EZ or the Infinity someone is using. This is crucial for support and troubleshooting purposes. These are two distinct products (commercially) and the firmware should clearly reflect this. Otherwise confusion ensues :)
@flabbergast Thanks for the info regarding ARM! That makes a lot of sense now. Yeah, the repo's pretty big, but I think it's shrunk a little with the pruning (maybe?) at 37.5MB compressed (without .git). The mbed stuff is still in there as well.
I'm comfortable using this format for boards other than the EZ:
<layout_name> # EG, ergodox
keymaps
<architecture_1>
<architecture_2>
...
<architecture_N>
When we get the actual functioning structure down for other projects, @ezuk can check it out and see if it's something he wants to switch the EZ to :)
Sounds good to me. :)
@fredizzimo I'd be happy to test the QMK port on my Infinity(ies) if you need an extra set of hands/eyes. I had ordered an EZ and the Infinity around the same time, the EZ showed up first so I did a lot of work on getting the Vagrantfile for easy compiling on Windows/Linux/OSX setup, but now I've traded that off and am using an Infinity, and I have the new USB-C model on order as well. The biggest thing I've missed has been the easy of tweaking the layouts in QMK vs Kiibohd/KLL. I could probably learn the KLL syntax, but they don't have some of the neat features that have been added to the QMK firmware recently (plus the awesome algernon layouts that I saw just as I was defaulting my EZ for my friend).
@dragon788,
I will mostly need help with testing the LED support, since I didn't install LED's in my board. I think I have the basis working, in my TMK fork as discussed here. But I haven't had any test reports for my latest fixes yet.
But of course I'm happy to see additional testing and bug reports.
Anyway, I will start the real porting now, as I have started my summer vacation, although at least the first week will be busy with other things as well.
I will start of with an almost direct port of what was initially done for ChibiOS support in TMK, and then proceed to the needed refactorings and other stuff. That leaves us with more time to discuss the direction we are going, and gives @jackhumbert, @ezuk and everyone else possibilities for their inputs.
ChibiOS will be added as a submodule, at least initially, as that way it won't grow the repository size, and we can always move to subtrees later.
I like that idea! Thank you, @fredizzimo :)
@fredizzimo I would love to help out here. I have an Infinity Ergodox Infinity from MassDrop/Input Club with LEDs installed. I'm happy to make and test things you are working on. In the process I hope to learn how to update/edit the firmware myself, and produce custom LED visualizations.
I also have an Ergodox Infinity from Massdrop, and would like to test. Can I brick the keyboard if I mess up, or is it always recoverable by flashing the original kiibohd firmware back?
Hello,
I'm making a keyboard with a teensy 3.2. I was able to compile using the Makefile file for lc but I couldn't do it with the Makefile.3.2
~/programming/qmk_firmware/keyboards/chibios_test/teensy_lc_onekey> make -f Makefile.3.2
Makefile.3.2:76: ../../tmk_core/tool/chibios/common.mk: No such file or directory
Makefile.3.2:77: ../../tmk_core/tool/chibios/chibios.mk: No such file or directory
make: *** No rule to make target '../../tmk_core/tool/chibios/chibios.mk'. Stop.
@WishCow There is typically a "boot magic" option that places the firmware into a writeable state, but sometimes that requires shorting pins which can be difficult on an assembled keyboard. The Ergodox EZ has the typical "Teensy reset switch" which provides this behavior. Your best bet would be to check with HaaTa from InputClub who helped with designing the Infinity Ergodox to see what the recovery options are on that board.
@leizzer,
Yes those makefiles comes straight from the TMK and haven't been modifyed so they should probably be deleted.
But don't worry, for Teensy 3.2 you just need to modify the rules.mk file. If you open the file you should be able to see some comments describing what to change.
Also note that the chibios_test "keyboard" has the structure for keyboards with subprojects, which is needlessly complex to serve as an example. Therefore I suggest that you start off by copying the infinity60 keyboard instead.
In fact I think I will send a pull request that deletes the whole chibios_test, as it no longer serves the original purpose. Instead I should probably create a template to be used for chibios projects, or what do you think @jackhumbert.
@WishCow It is very, very unlikely that you will brick the keyboard. The bootloader is a separate program, and it can't be overwritten by the normal dfu-util tool. So the physical reset button at the bottom of the PCB should always take you there, and you are able to flash another working firmware, including the original kiibohd firmware.
The biggest chance of bricking it would probably be to flash it more than 10000 times, after which the flash memory might start to fail. It's also possible to destroy the eeprom, by having some kind of weird bug that continuously writes there. But those two are very unlikely to happen during normal use.
@fredizzimo have you confirmed that this works for Teensy 3.2's? I just tried moving the infinity60 rules.mk file to a teensy3.2 layout setup. Upon flashing the teensy with it I get usb device not recognized errors.
@profet23, unfortunately I haven't tried it myself, since I don't have a a Teensy 3.2 myself. However I think that @flabbergast, who originally made it for TMK has been testing it (unfortunately the fact that he originally implemented it is not included in the history of this repository, since the commit was squased, so it shows that I did everything).
But in addition to just copying the rules.mk file, you also need to edit it, like I tried to explain in my previous message. Specifically these need to be changed
## chip/board settings
# - the next two should match the directories in
# <chibios>/os/hal/ports/$(MCU_FAMILY)/$(MCU_SERIES)
# - For Teensies, FAMILY = KINETIS and SERIES is either
# KL2x (LC) or K20x (3.0,3.1,3.2).
# - For Infinity KB, SERIES = K20x
MCU_FAMILY = KINETIS
MCU_SERIES = K20x
# Linker script to use
# - it should exist either in <chibios>/os/common/ports/ARMCMx/compilers/GCC/ld/
# or <this_dir>/ld/
# - NOTE: a custom ld script is needed for EEPROM on Teensy LC
# - LDSCRIPT =
# - MKL26Z64 for Teensy LC
# - MK20DX128 for Teensy 3.0
# - MK20DX256 for Teensy 3.1 and 3.2
# - MK20DX128BLDR4 for Infinity with Kiibohd bootloader
MCU_LDSCRIPT = MK20DX128BLDR4
# Startup code to use
# - it should exist in <chibios>/os/common/ports/ARMCMx/compilers/GCC/mk/
# - STARTUP =
# - kl2x for Teensy LC
# - k20x5 for Teensy 3.0 and Infinity KB
# - k20x7 for Teensy 3.1 and 3.2
MCU_STARTUP = k20x5
# Board: it should exist either in <chibios>/os/hal/boards/
# or <this_dir>/boards
# - BOARD =
# - PJRC_TEENSY_LC for Teensy LC
# - PJRC_TEENSY_3 for Teensy 3.0
# - PJRC_TEENSY_3_1 for Teensy 3.1 or 3.2
# - MCHCK_K20 for Infinity KB
BOARD = MCHCK_K20
# Cortex version
# Teensy LC is cortex-m0; Teensy 3.x are cortex-m4
MCU = cortex-m4
# ARM version, CORTEX-M0/M1 are 6, CORTEX-M3/M4/M7 are 7
# I.e. 6 for Teensy LC; 7 for Teensy 3.x
ARMV = 7
# Vector table for application
# 0x00000000-0x00001000 area is occupied by bootlaoder.*/
# The CORTEX_VTOR... is needed only for MCHCK/Infinity KB
OPT_DEFS = -DCORTEX_VTOR_INIT=0x00001000
@fredizzimo Yes, sorry I wasn't more clear, but I've made the indicated edits to the file. The only thing I was not sure about was OPT_DEFS = -DCORTEX_VTOR_INIT=0x00001000 i tried both leaving it alone and commenting it out. Both with the same result.
I have no idea what could be wrong, since Teensy 3.1 and 3.2 uses the same chip as the Infinity. The difference is the bootloader and the connections.
Do you have the file bootloader_defs.h in your directory? If you have, then delete it, as it enables the kiibohd one.
I did have the bootloader_defs.h in my directory. Removed it, same outcome.
I was expecting github to notify me if someone talks to me :/
I have my keyboard with teensy 3.2 working using tmk_firmware, I would like to make the changes here as well.
How do you handle the changes to tmk_core here? Because I have to change things there (chibios.mk and .gitignore). I want to fix a little the instructions too.
@fredizzimo I would like to see a chibios_keyboards or arm_keyboards folder so I can drop my keymap there.
@profet23 You can test your keybord with my fork of tmk_firmware here. In Master you should be able to run make -f Makefile.3.2 inside teensy_lc_onekey and in the Personal branch you can find "totoroboard" that is my keyboard with teensy 3.2.
I uploaded it with the TeensyLoader. Then we can come back to qmk_firmware once it is updated/fixed, it's just to test the concept :)
@leizzer hrm... I get device unrecognized in your repo as well. For both the one key and your "totoroboard".
kiibohd seems to compile and run fine on this controller.
@profet23 dont know what to tell you if you followed the instructions (cloning chibios and chibios-contrib)... I didn't change much code and I used TeensyLoader selecting the ch.hex inside the /build directory
@leizzer i didnt clone the two sub projects I just used the zip's as the instructions (link) noted that not all versions work. Maybe newer versions are required. I'll try when I get home.
@leizzer, for the organization of the keyboards, take a look at this. There's no final conclusion though, but for now I would suggest to put it in the handwired folder.
Regarding to the changes to tmk_core, just send a pull request. The review process would probably be a bit tighter than other pull requests, but otherwise there shouldn't be any difference.
But be aware that things are quite different here, so your changes might not apply anymore. Also note that ChibiOS and ChibiOS-Contrib are already included here as subprojects, and should automatically be synced with
git submodule sync --recursive followed by git submodule update --init --recursive. Only the second command is strictly needed, but the first one will make sure that everything is up to date in case the repository locations are changed.
@profet23,
I still have no idea why it doesn't work for you, if it does for @leizzer. Maybe the wiring is different or something?
May I ask what's the status of QMK for Infinity Ergodox? How stable is it? Which functionality is not working? I'm happy owner of Ergodox EZ and have a fork with custom layout and now I received Ergodox Infinity from Massdrop. So the question is, can just compile the code with my layout and flash it, or should I better recreate my layout in Infinity's web configurator?
PS. My layout is pretty simple and doesn't contain any advanced functionality like dual-role keys etc.
@sergei-dyshel,
The Infinity Ergodox support for normal keyboard functionality should be as stable as the QMK in general, and all basic and advanced features should work.
However I still haven't had time to properly fix the LCD and LED support(currently the right hand LCD stays on constantly while the left is dark). I have had slightly too big ambitions with the visualization support, and not much time to actually program it, so it has taken way longer than planned. But now I think I have came up with a simplified design, so it shouldn't take too long.
That said, I will soon(today) send a pull request that completely disables the LCD, LCD backlight, and the LEDs, so that it's completely usable without them, until I'm able to add proper support.
Thanks for update!
Thanks for your work on this @fredizzimo. Just as a data point: I haven't been using my Infinity because I a few times daily get stuck in one of the layers but have no feedback. So instead of just switching back to the layer I want to be in when I accidentally switch, I instead have to guess and mess around and frequently just unplug the Infinity to get back to a known state.
I don't need to be able to play tetris on the display or anything, but it would be nice to even just have colors of the backlight or something to say I'm in layer 0 or 1 or 2. :-) At the moment I'm using my Ergodox Ez, but I have this Infinity I'd like to be using more. :-) Just saying.
@linsomniac Have you tried using the lcd_backlight_hal_color function in infinity.c?
I hacked on this a little... https://github.com/ec1oud/qmk_firmware/commit/3fa9073f1af9ae2bd1e034bf766d2bdad2200458 re-enables the LCD and key backlighting and LCD text animations. (I don't like the LCD backlight animation, would rather have instant feedback when I switch layers instead of a slow fade.) And that needs https://github.com/ec1oud/qmk_firmware/commit/2506f5bb9843bfe161b36b6f4d7e937aee484b67 which is my custom keymap. Not that it's all correct and complete yet of course, but working well enough for me to use the keyboard. I don't know why backlight increment/decrement keys aren't working, either.
The support has been there for a while now... closing
So it looks like there were a lot of directory changes recently in QMK, which has made things a bit confusing. This readme now seems out of date: https://github.com/qmk/qmk_firmware/tree/master/keyboards/ergodox_infinity
How would I go about flashing the default ergodox infinty layout with QMK? The next thing would be to have the LCDs working.
Most helpful comment
@dragon788,
I will mostly need help with testing the LED support, since I didn't install LED's in my board. I think I have the basis working, in my TMK fork as discussed here. But I haven't had any test reports for my latest fixes yet.
But of course I'm happy to see additional testing and bug reports.
Anyway, I will start the real porting now, as I have started my summer vacation, although at least the first week will be busy with other things as well.
I will start of with an almost direct port of what was initially done for ChibiOS support in TMK, and then proceed to the needed refactorings and other stuff. That leaves us with more time to discuss the direction we are going, and gives @jackhumbert, @ezuk and everyone else possibilities for their inputs.
ChibiOS will be added as a submodule, at least initially, as that way it won't grow the repository size, and we can always move to subtrees later.