Linux – Make All Applications Respect Modified XKB Layout

Why are they not working

According to the ArchWiki’s article that you mentioned:

• X server gets keycodes from the input device and converts them to the state and keysym.

  • state is the bitmask of the X modifiers (Ctrl/Shift/etc).

  • keysym is (according to /usr/include/X11/keysymdef.h) the integer that

    identify characters or functions associated with each key (e.g., via the visible engraving) of a keyboard layout.

    Each printable character has its own keysym, like plus, a, A, or Cyrillic_a, but other keys also generate their keysyms, like Shift_L, Left or F1.

• The application in the key press/release events gets all of this information.

  Some applications track keysyms like Control_L by themselves, others just look for the modifier bits in the state.

So what happens, when you press AltGr+j:

• You press AltGr. Application gets KeyPressed event with keycode 108 (<RALT>) and keysym 0xfe03 (ISO_Level3_Shift), state is 0.

• You press j (which maps to “h” in dvorak without modifiers). Application gets KeyPressed event with keycode 44 (<AC07>), keysym 0xff51 (Left) and state 0x80 (modifier Mod5 is on).

• You release j. The application gets KeyRelease event for the key <AC07>/Left with the same parameters.

• Then release AltGr — KeyRelease event for AltGr. (By the way, the state here is still 0x80, but that doesn’t matter.)

This can be seen if you run xev utility.

So, that all means that, although the application gets the same keysym code (Left) as from the normal key <LEFT>, it also gets the keysym code and modifier state from the AltGr. Most probably, those programs that don’t work, watch the modifiers and don’t want to work when some are active.

How to make them work

Apparently, we can’t change every program to not to look for modifiers. Then the only option to escape this situation is to not generate modifiers’ keysyms and state bits.

1. Separate group

The only method that comes to my mind is: define cursor movement keys in a separate group and switch, with a separate key press, to that group prior to pressing the keys j, k, l, i (h, t, n, c) (group latching is the preferred method for one time group change, as I understand).

For example:

xkb_keymap {
    xkb_keycodes { include "evdev+aliases(qwerty)" };
    xkb_types { include "complete" };
    xkb_compatibility {
        include "complete"

        interpret ISO_Group_Latch { action = LatchGroup(group=2); };
    };
    xkb_symbols {
        include "pc+us(dvorak)+inet(evdev)"

        key <RALT> { [ ISO_Group_Latch ] };

        key <AC07> {
            type[Group2] = "ONE_LEVEL",
            symbols[Group2] = [ Left ]
        };
        key <AC08> {
            type[Group2] = "ONE_LEVEL",
            symbols[Group2] = [ Down ]
        };
        key <AC09> {
            type[Group2] = "ONE_LEVEL",
            symbols[Group2] = [ Right ]
        };
        key <AD08> {
            type[Group2] = "ONE_LEVEL",
            symbols[Group2] = [ Up ]
        };
    };
    xkb_geometry { include "pc(pc104)" };
};

Now, if you first press AltGr and then (separately) one of the movement keys, this should work.

However, this is not very useful, more appropriate would be to LockGroup instead of latch and press AltGr before and after group switch. Even better may be to SetGroup — then AltGr would select that group only while being pressed, but that discloses to the applications AltGr’s keysym (ISO_Group_Shift/ISO_Group_Latch/whatever is defined) (but modifier state stays clean).

But... there is also a possibility left that the application also reads keycodes (the codes of the real keys). Then it will notice the “fake” cursor keys.

2. Overlay

The more “low-level” solution would be the overlay (as the same article describes).

Overlay simply means that some (real keyboard )key returns the keycode of another key. The X server changes the keycode of a key and computes the modifier state and the keysym for that new keycode, so the application shouldn’t notice the change.

But overlays are very limited:

• There are only 2 overlay control bits in the X server (i.e. there can be maximum 2 overlays).
• Each key can have only 1 alternative keycode.

As for the rest, the implementation is quite similar to the method with a separate group:

xkb_keymap {
    xkb_keycodes { include "evdev+aliases(qwerty)" };
    xkb_types { include "complete" };
    xkb_compatibility {
        include "complete"

        interpret Overlay1_Enable {
            action = SetControls(controls=overlay1);
        };
    };
    xkb_symbols {
        include "pc+us(dvorak)+inet(evdev)"

        key <RALT> {
            type[Group1] = "ONE_LEVEL",
            symbols[Group1] = [ Overlay1_Enable ]
        };
        key <AC07> { overlay1 = <LEFT> };
        key <AC08> { overlay1 = <DOWN> };
        key <AC09> { overlay1 = <RGHT> };
        key <AD08> { overlay1 = <UP> };
    };
    xkb_geometry { include "pc(pc104)" };
};

SetControls means change the control bit while the key is pressed and restore it on the key release. There should be similar function LatchControls, but xkbcomp gives me

Error:            Unknown action LatchControls

on keymap compilation.

(By the way, I also use dvorak and also have remapped some movement keysyms to high levels of alphabetic keys. And also came across some broken functionality (selection in Xfce notes and desktop switch by Ctrl-Alt-Left/Right). Thanks to your question and this answer, now I know what an overlay is :).)