beta-tester
Posts: 1242
Joined: Fri Jan 04, 2013 1:57 pm
Location: de_DE

why on earth are USB keyboards still using scan codes

Tue May 24, 2016 1:35 pm

why on earth are USB keyboards still using scan codes these days?
why there is no new standard protocol for Input device available, that uses the real characters shown on the keyboard (for example as uft-8 or similar) to get rid of the damn stupid legacy keyboard layout issue.
or if it is not practicable to create a new standard propocoll then the manufacturer could add Information to the USB device description, what contains the layout as id to get the OS the chance to figure out the correct layout automatically...
so that you get is what you see?.
{ I only give negative feedback }
RPi B (rev1, 256MB), B (rev2, 512MB), B+, 2B, 3B, 3B+, ZeroW, ...

Heater
Posts: 13620
Joined: Tue Jul 17, 2012 3:02 pm

Re: why on earth are USB keyboards still using scan codes

Wed May 25, 2016 4:01 am

beta-tester,
why on earth are USB keyboards still using scan codes these days?
Strangely enough I was asking myself that same question recently.

A keyboard could just emit unicode, in whatever format. Job done. All languages covered, all keyboard layouts handled in the keyboard itself. Plug-n-play simplicity.

After all, when I connect to my Pi via ssh the Pi knows nothing of the keyboard layouts and scan codes of the keyboards I'm using on the clients machine.

Obviously it's a legacy thing. Inherited from the keyboards in use in 1970's. Shocking really.
Memory in C++ is a leaky abstraction .

User avatar
MarkHaysHarris777
Posts: 1820
Joined: Mon Mar 23, 2015 7:39 am
Location: Rochester, MN
Contact: Website

Re: why on earth are USB keyboards still using scan codes

Wed May 25, 2016 5:37 am

Heater wrote:beta-tester,
why on earth are USB keyboards still using scan codes these days?
Strangely enough I was asking myself that same question recently ...

Obviously it's a legacy thing. Inherited from the keyboards in use in 1970's. Shocking really.
Actually, its working correctly ...

... even philosophically correctly. At the keyboard level all we are interested in are buttons; and that
means button codes, or scan codes. At the keyboard level (platic parts and cheap eletronics) all we
care about is whether the button codes are going to the kernel.

At the system level we care about keyboard layouts (and we want those suckers to be cheap and programmable ). If you put the unicode and universal language stuff with dead keys, &c in the keyboard the keyboard now becomes hard to maintain (non programmable probably) and very expensive). With the keyboard layouts on the system the entire keyboard(s) for all languages and locales becomes not only cheap and programmable, but also very flexible and extensible.

Yes, scan codes are a legacy thing... and a very good legacy thing too !
marcus
:ugeek:

beta-tester
Posts: 1242
Joined: Fri Jan 04, 2013 1:57 pm
Location: de_DE

Re: why on earth are USB keyboards still using scan codes

Wed May 25, 2016 1:06 pm

even a keyboard is a bunch of buttons of the point of view of the hardware/kernel/pice of software...
that USB keyboard i mean is a HUMAN INTERFACR DEVICE...
i press a key and i want to see the letter on the screen or what ever, that is visibl on the keyboard key.
as standard user, i don't wanna care about what is going on in the background, how it is handled in hardware/software...
as programmer i wished each keyboard would have an defined identifier that describes, what keyboard layout i am dealing with - what is written on the keys.
but even this information is not available.
it would be so easy for the manufacturer, when he print/engrave the letters to the keyboard keys, when he also program an identifier to the USB device descriptor (or how it is named) - that is not really expensive to do so you even don't need extra hardware - simply program the device descriptor - but all the industry has to do it in the same way. then you can keep your scan codes, but then you know, what layout the keyboard have.
and then the OS could read out that identifier and switch to the corrrect keyboard layout automatically.

if you are not happy with the keyboard layout, or simply want to see the keyboard as a bunch of buttons, you can switch the keyboard layout in the os of use your scan code translation as usual.

these days there should be no need to setup an OS with the keyboard layout manually. :x
at the moment it is like...
you plug in a USB keyboard and the OS/kernel knows absolutely nothing absout the layout...
come on thats like stone age... :roll:
Last edited by beta-tester on Wed May 25, 2016 1:21 pm, edited 1 time in total.
{ I only give negative feedback }
RPi B (rev1, 256MB), B (rev2, 512MB), B+, 2B, 3B, 3B+, ZeroW, ...

User avatar
MarkHaysHarris777
Posts: 1820
Joined: Mon Mar 23, 2015 7:39 am
Location: Rochester, MN
Contact: Website

Re: why on earth are USB keyboards still using scan codes

Wed May 25, 2016 1:18 pm

beta-tester wrote:even a keyboard is a bunch of buttons of the point of view of the hardware/kernel/pice of software...
that USB keyboard i mean is a HUMAN INTERFACR DEVICE...
With scan codes we only need to manufacture one keyboard (with replaceable key tops). A keyboard has only one chip; and its basically the same one for every single keyboard manufactured. The only thing that needs to change are the plastic parts; and of course the keyboard layouts in the system.

Take a look at the sheer number of keyboard layouts in your Raspberry PI. Do you really think manufacturers can afford to build specific keyboard layouts into each and every keyboard they make? The physical device is not what's important; you only need one of those, and scan codes are fine.

The important piece is the mapping that occurs in the system with the keyboard layouts (and that's cheap and easy because its all software).
marcus
:ugeek:

beta-tester
Posts: 1242
Joined: Fri Jan 04, 2013 1:57 pm
Location: de_DE

Re: why on earth are USB keyboards still using scan codes

Wed May 25, 2016 1:31 pm

MarkHaysHarris777 wrote:
beta-tester wrote:even a keyboard is a bunch of buttons of the point of view of the hardware/kernel/pice of software...
that USB keyboard i mean is a HUMAN INTERFACR DEVICE...
With scan codes we only need to manufacture one keyboard (with replaceable key tops). A keyboard has only one chip; and its basically the same one for every single keyboard manufactured. The only thing that needs to change are the plastic parts; and of course the keyboard layouts in the system.

Take a look at the sheer number of keyboard layouts in your Raspberry PI. Do you really think manufacturers can afford to build specific keyboard layouts into each and every keyboard they make? The physical device is not what's important; you only need one of those, and scan codes are fine.

The important piece is the mapping that occurs in the system with the keyboard layouts (and that's cheap and easy because its all software).
scan codes are good... ok, accepted.
the hardware is the same/fix/general... ok, accepted.

but when a manufacturer can program the USB device descriptor with its own manufacturer ID, device ID, subdevice ID and a unique serial number for each keyboard he assembled, then he could spend 5 cent to add somewhere in the eeprom of the standard USB device chip the information of the en-UK, en-US, en-CA, ..., de-DE or something else (coded as a single BYTE, WORD or DWORD), that the OS knows what to use for initial setup.
{ I only give negative feedback }
RPi B (rev1, 256MB), B (rev2, 512MB), B+, 2B, 3B, 3B+, ZeroW, ...

User avatar
MarkHaysHarris777
Posts: 1820
Joined: Mon Mar 23, 2015 7:39 am
Location: Rochester, MN
Contact: Website

Re: why on earth are USB keyboards still using scan codes

Wed May 25, 2016 2:11 pm

beta-tester wrote: ... he could spend 5 cent to add somewhere in the eeprom of the standard USB device chip the information of the en-UK, en-US, en-CA, ..., de-DE or something else (coded as a single BYTE, WORD or DWORD), that the OS knows what to use for initial setup.


I'll try again... think about what you just said...

... picture a warehouse of keyboards stacked up someplace with en-UK, or en-US, or en-CA stamped on the end of the box ! ugh. Are you serious?

Can you picture some stock-girl running around mumbling to herself, "now where did I put those en-UK keyboards? ... amonst these hundreds of stacks of keyboard types ??"

Thats not feasible; and keyboard layouts are easily accommodating. Besides, and really think about this, sometimes I really want my en-US layout to change; and I'm not going to swap out my keyboard to make the swap... when I am doing my alternate language work (Hebrew, Latin, Greek, German, Russian... &c) I want a different layout with diacritics, dead-keys, special punctuation, and all the rest! With your proposal, my keyboard would be automatically locked into en-US... and somehow I would have to over-ride that anyways... the easiest solution is the way it is!
marcus
:ugeek:

W. H. Heydt
Posts: 10995
Joined: Fri Mar 09, 2012 7:36 pm
Location: Vallejo, CA (US)

Re: why on earth are USB keyboards still using scan codes

Wed May 25, 2016 2:27 pm

beta-tester wrote:even a keyboard is a bunch of buttons of the point of view of the hardware/kernel/pice of software...
that USB keyboard i mean is a HUMAN INTERFACR DEVICE...
i press a key and i want to see the letter on the screen or what ever, that is visibl on the keyboard key.
Not necessarily. I had a volunteer at ConReg earlier this year who was a teacher who touch-typed Dvorak layout. In her classroom, her keyboard was a QWERTY layout. The result is that her students are inhibited from trying to get into her computer because, while they can read the keycaps, that doesn't tell them what characters are actually in use. Now imagine her delight when I showed her how easy it is to set a Pi to treat a--standard--keyboard as a Dvorak keyboard.
as standard user, i don't wanna care about what is going on in the background, how it is handled in hardware/software...
Nor do you need to.

Heater
Posts: 13620
Joined: Tue Jul 17, 2012 3:02 pm

Re: why on earth are USB keyboards still using scan codes

Wed May 25, 2016 2:33 pm

W. H. Heydt
Nor do you need to.
Perhaps not the details. But for sure every Linux install I have done since 1997 has required messing around with keyboard configuration.

It's really annoying.

Can't we just have a keyboard with a Pi Zero in it that does all that messy stuff and pumps out unicode? Then if you want to mess with your your layout or language just log into to your keyboard and hack the config to your hearts content. :)
Memory in C++ is a leaky abstraction .

User avatar
Cancelor
Posts: 757
Joined: Wed Aug 28, 2013 4:09 pm
Location: UK

Re: why on earth are USB keyboards still using scan codes

Wed May 25, 2016 2:48 pm

beta-tester wrote:why on earth are USB keyboards still using scan codes these days?
why there is no new standard protocol for Input device available, that uses the real characters shown on the keyboard (for example as uft-8 or similar) to get rid of the damn stupid legacy keyboard layout issue.
or if it is not practicable to create a new standard propocoll then the manufacturer could add Information to the USB device description, what contains the layout as id to get the OS the chance to figure out the correct layout automatically...
so that you get is what you see?.
I'm with you on this one and praise you for thinking out of the box.
I have just looked at 8 keyboards that I have and they are all different.
The poor girl in the stock room has long since learnt how to pick more than one part.
Can't find the thread you want? Try googling : YourSearchHere site:raspberrypi.org

User avatar
Burngate
Posts: 6049
Joined: Thu Sep 29, 2011 4:34 pm
Location: Berkshire UK Tralfamadore
Contact: Website

Re: why on earth are USB keyboards still using scan codes

Wed May 25, 2016 2:55 pm

MarkHaysHarris777 wrote:I'll try again... think about what you just said...
... picture a warehouse of keyboards stacked up someplace with en-UK, or en-US, or en-CA stamped on the end of the box ! ugh. Are you serious?
Can you picture some stock-girl running around mumbling to herself, "now where did I put those en-UK keyboards? ... amonst these hundreds of stacks of keyboard types ??"
But ... isn't that exactly what happens now? She needs to find a keyboard with the correct key-caps on it, and the UK one has a pound-sign, which the US one doesn't.
Besides, and really think about this, sometimes I really want my en-US layout to change; and I'm not going to swap out my keyboard to make the swap... when I am doing my alternate language work (Hebrew, Latin, Greek, German, Russian... &c) I want a different layout with diacritics, dead-keys, special punctuation, and all the rest!
Fine.
So you need a bit of code that translates from the keyboard's default codes to the ones you want. Not a huge problem.
In fact it could translate a text file from English into Hebrew, or at least the Latin characters into Hebrew characters. I'm sure someone, somewhere wants to do that.

What I'm not sure I want to see is the robot you've programmed to re-engrave your keyboard key-caps when you decide to swap to Cyrillic, particularly if you've got your hands close by.
Gives a new meaning to "fingernail art"

User avatar
MarkHaysHarris777
Posts: 1820
Joined: Mon Mar 23, 2015 7:39 am
Location: Rochester, MN
Contact: Website

Re: why on earth are USB keyboards still using scan codes

Wed May 25, 2016 3:01 pm

Burngate wrote:
What I'm not sure I want to see is the robot you've programmed to re-engrave your keyboard key-caps when you decide to swap to Cyrillic, particularly if you've got your hands close by.
Gives a new meaning to "fingernail art"
Key caps mean nothing to me. I also use Dvorak, among the others I listed. My Hebrew and Greek fonts are a comical nightmare (but once I mesmerized the layout, and each key has four[4] characters!) its a piece of cake !

I did use cheat sheets for the layouts until I learned the ropes.
marcus
:ugeek:

User avatar
Burngate
Posts: 6049
Joined: Thu Sep 29, 2011 4:34 pm
Location: Berkshire UK Tralfamadore
Contact: Website

Re: why on earth are USB keyboards still using scan codes

Wed May 25, 2016 3:57 pm

I understand from that, that your memory is far far better than mine. I have to peer round the keyboard to find the ... there it is ... @
As for Dvorak keyboards, ... that's a whole - wait for it - New World :D :) :(

User avatar
MarkHaysHarris777
Posts: 1820
Joined: Mon Mar 23, 2015 7:39 am
Location: Rochester, MN
Contact: Website

Re: why on earth are USB keyboards still using scan codes

Wed May 25, 2016 4:25 pm

Burngate wrote:I understand from that, that your memory is far far better than mine. I have to peer round the keyboard to find the ... there it is ... @
As for Dvorak keyboards, ... that's a whole - wait for it - New World :D :) :(
Yeah, actually, its not much worse than any other touch typing you learn to do... just after a while it soaks in and you know where the keys are. I can't tell you how, but I just know where the @ is (except it was funny as hell when I got my first PI and didn't realize that it had the British layout and the @ sign was no where to be found !! I laughed out loud when that happened. Actually, I think the British layout makes a lot of sense, and I often use it !
marcus
:ugeek:

User avatar
scruss
Posts: 2548
Joined: Sat Jun 09, 2012 12:25 pm
Location: Toronto, ON
Contact: Website

Re: why on earth are USB keyboards still using scan codes

Wed May 25, 2016 4:34 pm

Burngate wrote:As for Dvorak keyboards, ... that's a whole - wait for it - New World :D :) :(
I see what you did there, Hovis Boy ...

Another pro-scancode reason is: how would you detect a Shift or Control press on its own using Unicode? You'd also have to wait for a potentially unknown number of bytes, as you could never tell the length of the UTF string you're going to receive.

Scancodes also make wonderful things like the Compose Key possible.
‘Remember the Golden Rule of Selling: “Do not resort to violence.”’ — McGlashan.

Heater
Posts: 13620
Joined: Tue Jul 17, 2012 3:02 pm

Re: why on earth are USB keyboards still using scan codes

Wed May 25, 2016 5:26 pm

scruss,

Another pro-scancode reason is: how would you detect a Shift or Control press on its own using Unicode?
If shift or control on their own have meaning then they should have code points in Unicode. If they don't already they should have. After all we have all kind of other junk in their, like emojis etc.
You'd also have to wait for a potentially unknown number of bytes, as you could never tell the length of the UTF string you're going to receive.
Who cares about the length of a string or bytes? Keyboards deliver characters. Use UTF-32 and we have a constant size of 32 bits.

Of course the problems with Unicode run deep. It is a brain damaged system. In general you cannot not the length of a string.

For example the Swedish "ö" can be a single composite character, (U+00F6), or it can be composed from an "o", (U+006F), and the dots, the diaeresis (U+0308).

In general it seems to be impossible to get the length in bytes or characters of a Unicode string until you have parsed it all.

But this problem exists if you are trying to type Unicode from a good old scan.code keyboard.
Memory in C++ is a leaky abstraction .

W. H. Heydt
Posts: 10995
Joined: Fri Mar 09, 2012 7:36 pm
Location: Vallejo, CA (US)

Re: why on earth are USB keyboards still using scan codes

Wed May 25, 2016 9:00 pm

Burngate wrote:I understand from that, that your memory is far far better than mine. I have to peer round the keyboard to find the ... there it is ... @
As for Dvorak keyboards, ... that's a whole - wait for it - New World :D :) :(
The US Federal Government was planning to convert to Dvorak (easier to type on --> faster typing --> fewer typists --> saves money) but WW2 intervened and they needed to buy pretty much every typewriter available to handle all the paperwork, and that meant...QWERTY.

So...now you know who to blame for continued existence (in the US, at least) of QWERTY keyboards.

User avatar
MarkHaysHarris777
Posts: 1820
Joined: Mon Mar 23, 2015 7:39 am
Location: Rochester, MN
Contact: Website

Re: why on earth are USB keyboards still using scan codes

Wed May 25, 2016 9:12 pm

W. H. Heydt wrote:
So...now you know who to blame for continued existence (in the US, at least) of QWERTY keyboards.
Good 'ol IBM didn't help either; when they introduced the Selectric typewriter they 'might' have gone with another keyboard, but NO

QWERTY is here to stay... WHY? ... there is no reason to keep this up/
marcus
:ugeek:

beta-tester
Posts: 1242
Joined: Fri Jan 04, 2013 1:57 pm
Location: de_DE

Re: why on earth are USB keyboards still using scan codes

Wed May 25, 2016 9:22 pm

scruss wrote:Another pro-scancode reason is: how would you detect a Shift or Control press on its own
oh yeah, i weren't thinking that far when i was pissed off after my "million and one" installation/setup of an OS (linux or windows doesn't matter) and i needed to select my used keyboard layout to have things working right (QWERTY, QWERTZ, or whatever and then what specific "flavor" of it).
so i was a bit too angry why things are like they are and not be better as they could...

and the other comments here goes a bit out of my intentios... :P
Last edited by beta-tester on Wed May 25, 2016 9:25 pm, edited 1 time in total.
{ I only give negative feedback }
RPi B (rev1, 256MB), B (rev2, 512MB), B+, 2B, 3B, 3B+, ZeroW, ...

stderr
Posts: 2178
Joined: Sat Dec 01, 2012 11:29 pm

Re: why on earth are USB keyboards still using scan codes

Wed May 25, 2016 9:25 pm

W. H. Heydt wrote:So...now you know who to blame for continued existence (in the US, at least) of QWERTY keyboards.
I'm pretty sure that most keyboards are the same, more or less. It's the idiotic messing with the size of the enter key and the backspace and thus the movement of some keys around that is trouble. And, of course, Apple has its own idiocy, for years putting the little raised bits that should be under the j and the f on the k and the d. And IBM moving the control key and putting the nearly never used caps lock in its place. The only need for QWERTY writing on a keyboard is to standardise in order to be able to talk about a specific key when complaining about keyboards.

User avatar
MarkHaysHarris777
Posts: 1820
Joined: Mon Mar 23, 2015 7:39 am
Location: Rochester, MN
Contact: Website

Re: why on earth are USB keyboards still using scan codes

Wed May 25, 2016 9:30 pm

stderr wrote: The only need for QWERTY writing on a keyboard is to standardise in order to be able to talk about a specific key when complaining about keyboards.
The QWERTY goes back to a study done on type-bar typewriters that figured out there would be less key clash on the QWERTY layout than other layouts !

no reason to keep that up
marcus
:ugeek:

Heater
Posts: 13620
Joined: Tue Jul 17, 2012 3:02 pm

Re: why on earth are USB keyboards still using scan codes

Wed May 25, 2016 9:34 pm

W. H. Heydt,
...easier to type on --> faster typing --> fewer typists --> saves money...
Except those claims are essentially bogus. Or at least unsubstantiated.

Thing is, what are you doing with your keyboard? After programming for decades I still mostly don't know where all the keys are by "finger memory". But that is not a problem. My simple mind cannot think of code write or fix at anything like a typists keying rate. It's certainly not the bottle neck in the process.

What would help me is not having "[", "]", "{", "}", "|", and so on buried away and so hard to use. "Alt Gr" and all that. Not to mention "`" and "~".

And why the hell do we waste space with the totally redundant "CapsLock" key? The "Windows" key? Not to mention the easy to get to "¤" which nobody ever uses, even if they know what it means.

Oh and why is the even "½" and "§" on my keyboard? Who ever uses that?

Qwerty is weird, but that is the least of our problems.
Last edited by Heater on Thu May 26, 2016 6:03 am, edited 1 time in total.
Memory in C++ is a leaky abstraction .

stderr
Posts: 2178
Joined: Sat Dec 01, 2012 11:29 pm

Re: why on earth are USB keyboards still using scan codes

Wed May 25, 2016 9:44 pm

MarkHaysHarris777 wrote:
stderr wrote: The only need for QWERTY writing on a keyboard is to standardise in order to be able to talk about a specific key when complaining about keyboards.
The QWERTY goes back to a study done on type-bar typewriters that figured out there would be less key clash on the QWERTY layout than other layouts !
no reason to keep that up
I'm saying that there is no reason for the actual printing on the keyboard, the letters marked as QWERTY or whatever, other than as a means to discuss and complain about keyboards. Anyone who knows how to type isn't going to be looking at the keyboard, it's just ridiculous. When one types, one looks at the screen, not the keyboard.

Regarding whether or not people should not be allowed to type using the QWERTY scheme, it's that really up to them? Now should they be taught that in a day and age when we can just install a different layout with ease? Couldn't we get a different ball for the Selectric back in the day? They even had one for APL!: http://www.computerhistory.org/collecti ... /102696478 . So maybe QWERTY should've been banned back then.

Some years back, I tested myself by putting in a Cyrillic key layout and typing that way. I knew how to type QWERTY and I knew the Cyrillic letters but I didn't have them together. It didn't take very long to get up to 10 wpm or something like that, I was typing and not looking at the keys. That's not fast, of course, but it is evidence that the difficulty in typing is the physicality more than what is written on the keys. So someone changing to Dvorak would probably not have a huge learning curve.

User avatar
mahjongg
Forum Moderator
Forum Moderator
Posts: 12339
Joined: Sun Mar 11, 2012 12:19 am
Location: South Holland, The Netherlands

Re: why on earth are USB keyboards still using scan codes

Wed May 25, 2016 9:46 pm

the MAIN reason (IMHO) that scan codes are used instead of sending a code for the key that is pressed is that scan codes send a code for each key that is PRESSED and another one for any key that is RELEASED!

This is important when you use the keyboard for things like gaming.

and yes, its very much a legacy thing, and as said its a thing where some thought went into!

dumbing it down to just a "send a key code when pressed", that was done before the invention of the key code mechanics, I remember older keyboards that simple sent ASCII codes, they became quickly obsolete.

many early homecomputers had the keyboard readout matrix directly incorporated (ZX81, TRS-80 and many many others all did that) so software could see when a key was pressed and when released, with correct software. for example the first BASIC (LEVEL 1 BASIC) of the TRS-80 was extremely "dumb" and would just send a key code when a key was released, so touch typing became impossible (no n-key rollover) LEVEL II BASIC fixed that.

And all TRS-80 games learned to use the keyboard keys to control the action, which meant they had to read at least two keys simultaneously, ( for example) an arrow key and a fire key, independently of each other.

so (among other PC designers) IBM decided its keyboard system should not loose that capability, hence key codes.

User avatar
scruss
Posts: 2548
Joined: Sat Jun 09, 2012 12:25 pm
Location: Toronto, ON
Contact: Website

Re: why on earth are USB keyboards still using scan codes

Thu May 26, 2016 2:27 am

mahjongg wrote:... TRS-80 games learned to use the keyboard keys to control the action, which meant they had to read at least two keys simultaneously
But the hardware wasn't quite up to perfection: if you pressed three keys that were on corners of the matrix, the fourth keypress was magically generated.Really helped you cheat^H^H^H^H^Hwin at MS Olympic Decathlon by wrong-footing your opponent in the two player running games.
‘Remember the Golden Rule of Selling: “Do not resort to violence.”’ — McGlashan.

Return to “Off topic discussion”