DarkElvenAngel
Posts: 1694
Joined: Tue Mar 20, 2018 9:53 pm

Re: Serial terminal project

Fri Apr 30, 2021 3:38 pm

Thank you for this,

I haven't setup the lock keys but this will help once I get to that!
Raspberry Pi Foundation could do everyone a favour by producing some decent documentation for tinyusb. I am sure it would be within their educational remit.
Were as I agree some good documentation is needed there might be issues with that since it's an external library and subject to changes. We need some good examples with good documentation around them. I can say for the examples we have it's easy enough to work out how to use those functions it's when we stray out of the light we find trouble.

I saw noted in another post that there is an update coming to tinyusb to address the host modes short comings and hopefully will drop in mid-May lets hope this update doesn't break our existing code.

DarkElvenAngel
Posts: 1694
Joined: Tue Mar 20, 2018 9:53 pm

Re: Serial terminal project

Fri Apr 30, 2021 11:30 pm

I've got nano working*!
P_20210430_191622_2.jpg
P_20210430_191622_2.jpg (52.6 KiB) Viewed 945 times
* mostly

So it works and is mostly usable when doing edits a file it's sometimes necessary to [ctrl] [L] to redraw the screen from time to time depending on what you're doing. The text doesn't scroll correctly when deleting a line for example. I'm thinking I've still got bugs in there but this is an improvement. I also realized that I was sending a line feed for the return key and that should have been carriage return.

Still have work to do and code to fix.

Edit : I don't know why the screen looks darker at the top in the picture it doesn't look that way in person.

fdufnews
Posts: 330
Joined: Fri Oct 07, 2011 5:37 pm

Re: Serial terminal project

Sat May 01, 2021 7:28 am

Edit : I don't know why the screen looks darker at the top in the picture it doesn't look that way in person.
Remanence.
Your eyes do not see the decrease of phosphorus but the camera does.

DarkElvenAngel
Posts: 1694
Joined: Tue Mar 20, 2018 9:53 pm

Re: Serial terminal project

Sat May 01, 2021 10:12 am

fdufnews wrote:
Sat May 01, 2021 7:28 am
Edit : I don't know why the screen looks darker at the top in the picture it doesn't look that way in person.
Remanence.
Your eyes do not see the decrease of phosphorus but the camera does.
It's an LCD screen though, all be it an older one. Does the same principle apply?

DarkElvenAngel
Posts: 1694
Joined: Tue Mar 20, 2018 9:53 pm

Re: Serial terminal project

Sun May 02, 2021 6:40 pm

A revisit to the font stretching modes of the vt100 results in
P_20210502_143535_1.jpg
P_20210502_143535_1.jpg (89.79 KiB) Viewed 868 times
This is the vttest of double wide and double height.

Slight error on the box test moving the cursor seems to still have errors.

DarkElvenAngel
Posts: 1694
Joined: Tue Mar 20, 2018 9:53 pm

Re: Serial terminal project

Tue May 04, 2021 4:23 am

I've run into an interesting problem,

I've had to lower the baud rate from 115200 I smashed it down to 9600. Why I'm not entirely sure the display driver isn't getting it's pixel buffer filled fast enough. The interesting part is the video driver runs in its own core so why is the serial data affecting it?

Now each time a byte is received it goes through some processing, we want to know if this is part of an escape sequence and if it is it needs more processing to decode it. However that's not causing it, at least not as far as I can tell. The bottom of the screen maybe 5 - 10 lines of text are in the blue so 1/3 mostly less.

This is a resent development before today this happened once and the screen was empty except for the cursor blinking and the blue was in time with the flash. Once everything was going it vanished. Would core 0 be able to cause this issue? There are no mutex, locks, fifo or any other safeguards in place for the video driver it has free access to the character buffer and setting flags. I did this because it the driver is read only. The only possible thing I can think of is that core 0 is hogging memory access and core 1 can't read when it needs to?

The solution to slow down the incoming data is not the best and I should figure out some more efficient ways to do some operations. I've already overclocked to 200 MHz this was because I could compute colours fast enough (character attributes and flags) There too I want to make things more efficient so I can run at stock clocks.

Otherwise, I've actual been able to use nano to edit some files and besides a scrolling issue where I need to redraw the screen it works quite well.

Does anyone know if wmemcpy is available on the pico and if it can work with 3 byte blocks? I have the CharBuff array and it's 24 bit I need to fill them with different values right now I memset everything 0 but ideally it needs { 0,7,0 } or something like that.

It mostly works, I'm thinking I need a break from this project now I have had tunnel vision on it from a while now and I can come back to it in a week or so. I'll keep tuned to this thread though so if you have idea's or questions I'll see them.

DarkElvenAngel
Posts: 1694
Joined: Tue Mar 20, 2018 9:53 pm

Re: Serial terminal project

Fri May 14, 2021 3:00 am

So I decided to give this project a once over and rethink it. I've scaled it way back just to get things working properly. I've rewritten the VGA driver, the character buffer and changed gears on the terminal emulator down to a VT52. Since VT52 is emulated on later terminals it made sense to restart with something simpler.

The new VGA driver will only display 24 lines of text so I've blanked the first and last 3 lines since at the current resolution I could have 30 lines. It's also very simple it doesn't support colour or text attributes. This is fine for the VT52 since it also doesn't support any either.

The terminal is serviceable for the command line and with the correct TERM=vt52 variable you can do most basic things nano doesn't work properly and haven't tested vim. I'm also failing the vttest in VT52 mode I'm missing something or one of the commands isn't doing what it suppose to, though it works perfectly in my loopback tests.

As long as the TERM=vt52 is set in the environment the terminal works just like it should! nano works just fine now.
P_20210514_002608_1.jpg
The box test
P_20210514_002608_1.jpg (106.42 KiB) Viewed 766 times
P_20210514_002628_1.jpg
Ident
P_20210514_002628_1.jpg (31.13 KiB) Viewed 766 times

I have the caps and num lock keys working and the LEDs even light up that's for the tip Memotech Bill.

I'm just trying to name the project something better that VGA Terminal I was thinking of PT52 once I get the VT100 support then it'll need a new name again so that's something to think on. Suggestions welcome for a good name.

hippy
Posts: 9957
Joined: Fri Sep 09, 2011 10:34 pm
Location: UK

Re: Serial terminal project

Fri May 14, 2021 9:46 am

Every good project gets a Version 2 'restart from clean paper', building on things learnt and experienced gained; it's a sure sign things have been learned and discovered. And it's usually quicker to get to where one is than how long it took to get there. So looking good.

As to naming; I don't see anything wrong with PT52/PT100. The "P" can be taken as both "Pico" and "Pseudo". It's easy to remember with the play on "VT". You could add a "-VGA" suffix to distinguish from "-DVI" etc should they emerge.

DarkElvenAngel
Posts: 1694
Joined: Tue Mar 20, 2018 9:53 pm

Re: Serial terminal project

Sat May 15, 2021 11:55 pm

Thanks hippy,

I have to agree PT52 is a good name it just fits.

DarkElvenAngel
Posts: 1694
Joined: Tue Mar 20, 2018 9:53 pm

Re: Serial terminal project

Mon May 17, 2021 11:10 pm

Almost there with the new version!

I had to change the font as I wanted to do something a little different since I have a 16 colour palette and can set all colours for background and foreground. With the Linux terminal I decided to use bright as it's lesser used bold form.
P_20210517_185223_2_1.jpg
Font test
P_20210517_185223_2_1.jpg (71.51 KiB) Viewed 673 times
There are a few characters I need to fix in the regular font.

I did find out there is a real terminal pt100 in the terminal information so I might need to rethink my name.

lars_the_bear
Posts: 203
Joined: Thu Jan 28, 2021 8:13 pm
Contact: Website

Re: Serial terminal project

Wed May 19, 2021 8:01 am

Hi

I can't wait to steal your work and use it in my own project ;)

But I'm a little confused about something. At the start of this thread you said you're using the Pimoroni demo board, but you also mention a 16-colour palette. As I understand it, the Pimoroni board uses a huge stack of GPIO pins to provide a kind of DAC, to get a large-ish colour gamut. So why only 16 colours? Is that a change you've made in your software, or in the wiring?

I ask because my application could manage quite well with 8 colours (or even 1, frankly) which I ought ot be able to generate with only five GPIO pins. Unfortunately, I find the demo "scanvideo" code completely incomprehensible, so I can't work out what to change in either the software or the wiring.

Kevin

DarkElvenAngel
Posts: 1694
Joined: Tue Mar 20, 2018 9:53 pm

Re: Serial terminal project

Wed May 19, 2021 2:55 pm

lars_the_bear wrote:
Wed May 19, 2021 8:01 am
Hi

I can't wait to steal your work and use it in my own project ;)

But I'm a little confused about something. At the start of this thread you said you're using the Pimoroni demo board, but you also mention a 16-colour palette. As I understand it, the Pimoroni board uses a huge stack of GPIO pins to provide a kind of DAC, to get a large-ish colour gamut. So why only 16 colours? Is that a change you've made in your software, or in the wiring?

I ask because my application could manage quite well with 8 colours (or even 1, frankly) which I ought ot be able to generate with only five GPIO pins. Unfortunately, I find the demo "scanvideo" code completely incomprehensible, so I can't work out what to change in either the software or the wiring.

Kevin
Kevin thanks for the interest and the openness about software theft! ;)

Let me explain Yes I'm using the Pimoroni demo board and I have a driver that can produce 65,535 colors, and yes the current code can produce a 16 colours for each text character and it's background. Now I have currently 5 Pallets the CGA 16 colour Text, 16 shade Monochrome, 16 shade Amber, 16 shade Green, and C64 16 colour pallets. So this is a software only change I only added a header so I could use serial in on GPIO21 and GPIO22. The limit on colours is purely a choice based on memory usage. once I've got the hard part of 16 colour mode fully working I'm moving to 256 colours this will also be a pallet to keep memory use down. In 256 colour mode the character structure is 4 bytes so standard memory operations are possible and I don't have to make custom ones! But I'm thinking to release the source after 16 colour mode is complete.

So I'm only using the demo board because it's ready made, it looked cool, it was ready made, it had sound output, it was ready made, others had got it working, and it was ready made! I have a Parallax Propeller that also has a VGA out wired up so I know how and have the parts. So if you do your own VGA I've seen it mentioned a few places on here how to use less pins. I've not looked into that aspect yet but a smaller DAC would open up more pins, if you need them.

Now something to think about monochrome can run a much higher resolution, however I just thought I should see what my limits are and if I overclock to 250 MHz I can get 80 columns by 50 lines in 16 colour mode with the resolution set to 640x480 no scaling in monochrome mode I can get the full 60 lines available. That's due to how my VGA driver fills the buffer in colour mode it has more processing to do in monochrome mode it doesn't.

If you want any other details or have more questions ask away.

lars_the_bear
Posts: 203
Joined: Thu Jan 28, 2021 8:13 pm
Contact: Website

Re: Serial terminal project

Wed May 19, 2021 4:20 pm

Thanks.

It seems to me that, if all you wanted was monochrome text, you could use (say) three pins (plus the two sync pins) to give an 8-level grey-scale. If (for simplicity) you stored four bits for each pixel, you could still get a 640x480 framebuffer in the Pico's memory. 8-level grey-scale would allow you (in principle) to anti-alias the text, which makes a huge difference to the appearance of the display. Whether the Pico has enough flash to store a font with anti-aliasing information, and whether it has enough CPU grunt to do the rendering, I really don't know.

These are the kinds of things I'd like to find out, but I don't want to use an add-on board that will devote 15 GPIO pins (or whatever it is) when I only need five. At present, though, I wouldn't know even how to start wiring this up, let alone munging the software.

To be honest, at present I'm using a Pi Zero for my terminal, which has an HDMI output, and is currently only a little more expensive than a Pico. So getting a Pico to work is not an urgent priority. But even with a custom kernel and yadda yadda yadda it still takes five seconds to start up, and I suspect the Pico would be nearly instant.

Kevin

PS. The other thing I'm looking into is the VGA32. This has a built-in VGA, and is dirt cheap -- but it only has a PS/2 keyboard connector.

DarkElvenAngel
Posts: 1694
Joined: Tue Mar 20, 2018 9:53 pm

Re: Serial terminal project

Wed May 19, 2021 6:21 pm

lars_the_bear wrote:
Wed May 19, 2021 4:20 pm
It seems to me that, if all you wanted was monochrome text, you could use (say) three pins (plus the two sync pins) to give an 8-level grey-scale. If (for simplicity) you stored four bits for each pixel, you could still get a 640x480 framebuffer in the Pico's memory. 8-level grey-scale would allow you (in principle) to anti-alias the text, which makes a huge difference to the appearance of the display. Whether the Pico has enough flash to store a font with anti-aliasing information, and whether it has enough CPU grunt to do the rendering, I really don't know.
Yes that would work you would need to wire the R G and B lines ton the VGA together and then choose resistors that would give you 0.7V max remember that each signal has 75 ohm impedance so all tied together that's 25 ohms. It's basically Ohm's Law.
lars_the_bear wrote:
Wed May 19, 2021 4:20 pm
These are the kinds of things I'd like to find out, but I don't want to use an add-on board that will devote 15 GPIO pins (or whatever it is) when I only need five. At present, though, I wouldn't know even how to start wiring this up, let alone munging the software.
Hence the prebuilt board for simplicity. I may spin my own with less colours but for now this works for me.
lars_the_bear wrote:
Wed May 19, 2021 4:20 pm
To be honest, at present I'm using a Pi Zero for my terminal, which has an HDMI output, and is currently only a little more expensive than a Pico. So getting a Pico to work is not an urgent priority. But even with a custom kernel and yadda yadda yadda it still takes five seconds to start up, and I suspect the Pico would be nearly instant.
Well the pico is quick to boot but my VGA screen takes about 5 seconds to switch on / come out of sleep mode and sync. I've gotten a pi zero to boot to the console in 1 or 2 seconds, using buildroot and boot time optimization. It comes down to the extras needed to get one or the other going.

Pi zero : HDMI Screen, HDMI Cable, SD card

Pico : VGA Monitor, some kind of PCB with a VGA,

BOTH: OTG micro B to USB A, USB Keyboard*

*The pico has a hard time with USB Keyboards I have a few and only one works with it.

DarkElvenAngel
Posts: 1694
Joined: Tue Mar 20, 2018 9:53 pm

Re: Serial terminal project

Thu May 20, 2021 5:11 am

Little update,

16 Colour mode is complete. I've added some extra escape sequences mainly to do to with resetting arrays of the buffer namely the entire line and screen these are not present on the VT52. I've also been really thinking over the ESC [ CSI ansi standard support, most notably with colours So the pt52 will now all ignore the CSI codes this fixes issues with bad programs that don't check if the terminal accepts these codes or not. These programs are a lot of core linux commands and even man seems to have a hard coded \e[m to turn off bold funny thing is it still calls the correct code to turn it off as well so better than most. Ansi escape codes will be available when I do the PT52E version.

Somethings I need to fix

Keyboard mapping some keys do nothing and or the wrong thing ( I'm looking at you delete key who thinks its backspace )
PT52 doesn't have a graphics set in it's font! Or any characters beyond 0x7F, Got to get those PETSCII in there.

A few people have expressed interest in my source code and I haven't published any yet, to be honest I've put a lot of work in to this and my hope is if I publish it new idea's and perspectives would see ways to improve things. This hasn't really happened with my other published projects I see the source code is forked and then nothing is done with it, nothing is contributed back. So I don't know what I want to do, on one hand you've all been awesome and given me these great ideas and I feel like that needs something back. On the other I post my code and it sits there someone picks the meat off the bones or complains that something isn't working or complete. It's all of you that have commented on this project and some of the my other threads I've been asking for assistance and advice, your help and interest kept me going so I have that on my mind too.

And this hasn't got anything to do with Kevin's stealing the source joke, I've been apprehensive of sharing the source code before I did my big rewrite.

So I need time to think this over and decide what to do. besides fix that delete key before I press it again!

Thanks for your interest and support, means a lot!

hippy
Posts: 9957
Joined: Fri Sep 09, 2011 10:34 pm
Location: UK

Re: Serial terminal project

Thu May 20, 2021 9:53 am

DarkElvenAngel wrote:
Thu May 20, 2021 5:11 am
A few people have expressed interest in my source code and I haven't published any yet, to be honest I've put a lot of work in to this and my hope is if I publish it new idea's and perspectives would see ways to improve things. This hasn't really happened with my other published projects I see the source code is forked and then nothing is done with it, nothing is contributed back. So I don't know what I want to do, on one hand you've all been awesome and given me these great ideas and I feel like that needs something back.
The best I can suggest is to publish and accept that people will take and not contribute back, settle for knowing you have contributed something to the world even if you receive no thanks in return. At least you will have an answer to "What good have you ever done?" and can be proud of that.

A big part of it is asking yourself whether you are doing something to get praise or doing it just to help others, even if recipients are not grateful enough to offer thanks.

You have to decide if you are altruistic or after something more. That will help guide your decision of what to do.

And the good thing is, once you are happy with whatever you decide, it will likely apply to every project you work on so you will rarely have to worry about what to do for those.

And you may even find some answers to the big questions; who am I, what am I, why am I here ?
DarkElvenAngel wrote:
Thu May 20, 2021 5:11 am
On the other I post my code and it sits there someone picks the meat off
the bones or complains that something isn't working or complete.
That's something one has to accept and the best way to deal with it is to adopt morally justified principles you can stand by.

You have a perfect right to say; "it is what it is, given away for free with no obligation to fix or improve".

You can settle for that, and say no more, learn to ignore those who pressure you to do what they want you to do.

Or you can propose that if anyone wants something they can pay for it, just as they would need to do if employing a developer or contractor to get what they want.

Accepting sponsorship for a project is an option but is most appropriate for projects you intend to fix and improve, are prepared to keep working on regularly. If not, it invites and legitimises sponsors putting pressure on you to do what they demand, hands them the moral high ground. It is better to negotiate any cost of work required on a case by case basis.

DarkElvenAngel
Posts: 1694
Joined: Tue Mar 20, 2018 9:53 pm

Re: Serial terminal project

Thu May 20, 2021 6:55 pm

Wise advice as always hippy,

I'm not in this for glory and fame, I do it for fun. So with that I guess I have to just hope that this project helps to inspire someone else as I was inspired to create it by another project.

With that out of the way I need a licence on my code so most of the time I use MIT however I was think of BSD 3-Clause License I don't know if it matters really between the two I just thought it sounds more in line with keeping the project open.

Once I figure that little bit out the code will go live on gitlab I have the repo created and initialized I'm happy with the naming scheme I've settled on. Then the grand plan will be reveled!

hippy
Posts: 9957
Joined: Fri Sep 09, 2011 10:34 pm
Location: UK

Re: Serial terminal project

Thu May 20, 2021 7:40 pm

MIT or BSD - I don't think it matters much. MIT is well known for being "do whatever you want with it", allowing commercial and proprietary use, where BSD may be less well known and, not being MIT, may give the first impression it is more restrictive than MIT.

BSD-2 is I believe the closest equivalent of MIT. I am personally not sure about the 3 and 4 clause versions as those move towards imposing obligations which is the wrong direction IMO, but I am sure there is merit for them.They aren't unreasonable nor onerous and I wouldn't call them objectionable.

The Pico-SDK is BSD-3, and I believe code should be BSD-3 to be accepted within the official eco-system if that were an intent. That would suggest BSD-3 as the best choice if C / Pico-SDK based.

MicroPython and CircuitPython are MIT so that would probably be the best choice there.

lars_the_bear
Posts: 203
Joined: Thu Jan 28, 2021 8:13 pm
Contact: Website

Re: Serial terminal project

Thu May 20, 2021 8:59 pm

DarkElvenAngel wrote:
Thu May 20, 2021 5:11 am
Keyboard mapping some keys do nothing and or the wrong thing ( I'm looking at you delete key who thinks its backspace )
My gut feeling is that there needs to be a measure of configurability here, both in the keyboard codes that are received, and the serial codes that are sent. BS/delete are a particular bear -- I don't think there's every been any general agreement on what these things should do. Most software (in my experience) that was designed to work with a terminal treats an ASCII backspace (8) as destructive, and yet expects the terminal it's sending data to to treat this character as non-destructive. I can't see any logic to that, or many of the other oddities in terminal behaviour.

For example, what should a terminal do if the cursor is in the first column, and it receives a backspace code? Nothing? Backspace into the previous line? The world of the serial terminal is full of things like this that nobody seems to agree on.

As for the more general issue of sharing source code -- yeah. Everything I write that isn't part of my commercial work, I post source code on GitHub. I do it in the hope that somebody will find it useful. I get quite a lot of helpful feedback, but I also get people complaining that it doesn't do this thing or that thing. My response to these folks is always the same: feel free to submit a PR.

Kevin.

DarkElvenAngel
Posts: 1694
Joined: Tue Mar 20, 2018 9:53 pm

Re: Serial terminal project

Fri May 21, 2021 1:43 am

Well the backspace is non destructive in the VT52 and delete does nothing it's ignored all together on the VT52. I believe I'm sending the correct keycode and that the way nano has decided it should behave.

I have more love for this project is over the others don't tell them I said that I'll deny everything one goal is to get some programs to run directly on it and not just a serial terminal have to see how it goes.

But for sure I feel like sharing my work will inspire someone else.

User avatar
Gavinmc42
Posts: 5677
Joined: Wed Aug 28, 2013 3:31 am

Re: Serial terminal project

Fri May 21, 2021 4:17 am

Enough room to port Notcurses?
https://github.com/dankamongmen/notcurses

A Pico display terminal maxed out?
Digital display signage on a budget?
If RS485 is used on an Uart then long distance could be done.

I think I need to get more Pico's.
I'm dancing on Rainbows.
Raspberries are not Apples or Oranges

DarkElvenAngel
Posts: 1694
Joined: Tue Mar 20, 2018 9:53 pm

Re: Serial terminal project

Fri May 21, 2021 2:25 pm

Gavinmc42 wrote:
Fri May 21, 2021 4:17 am
Enough room to port Notcurses?
https://github.com/dankamongmen/notcurses

A Pico display terminal maxed out?
Digital display signage on a budget?
If RS485 is used on an Uart then long distance could be done.

I think I need to get more Pico's.
This this could run on a Pi Zero and then use the PT52 as the terminal. I don't know if 16 colours is enough might need to wait for me to sort out 256 colour (spoiler I have just getting the code ready to publish is time consuming). Now I want to check this out but I will resist.

The font problem they mentioned would be interesting since the PT52 only has a 8bit character space and no Unicode support.

You could always make a custom font.

EvanEscent
Posts: 28
Joined: Thu Apr 01, 2021 10:45 pm
Location: Blighty
Contact: Website Yahoo Messenger Facebook Skype Twitter

Re: Serial terminal project

Tue May 25, 2021 6:02 pm

The font problem they mentioned would be interesting since the PT52 only has a 8bit character space and no Unicode support.
It's looking more and more amazing. I imagine there are a lot of people
with twitching baited fingers ...

Not intending to further push this but does your code base so far
have any potential for Sixels?

This would very much be a cherry-on-the-cake feature but it would make
your terminal a truly wide-ranging appliance.
I read, I tinker, I test, I repeat.
Sometimes I learn.

DarkElvenAngel
Posts: 1694
Joined: Tue Mar 20, 2018 9:53 pm

Re: Serial terminal project

Tue May 25, 2021 7:27 pm

EvanEscent wrote:
The font problem they mentioned would be interesting since the PT52 only has a 8bit character space and no Unicode support.
It's looking more and more amazing. I imagine there are a lot of people
with twitching baited fingers ...

Not intending to further push this but does your code base so far
have any potential for Sixels?

This would very much be a cherry-on-the-cake feature but it would make
your terminal a truly wide-ranging appliance.
I hadn't heard of Sixels until you just mentioned.
More to come
I'm still working to improve this project and I hope you find it useful and or educational.
Looking to add some menu support to access things like serial setting or run little games or programs. Some character RAM for custom characters, this could be used for simple animations.
When the screen isn't fully in use with the text buffer status or UI bars in the unused areas.
And more if it will fit!


An except from README.md

I guess we need to add sixels to the list!

The code is in a repo there are some errors in the README.md I need to fix I got someone to proof read it. so I need to fix that. looks like I have the 256 colour mode already to go.
There are issues with the keyboard keys but I'm going to fix the readme and put the repo live first! should be soon.

DarkElvenAngel
Posts: 1694
Joined: Tue Mar 20, 2018 9:53 pm

Re: Serial terminal project

Tue May 25, 2021 8:01 pm

CODE IS RELEASED!!!

Find it HERE!


Please if you find this useful and make improvements post and share them!

There are many TODOs but this works as it is so enjoy!

256 Colour mode is now available.

Return to “General”