shacharshemesh
Posts: 15
Joined: Wed Dec 11, 2019 7:34 am
Contact: Website

Writing an OS from scratch

Fri Apr 16, 2021 1:43 pm

Hi all,

For educational reasons, I would like to write an OS from scratch for the RaspberryPi 4. I found a bunch of resources, but there are a bunch of resources that I feel I should have found and didn't. Any help would be appreciated.

The most important resource, I believe, is that BCM2711 data sheet, which I found at https://www.raspberrypi.org/documentati ... 11_1p0.pdf.

While it contains a lot of what I'd need, I feel there is stuff that should have been there but isn't. For example, I know the SoC has an ethernet MAC, but it does not appear to be documented anywhere. I kind of picked up that it is PCI connected, but then I expected to see a PCI controller, which is not documented either. I'm sure I'm missing something basic, but I'm not sure what.

Something else I'm used to SoCs datasheets containing but this one doesn't, is the pin-out of the chip. This is essential for understanding pin multiplexing and reading the board schematics, but does not appear to be present in the datasheet.

Which brings me to the board schematics. I found something that purports to be it at https://www.raspberrypi.org/documentati ... educed.pdf. It does not, however, seem to have the actual BCM2711 chip on it. Worse, when I followed the Ethernet port, I see references to a line called "TRD0_P". That line appears only once in the schematics, implying that it is not connected to anything. That label also doesn't appear in the BCM2711 datasheet.

Last, but by no means least, there is the question of hooking the boot process. It is not clear to me where is the right place to stop the normal RP4 boot process into Linux and start running my own code. The eeprom and start.elf files don't seem to be it. Do I need to write my code as "vmlinuz"? If so (and I'm sorry if I'm asking something trivial here), what state is the chip when I get control? Am I running 32 or 64 bit? Is there any hardware I should assume is already initialized?

Thank you for any help you may offer,
Shachar
Likes to engage in lost causes and doomed projects.
Author and developer of the Practical programming language (https://practical-pl.org)

The longer you spend arguing online with a fool, the greater the chances he's doing the same thing.

fruitoftheloom
Posts: 26564
Joined: Tue Mar 25, 2014 12:40 pm
Location: Delightful Dorset

Re: Writing an OS from scratch

Fri Apr 16, 2021 2:15 pm

shacharshemesh wrote:
Fri Apr 16, 2021 1:43 pm
Hi all,

For educational reasons, I would like to write an OS from scratch for the RaspberryPi 4. I found a bunch of resources, but there are a bunch of resources that I feel I should have found and didn't. Any help would be appreciated.

The most important resource, I believe, is that BCM2711 data sheet, which I found at https://www.raspberrypi.org/documentati ... 11_1p0.pdf.

While it contains a lot of what I'd need, I feel there is stuff that should have been there but isn't. For example, I know the SoC has an ethernet MAC, but it does not appear to be documented anywhere. I kind of picked up that it is PCI connected, but then I expected to see a PCI controller, which is not documented either. I'm sure I'm missing something basic, but I'm not sure what.

Something else I'm used to SoCs datasheets containing but this one doesn't, is the pin-out of the chip. This is essential for understanding pin multiplexing and reading the board schematics, but does not appear to be present in the datasheet.

Which brings me to the board schematics. I found something that purports to be it at https://www.raspberrypi.org/documentati ... educed.pdf. It does not, however, seem to have the actual BCM2711 chip on it. Worse, when I followed the Ethernet port, I see references to a line called "TRD0_P". That line appears only once in the schematics, implying that it is not connected to anything. That label also doesn't appear in the BCM2711 datasheet.

Last, but by no means least, there is the question of hooking the boot process. It is not clear to me where is the right place to stop the normal RP4 boot process into Linux and start running my own code. The eeprom and start.elf files don't seem to be it. Do I need to write my code as "vmlinuz"? If so (and I'm sorry if I'm asking something trivial here), what state is the chip when I get control? Am I running 32 or 64 bit? Is there any hardware I should assume is already initialized?

Thank you for any help you may offer,
Shachar

https://intestinate.com/pilfs/
The information is out there....you just have to let it in.

My other Linux machines: ChromeBox
https://www.aliexpress.com/item/32966393971.html
& Stone Desktop Intel CoreDuo circa 2010

cleverca22
Posts: 3610
Joined: Sat Aug 18, 2012 2:33 pm

Re: Writing an OS from scratch

Fri Apr 16, 2021 3:34 pm

shacharshemesh wrote:
Fri Apr 16, 2021 1:43 pm
For example, I know the SoC has an ethernet MAC, but it does not appear to be documented anywhere. I kind of picked up that it is PCI connected, but then I expected to see a PCI controller, which is not documented either. I'm sure I'm missing something basic, but I'm not sure what.
the ethernet is handled via a genet controller, it is not part of the pci bus, its just directly in the address space

the linux source is under https://github.com/raspberrypi/linux/tr ... dcom/genet
the UEFI source is under https://github.com/tianocore/edk2-platf ... -pi4-genet
the windows source is under https://github.com/raspberrypi/windows- ... 1/bcmgenet

reading the UEFI/windows driver may help, because its targeting exactly what is on the rpi4, while linux is targeting any genet based device

the pci-e controller is only used to talk to the vl805 pcie<->usb controller (which is xhci based)
but there is also another usb port, on the usb-c jack, controlled by either the dwc2 (follow the old tutorials) or a broadcom xhci (add otg_mode=1 to config.txt)
you will need to respect the device-tree the firmware passes you, to know which usb controllers can be used

User avatar
DougieLawson
Posts: 41204
Joined: Sun Jun 16, 2013 11:19 pm
Location: A small cave in deepest darkest Basingstoke, UK
Contact: Website Twitter

Re: Writing an OS from scratch

Fri Apr 16, 2021 4:05 pm

Any language using left-hand whitespace for syntax is ridiculous

Any DMs sent on Twitter will be answered next month.
Fake doctors - are all on my foes list.

Any requirement to use a crystal ball or mind reading will result in me ignoring your question.

shacharshemesh
Posts: 15
Joined: Wed Dec 11, 2019 7:34 am
Contact: Website

Re: Writing an OS from scratch

Fri Apr 16, 2021 7:46 pm

Just in case anyone else is in the same boat as me: none of the above links bear any relation to the question asked. They are generic "how to build an operating system" resources. Many of the statements there even assume building on X86, which I, obviously, do not.

I guess Dougie meant well? Maybe? If so, maybe not start from the page Dougie linked to, which kinda tries to discourage you from even trying.
Likes to engage in lost causes and doomed projects.
Author and developer of the Practical programming language (https://practical-pl.org)

The longer you spend arguing online with a fool, the greater the chances he's doing the same thing.

User avatar
DougieLawson
Posts: 41204
Joined: Sun Jun 16, 2013 11:19 pm
Location: A small cave in deepest darkest Basingstoke, UK
Contact: Website Twitter

Re: Writing an OS from scratch

Fri Apr 16, 2021 8:15 pm

shacharshemesh wrote:
Fri Apr 16, 2021 7:46 pm
I guess Dougie meant well? Maybe? If so, maybe not start from the page Dougie linked to, which kinda tries to discourage you from even trying.
I really meant don't waste your time (and by dint of that don't waste the time of folks on this forum). I have forty+ years of programming (C/C++, COBOL, PL/I, Java, Perl, REXX, Python, Javascript), including countless years writing assembler on Zilog Z80, MOS 6502, IBM S/370 (& IBM zSeries) and RPi ARM6. I wouldn't even dream of trying to write an OS - there's no money in it.
Any language using left-hand whitespace for syntax is ridiculous

Any DMs sent on Twitter will be answered next month.
Fake doctors - are all on my foes list.

Any requirement to use a crystal ball or mind reading will result in me ignoring your question.

shacharshemesh
Posts: 15
Joined: Wed Dec 11, 2019 7:34 am
Contact: Website

Re: Writing an OS from scratch

Fri Apr 16, 2021 8:21 pm

cleverca22 wrote:
Fri Apr 16, 2021 3:34 pm
shacharshemesh wrote:
Fri Apr 16, 2021 1:43 pm
For example, I know the SoC has an ethernet MAC, but it does not appear to be documented anywhere. I kind of picked up that it is PCI connected, but then I expected to see a PCI controller, which is not documented either. I'm sure I'm missing something basic, but I'm not sure what.
the ethernet is handled via a genet controller, it is not part of the pci bus, its just directly in the address space

the linux source is under https://github.com/raspberrypi/linux/tr ... dcom/genet
the UEFI source is under https://github.com/tianocore/edk2-platf ... -pi4-genet
the windows source is under https://github.com/raspberrypi/windows- ... 1/bcmgenet

reading the UEFI/windows driver may help, because its targeting exactly what is on the rpi4, while linux is targeting any genet based device

the pci-e controller is only used to talk to the vl805 pcie<->usb controller (which is xhci based)
but there is also another usb port, on the usb-c jack, controlled by either the dwc2 (follow the old tutorials) or a broadcom xhci (add otg_mode=1 to config.txt)
you will need to respect the device-tree the firmware passes you, to know which usb controllers can be used
Okay, that is very useful. Thank you. Specifically, the mere knowledge of the repo already pushes me further along.

I have to admit, though, I am disappointed by the level of documentation available. Having to reverse-engineer how to operate the network from code, rather than having a datasheet that tells you what each register does and how to access it, is not what I expected. I understand that the BCM2711 is only used by PI, but to have a whole component that simply has no mention in the official SoC datasheet (it does state it gets interrupts 29 and 30, but otherwise a text search for GENET returns no results) makes the potential platform choice questionable.

I picked the pi because it is easily available to me and to my potential audience. One of the purposes of the exercise, however, is to build a tutorial that shows how OS development turns datasheets into driver code. That's hard to do if I cannot point my potential audience toward a datasheet they can follow along with.

Too see what I expected to find here, check out the BananaPi equivalent page:
http://wiki.banana-pi.org/Banana_Pi_BPI-M64#documents

You get board schematics that actually include the components on the board. You get a SoC datasheet that includes the pin out (so you can cross reference that with the board's schematics), as well the registers and access to all in SoC components. I expected something similar here, except it seems to be available from neither raspberrypi web site nor the Broadcom one.

Am I missing something?
Likes to engage in lost causes and doomed projects.
Author and developer of the Practical programming language (https://practical-pl.org)

The longer you spend arguing online with a fool, the greater the chances he's doing the same thing.

User avatar
jahboater
Posts: 6949
Joined: Wed Feb 04, 2015 6:38 pm
Location: Wonderful West Dorset

Re: Writing an OS from scratch

Fri Apr 16, 2021 9:22 pm

DougieLawson wrote:
Fri Apr 16, 2021 8:15 pm
I wouldn't even dream of trying to write an OS - there's no money in it.
No money, but it could be fun and educational!
I bet the first time something you have written boots and runs is a high spot in your life.

There is also the possibility of writing something better than the popular options today:-

A text editor - easy
A compiler or interpreter - possible, but likely better to extend a compiler "collection" such as GCC.
An operating system - no, something like Linux has had thousands of developers working on it for years.

timrowledge
Posts: 1373
Joined: Mon Oct 29, 2012 8:12 pm
Location: Vancouver Island
Contact: Website

Re: Writing an OS from scratch

Fri Apr 16, 2021 11:06 pm

If you’re actually writing an OS from scratch you are likely to be at least a year from actually caring even peripherally about peripherals. What is this OS going to do? How? How do you intend to handle memory?Loading & starting programs? Dealing with errors? How will you produce code to run?

By the time you might be needing to care about cpu pinouts the hardware could be very different to the current state. Been there, done that. It’s extremely hard work. Not going to do it again, thank you.
Making Smalltalk on ARM since 1986; making your Scratch better since 2012

User avatar
Gavinmc42
Posts: 5490
Joined: Wed Aug 28, 2013 3:31 am

Re: Writing an OS from scratch

Fri Apr 16, 2021 11:36 pm

Have you looked in the Baremetal posts?
viewforum.php?f=72

There is no complete documentation for the Pi chips.
Pick another SoC if you want bigger datasheets.

It also depends on what you means by OS.
I use Ultibo, which is more like an OS construction kit.
No need for me to even look at register level code.

But Ultibo is not released yet for the Pi4.
Lots of stuff has to be "reverse" engineered from the Linux open source code.
VideoCore 6, Ethernet, USB etc are different from previous Pi's.

Learning how to write an OS is not the same as writing one on a Pi4.
Why not do it on the Pico?
It has more documentation and even now examples of RTOS/Fuzix.
I'm dancing on Rainbows.
Raspberries are not Apples or Oranges

shacharshemesh
Posts: 15
Joined: Wed Dec 11, 2019 7:34 am
Contact: Website

Re: Writing an OS from scratch

Sat Apr 17, 2021 5:01 am

jahboater wrote:
Fri Apr 16, 2021 9:22 pm
No money, but it could be fun and educational!
I bet the first time something you have written boots and runs is a high spot in your life.

There is also the possibility of writing something better than the popular options today:-

A text editor - easy
A compiler or interpreter - possible, but likely better to extend a compiler "collection" such as GCC.
An operating system - no, something like Linux has had thousands of developers working on it for years.
I wouldn't write a new compiler. At least, not without some very good reason. Sadly, there is no other way to create a new programming language other than to write the compiler for it, so I had no choice:
https://github.com/practical

What I really don't understand is people seeing someone else wanting to do something ambitious, and responding with automatic discouragements. My best guess is that Dougie wishes to write an OS from scratch, but since he doesn't, it seems logical to him that no one else should either.
Likes to engage in lost causes and doomed projects.
Author and developer of the Practical programming language (https://practical-pl.org)

The longer you spend arguing online with a fool, the greater the chances he's doing the same thing.

shacharshemesh
Posts: 15
Joined: Wed Dec 11, 2019 7:34 am
Contact: Website

Re: Writing an OS from scratch

Sat Apr 17, 2021 5:20 am

timrowledge wrote:
Fri Apr 16, 2021 11:06 pm
If you’re actually writing an OS from scratch you are likely to be at least a year from actually caring even peripherally about peripherals.
Your experience is vastly different than mine. The very first thing you want, after getting the hardware to run your code, is to start jiggling GPIOs so you can externalize what your code is doing. Right next, probably, is getting a UART under control, for pretty much the same reason.

Once multi processing and timers start getting involved, single-step debugging simply stops being useful, assuming it even still works.
timrowledge wrote:
Fri Apr 16, 2021 11:06 pm
What is this OS going to do? How? How do you intend to handle memory?Loading & starting programs? Dealing with errors? How will you produce code to run?
The purpose of this OS is to show how operating systems are built. There is no intention of making it complete by any stretch. Generally speaking, anything that can be shown on Linux in user space, I'll probably show on Linux. By the time we get to loading programs, it is likely I'll switch.

So, as far as plans now go, if my most ambitious plans pan out, this OS will:
  • Boot
  • Some peripheral support. Blinken leds and UART almost certainly. We'll see about anything else.
  • Run programs from ROM (or otherwise precompiled with the OS itself).
  • Have a user space, kernel space and system calls
  • Timers
  • Preemptive multi-tasking.
  • Virtual address space. Maybe virtual memory, but that depends on whether I'll enable external storage.
  • I'd like to show working with interrupts/DMA combination, so probably networking at UDP levels.
timrowledge wrote:
Fri Apr 16, 2021 11:06 pm
By the time you might be needing to care about cpu pinouts the hardware could be very different to the current state. Been there, done that. It’s extremely hard work. Not going to do it again, thank you.
That's not how it works. It sounds like you were working on writing software for hardware as it was being developed. That's very different than my case. From my experience, getting any of your code to run shouldn't take more than a couple of days, and that's assuming the manufacturer didn't provide sources that you can just copy that do the initialization.

Since I'm not even building the board, the pins are actually important at the early stages, when you either want a jtag in there, or want to connect a logic to see whether you're jiggling the right things. Once the peripherals are in place, dumping stuff over the UART is easier to see whether you've hit your marks.

You actually need the pinouts at the earlier stages, not the later ones. That's why the BCM2711 can get away with not having them: they are assuming everyone are using Linux.
Likes to engage in lost causes and doomed projects.
Author and developer of the Practical programming language (https://practical-pl.org)

The longer you spend arguing online with a fool, the greater the chances he's doing the same thing.

shacharshemesh
Posts: 15
Joined: Wed Dec 11, 2019 7:34 am
Contact: Website

Re: Writing an OS from scratch

Sat Apr 17, 2021 6:11 am

Gavinmc42 wrote:
Fri Apr 16, 2021 11:36 pm
Have you looked in the Baremetal posts?
viewforum.php?f=72

There is no complete documentation for the Pi chips.
Pick another SoC if you want bigger datasheets.

It also depends on what you means by OS.
I use Ultibo, which is more like an OS construction kit.
No need for me to even look at register level code.

But Ultibo is not released yet for the Pi4.
Lots of stuff has to be "reverse" engineered from the Linux open source code.
VideoCore 6, Ethernet, USB etc are different from previous Pi's.

Learning how to write an OS is not the same as writing one on a Pi4.
Why not do it on the Pico?
It has more documentation and even now examples of RTOS/Fuzix.
Using a tool that auto-generates the OS is counter-productive to what I want to do: create a tutorial on how operating systems are built.

I had a brief look at the Pico, and it may actually tick a lot of what I'm trying to do. It's definitely better documented than the rest of them. The Omega2 also piqued my interest, in that developing this on a breadboard is a nice touch.

Thank you. That has been most helpful.

One thing I'll have to give up from my grandiose plans is that I was hoping of demonstrating working with an MMU, which the pico doesn't have.
Likes to engage in lost causes and doomed projects.
Author and developer of the Practical programming language (https://practical-pl.org)

The longer you spend arguing online with a fool, the greater the chances he's doing the same thing.

User avatar
Gavinmc42
Posts: 5490
Joined: Wed Aug 28, 2013 3:31 am

Re: Writing an OS from scratch

Sat Apr 17, 2021 8:16 am

I would be interested in an OS tut on Pico.
Just ordered my second one as it can be a programmer/debugger for the first one.

No MMU can be a pain, I think that's why uLib was made?
Goes back to 68000 days?

I have been impressed with the Pico so far, 2 cores plus two tiny PIO "ALUs".
Much better than my first 16MHz 68000 computer that ran Minix.

Intel chips run Minix 3 internally?
https://en.wikipedia.org/wiki/Minix#Minix_3
I'm dancing on Rainbows.
Raspberries are not Apples or Oranges

LTolledo
Posts: 5155
Joined: Sat Mar 17, 2018 7:29 am
Location: Anime Heartland

Re: Writing an OS from scratch

Sat Apr 17, 2021 12:53 pm

the first thing that came to my mind when I saw the title....

"....making a new OS using Scratch....." ? :shock:
(or Scratch 2 or 3) ;)

well.... if its possible using whatever tools....then why not....
its the OP's project.... the OP will bear all the hardships (and rewards) in making one.....

I just hope the OP makes it in time before RPi9 comes out... :mrgreen:
"Don't come to me with 'issues' for I don't know how to deal with those
Come to me with 'problems' and I'll help you find solutions"

Some people be like:
"Help me! Am drowning! But dont you dare touch me nor come near me!"

ejolson
Posts: 7114
Joined: Tue Mar 18, 2014 11:47 am

Re: Writing an OS from scratch

Sat Apr 17, 2021 3:20 pm

shacharshemesh wrote:
Sat Apr 17, 2021 5:20 am
The purpose of this OS is to show how operating systems are built. There is no intention of making it complete by any stretch. Generally speaking, anything that can be shown on Linux in user space, I'll probably show on Linux. By the time we get to loading programs, it is likely I'll switch.
Could it be you are writing a book about writing an operating system, rather than just writing an operating system?

It does seem teaching batch after batch of students the reading skills needed to convert a datasheet into a working Ethernet driver, for example, is something that could benefit the industry as a whole. Such things are not part of even the most practical courses on operating systems at the universities I know about.

My friend used to say a difficult problem is easy after you know the solution. All the people who think this stuff is too difficult for anyone would not think so if they knew well how to do it themselves.

Of course it is difficult (and pointless) to teach how to read something that doesn't exist or is too expensive to obtain. From this point of view, one can understand why MIT used a PDP11 emulator for their operating systems class in the past. The documents needed for writing an Ethernet driver or configuring the MMU on a PDP11 are available and possible to read. One problem was pushback from students for the architecture being so old.

For some reason the educational focus of the Pi shifted away from publishing the drivers, schematics and documentation for people to to read and learn from. I think a practical technical focus was the most original thing about the Pi and that this focus needs to be regained.

My understanding is that the people who grew up and mastered computers when the learning curve was much flatter are getting old and that essential skills such as how to read a datasheet have not been passed on, at least in many societies. This technical know-how needs to be conveyed to the next generation if one wants to maintain current levels of prosperity or improve them.

User avatar
Gavinmc42
Posts: 5490
Joined: Wed Aug 28, 2013 3:31 am

Re: Writing an OS from scratch

Sun Apr 18, 2021 12:16 am

20 years ago I was reading the datasheet of a microcontroller.
Had an epiphany on a weird way to use one feature of the chip.
That led to a company start up and that company still exists and still makes product based on that idea.
I have yet to find a competing product that's got better performance on battery power.
New tech is only now catching up.

Cannot do that with the Pi SoC so much.
2000+ pages on just the Arm part.....
Much easier with those 600+ pages of the Pico?

I am happy enough now with the Pi4/400 that it is perfectly able to write/develop the next OS for a Pico.
Not convinced there is enough data yet to do that on a Pi by an average user.
Those guys in the Baremetal forum are not average.

Uni "Kids" these days just googoo and say it cannot be done because they found no way to do the software problem.
Someone needs to show them ;)
I'm dancing on Rainbows.
Raspberries are not Apples or Oranges

shacharshemesh
Posts: 15
Joined: Wed Dec 11, 2019 7:34 am
Contact: Website

Re: Writing an OS from scratch

Sun Apr 18, 2021 3:48 am

ejolson wrote:
Sat Apr 17, 2021 3:20 pm
Could it be you are writing a book about writing an operating system, rather than just writing an operating system?
Books today are called "Youtube channels".

There is no way I'm undertaking another book. I have 120 pages of (sometimes very) raw draft of a book. I used to have a half hour train commute to work where I would write it. Switch jobs, no chance I can master the time commitment any more.

But the beauty of a Youtube channel is that you don't have to complete the whole thing to publish.
ejolson wrote:
Sat Apr 17, 2021 3:20 pm
Of course it is difficult (and pointless) to teach how to read something that doesn't exist or is too expensive to obtain. From this point of view, one can understand why MIT used a PDP11 emulator for their operating systems class in the past. The documents needed for writing an Ethernet driver or configuring the MMU on a PDP11 are available and possible to read. One problem was pushback from students for the architecture being so old.
My Uni taught machine language on the PDP11 (circa 1997). I didn't understand why at the beginning, but the machine language itself is modern in architecture (by which I mean lots of registers covering the full address space) without having the.... idiosyncrasies of the Intel platform, and also not the insane cross-command dependencies of RISC. The 68000 was the only CPU I could offer as an alternative, and by 1997 it was no longer considered modern itself.

I was not aware that PDP11 had an MMU, and neither is Wikipedia. Are you sure it wasn't something added by the emulator?
ejolson wrote:
Sat Apr 17, 2021 3:20 pm
For some reason the educational focus of the Pi shifted away from publishing the drivers, schematics and documentation for people to to read and learn from. I think a practical technical focus was the most original thing about the Pi and that this focus needs to be regained.

My understanding is that the people who grew up and mastered computers when the learning curve was much flatter are getting old and that essential skills such as how to read a datasheet have not been passed on, at least in many societies. This technical know-how needs to be conveyed to the next generation if one wants to maintain current levels of prosperity or improve them.
Like I said above, I will not be doing this effort on the Pi. It's a good platform, and highly available, but simply not suited for this purpose. Even if I manage to tease out how to do things, I cannot ask anyone trying to follow along to do the same. I'm hoping the Omega2 will prove a better fit. At $16 per board (though no physical Ethernet port at that price range), the price, at least, is right.
Likes to engage in lost causes and doomed projects.
Author and developer of the Practical programming language (https://practical-pl.org)

The longer you spend arguing online with a fool, the greater the chances he's doing the same thing.

ejolson
Posts: 7114
Joined: Tue Mar 18, 2014 11:47 am

Re: Writing an OS from scratch

Sun Apr 18, 2021 5:10 am

shacharshemesh wrote:
Sun Apr 18, 2021 3:48 am
My Uni taught machine language on the PDP11 (circa 1997). I didn't understand why at the beginning, but the machine language itself is modern in architecture (by which I mean lots of registers covering the full address space) without having the.... idiosyncrasies of the Intel platform, and also not the insane cross-command dependencies of RISC. The 68000 was the only CPU I could offer as an alternative, and by 1997 it was no longer considered modern itself.

I was not aware that PDP11 had an MMU, and neither is Wikipedia. Are you sure it wasn't something added by the emulator?
A simple description of memory management on a PDP-11 is given at

https://gunkies.org/wiki/PDP-11_Memory_Management

As Unix evolved into a robust multi-user time-sharing system on the PDP-11, it is not surprising that the hardware provides different privilege levels with separate memory mappings.
Last edited by ejolson on Sun Apr 18, 2021 4:08 pm, edited 1 time in total.

JamesPi123
Posts: 121
Joined: Fri Sep 23, 2016 10:02 pm
Location: Inside my Pi
Contact: Website

Re: Writing an OS from scratch

Sun Apr 18, 2021 8:04 am

Not trying to be discouraging, but I agree with Dougie. Writing an OS is incredibly complex and time consuming. Most operating systems you know like Linux and BSD were not made by one person. That one person wrote a small, but good piece of code that others liked and improved. Not to mention the complexity of modern computers compared to what they had back in the day. Write baremetal projects? Sure. Write a whole OS for a relatively undocumented single board computer? I don't think it is a good use of time or effort.

shacharshemesh
Posts: 15
Joined: Wed Dec 11, 2019 7:34 am
Contact: Website

Re: Writing an OS from scratch

Sun Apr 18, 2021 8:50 am

There are a couple of things I'm genuinely curious about.

First, even assuming you missed the part about this being an educational project, I still have to wonder something. What's it to you? Why do you care how I spend my time? I've had someone come to a mailing list of a project I started and written from scratch, and start berating me for picking C++, claiming he'll now have to re-write the project in C. My reply to him was "good luck. Let me know when you have something working so I can link to it from my project".

Yes, he was delusional about the amount of effort it took to get to where my project was. No, it was not my job to disillusion him.

The second has to do with this statement you made:
JamesPi123 wrote:
Sun Apr 18, 2021 8:04 am
Not trying to be discouraging, but I agree with Dougie.
I missed how that restraint you put on yourself changed the final outcome. For educational purposes, I hereby release you from it. Please do phrase this message as discouragingly as you please, so I can see the difference.
Likes to engage in lost causes and doomed projects.
Author and developer of the Practical programming language (https://practical-pl.org)

The longer you spend arguing online with a fool, the greater the chances he's doing the same thing.

fruitoftheloom
Posts: 26564
Joined: Tue Mar 25, 2014 12:40 pm
Location: Delightful Dorset

Re: Writing an OS from scratch

Sun Apr 18, 2021 9:03 am

shacharshemesh wrote:
Sun Apr 18, 2021 8:50 am
There are a couple of things I'm genuinely curious about.

First, even assuming you missed the part about this being an educational project, I still have to wonder something. What's it to you? Why do you care how I spend my time? I've had someone come to a mailing list of a project I started and written from scratch, and start berating me for picking C++, claiming he'll now have to re-write the project in C. My reply to him was "good luck. Let me know when you have something working so I can link to it from my project".

Yes, he was delusional about the amount of effort it took to get to where my project was. No, it was not my job to disillusion him.

The second has to do with this statement you made:
JamesPi123 wrote:
Sun Apr 18, 2021 8:04 am
Not trying to be discouraging, but I agree with Dougie.
I missed how that restraint you put on yourself changed the final outcome. For educational purposes, I hereby release you from it. Please do phrase this message as discouragingly as you please, so I can see the difference.


Just start with the Linux from Scratch project, it is the easiest way to get a basic understanding of creating an Operating System, that way you can make an informed decision whether or not it meets your need to learn ;)
The information is out there....you just have to let it in.

My other Linux machines: ChromeBox
https://www.aliexpress.com/item/32966393971.html
& Stone Desktop Intel CoreDuo circa 2010

manu2007
Posts: 24
Joined: Fri Apr 02, 2021 1:13 pm

Re: Writing an OS from scratch

Sun Apr 18, 2021 9:10 am

I think I would just ignore the discouraging BS and I would study Minix :

https://github.com/gdevic/minix1

User avatar
DougieLawson
Posts: 41204
Joined: Sun Jun 16, 2013 11:19 pm
Location: A small cave in deepest darkest Basingstoke, UK
Contact: Website Twitter

Re: Writing an OS from scratch

Sun Apr 18, 2021 9:33 am

manu2007 wrote:
Sun Apr 18, 2021 9:10 am
I think I would just ignore the discouraging BS and I would study Minix :

https://github.com/gdevic/minix1
You've completely missed the whole point.

If the OP builds with gcc on a Linux or Minix base that isn't an OS built "from scratch". That's a derivative work. Any effort done on that would be better spent looking at ways to improve Linux.

To me "from scratch" means you have a piece of hardware that will start running instructions from a fixed address (determined by the hardware or the boot loader or the switches on the front panel) and nothing more. If you do nothing it branches to, for example, 0x4000 and does nothing more (until you load a "kernel" at 0x4000) it's ready to run machine code. You're not allowed to use someone else's compiler, if you need a compiler you have to start by writing an assembler. To write an assembler you'd have to hand assemble it into machine code. Once you have a compiler running you can use that to compile a better compiler (recursively until it turns into gcc or clang). That's the stuff the Bell Labs and AT&T folks did to get Unix running.

That, of course, is entirely unlike Linus' first attempts at Linux which were derived from Minix. So even the world's superstar OS creator didn't build "from scratch". He was basically stating that some bits of Minix work OK, some bit are junk, so I've rewritten the junk pieces.

I still think the OP is wasting his time.
Any language using left-hand whitespace for syntax is ridiculous

Any DMs sent on Twitter will be answered next month.
Fake doctors - are all on my foes list.

Any requirement to use a crystal ball or mind reading will result in me ignoring your question.

shacharshemesh
Posts: 15
Joined: Wed Dec 11, 2019 7:34 am
Contact: Website

Re: Writing an OS from scratch

Sun Apr 18, 2021 10:02 am

DougieLawson wrote:
Sun Apr 18, 2021 9:33 am
To me "from scratch" means you have a piece of hardware that will start running instructions from a fixed address (determined by the hardware or the boot loader or the switches on the front panel) and nothing more. If you do nothing it branches to, for example, 0x4000 and does nothing more (until you load a "kernel" at 0x4000) it's ready to run machine code. You're not allowed to use someone else's compiler, if you need a compiler you have to start by writing an assembler. To write an assembler you'd have to hand assemble it into machine code. Once you have a compiler running you can use that to compile a better compiler (recursively until it turns into gcc or clang). That's the stuff the Bell Labs and AT&T folks did to get Unix running.
You're not starting from scratch unless you build the CPU from transistors.

Hell, how did the transistors get there? You need your own clean room and damping the silicon.

What am I talking about? Silicon????? Where did that come from? Better starting gluing those protons and electrons.

https://www.youtube.com/watch?v=7s664NsLeFM

Or, you can acknowledge that I'm only after creating the operating system from scratch. Not the hardware. Not the compiler. Just the operating system.
DougieLawson wrote:
Sun Apr 18, 2021 9:33 am
I still think the OP is wasting his time.
There is no contest that someone is wasting my time.
Likes to engage in lost causes and doomed projects.
Author and developer of the Practical programming language (https://practical-pl.org)

The longer you spend arguing online with a fool, the greater the chances he's doing the same thing.

Return to “Advanced users”