Heater
Posts: 12969
Joined: Tue Jul 17, 2012 3:02 pm

Re: "Nothing ever works"

Wed Oct 31, 2018 9:59 pm

Oh boy, endianness is a whole other can of worms.

If your C/C++ code, or whatever language, is making assumptions about the order in which the bytes that make up your integers and such are stored in memory then it is relying on undefined behavior of the language. Anything can happen. In short, it's badly written code. As I had the pleasure of finding out back in the day when trying to port code written for Intel to Motorola 68000 machines.

Luckily that is not such a big problem today, people learned how to write endianness independent code.

Much the same could be said of 32 bit vs 64 bit code. A coder should not be making assumptions about the size of an int or the size of a pointer etc.

Again, that is not such a big deal today. Which is why we have so much software that runs fine on 32 or 64 bit machines.

After we have all that out of the way, we still have a huge diversity in platforms when it comes to kernel versions, library versions, configurations, package managers, desktops and so on. It's amazing anything ever works. But it does...

There is a reason why IBM just paid 34 billion dollars for RedHat. RedHat does a lot or work in making all this mess work and supporting it.

The Pi Foundation and Raspbian don't have that much resources to apply to the problem.

ejolson
Posts: 3241
Joined: Tue Mar 18, 2014 11:47 am

Re: "Nothing ever works"

Thu Nov 01, 2018 4:17 pm

Heater wrote:
Wed Oct 31, 2018 9:59 pm
Luckily that is not such a big problem today, people learned how to write endianness independent code.
You might be right. Even the binary log files written by systemd seem to be endian independent. The source code is sprinkled with endian conversions such as

Code: Select all

        h.header_size = htole64(ALIGN64(sizeof(h)));

        h.incompatible_flags |= htole32(
                f->compress_xz * HEADER_INCOMPATIBLE_COMPRESSED_XZ |
                f->compress_lz4 * HEADER_INCOMPATIBLE_COMPRESSED_LZ4);

        h.compatible_flags = htole32(
                f->seal * HEADER_COMPATIBLE_SEALED);
throughout (though mixing integer multiplies with a logical or feels a bit weird to me). One can see why the original Unix developers used text files wherever possible. It seems big endian used to be more popular but now little endian is more popular. Similarly 32-bit used to be more popular, but now 64-bit is more popular (with the exception of Raspberry Pi). I also find it amazing when anything ever does work, especially when I see how complicated simple things have become.

hippy
Posts: 5595
Joined: Fri Sep 09, 2011 10:34 pm
Location: UK

Re: "Nothing ever works"

Thu Nov 01, 2018 4:41 pm

ejolson wrote:
Thu Nov 01, 2018 4:17 pm
I also find it amazing when anything ever does work.
As Heater suggests; one really has to go out of one's way to create something which is so platform dependent that it won't work, can't work, on other platforms. Particularly so when using languages which are intended to be endian and platform agnostic, are designed to overcome such problems.

And, as I keep saying, in most cases that's not the problem. It's not that the programmer has done something to deliberately or inadvertently tie themselves to particular platforms or excluded themselves from others.

I obviously go against the grain: I don't find it surprising that anything works; I find it surprising when it doesn't. Well crafted code written for one thing on one platform should just work without problem with that same thing on another platform. If it wasn't well crafted code I would not be surprised if it didn't. But that's not what we are talking about.

ejolson
Posts: 3241
Joined: Tue Mar 18, 2014 11:47 am

Re: "Nothing ever works"

Thu Nov 01, 2018 5:43 pm

hippy wrote:
Thu Nov 01, 2018 4:41 pm
ejolson wrote:
Thu Nov 01, 2018 4:17 pm
I also find it amazing when anything ever does work.
As Heater suggests; one really has to go out of one's way to create something which is so platform dependent that it won't work, can't work, on other platforms. Particularly so when using languages which are intended to be endian and platform agnostic, are designed to overcome such problems.
As a simplification to the variety of typed fixed-record random-access files common on earlier operating systems, Unix and subsequently Linux as well as Microsoft Windows view files as a collection of bytes. Any programming language that has to read or write such files immediately runs into architectural issues such as the machine word size and endianness. While there are programming languages which don't do such files, many useful applications need to. As a result, unless your program only works with text files, it will have to consider architectural issues when reading and writing files on Linux or Windows. Even text files encoded using UTF-16 or UTF-32 are affected by endian considerations, which is why a BOM byte order mark is sometimes inserted in such files. Never mind the few who are still dealing with EBCDIC and virtual punch card queues.

When something becomes so complicated that few if any understand how and why it works, then I think there is a greater chance that it actually doesn't work. That is one of the reasons why I have remarked that Raspberry Pi is becoming too complicated.

Heater
Posts: 12969
Joined: Tue Jul 17, 2012 3:02 pm

Re: "Nothing ever works"

Thu Nov 01, 2018 7:46 pm

hippy,
As Heater suggests; one really has to go out of one's way to create something which is so platform dependent that it won't work, can't work, on other platforms.
No, no. I did not suggest that. Windows and Mac guys do that everyday, without ever thinking about it. Their programs run on Windows or Mac and that is it. Same can be said of iOS or Android app creators. In fact mobile app creators go far out their way to make something that works on iOS and Android and wherever.

Meanwhile, I might publish code that runs on Debian but fails to build or run on your RedHat, Ubuntu or whatever other Linux distro you have. For sure I cannot test my code and build system on every Linux distro. Never mind make a package for all of them.
Particularly so when using languages which are intended to be endian and platform agnostic, are designed to overcome such problems.
Endiannes is only a small part of the problem of creating cross-platform code.
Well crafted code written for one thing on one platform should just work without problem with that same thing on another platform.
That is the "Holy Grail". Despite the fact that we have been developing computers and operating systems for the best part of a century we still have not achieved that.

This whole debate is ignoring the turmoil that is the mass of Free and Open Source programs and libraries we depend on.

Let's take some concrete examples:

1) My colleague, an excellent software engineer, made some Python code that used the MQTT protocol. One day his code could no longer connect to an Mosquito server. There was no error message from the Python library he used. After much googling around we find the MQTT protocol has been revised and his code no longer works. And with the new library we have we can make it work by specifying an MQTT protocol version in the configuration. Who would know that would happen ahead of time?

2) Code that people have written in Python suddenly does not work. Why, because no matter how well your code was written if your code works with Python V2 and your platform has Python V3 it will fail. Or vice versa.

3) I was using OpenSSL some years ago to create certificates on 64 bit machines. The same procedure failed on 32 bit machines.

4) Graphical GUI code written against Qt 3 will likely not work on a platform that has Qt 4. And again now we have Qt 5.

5) Closer to home. All kind of changes in the way Raspbian has changed over the years has caused all kind of failures in peoples projects and a lot of debate on how to fix them on this forum.

My point here is that no matter how good a software engineer you are, your program will have dependencies on all kind of libraries and such. Every single Linux distro and version of distro is different in the libraries and such it provides and the way it is configured. Ergo, getting your program to build and run on all of them is hard.

As I said, it's amazing things work as well as they do.

Heater
Posts: 12969
Joined: Tue Jul 17, 2012 3:02 pm

Re: "Nothing ever works"

Thu Nov 01, 2018 8:08 pm

ejolson,
When something becomes so complicated that few if any understand how and why it works, then I think there is a greater chance that it actually doesn't work.
I feel that for sure that must be true.
That is one of the reasons why I have remarked that Raspberry Pi is becoming too complicated.
I would not pick on the Raspberry Pi here. It's true of all machines we use today.

We are far away from the first 8 bit personal computers where potentially one could understand the whole thing in a single human life span.

If the Pi were simplified to make it understandable then it might not have a network stack, no ethernet or WiFi. It might not have 3D graphics acceleration. Let alone a window manager. It would not have all the language systems available now.

It would be reduced to an Arduino,

User avatar
topguy
Posts: 5624
Joined: Tue Oct 09, 2012 11:46 am
Location: Trondheim, Norway

Re: "Nothing ever works"

Fri Nov 02, 2018 2:01 pm

That is the "Holy Grail". Despite the fact that we have been developing computers and operating systems for the best part of a century we still have not achieved that.
Java was definitely an attempt to come closer to "write once run everywhere", I'll let other people decide if it was a successful attempt or not. It definitely had a big effect on worldwide software development.

If every manufacturer of computers and software just did the same as what had been done before because they never wanted anything to break, then there would never be any technical progress, of hardware or software. 4 out of the 5 examples given by Heater was caused by progressively better, newer versions of software, and one of the costs of progress is often loss of backwards compability.

ejolson
Posts: 3241
Joined: Tue Mar 18, 2014 11:47 am

Re: "Nothing ever works"

Fri Nov 02, 2018 5:45 pm

topguy wrote:
Fri Nov 02, 2018 2:01 pm
Java was definitely an attempt to come closer to "write once run everywhere", I'll let other people decide if it was a successful attempt or not. It definitely had a big effect on worldwide software development.
Historically the engineers developing new computer processors have been far better at improving things without breaking reverse compatibility than those developing software. In the context of Raspberry Pi, the ARMv8 processor used in the 3B+ can run the same code as the ARMv6 in the Zero. More surprisingly, the latest Xeon and Ryzen processors can still execute 16-bit segmented code written for the original 8086, though with EFI they no longer boot that way. That 40 years of reverse compatibility has not prevented significant improvements.

Java introduces an additional software layer between the hardware and the running application. Theoretically, this could insulate an application from hardware changes. In practice, it simply adds yet another link in the chain of dependencies that can go wrong and lead to the observation that nothing ever works. While 8086 binaries still run on current x86 hardware, I'm not so sure byte code from the 90's will run on current JVMs, but maybe it does.

Given the proper priorities, software (especially Python) could achieve greater reverse compatibility. Unfortunately, some software development efforts seem to award bonus points for changes which break or disrupt a competitors product. In the free software community points are also awarded for introducing build dependencies in other projects which make your own project more influential. Both of these activities can lead to the observed phenomena that nothing ever works.
Last edited by ejolson on Fri Nov 02, 2018 7:54 pm, edited 8 times in total.

timrowledge
Posts: 1272
Joined: Mon Oct 29, 2012 8:12 pm
Location: Vancouver Island
Contact: Website

Re: "Nothing ever works"

Fri Nov 02, 2018 6:51 pm

Smalltalk has been doing the “write once run everywhere “ thing quite well for almost 40 years. It takes some work to keep it up but it’s worth it. The system has been run on Dorados, 68000, x86, spare, arm, mips, obscure custom cpus, IBM mainframes...
Making Smalltalk on ARM since 1986; making your Scratch better since 2012

Heater
Posts: 12969
Joined: Tue Jul 17, 2012 3:02 pm

Re: "Nothing ever works"

Fri Nov 02, 2018 8:09 pm

@topguy,
Java was definitely an attempt to come closer to "write once run everywhere", I'll let other people decide if it was a successful attempt or not. It definitely had a big effect on worldwide software development.
Java was a great idea.

However in my experience it has been the worst offender in terms of failing to work on different CPU architectures, different operating systems, different versions of the same OS, different versions of the Java engine.

In that respect Java had a big effect on the software world. A great example of how not to do it !

@ejolson
Historically the engineers developing new computer processors have been far better at improving things without breaking reverse compatibility than those developing software.
I know what you are getting at but I'm not sure it is totally true.

Because:

1) Back in the late 70's Intel spent a fortune on developing the iAPX 432 as the future of micro-processors. Not at all compatible with the instruction set of their 8 bit machines, which internally they described as "a dog".

2) In the late 80's Intel spent a fortune on developing the i860 processor as the future of micro-processors as we move to 32 bits. Not at all compatible with the instruction set of their 16 bit machines at the time, which internally they described as "a dog".

3) In the 2000's Intel spent a fortune on developing the Itanium as the future of micro-processors as we move to 64 bits. Not at all compatible with the instruction set of their 32 bit machines at the time, which internally they described as "a dog".

I conclude that the hardware guys are very good at breaking reverse compatibility. Some how market forces keep them in line.

Said market forces do not play out the same in the open source software world.

User avatar
DougieLawson
Posts: 35550
Joined: Sun Jun 16, 2013 11:19 pm
Location: Basingstoke, UK
Contact: Website Twitter

Re: "Nothing ever works"

Fri Nov 02, 2018 9:40 pm

The way to fix Java (which is an elegant language with all of the ugly C++ complexity removed) would be to build a Java compiler to emit GCC C/C++ code. GCC can then generate a machine specfic run-time module that would get all the benefits of being compiled for the architecture it's running on (including using locally optimised libraries). You'd win both ways. Java's biggest problem is the JVM.
Note: Having anything remotely humorous in your signature is completely banned on this forum. Wear a tinfoil hat and you'll get a ban.

Any DMs sent on Twitter will be answered next month.

This is a doctor free zone.

Heater
Posts: 12969
Joined: Tue Jul 17, 2012 3:02 pm

Re: "Nothing ever works"

Sat Nov 03, 2018 2:17 am

Java elegant?

Only when compared to the monstrosity that is C++.

Interesting idea. Problem is it would take all day to build a "Hello World" program that way. Debugging the output would be interesting...

ejolson
Posts: 3241
Joined: Tue Mar 18, 2014 11:47 am

Re: "Nothing ever works"

Sat Nov 03, 2018 5:43 am

DougieLawson wrote:
Fri Nov 02, 2018 9:40 pm
The way to fix Java (which is an elegant language with all of the ugly C++ complexity removed) would be to build a Java compiler to emit GCC C/C++ code.
I believe fixing Java in this way would break enough existing code to create the effect that nothing ever works.

For the sake of having something appear to work, it is better not ever to change anything. Over time people get used to old bugs and don't mind them anymore. Such bugs become familiar pets that provide a comforting part of the look and feel of a program. If changes become necessary, they should be made with great care in small patches no greater than 10 lines at a time with full regression and unit tests between each patch to make sure each program continues to function in exactly the way it did before and as users have come expect. This is extremely important, because only after things stop changing can significant progress be made (towards understanding).

User avatar
bensimmo
Posts: 4125
Joined: Sun Dec 28, 2014 3:02 pm
Location: East Yorkshire

Re: "Nothing ever works"

Sat Nov 03, 2018 1:51 pm

One problem is security, when the fixes are to improve security, sometime that will break backwards compatibility.

ejolson
Posts: 3241
Joined: Tue Mar 18, 2014 11:47 am

Re: "Nothing ever works"

Sat Nov 03, 2018 5:45 pm

bensimmo wrote:
Sat Nov 03, 2018 1:51 pm
One problem is security, when the fixes are to improve security, sometime that will break backwards compatibility.
For example, ssh was created in 1995. The original version 1 protocol was patched in a reverse compatible way to mitigate an injection attack and again patched to mitigate a timing attack that could be used to recover session keys. Although a new version 2 protocol was developed in 2006, existing clients continued to speak version 1 if needed. Automatic fallback to the earlier protocol was disabled by default in most clients about 7 years ago, since most servers could speak version 2 by then. More recently the version 1 protocol has been removed from many server implementations. However, current client software can still connect to very old servers if the version 1 protocol is explicitly requested or if version 1 fallback is enabled in the configuration.

Today the versions of ssh and slogin that ship with Raspbian can still connect to a very old ssh server running on Slackware Linux 3.0 from 1996. Note that the TCP/IP protocol itself has also changed in a reverse compatible way for security reasons since then, yet it still works.

hippy
Posts: 5595
Joined: Fri Sep 09, 2011 10:34 pm
Location: UK

Re: "Nothing ever works"

Sat Nov 03, 2018 7:00 pm

bensimmo wrote:
Sat Nov 03, 2018 1:51 pm
One problem is security, when the fixes are to improve security, sometime that will break backwards compatibility.
A change with Samba default configurations broke a good many setups when it came to Windows networks, and how to get back to how it was wasn't at all well documented nor obvious. I'm still being asked for a password every session when I connect to my Pi when I never used to. Some might consider that advantageous, but I don't. It may also be a minor annoyance but it's an annoyance, and a continual one, none the less.

ejolson
Posts: 3241
Joined: Tue Mar 18, 2014 11:47 am

Re: "Nothing ever works"

Sat Nov 03, 2018 8:34 pm

hippy wrote:
Sat Nov 03, 2018 7:00 pm
bensimmo wrote:
Sat Nov 03, 2018 1:51 pm
One problem is security, when the fixes are to improve security, sometime that will break backwards compatibility.
A change with Samba default configurations broke a good many setups when it came to Windows networks, and how to get back to how it was wasn't at all well documented nor obvious. I'm still being asked for a password every session when I connect to my Pi when I never used to. Some might consider that advantageous, but I don't. It may also be a minor annoyance but it's an annoyance, and a continual one, none the less.
From my understanding version 1 was originally implemented in the 80's as a protocol layered on top of NetBIOS. Version 1 has been deprecated for at least a decade, however, there were some high profile malware attacks that led to the removal of automatic fallback from current software.

In my opinion, the resulting mayhem was mostly caused by the extremely poor error messages when a protocol mismatch occurred. For example, in Raspbian the message is

mount error(95): Operation not supported
Refer to the mount.cifs(8) manual page (e.g. man mount.cifs)

Unfortunately, the man page does not list what error code 95 means. However, a more reasonable message is delivered to the system log viewable using dmesg which states

[511206.467637] No dialect specified on mount. Default has changed to a more secure dialect, SMB2.1 or later (e.g. SMB3), from CIFS (SMB1). To use the less secure SMB1 dialect to access old servers which do not support SMB3 (or SMB2.1) specify vers=1.0 on mount.
[511206.469858] CIFS VFS: Dialect not supported by server. Consider specifying vers=1.0 or vers=2.0 on mount for accessing older servers
[511206.469931] CIFS VFS: cifs_mount failed w/return code = -95

Aside from the weird fact that the error code is -95 in the logs but positive as reported by the mount command, it is possible many affected users did not know how to view the above messages. It is also possible that some users had unwisely added SMB mount entries to /etc/fstab which then prevented their Raspberry Pi from booting at all. Therefore, even though it worked before, the observed result is nothing ever works (for long).

User avatar
DavidS
Posts: 4334
Joined: Thu Dec 15, 2011 6:39 am
Location: USA
Contact: Website

Re: "Nothing ever works"

Fri Nov 09, 2018 4:17 pm

And this thread is a highlight as to why I will use Linux for a while, or Windows, or Haiku OS, or whatever modern alternative operating system. I always end up giving up the alternative systems and running home to RISC OS. The worse alternative OS when it comes to making things work is Windows.

The dependancy tangles of other OS's is bad enough by and of it self. I disslike Python as well for the same reason, though it's dependancy tangles are even worse than those of other OS's.

Then there is the extreme version dependancies of many other OS's, often requiring a very specific version of a given library, and refusing to run on a minor revision, that changed nothing and only fixed a single bug.

And it gets even worse with the tangle of dependancies if you attempt to use something like Windows, as often you will have a library replaced by an application installer with the version it uses, and this will break 20 or more other programs that you absolutely need to have to do your daily tasks.

I do not have these issues when running RISC OS. RISC OS is the reason I purchased my first Raspberry Pi, and the reason that I continue to like the Raspberry Pi (though I have added reasons in the GPIO capabilities of the Raspberry Pi series).

There are other OS's that do not have these issues very often, though they are either dead, or not widely enough used to have a usable level of software running on them. So RISC OS is the balancing point for me.
RPi = The best ARM based RISC OS computer around
More than 95% of posts made from RISC OS on RPi 1B/1B+ computers. Most of the rest from RISC OS on RPi 2B/3B/3B+ computers

ejolson
Posts: 3241
Joined: Tue Mar 18, 2014 11:47 am

Re: "Nothing ever works"

Fri Nov 09, 2018 9:15 pm

DavidS wrote:
Fri Nov 09, 2018 4:17 pm
And this thread is a highlight as to why I will use Linux for a while, or Windows, or Haiku OS, or whatever modern alternative operating system. I always end up giving up the alternative systems and running home to RISC OS.
I suspect the difficulty for me with operating systems other than old versions of Debian Linux and Slackware is not that things don't work, but that I don't know how to make them work.

I originally thought with open source that since the code was available one could just recompile and things would always work the same way in the future. However, in my experience, once a particular open-source software becomes popular it attracts the attention of multitudes who change it for different reasons. This is similar to what happens when a technology company such as Google, Microsoft or Apple change commercial software to meet a new vision of marketable technology that is different from what people already know.

The end result is that nothing works because I don't know how to make it work.

Heater
Posts: 12969
Joined: Tue Jul 17, 2012 3:02 pm

Re: "Nothing ever works"

Fri Nov 09, 2018 10:40 pm

ejolson,
This is similar to what happens when a technology company such as Google, Microsoft or Apple change commercial software to meet a new vision of marketable technology that is different from what people already know.

The end result is that nothing works because I don't know how to make it work.
You know what? You may have a point there.

Except, it turns out that it's those same big players that have moved in and taken over the Linux world causing all the grief.

Bryan Lunduke explains what has been going on in this recent presentation: https://www.youtube.com/watch?v=TVHcdgrqbHE

It's a bit of an eye opener.

The short of it is that said big players are investing tons of money into Linux, paying Torvalds via the Linux Foundation for example, so they get to have a lot of influence on what goes on.

Roken
Posts: 289
Joined: Sun Dec 31, 2017 4:35 pm
Location: UK

Re: "Nothing ever works"

Fri Nov 09, 2018 10:59 pm

I honestly couldn't be bothered reading through the whole thread, but I can answer the original post. Nothing ever works, because "nothing" was never intended to work. Only "Something" can work.
Headless PI. OMG, someone cut it's head off. Oh, hang on. it didn't have one to start with.

Heater
Posts: 12969
Joined: Tue Jul 17, 2012 3:02 pm

Re: "Nothing ever works"

Fri Nov 09, 2018 11:44 pm

Roken,
Nothing ever works, because "nothing" was never intended to work. Only "Something" can work.
That sounds deeply profound. Useless but profound.

Let's deconstruct the statement like this:

"Nothing" = That which is not a thing.

"ever" = Now, in the past and in the future. All of time in fact.

So "Nothing ever works" = "That which is not a thing works, now, in the past and in the future".

If we invert some logic there we get "That which are things may work sometimes"

Which about sums up the situation.

jamesh
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 23079
Joined: Sat Jul 30, 2011 7:41 pm

Re: "Nothing ever works"

Sat Nov 10, 2018 8:17 am

Thread has turned stupid.
Principal Software Engineer at Raspberry Pi (Trading) Ltd.
Contrary to popular belief, humorous signatures are allowed. Here's an example...
"My grief counseller just died, luckily, he was so good, I didn't care."

Return to “Off topic discussion”