hippy
Posts: 5620
Joined: Fri Sep 09, 2011 10:34 pm
Location: UK

"Nothing ever works"

Wed Oct 24, 2018 1:22 pm

Why does "nothing ever seem to work" for anything I try on the Pi ?

I know I'm not alone in holding such sentiments when I recall the "Raspberry Pi is a waste of time (and money)" thread -

viewtopic.php?t=221876
You spend hours upon hours googling around why things doesnt work, boot, start et.c. Then you spend even more hours figuring out errors and when it starts working its always a disappointment. ... Im simply giving up. Im tired of googling, tired of guessing what people mean, tired of errors, tired of things not working 100% ... You basically have to be a Linux wizard to correct anything that doesnt go as described".
That mostly got dismissed as mere ranting and venting frustrations, someone even, rather dismissively and offensively I thought, suggested it was just because the OP were unhappy in their life and were lashing out.

In that thread I did mention I had sympathy for the view that "nothing ever seems to work" which others have also expressed. And guess what ? Having thrown in the towel on diskless booting for now; I moved on to investigating VC4 programming. But, with no suprise at all, that's also turned into a frustrating adventure through further Circles of Hell.

https://github.com/nineties/py-videocore.git

Code: Select all

git clone https://github.com/nineties/py-videocore.git
cd py-videocore
sudo python setup.py install
Yay! On to "getting started" ...

Code: Select all

sudo python examples/hello_world.py
"ImportError: No module named rpi_vcsm.VCSM"

No surprise. Never mind ...

https://github.com/Idein/rpi-vcsm

Code: Select all

git clone https://github.com/Idein/rpi-vcsm.git
cd rpi-vcsm/
pip install -r requirements.txt
python setup.py install
Nope. Fails on the "setup.py install with a permissions error. Change it to a "sudo" and it appears to work. So back to trying to run that "hello_world.py" and, nope -

"ImportError: No module named ioctl_opt"

Same with "sudo".

Okay. Maybe try a different tack. Something easier like -

viewtopic.php?t=78414#p560315
1. Download VASM by Volker Barthelmann here: http://www.ibaug.de/vasm/vasm.tar.gz
2. Compile it using "make" with the options: CPU = vidcore
Nope. Not even with the erroneous spaces removed. Maybe this will help -

http://sun.hasenbraten.de/vasm/index.php?view=compile

Code: Select all

make CPU=vidcore SYNTAX=std
Nope.

So why does "nothing ever seem to work" ? Why does it always feel like an exercise in futility and endless frustration with hours wasted ?

And how do people live with that or, better still, overcome it, resolve their problems ?

The list of things which 'should just work' which I have tried and failed at, have never managed to resolve, just seems to be growing ever longer, and everything I try seems to add to that list.

It's not hard to come to the conclusion that it's simply not worth it. That anything one attempts will result in nothing more than wasted time and effort which could be better spent elsewhere.

If anyone knows how to get any of the above to work please do let me know.

hippy
Posts: 5620
Joined: Fri Sep 09, 2011 10:34 pm
Location: UK

Re: "Nothing ever works"

Wed Oct 24, 2018 2:03 pm

I'm not sure why this got moved to "off-topic" because the way I see it is that it's very much on-topic with respect to my using the Pi and software available. But no problem; I simply want to make things work. I'll post separately on those issues.

User avatar
PeterO
Posts: 4882
Joined: Sun Jul 22, 2012 4:14 pm

Re: "Nothing ever works"

Wed Oct 24, 2018 2:09 pm

If the dependencies are not correctly given, moan at the author of the package not at us,
It's off topic here as it has nothing to do with RPF/RPT/or anyone not using the package in question.

PeterO
Discoverer of the PI2 XENON DEATH FLASH!
Interests: C,Python,PIC,Electronics,Ham Radio (G0DZB),1960s British Computers.
"The primary requirement (as we've always seen in your examples) is that the code is readable. " Dougie Lawson

jamesh
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 23083
Joined: Sat Jul 30, 2011 7:41 pm

Re: "Nothing ever works"

Wed Oct 24, 2018 2:10 pm

hippy wrote:
Wed Oct 24, 2018 2:03 pm
I'm not sure why this got moved to "off-topic" because the way I see it is that it's very much on-topic with respect to my using the Pi and software available. But no problem; I simply want to make things work. I'll post separately on those issues.
I moved it. Debated deleting it, as it really doesn't help anyone or even yourself fix the issues. It's just a whinge. But hey, feeling generous today.
Principal Software Engineer at Raspberry Pi (Trading) Ltd.
Contrary to popular belief, humorous signatures are allowed. Here's an example...
"My grief counseller just died, luckily, he was so good, I didn't care."

User avatar
topguy
Posts: 5627
Joined: Tue Oct 09, 2012 11:46 am
Location: Trondheim, Norway

Re: "Nothing ever works"

Wed Oct 24, 2018 2:11 pm

Read documentation closely is my tip..
( and dont expect people to always list dependencies of dependencies... )

https://github.com/nineties/py-videocore mentions "rpi-vcsm >= 2.0.0" as a dependency.
https://github.com/Idein/rpi-vcsm has "ioctl-opt>=1.2" listed as requiement.
Which I guess is this module: https://pypi.org/project/ioctl-opt/

hippy
Posts: 5620
Joined: Fri Sep 09, 2011 10:34 pm
Location: UK

Re: "Nothing ever works"

Wed Oct 24, 2018 2:35 pm

topguy wrote:
Wed Oct 24, 2018 2:11 pm
Read documentation closely is my tip..
( and dont expect people to always list dependencies of dependencies... )

https://github.com/nineties/py-videocore mentions "rpi-vcsm >= 2.0.0" as a dependency.
https://github.com/Idein/rpi-vcsm has "ioctl-opt>=1.2" listed as requiement.
Which I guess is this module: https://pypi.org/project/ioctl-opt/
"sudo pip install ioctl-opt" does seem to have solved that one so many thanks for that.

In my defence I will say that I thought the "pip install -r requirements.txt" was meant to install the required ioctl-opt and anything else it needed. Perhaps not, but I'd have expected it to have done something even if just telling me what was required wasn't installed. It might also have helped if the instructions for py-videocore had also included a "pip install -r requirements.txt" itself.

hippy
Posts: 5620
Joined: Fri Sep 09, 2011 10:34 pm
Location: UK

Re: "Nothing ever works"

Wed Oct 24, 2018 2:44 pm

PeterO wrote:
Wed Oct 24, 2018 2:09 pm
If the dependencies are not correctly given, moan at the author of the package not at us,
It's off topic here as it has nothing to do with RPF/RPT/or anyone not using the package in question.
I can understand that perspective but a fair chunk of the questions which get asked on here have nothing to do with the RPF/RPF or anyone not interested in whatever those asking are using. Most Python, C, Kodi, Samba, issues and many other topics could be considered off-topic as Dougie has noted in the past. But they aren't treated as such.

DirkS
Posts: 9880
Joined: Tue Jun 19, 2012 9:46 pm
Location: Essex, UK

Re: "Nothing ever works"

Wed Oct 24, 2018 2:45 pm

hippy wrote:
Wed Oct 24, 2018 2:35 pm
In my defence I will say that I thought the "pip install -r requirements.txt" was meant to install the required ioctl-opt and anything else it needed. Perhaps not, but I'd have expected it to have done something even if just telling me what was required wasn't installed. It might also have helped if the instructions for py-videocore had also included a "pip install -r requirements.txt" itself.
If you install the requirements locally (no sudo) it can fail. It depends on where the install routing wants to copy / move / create files.
If it succeeds (perfectly possible) then you would still be in trouble if you run your script with sudo because then it cannot find the locally installed modules.
Nothing to do with the Pi, just the way Python works on Linux.

So install python modules with sudo (or better: use virtual environments)

User avatar
topguy
Posts: 5627
Joined: Tue Oct 09, 2012 11:46 am
Location: Trondheim, Norway

Re: "Nothing ever works"

Wed Oct 24, 2018 2:48 pm

I dont mind Python as language but all the different methods for install modules are sometimes seriously frustrating.
- apt/apt-get
- easyinstall
- pip
- sudo pip
- setup.py

If anyone know a "best practice" guide for installing a variety of Python modules, please link it.

DirkS
Posts: 9880
Joined: Tue Jun 19, 2012 9:46 pm
Location: Essex, UK

Re: "Nothing ever works"

Wed Oct 24, 2018 3:08 pm

topguy wrote:
Wed Oct 24, 2018 2:48 pm
If anyone know a "best practice" guide for installing a variety of Python modules, please link it.
My personal best practices. Use
- sudo pip
- sudo python setup.py (mostly for modules that are not available via pip but e.g. directly from github)

- don't use 'bare' pip (to avoid local installs)
- don't use easyinstall (AFAICT that's pretty much obsolete)
- don't use apt (mostly older versions and can interfere when you try to install an update with pip)

And if possible use virtual environments. In that case you don't need sudo.

User avatar
bensimmo
Posts: 4129
Joined: Sun Dec 28, 2014 3:02 pm
Location: East Yorkshire

Re: "Nothing ever works"

Wed Oct 24, 2018 3:16 pm

rule number one and the only rule I know is.
Do not PIP if APT has already done it.
I have broken my setup many times doing that and certainly do not pip install pip, all hell lets loose.
Mainly as I didn't know apt had installed them.


sudo pip /sudo pip3 is needed on the Pi
pip is used on most other things and virtual environments I believe.

That's it, not specifically use environments, it's not the Pi way.


perhaps something for Ben N to explain in the MagPi issues?

hippy
Posts: 5620
Joined: Fri Sep 09, 2011 10:34 pm
Location: UK

Re: "Nothing ever works"

Wed Oct 24, 2018 3:21 pm

topguy wrote:
Wed Oct 24, 2018 2:48 pm
I dont mind Python as language but all the different methods for install modules are sometimes seriously frustrating.
That's rather an understatement. And compounded by the version 10 upgrade which can lead to 'pip' potentially crashing with an internal Syntax Error when it's screwed itself up.

Plus as DirkS notes, 'pip' does different things depending whether it's invoked with 'sudo' and without.

When I ran the 'pip' install which seems to have solved my problem it recommended I run some sort of "pip upgrade" which the developers of pip say not do do because it may cause more problems than it solves - https://github.com/pypa/pip/issues/5599

Even when arguing the best solution there's rarely any consensus - I'm not having a go at anyone above who is simply doing their best to help.
bensimmo wrote:
Wed Oct 24, 2018 3:16 pm
perhaps something for Ben N to explain in the MagPi issues?
Someone needs to. Circles of Hell as I said.

ShiftPlusOne
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 5790
Joined: Fri Jul 29, 2011 5:36 pm
Location: The unfashionable end of the western spiral arm of the Galaxy

Re: "Nothing ever works"

Wed Oct 24, 2018 3:21 pm

topguy wrote:
Wed Oct 24, 2018 2:48 pm
I dont mind Python as language but all the different methods for install modules are sometimes seriously frustrating.
- apt/apt-get
- easyinstall
- pip
- sudo pip
- setup.py

If anyone know a "best practice" guide for installing a variety of Python modules, please link it.
https://xkcd.com/1987/

Depends who you ask. Python devs tend to use pip to get the latest modules. I'm not a fan of that approach because it's a free for all with no quality control. pip packages are difficult to remove and upgrade cleanly as a result.

I'd recommend always using the apt package if it's available and if you can work with that version of the package. If you want your software to be included in apt repos, you'll normally have to work with packages which already exist in apt. One of the headaches of maintaining our Raspbian repo is developers who work against the latest bleeding edge packages and then want their packages to apt. The answer is usually no, because we are not going to package all of your dependencies for you and we're not going to change libraries already used by other applications in the repo.

It really depends on what you're doing. If you're developing software targeting a certain distro, work with what's available in apt. If you're just playing around with python and don't care about keeping your development environment clean and would like the latest bleeding edge software, use pip. 'sudo pip' is used to install packages globally, which some packages require. I'd be cautious with that because you're trusting strangers on the internet with root access to your machine when you do that.

setup.py and easyinstall are usually not required. They're the tools used to generate/distribute python packages. It's what everything else uses under the hood. You'll normally only need to do that if the package is not already available for pip install. Because the barrier to entry for pip is so low, you'll normally find that everything is already there.

hippy
Posts: 5620
Joined: Fri Sep 09, 2011 10:34 pm
Location: UK

Re: "Nothing ever works"

Wed Oct 24, 2018 5:07 pm

ShiftPlusOne wrote:
Wed Oct 24, 2018 3:21 pm
Depends who you ask. Python devs tend to use pip to get the latest modules. I'm not a fan of that approach because it's a free for all with no quality control. pip packages are difficult to remove and upgrade cleanly as a result.
That's another Circle of Hell right there. And the fact that Python is multi-platform, so what works and is most appropriate on one system isn't for another, adds to the confusion.

Heater
Posts: 12976
Joined: Tue Jul 17, 2012 3:02 pm

Re: "Nothing ever works"

Wed Oct 24, 2018 5:17 pm

hippy,

I understand your frustration. It's not just a Pi problem though. So many OS and so much software I have ever used has the same frustrations. You have to fight with it to get it to work how you think it should. Or even to find out if it can do that or not. Ever tried doing anything off the beaten track on Windows 10? Nightmare. And that is a "user friendly" OS that has been honed to perfection of four decades at the cost of billions of dollars.
There is your problem.

Nothing to do with the Pi, or Raspian or even Linux of any other operating system.

Who is this "nineties" guy? What is this "py-videocore"? Anyone can write whatever code they like and stick into gihub. I know I do. Maybe it works, maybe it's garbage. There are 30 million or so such repositories on github. (Note: I don't mean to say anything bad about "nineties" or "py-videocore". It's just an example)

Do you really want to blame the Pi or the Foundation or Raspbian or Linux or any other OS for this chaos?

It's frustrating but I celebrate it. This ocean of Free and Open Source Software chaos has provided me with all kind of systems and software that I would never have imagined I could make use of. All given very kindly for free. Amazing.

jahboater
Posts: 4595
Joined: Wed Feb 04, 2015 6:38 pm

Re: "Nothing ever works"

Wed Oct 24, 2018 5:30 pm

Compared with some of the horrid platforms I have had to use over the last 40 years, Linux and the Pi are astonishing easy to work with!

ejolson
Posts: 3263
Joined: Tue Mar 18, 2014 11:47 am

Re: "Nothing ever works"

Wed Oct 24, 2018 6:05 pm

Heater wrote:
Wed Oct 24, 2018 5:17 pm
So many OS and so much software I have ever used has the same frustrations.
In software development (both proprietary and open source) there is a philosophy that the best time to improve something is when it is currently working. Variations of "if it's not broke, fix it anyway" are used to express this idea. The result, however, is often not an improvement but regressions and incompatibility.

Python, it's versions and libraries appear particularly susceptible as do the non-kernel components of GNU/Linux and Windows. At the same time, many binary executables have continued to run on either operating system unchanged for the last 10 years. Therefore, if the right priorities and techniques are applied, it is possible to improve one software without introducing regressions and breaking compatibility with other software.

hippy
Posts: 5620
Joined: Fri Sep 09, 2011 10:34 pm
Location: UK

Re: "Nothing ever works"

Wed Oct 24, 2018 7:22 pm

Heater wrote:
Wed Oct 24, 2018 5:17 pm
Do you really want to blame the Pi or the Foundation or Raspbian or Linux or any other OS for this chaos?
No; I'm not blaming the Pi, the Foundation, Raspbian or Linux, but there does seem to be something about the ecosystem, culture, or whatever, which they are all a part of which appears to be more problematic than I have experienced elsewhere.

It seems to me that the problem of no instructions, incomplete or incorrect instructions, is far more prevalent when it comes to this ecosystem than it is for other development platforms. When it comes to near everything one tries failing at the first hurdle it becomes pretty annoying and frustrating.

Sometimes the developer may be to blame but I would mostly say not. Most seem competent given the scope of what they are working on, provide instructions, strive to do the best they can, but it often still doesn't work as expected.

It's not just 'rubbish' being downloaded but solid code, tested and working for the developer, intended to work for others, and it should. And it usually does. It's the getting it installed, built, the steps to having it actually work, where the problems lies.

There seems to be a fundamental problem somewhere.

It really shouldn't be like this. Not every time, not what feels like every time. That's probably what grates and frustrates most.
Heater wrote:
Wed Oct 24, 2018 5:17 pm
Ever tried doing anything off the beaten track on Windows 10? Nightmare.
Indeed. And if one goes off the beaten track one has to expect there to be some frustrations along the way.

But I'm talking about cloning a git repository and typing 'make', downloading a .tar.gz and following three lines of simple instructions, which often doesn't work. Things which really should.

There has got to be some cause and reason for that.

User avatar
topguy
Posts: 5627
Joined: Tue Oct 09, 2012 11:46 am
Location: Trondheim, Norway

Re: "Nothing ever works"

Wed Oct 24, 2018 7:48 pm

Indeed. And if one goes off the beaten track one has to expect there to be some frustrations along the way.

But I'm talking about cloning a git repository and typing 'make', downloading a .tar.gz and following three lines of simple instructions, which often doesn't work. Things which really should.
I'll claim that if you want to build something from source you are already outside the beaten track. Its taken me a lot of years to do this with some confidence that I can fix most of the problems that might occur. And the list of potential problems is loooong.
For C/C++ code, the automake tools ( ./configure;make;make install ) has made this much more foolproof than it used to be, but still the success rate is only as good as the person writing the code.
There has got to be some cause and reason for that.
Yes, its called "assumptions" and "human nature". The author of the code will make ( deliberate or involuntary ) assumptions about the system his code will be compiled/run on. And you will make the assumption that he hasn't made any assumptions. ;-)

Heater
Posts: 12976
Joined: Tue Jul 17, 2012 3:02 pm

Re: "Nothing ever works"

Wed Oct 24, 2018 8:16 pm

Hippy,

I do see your point. We could talk around the generalities of this problem forever. But lets take the specific case that you opened with here. Why did it not work for you?

Firstly I note that the first statement in the README is:

"This is work in progress project. Backward compatibility is not guaranteed."

Hmm... OK that is a warning that this is in development and could well be broken at any point in time. Especially if any platform changes occur.

What is wrong with the instruction?

On the face of it they look reasonable as an installation and get started guide. What is missing is a statement about exactly what Pi it has been tested on. Exactly what versions of the kernel, OS, language and other dependencies it was tested against. For example which commit of rpi-vcsm? Exactly what relevant configuration files does the developer have?

Without being able to bit for bit reproduce the entire environment the developer had when he last tested it there is always a chance it will fail.

Anecdote: As it happens this week I'm struggling with an example of this frustration from the closed source, commercial software world:

Two years ago I made this: https://github.com/ZiCog/xoro It's a design in Verilog for an FPGA. Two years ago it was tested and it worked. On Monday I decided to get back to it. So I clone the repo, build the code using the Quartus Verilog logic synthesizer and load it to the exact same FPGA board I have. Sure enough it does not work!

What changed?

So far I have no idea. Most likely candidate is the fact that two years ago I was using Quartus 16 and now I have Quartus 18. Of course the Win 10 OS I run it on has been updated since then but Verilog has no dependencies on Windows libraries and such.

I could install an old Quartus 16 and check it there. But no, I'm going to roll my sleeves up and test every little part until I find what has broken.

This kind of problem is so prevalent with those designing PCBs or for FPGA, ASICs etc, also in mechanical CAD, that they archive their entire system, OS, tools, etc along with their board, logic or other designs. Just so they stand some chance of being able to reproduce things if they have to. They even warn to "never ugrade your tools in the middle of a project"

Heater
Posts: 12976
Joined: Tue Jul 17, 2012 3:02 pm

Re: "Nothing ever works"

Wed Oct 24, 2018 8:32 pm

topguy,
I'll claim that if you want to build something from source you are already outside the beaten track...
I'll agree with that. Especially when the code in question is still under early development and comes with a warning label to that effect.
...the list of potential problems is loooong.
For C/C++ code, the automake tools ( ./configure;make;make install ) has made this much more foolproof than it used to be, but still the success rate is only as good as the person writing the code.
I think that is a bit hard on the developer. It's impossible for them to test their code and build process on every possible platform. Hardware changes, operating systems change, library dependencies change, languages and compilers change. As you say "the list of potential problems is loooong".

What they can do is specify exactly what versions of everything they have used to build and test against. That does not solve the problem exactly because we may not want to go backwards in time to make their stuff work. But it can be a great help in pin pointing problems and finding solutions.

hippy
Posts: 5620
Joined: Fri Sep 09, 2011 10:34 pm
Location: UK

Re: "Nothing ever works"

Thu Oct 25, 2018 12:32 am

topguy wrote:
Wed Oct 24, 2018 7:48 pm
I'll claim that if you want to build something from source you are already outside the beaten track.
I would disagree seeing as that's the standard model in the Linux world if one wants to go beyond using stuff which is only obtained via apt-get.

Besides; we're talking Python here. There's nothing to compile per se; it should just run when the Python interpreter is pointed at it. Dependencies should just install using the provided instructions.

When it comes to compiling stuff on Windows it's just download this, open in Visual Studio ( or whatever ), click build, and I have rarely had the problems with that which I've had on Linux.

Maybe I have been incredibly lucky with Windows and incredibly unlucky with Linux but I can only call it how I have experienced it.
topguy wrote:
Wed Oct 24, 2018 7:48 pm
its called "assumptions" and "human nature". The author of the code will make ( deliberate or involuntary ) assumptions about the system his code will be compiled/run on. And you will make the assumption that he hasn't made any assumptions. ;-)
"Download this, run make". That really shouldn't be as problematic as it is. Especially when the code is specifically targeted, tried and tested, on a Pi.

User avatar
topguy
Posts: 5627
Joined: Tue Oct 09, 2012 11:46 am
Location: Trondheim, Norway

Re: "Nothing ever works"

Thu Oct 25, 2018 2:52 pm

I would disagree seeing as that's the standard model in the Linux world if one wants to go beyond using stuff which is only obtained via apt-get.
I agree that its the standard model, but I have never seen anyone claim it will work 100% on all systems, or that this is even a realistic goal.

Windows is very different in structure and composition than Linux. Linux gives you much greater freedom regarding how you will set up your system, but there is no single team of developers to make sure that everything works all the time like Microsoft have.

ejolson
Posts: 3263
Joined: Tue Mar 18, 2014 11:47 am

Re: "Nothing ever works"

Thu Oct 25, 2018 6:05 pm

hippy wrote:
Thu Oct 25, 2018 12:32 am
Besides; we're talking Python here. There's nothing to compile per se; it should just run when the Python interpreter is pointed at it.
Python was originally conceived as a teaching language rather than a production language. BASIC is another well-known example. In theory changes to production programming languages are made carefully to ensure that valuable working code continues to work. With a teaching language changes are made in arbitrary ways that break reverse compatibility to obtain a syntax which is better for teaching.

Although people have been using Python in production settings for some time, Guido van Rossum decided to go ahead with changes that break compatibility between Python 3 and Python 2. While the culture of move fast and break things is still a bit exaggerated in the Python community, it is notable that Guido stepped down this year as the benevolent dictator of Python and that role will now be played by a committee, probably formed by industry representatives with a vested interest in stability. Whether things get better or worse is yet to be seen.

hippy
Posts: 5620
Joined: Fri Sep 09, 2011 10:34 pm
Location: UK

Re: "Nothing ever works"

Thu Oct 25, 2018 6:23 pm

topguy wrote:
Thu Oct 25, 2018 2:52 pm
I would disagree seeing as that's the standard model in the Linux world if one wants to go beyond using stuff which is only obtained via apt-get.
I agree that its the standard model, but I have never seen anyone claim it will work 100% on all systems, or that this is even a realistic goal.
No that's not realistic and I wouldn't expect anyone to claim things will always work perfectly. But the issue is that it so often fails to work, most times for me it seems. That's the real problem.

One seems to need to be an expert to get anything done, have years of experience in being able to figure out what's gone wrong and how to fix it. It shouldn't be like that even for what should be simple things.

And it's not like everyone is having to craft their own installation tools or they are fundamentally different on every system. Git is git, make is make, sed is sed, etc, etc, pretty much everywhere. It shouldn't be like the wild west out there, luck of the draw whether the installation tools and instructions work or not, a good chance they won't.

One could venture that it's not 'fit for purpose', easily imagine it might be one of the reasons people don't take up coding, or don't stick with it, become discouraged or disillusioned. After all, who in their right mind wants to deal with such a nightmare when one never knows if something will work or not, not knowing if it's just going to be another day of wasted effort and frustration ?

Return to “Off topic discussion”