swampdog
Posts: 755
Joined: Fri Dec 04, 2015 11:22 am

Large free upload space?

Thu Aug 05, 2021 3:31 pm

Hi Folks,

I've either got stuck in a googling rut or there isn't a site but just in case..

..it would need to be (currently) 2.2Gb (about 7Gb uncompressed) so I guess 5Gb to hold at least two versions.

trejan
Posts: 3588
Joined: Tue Jul 02, 2019 2:28 pm

Re: Large free upload space?

Thu Aug 05, 2021 3:38 pm

For what? Is it public? Private? Open source? Commercial? Will lots of people download this?

Will Google Drive be sufficient? Github doesn't like you hosting huge files and will limit each file to 2GB max.

swampdog
Posts: 755
Joined: Fri Dec 04, 2015 11:22 am

Re: Large free upload space?

Thu Aug 05, 2021 4:14 pm

Public. Currently a tarball of..

Code: Select all

foo@pi20:~/usr/src/QT $ cat go | egrep "_PKG|_VER" | grep ^:
: ${Z3_PKG:="z3"}
: ${Z3_VER:="4.8.6"}
: ${LV_PKG:="llvm"}
: ${LV_VER:="13.0.0"}
: ${CM_PKG:="cmake"}
: ${CM_VER:="3.20.3"}
: ${PY_PKG:="Python"}
: ${PY_VER:="3.9.5"}
: ${DB_PKG:="mariadb"}
: ${DB_VER:="10.5.10"}
: ${QT_PKG:="qt"}
: ${QT_VER:="5.15.2"}
: ${QC_PKG:="qtc"}
: ${QC_VER:="5.0.b1"}
: ${GC_PKG:="gcc"}
: ${GC_VER:="11.1.0"}
..bootstrapped llvm (so more options can be enabled) and rebuilt with its own clang.

I don't envisage many takers but thought I'd find out by uploading it somewhere. I keep building it for myself in order to learn QT programming, something I've been meaning to do for over a decade.

trejan
Posts: 3588
Joined: Tue Jul 02, 2019 2:28 pm

Re: Large free upload space?

Thu Aug 05, 2021 4:45 pm

Probably something like Google Drive or Mediafire? There is Mega as well. The other big file hosting services like Dropbox and Box don't allow 2GB+ files on their free tiers.

swampdog
Posts: 755
Joined: Fri Dec 04, 2015 11:22 am

Re: Large free upload space?

Fri Aug 06, 2021 4:46 pm

Cheers. I'll investigate google drive over the weekend. I had read something (can't recall) that caused me to dismiss me even looking into it.

Heater
Posts: 18508
Joined: Tue Jul 17, 2012 3:02 pm

Re: Large free upload space?

Fri Aug 06, 2021 5:25 pm

swampdog wrote:
Thu Aug 05, 2021 4:14 pm
..bootstrapped llvm (so more options can be enabled) and rebuilt with its own clang.

I don't envisage many takers but thought I'd find out by uploading it somewhere. I keep building it for myself in order to learn QT programming, something I've been meaning to do for over a decade.
How about just publishing the instructions required so that people can reproduce the creation of that huge tar ball for themselves, rather just putting a big binary blob out there?

Perhaps here or on a blog post some place?

From a security perspective we should not encourage people to download and install random executables from whatever file sharing site. I'm not suggesting you have any bad intensions but in general it's a bad idea for people to do that.

From an informational/educational perspective it's better people know how to create the thing and how to overcome whatever problems you had along the way.
Memory in C++ is a leaky abstraction .

swampdog
Posts: 755
Joined: Fri Dec 04, 2015 11:22 am

Re: Large free upload space?

Sat Aug 07, 2021 2:41 am

I do intend to post the script but it will require the user to to know many things. It's a pig to build correctly as it's so easy to accidentally pull in an unwanted system dependency against a wanted system system dependency. Consider, for instance, the build pulls in an "apt" QT5 lib in favour of the "sideloaded" one. That's just the obvious problem.

The whole idea of the thing is to have the latest of everything such that I don't have to mess with it for a year or two while I learn QT. Thus, initially the script hunts for the latest gcc in /usr/local/GCC/* then uses that to build (say) gcc 11.1.0 into /usr/local/QT/5152r/ which it then repeats using /usr/local/QT/5152r/ gcc to rebuild itself to remove dependencies on (say) /usr/local/GCC/11.1.0/. Even I can't be bothered to do that most of the time so the script looks for a prebuilt /usr/local/QT/5152r/ gcc tarball and unpacks that.

I do most of the work on linux mint 18.3 then expect the script to work without change on rpi. Obviously the script has to make some decisions but I try to keep them generic: a new gcc build hunts for its predecessor gcc and greps part of its config into the new gcc and uses its predecessor gcc to build the new gcc. I also build gmp/mpfr/mpc/isl out-of tree of historic reasons.

All we have so far is gcc 11.1.0 in /usr/local/QT/5152r/ and we need to bootstrap llvm. My old PC died. New PC, 32Gb ram OOM because gcc linker was using 7.9 Gb and the "make -j15" had too many linkers collide at the same time. I don't recall how much the rpi4 needed except I fudged both through manually before discovering "-DLLVM_PARALLEL_LINK_JOBS=1", enough to produce an llvm linker. Now we can dispense with gcc and use clang to compile itself with features gcc cannot compile. Eventually we're at a point we can use clang to build all the tools to let us build QT.

Mint 18.3 is now out of support. Some stuff I build there (like python) could be removed for rpi or newer mint. On the plus side, all I had to do was remove vulkan off the rpi64 build and it worked. The downside is some QT stuff was pulled out of "master" and needed patches.

The binary works and could be used to build rebuild QT with almost none of the above hassle, aside from installing the required dev packages.

One thing I am sure of.. I won't be supporting it. The source build is too complex and either the binary works or it doesn't. I figured, if I can find free upload space, may as well post it before I blat mint 18.3(*) because after all, copy onto a rpi4 with no internet and wipe the sdcard after.

(*) I will lose stuff upgrading. I'm fully retired now and have decided to be ruthless. The 24/7 stuff in the cellar is getting culled to bare minimum.

Heater
Posts: 18508
Joined: Tue Jul 17, 2012 3:02 pm

Re: Large free upload space?

Sat Aug 07, 2021 4:14 am

It's not clear to me why one needs to build a new version of GCC then build LLVM then build Qt just to get on with learning Qt?
Memory in C++ is a leaky abstraction .

ejolson
Posts: 8037
Joined: Tue Mar 18, 2014 11:47 am

Re: Large free upload space?

Sat Aug 07, 2021 4:34 am

Heater wrote:
Sat Aug 07, 2021 4:14 am
It's not clear to me why one needs to build a new version of GCC then build LLVM then build Qt just to get on with learning Qt?
I think it's the same reason one needs the latest version of the rust compiler to get on with learning rust. Some tools haven't stabilised to the point where the Debian packages are useful.

Heater
Posts: 18508
Joined: Tue Jul 17, 2012 3:02 pm

Re: Large free upload space?

Sat Aug 07, 2021 9:39 am

ejolson wrote:
Sat Aug 07, 2021 4:34 am
I think it's the same reason one needs the latest version of the rust compiler to get on with learning rust. Some tools haven't stabilised to the point where the Debian packages are useful.
That might explain it.

Given that a usable version of Qt 5 is available as a Pi OS package it seems like a lot of work just to get to the point of leaning some Qt.

As for Rust, that has been stable since 2015. Sure it is still evolving but backward compatibility is assured. Besides grabbing the lates version is only a single command away: https://www.rust-lang.org/tools/install
Memory in C++ is a leaky abstraction .

swampdog
Posts: 755
Joined: Fri Dec 04, 2015 11:22 am

Re: Large free upload space?

Thu Aug 12, 2021 6:55 pm

Apologies to all for not replying sooner. I lost my glasses. I do, of course have a backup pair but they managed to migrate offsite: they were in my car which went in for maintenance!

It's primarily as ejolson surmised. I also have an "sdlib" with a lot of bitrot and a complex build. It started off life when C++ Builder v1 came out and has supported Borland, MSVC, cygwin, BSD, linux, aix in its day. It's purpose is wrappers to make things appear the same and very importantly back then, compile fast. Even pre-compiled headers were buggy for instance so the way around that was to have"sdlib" build with them disabled then hide system headers when the app used "sdlib". Some targets corrupted memory if (eg) std::string was used with native types: Builder was particularly good at that (it was Delphi, aka turbo pascal underneath). End result, a complex mess of macros, sometimes having to deal with particular patch versions of certain compilers. Worse, you'd be looking at runtime checks for particular OS versions (particularly windoze) where M$ "fix" something but their docs never admit to it.

The whole lot needs to go. It only builds under linux now. Additionally, a former employer sold "sdlib" as their own so there's potential legal issues which is why you've never seen open-source from me. They also think they own my "go" build system (nothing to do with the language) but two good things have come out of covid: (a) that employer's company doesn't appear to be trading any more. (b) I don't have to keep providing gcc 4.02 compatible versions of "sdlib".

Now I'm fully retired I can do my own thing. Most of "sdlib" has been superseded by better more functional code and as QT is moving towards cmake, I can learn that as well.

I never got round to checking out storage last weekend and won't be able to this weekend because I've decided this weekend is gaming weekend. My new PC won't run win7 so have been missing Skyrim. Here goes trying it with unregistered win10. There may be beer involved. No sane person would interact with win10 without beer! :-)

Heater
Posts: 18508
Joined: Tue Jul 17, 2012 3:02 pm

Re: Large free upload space?

Fri Aug 13, 2021 1:35 am

So I'm wondering what actual feature is it you need from the latest builds of Qt 5 or GCC or LLVM?

Conversely what is missing with the version of Qt 5 that comes in the Pi OS package(s)?
Memory in C++ is a leaky abstraction .

swampdog
Posts: 755
Joined: Fri Dec 04, 2015 11:22 am

Re: Large free upload space?

Tue Aug 17, 2021 6:06 pm

I want the latest QT5 builds because once I get round to sitting down with it, I do not want it changing unless it's under my control. I may not know how to use QT but I have built it many times for others in the past and almost all support issues have arisen from the underlying OS changing.

Additionally I do not know where QT6 is heading. It may be that I'll stick with QT5 for a very long time. As for, LLVM, same as GCC, if there's not a ton of different versions installed, less headaches. After all, we already have clang[5..9] on rpi.

I'm also a bit sick of cmake doing things it shouldn't. I've been using 'configure' until the latest build and on linux mint 18.3, where some system stuff is too old, no problem because the new stuff is earlier in the PATH etc. cmake likes to ignore that and use the ancient system stuff. I then get to root around "CMakeCache.txt" for hints on how to proceed.

Heater
Posts: 18508
Joined: Tue Jul 17, 2012 3:02 pm

Re: Large free upload space?

Tue Aug 17, 2021 6:45 pm

swampdog wrote:
Tue Aug 17, 2021 6:06 pm
I want the latest QT5 builds because once I get round to sitting down with it, I do not want it changing unless it's under my control.
Sounds reasonable.
swampdog wrote:
Tue Aug 17, 2021 6:06 pm
I may not know how to use QT but I have built it many times for others in the past and almost all support issues have arisen from the underlying OS changing.
That implies you need a fixed operating system version. Or at least have it under your control.
swampdog wrote:
Tue Aug 17, 2021 6:06 pm
As for, LLVM, same as GCC, if there's not a ton of different versions installed, less headaches. After all, we already have clang[5..9] on rpi.
And the compilers need to be fixed and under control.

I can understand your motivation here. Although that sounds like a lot of work to supportive OS, compiler libraries, just for some GUI application you seen to have not written yet. If you are providing support into the future for someone else I hope they are paying well.

Would it not be easier to just use whatever Qt support is available and rework the application as and when the environment changes?
Memory in C++ is a leaky abstraction .

ejolson
Posts: 8037
Joined: Tue Mar 18, 2014 11:47 am

Re: Large free upload space?

Tue Aug 17, 2021 7:04 pm

Heater wrote:
Tue Aug 17, 2021 6:45 pm
swampdog wrote:
Tue Aug 17, 2021 6:06 pm
I want the latest QT5 builds because once I get round to sitting down with it, I do not want it changing unless it's under my control.
Sounds reasonable.
swampdog wrote:
Tue Aug 17, 2021 6:06 pm
I may not know how to use QT but I have built it many times for others in the past and almost all support issues have arisen from the underlying OS changing.
That implies you need a fixed operating system version. Or at least have it under your control.
swampdog wrote:
Tue Aug 17, 2021 6:06 pm
As for, LLVM, same as GCC, if there's not a ton of different versions installed, less headaches. After all, we already have clang[5..9] on rpi.
And the compilers need to be fixed and under control.

I can understand your motivation here. Although that sounds like a lot of work to supportive OS, compiler libraries, just for some GUI application you seen to have not written yet. If you are providing support into the future for someone else I hope they are paying well.

Would it not be easier to just use whatever Qt support is available and rework the application as and when the environment changes?
While Qt may be one of the more stable GUI toolkits, my understanding is that such things are typically in greater flux with more bugs than most compilers. In any case, if any bugs are encountered, the first thing the upstream developers usually ask is if you tried the latest version. For this reason among others it's better to use the latest version when making anything new.

swampdog
Posts: 755
Joined: Fri Dec 04, 2015 11:22 am

Re: Large free upload space?

Wed Aug 18, 2021 7:52 pm

Heater wrote:
Tue Aug 17, 2021 6:45 pm
swampdog wrote:
Tue Aug 17, 2021 6:06 pm
I want the latest QT5 builds because once I get round to sitting down with it, I do not want it changing unless it's under my control.
Sounds reasonable.
swampdog wrote:
Tue Aug 17, 2021 6:06 pm
I may not know how to use QT but I have built it many times for others in the past and almost all support issues have arisen from the underlying OS changing.
That implies you need a fixed operating system version. Or at least have it under your control.
Yes. My premise is, though, the situation where you haven't. My old PC uses raid1 and LVM which linux mint 18 installer did not support. I made that happen post install. My new PC is still booting that mint18, albeit off a pair of ssd drives (imo a waste).

New PC will eventually boot mint20 off an nvme drive so raid1 can go (I'll use timeshift) but still use LVM. I also want to experiment with mint20 not being a development box but instead using KVM to run another mint20 inside itself which is the dev box. The ability to be able to simply copy /usr/local/QT/ around for a while will be very useful.
Heater wrote:
Tue Aug 17, 2021 6:45 pm
swampdog wrote:
Tue Aug 17, 2021 6:06 pm
As for, LLVM, same as GCC, if there's not a ton of different versions installed, less headaches. After all, we already have clang[5..9] on rpi.
And the compilers need to be fixed and under control.

I can understand your motivation here. Although that sounds like a lot of work to supportive OS, compiler libraries, just for some GUI application you seen to have not written yet. If you are providing support into the future for someone else I hope they are paying well.

Would it not be easier to just use whatever Qt support is available and rework the application as and when the environment changes?
It appears I didn't make myself clear on that. The /usr/local/QT/ tarball would only be used during development. The expectation is the "finished" app would be compiled normally, against the system QT.

swampdog
Posts: 755
Joined: Fri Dec 04, 2015 11:22 am

Re: Large free upload space?

Wed Aug 18, 2021 8:06 pm

ejolson wrote:
Tue Aug 17, 2021 7:04 pm
Heater wrote:
Tue Aug 17, 2021 6:45 pm
swampdog wrote:
Tue Aug 17, 2021 6:06 pm
I want the latest QT5 builds because once I get round to sitting down with it, I do not want it changing unless it's under my control.
Sounds reasonable.
swampdog wrote:
Tue Aug 17, 2021 6:06 pm
I may not know how to use QT but I have built it many times for others in the past and almost all support issues have arisen from the underlying OS changing.
That implies you need a fixed operating system version. Or at least have it under your control.
swampdog wrote:
Tue Aug 17, 2021 6:06 pm
As for, LLVM, same as GCC, if there's not a ton of different versions installed, less headaches. After all, we already have clang[5..9] on rpi.
And the compilers need to be fixed and under control.

I can understand your motivation here. Although that sounds like a lot of work to supportive OS, compiler libraries, just for some GUI application you seen to have not written yet. If you are providing support into the future for someone else I hope they are paying well.

Would it not be easier to just use whatever Qt support is available and rework the application as and when the environment changes?
While Qt may be one of the more stable GUI toolkits, my understanding is that such things are typically in greater flux with more bugs than most compilers. In any case, if any bugs are encountered, the first thing the upstream developers usually ask is if you tried the latest version. For this reason among others it's better to use the latest version when making anything new.
I did once build a QT5 full-debug. I don't recall how big it eventually turned out as - but the LVM partition I assigned for the object files is 64Gb. This would mean that part was either >32Gb or >48Gb as I either would have doubled it when it ran out of space or added another 16Gb, depending how much LVM free space was around at the time.

That's the trouble with being ruthless with backups. I dunno how big the end install was. The moment you delete it, you need it. The mountpoint for the install, /usr/local/QT/5125d/ is dated Jan 25th 2020 and when did I delete it? Last weekend!

Heater
Posts: 18508
Joined: Tue Jul 17, 2012 3:02 pm

Re: Large free upload space?

Thu Aug 19, 2021 4:23 am

swampdog wrote:
Wed Aug 18, 2021 7:52 pm
It appears I didn't make myself clear on that. The /usr/local/QT/ tarball would only be used during development. The expectation is the "finished" app would be compiled normally, against the system QT.
I don't understand that.

If you need the latest and greatest version of Qt and GCC and LLVM to develop a Qt application then it will not be possible to compile the finished application with the system Qt.

Conversely if the finished application will compile with the systems Qt then there was no point in going to all the trouble of building the latest Qt and GCC and LLVM etc.

That was certainly the case when I was built the Qt 5 libs with frame buffer and 3d acceleration support back the day when the Raspbian package was still Qt 4 and did not support frame buffer rendering.
Memory in C++ is a leaky abstraction .

swampdog
Posts: 755
Joined: Fri Dec 04, 2015 11:22 am

Re: Large free upload space?

Thu Aug 19, 2021 10:15 am

Heater wrote:
Thu Aug 19, 2021 4:23 am
swampdog wrote:
Wed Aug 18, 2021 7:52 pm
It appears I didn't make myself clear on that. The /usr/local/QT/ tarball would only be used during development. The expectation is the "finished" app would be compiled normally, against the system QT.
I don't understand that.

If you need the latest and greatest version of Qt and GCC and LLVM to develop a Qt application then it will not be possible to compile the finished application with the system Qt.

Conversely if the finished application will compile with the systems Qt then there was no point in going to all the trouble of building the latest Qt and GCC and LLVM etc.

That was certainly the case when I was built the Qt 5 libs with frame buffer and 3d acceleration support back the day when the Raspbian package was still Qt 4 and did not support frame buffer rendering.
Methinks you're neglecting to consider the timeframe. While I'm learning how to program QT and use it's codebase, the system QT will be progressing. I'll likely start with porting a few of the more useful features from "sdlib", none of which are GUI related plus make other stuff work again.

For instance, the sdlib socket code (recall originally compilable for windoze) is horrible because non of it is blocking. For those unaware, if the main thread under windoze dies the app does not stop running: windoze picks a still running thread (effectively random) and promotes that to the main thread.

There are more fundamental issues going back to windoze TCHAR (8bit or 16bit char) so the whole sdlib codebase is littered with nasty macros because the borland and/or/both visualc++ compilers were not interested in standards. Simply attempting to use std::string in VCL (borland Visual Component Library) could cause a crash, the VCL is actually delphi so you've got turbo pascal AnsiString. Sometimes something as simple as..

Code: Select all

AnsiString as;
std::string cs;
as=cs //or vice-versa
..could crash when used in the wrong context, especially threads. Note pascal strings are not nul terminated, they have a length field and are indexed from [1] not [0] and implemented by reference. Most of that code from sdlib has been deleted but the infrastructure still remains. I had to look at http://docwiki.embarcadero.com/RADStudi ... s_(Delphi) myself to refresh my memory on how bad it was. Note the other string types that needed to be dealt with also. Then throw visualc++ into the mix. Utter madness.

Consequently, sdlib does not support utf8. QT infrastructure will inherently fix these issues by virtue of them not being a problem in the first place.

By the time I get round to writing a non-trivial GUI app, /usr/local/QT/ will likely be behind the system QT so no problem compiling. In the meantime I can make use of newer language features and tools. I only vaguely know about C++ "auto" and lamba for instance because sdlib had to work all the way back to gcc 4.0.2.

Return to “Off topic discussion”