User avatar
mpthompson
Posts: 620
Joined: Fri Feb 03, 2012 7:18 pm
Location: San Carlos, CA
Contact: Website

Re: Debian Hard Float (armhf) for RPi

Mon Apr 09, 2012 4:36 pm

Plugwash, thanks for checking in.  Much of the advice you gave me weeks ago was very valuable in getting things to this point.  Getting the iMX53 up and running was a fun adventure.

plugwash said:

Frankly even with official debian ports you have no real gaurantee that everything you install will function as advertised. Some packages have build time test suites but many don't (and sometimes disabling a failing test with unknown real world applicability is considered a lesser evil than not building the package at all) so really the only way to get problems rooted out is to get users using the port and reporting issues.
Yeah, I kind of suspected as much.

The three years requirement only applies in the specific case of distributing software in binary only form with a written offer to supply the source code it is not relevant to online distribution.
For digital download the relavent clause in version 2 of the GPL says

"If distribution of executable or object code is made by offering access to copy from a designated place, then offering equivalent access to copy the source code from the same place counts as distribution of the source code, even though third parties are not compelled to copy the source along with the object code. "

GPLv3 is actually a bit looser in this regard but since there is some software in debian that is GPLv2 only you need to comply with the requirements of GPLv2.

So basically if you are posting binary packages then you need to post the corresponding source packages but you only need to keep the source packages for as long as you keep the binary packages. You do not need to keep a historic archive of source packages.


OK, thanks for the clarification.  I ran into some folks on the IRC were hardcore about the three year requirement.  I think the basic answer is I need to take the time to learn this.  It's just much less fun that building code and getting things work.


If you haven't already done so your repositry should be setup to handle both source and binary packages. You should also adopt a versioning convention that makes it clear which packages you have modified (e.g. using +rpi1 in the version numbers of packages you modify)


Ah, terrific suggestion.  Fortunately, I've not had to modify any package yet except for the gcc compiler and one or two others to coax the compiler to build with certain optimization options. BTW, I noticed some of the armh packages are stamped with ":armhf" in the binary package name/version.  Is this for the same reason?

With regards to my repository, I've just got things setup at the most basic level on my internal server for binary files.  I obviously need to get more sophisticated with things.  Are there specific tools I should be looking to help manage upload and storage of packages in a repository?  Right now, I'm just using apt-ftparchive to lay things out and build the various indexes.

Likely you would have to coordinate with the foundation to get hardfloat versions of the closed binaries built.
Yeah, I have faith that once enough progress is made on a hard float version of Debian that shouldn't be too much of an issue.  It really should be just a recompile.  Perhaps I'm being naive though.


Four i.mx53 quickstart boards are enough to pretty much keep up with debian sid though with that number there will be backlogs when big changes come through. Keeping up with wheezy should be easier though I don't have any figures for exactly how much easier.


I've observed about a few dozen package changes in wheezy since I began working with it a few weeks ago -- at least in the small set of packages I have installed on my systems.   Keeping up with wheezy should be possible once things are automated.

(nearly) everything debian uses is donated by someone and it's mostly spoken for. There isn't some big pool of arm boxes sitting around to build another arm variant.
Yep, pretty much expected this was the case.


In general autobuilder hardware is donated by those who care about the port.

Afaict in the case of armel and armhf the build hardware was donated by linaro but since broadcom is not a member of linaro and almost everyone else seems to have moved on from armv6 it seems unlikely they would provide financial support for an armv6 variant of armhf.


As long as the costs are reasonable (ie. can be kept under a few thousand), I can help build out the infrastructure.  The only thing holding me back is some worries that the future of the RPi is somewhat murky and the hardware delays aren't exactly encouraging in this regards.  However, hopefully it's all just a bump in the road, and RPi will get back its mojo.


I think it's unlikely that this flavour will ever be accepted into the official archive. Firstly because the archive can't handle two sets of packages with the same architecture name (and it is correct to maintain the armhf architecture name because these packages are compatible with the official armhf packages) and secondly because it's really only targeted at one particular (though admittedly very popular) board.


I expect this as well.  Where I could really get support from Debian is just having a few interested mentors/advisors who can help steer development of the port so that it honors the spirit of Debian and spend the time to give guidance to myself and others over whatever bumps in the road we're likely to encounter in trying to launch a full Debian port for the RPi.  Would you, or someone you know, be interested in mentoring/advising in this manner?  I realize it's these things sometimes turn into a major time commitment as newbies come up to speed.


The binary packages themselves are not normally signed, only the changes files used to upload them to a repository and the packages files in the repository. It is generally suggested to generate keys explicitly for these purposes (and sign them with your main key) to reduce the risk of compromise to your main key.


I know next to nothing about this stuff, but I was able to get a key generated for my mini-repository I have setup.  I'll look into this more.


Speaking of digital signatures if you have not already done so you should get your key into the web of trust and preferably signed by some debian developers.


More stuff to learn about .  I have zero clue what the "web of trust" is.  I'll hopefully be attending a Debian user group meeting shortly and these are some great questions for me to bring there.


Afaict lintian is not normally run as part of the package build process.


I think lintian is run everytime I build a package.  I turn it off though using a switch in debuild as it is a major time sink and I'm not changing the packages.

Thanks again for all the advice.

User avatar
mpthompson
Posts: 620
Joined: Fri Feb 03, 2012 7:18 pm
Location: San Carlos, CA
Contact: Website

Re: Debian Hard Float (armhf) for RPi

Mon Apr 09, 2012 4:38 pm

pepedog said:


I love thump2



Typos, typos, typos. There is always a wiseguy.

cgenner
Posts: 1
Joined: Mon Apr 09, 2012 5:07 pm

Re: Debian Hard Float (armhf) for RPi

Mon Apr 09, 2012 6:24 pm

mpthompson, I've a friend who is a Debian developer and Debian Sys Admin.  I see him most Fridays so I'll ask him what the process is if you don't have much luck with the San Fran Debian group on Wednesday.

From previous conversations most of the hardware they have is donated by people who want Debian to work on it, for example HP donate a load of servers and hosAs for the hardware ting so that there are debian hardware packages available in the core distro.

I'm interested in the work you are doing and want to encourage to keep at it.  If you need any help then please drop me a line.  I would like to be able to donate the resources of my RPi to a Debian build cluster.

User avatar
abishur
Posts: 4477
Joined: Thu Jul 28, 2011 4:10 am
Location: USA
Contact: Website

Re: Debian Hard Float (armhf) for RPi

Mon Apr 09, 2012 6:45 pm

mpthompson said:


Abishur said:


When it comes to recompiling the packages with hard float instructions, is that as simple a matter as downloading the source code and compiling it on an distribution designed to take advantage of hard floats, or are there some special switches you need to use when you do this?


....

However, the bottom line, the packages are downloaded and simply recompiled.  When I do run into issues, I'm learning enough to better handle the problems than I was just a week or two ago.



That's what I thought, as the R-Pi's are released into the wild, I certainly wouldn't mind devoting some time to recompiling a set of packages.  I lack the experience to handle all but the most basic issues that might arise, but I could at least try to work things such that I was a "tier 1" kind of person.  Re-compile it, if it works, upload the working package somewhere, and if it doesn't tell the "tier 2" person so they can work on it.

Enough people like me and we could (hopefully) make short(ish) work of those packages.

I dunno if that's something feasible or not, but I thought I'd throw my support out there.
Dear forum: Play nice ;-)

User avatar
SN
Posts: 1014
Joined: Mon Feb 13, 2012 8:06 pm
Location: Romiley, UK
Contact: Website

Re: Debian Hard Float (armhf) for RPi

Mon Apr 09, 2012 7:08 pm

+1 abishur
Steve N – binatone mk4->intellivision->zx81->spectrum->cbm64->cpc6128->520stfm->pc->raspi ?

User avatar
mpthompson
Posts: 620
Joined: Fri Feb 03, 2012 7:18 pm
Location: San Carlos, CA
Contact: Website

Re: Debian Hard Float (armhf) for RPi

Mon Apr 09, 2012 10:00 pm

Abishur said:

That's what I thought, as the R-Pi's are released into the wild, I certainly wouldn't mind devoting some time to recompiling a set of packages.  I lack the experience to handle all but the most basic issues that might arise, but I could at least try to work things such that I was a "tier 1" kind of person.  Re-compile it, if it works, upload the working package somewhere, and if it doesn't tell the "tier 2" person so they can work on it.
Enough people like me and we could (hopefully) make short(ish) work of those packages.


Abishur, thanks for the offer.  Let's hope the RPi's come out soon so that we can find out how they handle compilations.  The minimal memory on the RPi is going to be a challenge when it comes to many of the packages which are painful to build even on a system running at 1GHz with 1GB of RAM.  However, there are many small packages which would build find on the RPi.

User avatar
abishur
Posts: 4477
Joined: Thu Jul 28, 2011 4:10 am
Location: USA
Contact: Website

Re: Debian Hard Float (armhf) for RPi

Mon Apr 09, 2012 10:08 pm

Is the memory issue something that would just make everything compile *really* slowly, or would it cause it to not be able to compile at all?  If it's just a matter of time... well it would suck but you leave it and come back to it, enough people willing to let it run like that and things would eventually get there....
Dear forum: Play nice ;-)

User avatar
mpthompson
Posts: 620
Joined: Fri Feb 03, 2012 7:18 pm
Location: San Carlos, CA
Contact: Website

Re: Debian Hard Float (armhf) for RPi

Mon Apr 09, 2012 10:30 pm

Here are the things on my plate this week to move things along:

1. Hopefully meet with some Debian people at the user group meeting in San Francisco and solicit help, advice or whatever.  Perhaps I'll find some people really interested in helping make something happen with the RPi.

2. Manually build the packages required to bring up sbuilder -- the tool that will automate the building of packages.  This is an additional 160 packages, but I need to see if they all need to be ARMv7 clean.  If not, it will a lot.

3. Get a better understanding of code signing related to building and distributing packages via repository.  I'm woefully ignorant of how things work in this area.

4. Look into getting a hosted server where I can post the packages and source code I have built and share with others. This will probably take a week or more, but I'll get it started.  I'll also post tools for getting everything running under QEMU as well so that even people without ARM systems yet can examine and play the packages.

User avatar
mpthompson
Posts: 620
Joined: Fri Feb 03, 2012 7:18 pm
Location: San Carlos, CA
Contact: Website

Re: Debian Hard Float (armhf) for RPi

Mon Apr 09, 2012 10:39 pm

Abishur said:


Is the memory issue something that would just make everything compile *really* slowly, or would it cause it to not be able to compile at all?  If it's just a matter of time... well it would suck but you leave it and come back to it, enough people willing to let it run like that and things would eventually get there....


I believe the issue would be swapping.  It would making things go really slow.  I mean REALLY, REALLY slow.  Also, in the email threads over at the debian-arm list, I've seen postings about USB disks not be very suitable for large compilations as so much intense disk activity over the USB port leads to kernel instability.  Not sure why this is, but I do know that, in general, the people at Debian look for SATA devices on the build machines for armel and armhf.

Since it's still unknown how long it will take to get RPi devices, using QEMU on a PC may be a reasonable substitute in the meantime.  On a fast PC, compilations will be as fast or faster than occur on the Pi -- especially because you can artificially create large memory ARM systems.  As I mentioned above, if I can post the packages, I'll also post a QEMU kernel and filesystem image running an emulated ARMv6+VFP system that should be suitable for doing builds.

If I can get sbuild working well, I'm going to look into purchasing a handful of additional Freescale iMX53 QSB systems to turn into a build farm.  Then we'll really start cranking.

plugwash
Forum Moderator
Forum Moderator
Posts: 3439
Joined: Wed Dec 28, 2011 11:45 pm

Re: Debian Hard Float (armhf) for RPi

Mon Apr 09, 2012 10:40 pm

mpthompson said:


OK, thanks for the clarification.  I ran into some folks on the IRC were hardcore about the three year requirement.  I think the basic answer is I need to take the time to learn this.  It's just much less fun that building code and getting things work.


I wouldn't worry too much about it. I've never heard of someone getting into any trouble over minor technical violations of a license as long as when push comes to shove they do actually release the source code in question.


Ah, terrific suggestion.  Fortunately, I've not had to modify any package yet except for the gcc compiler and one or two others to coax the compiler to build with certain optimization options. BTW,


Even if the modifications are trivial though (e.g changing optimisation levels) you still need to keep track of them so you can port them forward to new versions of packages.


I noticed some of the armh packages are stamped with ":armhf" in the binary package name/version.  Is this for the same reason?


The filenames of binary packages contain _armhf to identify them as armhf packages in the archive. This has nothing to do with architecture specific modificaitons. Packages for all architectures in the official archive are built from the same source packages (though those source packages may contain conditionals that make them behave differently on different architectures).

:armhf in the package manager interface is notation related to multiarch, don't worry about that for now.


With regards to my repository, I've just got things setup at the most basic level on my internal server for binary files.  I obviously need to get more sophisticated with things.  Are there specific tools I should be looking to help manage upload and storage of packages in a repository?


There are many tools that can be used to manage a repositry. http://wiki.debian.org/HowToSe.....able_Tools

dak is the one used by the official archive but it's not normally used for unofficial repositries due to it's complexity.

mini-dak ( http://www.hadrons.org/~guillem/debian/mini-dak/ ) is the tool used to run the debian-ports.org archive and is probablly where I would look first if I was doing a project like this.


Right now, I'm just using apt-ftparchive to lay things out and build the various indexes.


apt-ftparchive is fine for a local mini-repository but I don't think it's really designed for a project of this scale/type


Yeah, I have faith that once enough progress is made on a hard float version of Debian that shouldn't be too much of an issue.  It really should be just a recompile.  Perhaps I'm being naive though.


Yeah hopefully it should.


As long as the costs are reasonable (ie. can be kept under a few thousand), I can help build out the infrastructure.  The only thing holding me back is some worries that the future of the RPi is somewhat murky and the hardware delays aren't exactly encouraging in this regards.  However, hopefully it's all just a bump in the road, and RPi will get back its mojo.


I know the feeling, it's difficult to put too much into a project like this when one doesn't actually have the hardware in question and are looking at unknown lead times to get it.


I expect this as well.  Where I could really get support from Debian is just having a few interested mentors/advisors who can help steer development of the port so that it honors the spirit of Debian and spend the time to give guidance to myself and others over whatever bumps in the road we're likely to encounter in trying to launch a full Debian port for the RPi.  Would you, or someone you know, be interested in mentoring/advising in this manner?


I will certainly continue to advise you as best I can, once you make the repositry available I may even help with some actual troubleshooting.

As for following the spirit of debian I think the main things are making sure you release source, pushing any changes that are not specific to the Pi port back to debian, and generally trying to keep the differences between your port and normal debian to a minimum .


More stuff to learn about .  I have zero clue what the "web of trust" is.


With PGP/GPG each key has one or more UIDs associated with it. Each UID usually contains a name and an email address. UIDs on a key can be signed by other keys.

When you sign a UID on someones key you are basically saying you have checked their ID and that it matches who they claim to be in their UID. Typically keysigning is reciprocal, you check their ID and sign their key and they check your ID and sign your key. It is also common to send the signatures by email to the email addresses in the UIDs rather than uploading them direct to the keyservers to show that the person has access to the email address they have placed in their UID.

These signatures form a directed graph known as the "web of trust", the more independent paths there are from your key to another key in the web of trust and the shorter those paths are the greater the assurance you can have that you are dealing with who you think you are dealing with.

So your personal key (which should at the very least be protected with a passphrase and not kept on systems that other people have access to) should be signed by some other people in the web of trust (preferably including debian developers if at all possible*). The keys used for autosigning the repo and uploads to it (if required by the archive setup you are using) should have UIDs that clearly identify them as automatic signing keys and should be signed by your personal key.

http://wiki.debian.org/Keysigning


I think lintian is run everytime I build a package.  I turn it off though using a switch in debuild as it is a major time sink and I'm not changing the packages.


Ah never used debuild myself, i've always used dpkg-buildpackage directly.

Anyway lintian is not normally run as part of the build process on autobuilders and I see no reason for you to run it on yours.

* You will need your key to be signed by debian developers if you ever want to become a DM or DD, further debian developers tend to be well connected in the web of trust and having your key signed by them should give your users more confidence that you are who you say your are.

plugwash
Forum Moderator
Forum Moderator
Posts: 3439
Joined: Wed Dec 28, 2011 11:45 pm

Re: Debian Hard Float (armhf) for RPi

Mon Apr 09, 2012 11:05 pm

Abishur said:


Is the memory issue something that would just make everything compile *really* slowly,


If you run out of memory and your system has swap setup your system will start using the swap. A little bit of swap to deal with memory that is still allocated but hasn't been touched in a while is not necessarily too bad but if the working set (that is the stuff you need now or in the near future) gets bigger than memory things will slow down massively and the smaller memory gets relative to the working set the more often things will need to be swapped in and out and the slower things will get.

Bear in mind that even on an I.MX53 quickstart board with 1 gigabyte of ram and native SATA some packages are already swapping heavily to the point that they take days to build. On a Pi i'd expect it to take far longer than that and that is assuming that the USB hard drive was stable enough to stay the course.

If you run out of both memory and swap things will fail.

Building in emulation may be worth considering/benchmarking. Debian themselves don't it because they have considered the ability to run stable autobuilders to be an important part of proving an architecture.

It's also theoretically possible to use qemu in user mode for building packages though I doubt that it's a well tested configuration

plugwash
Forum Moderator
Forum Moderator
Posts: 3439
Joined: Wed Dec 28, 2011 11:45 pm

Re: Debian Hard Float (armhf) for RPi

Mon Apr 09, 2012 11:18 pm

mpthompson said:


2. Manually build the packages required to bring up sbuilder -- the tool that will automate the building of packages.  This is an additional 160 packages, but I need to see if they all need to be ARMv7 clean.  If not, it will a lot.


Really the only stuff that absoloutely has to be free of ARMv7 code is stuff that may get incoportated into newly built packages. That is any static libraries inside the build chroot (probablly best if everything in the chroot is ARMv6 just to be sure). Stuff outside the build chroot can happilly be armv7 (assuming the system running the autobuilder has an armv7 processor).

User avatar
mpthompson
Posts: 620
Joined: Fri Feb 03, 2012 7:18 pm
Location: San Carlos, CA
Contact: Website

Re: Debian Hard Float (armhf) for RPi

Tue Apr 10, 2012 3:14 am

plugwash said:

Really the only stuff that absoloutely has to be free of ARMv7 code is stuff that may get incoportated into newly built packages. That is any static libraries inside the build chroot (probablly best if everything in the chroot is ARMv6 just to be sure). Stuff outside the build chroot can happilly be armv7 (assuming the system running the autobuilder has an armv7 processor).
Yeah, I'm looking at sbuild right now.  There are a few extra packages it has debootstrap include that I need to get built, but virtually all the extra packages look to be on the ARMv7 side of chroot.

User avatar
mpthompson
Posts: 620
Joined: Fri Feb 03, 2012 7:18 pm
Location: San Carlos, CA
Contact: Website

Re: Debian Hard Float (armhf) for RPi

Tue Apr 10, 2012 6:45 am

Having fun with sbuild this evening.  It was kinda spooky when it worked for the first package I built as there is a LOT going on under the hood.  On the second package I ran into an issue where I need to build packages that it depends on.  Hmmm.  There must be a way to automate/optimize this process.

Found some good references on the web that seem to point me in the right direction.  Will probably take me another day or two to sort things out.  However, it will be worth it.  Even though I don't have any infrastructure built around sbuild yet to automatically feed packages to it to be built, it should greatly simplify the procedures I'm currently using to manually build packages.

jamesh
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 23424
Joined: Sat Jul 30, 2011 7:41 pm

Re: Debian Hard Float (armhf) for RPi

Tue Apr 10, 2012 8:31 am

Abishur said:


Is the memory issue something that would just make everything compile *really* slowly, or would it cause it to not be able to compile at all?  If it's just a matter of time... well it would suck but you leave it and come back to it, enough people willing to let it run like that and things would eventually get there....


Just as a idea of compile times.

I compiled Stellarium on a desktop PC (AMD Athlon(tm) II X2 215 Processor × 2, 4GB RAM) and it took 10 minutes. On a Raspi (old board, old kernel, 4GB SD card, Swap on SD card partition, no HDD) it took over 5 hours. It built fine though.

If I wanted to build lots of packages, rather than QEMU, I'd use a cross compiler. Much faster, but you do have the rare issue of possible incompatibilities)
Principal Software Engineer at Raspberry Pi (Trading) Ltd.
Contrary to popular belief, humorous signatures are allowed. Here's an example...
"My grief counseller just died, luckily, he was so good, I didn't care."

SirLagz
Posts: 1705
Joined: Mon Feb 20, 2012 8:53 am
Location: Perth, Australia
Contact: Website

Re: Debian Hard Float (armhf) for RPi

Tue Apr 10, 2012 9:02 am

Once I get my Pi(s) I would happily contribute to the build effort, or use crosscompilers if there is an easy way for me to start dipping my toes into this.

Thank you for all the effort you have put in so far mpthompson

As a Debian user myself, you have my full support and if I can help in any way, I want to
My Blog - http://www.sirlagz.net
Visit my blog for Tips, Tricks, Guides and More !
WiFi Issues ? Have a look at this post ! http://www.raspberrypi.org/phpBB3/viewtopic.php?f=28&t=44044

User avatar
jojopi
Posts: 3079
Joined: Tue Oct 11, 2011 8:38 pm

Re: Debian Hard Float (armhf) for RPi

Tue Apr 10, 2012 9:15 am

JamesH said:

If I wanted to build lots of packages, rather than QEMU, I'd use a cross compiler. Much faster, but you do have the rare issue of possible incompatibilities)
I would not be too worried about incompatibilities -- it is the same compiler code either way.

The problem with cross compiling is that the build systems for most packages do not support it very well.  All the configuration tests examine the build platform instead of the target; intermediate binaries required during the build are compiled for the wrong architecture; and finally any validation test suites cannot be executed at all.

"distcc" apparently offers a compromise where the build system runs on the target host, but passes preprocessed C/C++ code over the network to be cross-compiled.  I have not yet tried this, however.

Chris.Rowland
Posts: 239
Joined: Thu Jan 12, 2012 5:45 pm

Re: Debian Hard Float (armhf) for RPi

Tue Apr 10, 2012 9:29 am

I'm also happy to contribute to a build process but don't understand how this is OK but the distributed build process I suggested in post 60 isn't.

shirro
Posts: 248
Joined: Tue Jan 24, 2012 4:54 am

Re: Debian Hard Float (armhf) for RPi

Tue Apr 10, 2012 10:02 am

jojopi said:


"distcc" apparently offers a compromise where the build system runs on the target host, but passes preprocessed C/C++ code over the network to be cross-compiled.  I have not yet tried this, however.


I have been using distcc to compile far too many kernels and u-boots just because I wanted to sit in front of an arm box and it is awesome. I can't believe I haven't used it before. I suspect cross-compiling on a decent box is still a fair bit faster but as you say there are possible config issues.

I have been planning to make a little Pi image for myself – will probably cross compile the bootstrap now (my natively built armv6hf bootstrap hasn't been working out so well) and then build the rest in an armv6 chroot on an imx using distcc and a bunch of old laptops, desktops and even my mythbox. If it doesn't build properly it isn't a big deal because it is just a silly personal project.

I appreciate the compile all native approach that mpthompson is taking. It must take a lot of patience but it removes a potential source of problems and is how Debian, Ubuntu and Fedora do it.

plugwash
Forum Moderator
Forum Moderator
Posts: 3439
Joined: Wed Dec 28, 2011 11:45 pm

Re: Debian Hard Float (armhf) for RPi

Tue Apr 10, 2012 12:44 pm

Abishur said:


That's what I thought, as the R-Pi's are released into the wild, I certainly wouldn't mind devoting some time to recompiling a set of packages.  I lack the experience to handle all but the most basic issues that might arise, but I could at least try to work things such that I was a "tier 1" kind of person.  Re-compile it, if it works, upload the working package somewhere, and if it doesn't tell the "tier 2" person so they can work on it.


IMO whereever possible packages should be built by autobuilders (some packages will have to be built manually because of loops in the build-dependencies but those should be by far the exception). Having a load of people manually building and uploading packages seems like a recipie for trouble to me.

Where help is likely to be needed is in figuring out how to deal with failures and/or dealing with packages that build successfully but fail the test for armv7 code. Often it's pretty simple stuff like changing the optimisation level to work arround an internal compiler error or removing ill-advised compiler flags added by misguided maintainers (either in debian or upstream) but someone still needs to do the work of finding out what optimisation levels work and how to persuade the package's build system to use them.

I think the priority for this project needs to be to get a proper public repositry (with source) online with at least one autobuilder running. Once that is done collaboration on rooting out the issues can start.

User avatar
mpthompson
Posts: 620
Joined: Fri Feb 03, 2012 7:18 pm
Location: San Carlos, CA
Contact: Website

Re: Debian Hard Float (armhf) for RPi

Tue Apr 10, 2012 6:25 pm

plugwash said:

I think the priority for this project needs to be to get a proper public repositry (with source) online with at least one autobuilder running. Once that is done collaboration on rooting out the issues can start.
Thanks plugwash, this is the approach I'm working towards. My own preference is to not stray too far from the beaten path when it comes to creating a Debian repository and the autobuild process.  To the extent possible, simply replicate what has already been done with other unofficial Debian ports.  This way, as issues are encountered, there is a much better chance of getting help from knowledgeable people.

I didn't plan on kicking off working on a repository so soon, but I can see there is a demand for the packages to be made available as soon as possible and avenues by which others can contribute to making all this happen.

I'm setting a goal to have the first public repository up by the end of next week (say around April 20th) that people can start looking at.  I'm looking at hosting providers, registering a domain name and setting up DAK (Debian Archive Kit) to manage the repository now.  At a minimum, the repository will have all the packages necessary to do a debootstrap installation of Debian armhf, install build essential, run an ssh client/server, and a handful of other miscellaneous packages needed to support Debian building itself.  It's doubtful I'll have a kernel package at that time, but I'll have information on bringing up Debian on an emulated ARMv6+VFP under QEMU or converting the Foundations Debian armel release to running armhf packages on real RPi hardware -- that's how I created the test image for Dom.  That can be done either standalone or under chroot.

For people who are ambitious and want to help out, the following is my recommendation.  There will be enough packages available in the repository to get your own test and build system running either under QEMU or, preferably, on a capable ARM system such as the Freescale iMX53 Quick Start Board.  As the autobuilder encounters troublesome packages we'll flag those and so that someone can examine what the issues might be using their local test and build environment.  Once a solution is found (whether it's simply adjusting optimization settings, making sure that dependent libraries are built ARMv6 first or working through upstream source issues) we can then feed whatever changes need to be made to the autobuilder so the troublesome package is built and put into the repository.  The more eyes we have looking at troublesome packages, the faster we'll get to the point where an interesting subset of Debian armhf becomes available for the RPi.

I believe trying to come up with some type of distributed model for building packages will introduce more overhead and avenues for error than it will be worth.  Perhaps we can evolve to that, but keeping it simple and with the tried & true approach is probably best for now.  I don't want to put a damper on anyone's ambition to help, but in my experience having too many moving parts in the system when first set up is usually a recipe for failure. Gotta first crawl, walk and then we can run.  I'm still very much at the crawling stage.

I'll keep people up to date in this thread as the pieces come together over the next week or two.  I'll do my very best to make that interested people have live RPi optimized Debian armhf packages in their own hands by the end of this month at the latest.

User avatar
abishur
Posts: 4477
Joined: Thu Jul 28, 2011 4:10 am
Location: USA
Contact: Website

Re: Debian Hard Float (armhf) for RPi

Tue Apr 10, 2012 6:47 pm

Sound great!  I don't suppose while we're tweaking things, we could make it so that Debian doesn't stall for a long time when loading the R-Pi when it's not connected to a network?  Probably the wrong place to bring this up, but I thought I'd throw it out there for what it's worth
Dear forum: Play nice ;-)

User avatar
mpthompson
Posts: 620
Joined: Fri Feb 03, 2012 7:18 pm
Location: San Carlos, CA
Contact: Website

Re: Debian Hard Float (armhf) for RPi

Wed Apr 11, 2012 2:14 am

Abishur said:


I don't suppose while we're tweaking things, we could make it so that Debian doesn't stall for a long time when loading the R-Pi when it's not connected to a network?  Probably the wrong place to bring this up, but I thought I'd throw it out there for what it's worth


As it seems the Foundation will have a new Debian armel image out soon, you may want to send them suggestion to look into that issue.  Since I and 99.999% of everyone else here doesn't have any real hardware we have no way off addressing the issue.  In any case, we will be using the same exact kernel that Debian armel is using.

User avatar
mpthompson
Posts: 620
Joined: Fri Feb 03, 2012 7:18 pm
Location: San Carlos, CA
Contact: Website

Re: Debian Hard Float (armhf) for RPi

Wed Apr 11, 2012 2:24 am

I'm learning a lot about Debian repositories and how to automate the inclusion of new packages into the repositories.

An issue I ran into is that while I've been manually building the various packages up until now I haven't been creating what are known as .changes files that direct the various Debian package building tools how to handle the source code and binaries as they are inserted into the repository.  This means that unless I rebuild all my packages, there will not be source code in the repository until the packages get rebuilt using the automated tools.  I'll have to see if I can work around this or determine if a small subset of the packages have GPL so that only those get rebuilt.  Fortunately, the import of binary .deb files can work independent of the .changes files so I've created a much more robust local repository and imported all my packages into it.

[Update: Nevermind, I found the option to import source from a .dsc file. I should have known better to carefully read through all the options for adding files to a repository.]

plugwash
Forum Moderator
Forum Moderator
Posts: 3439
Joined: Wed Dec 28, 2011 11:45 pm

Re: Debian Hard Float (armhf) for RPi

Wed Apr 11, 2012 10:19 am

I beleive the most debian-like way to run a repo like this would be to import the source first. Then wanna-build and buildd can pick up packages that haven't been built and build them.

Return to “Raspbian”