User avatar
mpthompson
Posts: 620
Joined: Fri Feb 03, 2012 7:18 pm
Location: San Carlos, CA
Contact: Website

Re: Debian Hard Float (armhf) for RPi

Mon May 07, 2012 2:37 pm

atcapollo said:


Sorry novice question what's the difference between Debian and Raspbian armhf? I assumed this was the core packages but looking at the same build targets I guess not.


Raspbian is an unofficial port of Debian Wheezy armhf with compilation settings adjusted to produce optimized hard float code that will run on the Raspberry Pi.  The official Debian Wheezy armhf release is only meant for ARM CPUs that will run ARMv7+VFPv3-D16+Thumb2 code.  The Raspberry Pi, having an ARMv6 CPU, cannot run such code which is the reason Raspbian requires a full rebuild of all the packages.

User avatar
mpthompson
Posts: 620
Joined: Fri Feb 03, 2012 7:18 pm
Location: San Carlos, CA
Contact: Website

Re: Debian Hard Float (armhf) for RPi

Mon May 07, 2012 3:19 pm

Mouser is telling me that my order for the additional Freescale iMX53 QSB boards should ship in two days, on May 9th.  That is good news as I could have them on-line by the end of the week if they do indeed ship on Wednesday.  As we are now hitting larger, more complex packages that take longer to build, we could certainly use the additional horse-power.

User avatar
fjen
Posts: 14
Joined: Mon May 07, 2012 7:15 pm
Location: Germany

Re: Debian Hard Float (armhf) for RPi

Mon May 07, 2012 8:45 pm

any plans to include contrib or non-free? i would need some firmware files and rar/unrar from there.

User avatar
mpthompson
Posts: 620
Joined: Fri Feb 03, 2012 7:18 pm
Location: San Carlos, CA
Contact: Website

Re: Debian Hard Float (armhf) for RPi

Mon May 07, 2012 9:02 pm

fjen said:


any plans to include contrib or non-free? i would need some firmware files and rar/unrar from there.


Right now we are just concentrating on main as that is the core of the system, but it's very likely we'll include as much of contrib and non-free as possible once the initial rebuild of main is complete.  I haven't looked at those repositories yet under Wheezy armhf so I'm really not sure how much incremental work it will be to include them.  Hopefully, not too much.

I do use contrib and non-free under Squeeze and I too would like to see them be part of Raspbian as well.  I'm just not certain how much effort we would put into fixing contrib and non-free packages that might have build problems.

User avatar
mpthompson
Posts: 620
Joined: Fri Feb 03, 2012 7:18 pm
Location: San Carlos, CA
Contact: Website

Re: Debian Hard Float (armhf) for RPi

Mon May 07, 2012 9:12 pm

I'm working on a Raspbian FAQ here:  http://www.raspbian.org/RaspbianFAQ

Let me know if there is additional information about Raspbian that I can provide in a FAQ format.  So far I've included a number of questions that have popped up in this thread.

plugwash
Forum Moderator
Forum Moderator
Posts: 3439
Joined: Wed Dec 28, 2011 11:45 pm

Re: Debian Hard Float (armhf) for RPi

Mon May 07, 2012 10:28 pm

One thing I will say is that while high package counts are nice they are IMO far from the most important thing.

To me once we have a somewhat usable system  the most important thing is to reach the point of "self contained buildability". That is for every source package for which we have architecture dependent binaries we should be able to satisfy the build-depends without going outside raspbian.

This is important for two reasons. Practically because it makes handling updates much easier and philosophically because people should be able to modify and recompile the software we supply without undue difficulty (though ram will always be an issue with some packages ).

User avatar
mpthompson
Posts: 620
Joined: Fri Feb 03, 2012 7:18 pm
Location: San Carlos, CA
Contact: Website

Re: Debian Hard Float (armhf) for RPi

Tue May 08, 2012 1:59 am

Point taken.  While package counts may not be so useful at the practical level of measuring "how ready Raspbian is", it does serve a purpose in helping communicate to the broader audience following this project steady progress is being made with regards to the builds. This helps keep people engaged and interested in the project and reassures them it will be seen through to completion.

As you note, we are not yet at "self contained buildability" which will  indeed be an important milestone.  For those following this project, this means that in order to build certain packages we are still relying on regular Debian Wheezy armhf packages to satisfy build dependencies. Although we can't use these Debian packages in our repository, we can use them to satisfy build dependencies as long as we make sure to verify anything built with them that ARMv7 code didn't leak into the packages that are being built.  When we no longer have to do this, the repository will be self-contained.

Just to be consistent with earlier reporting and with the understanding these number don't necessarily reflect how "done" Raspbian is, on early Monday evening the package counts now stand at:

Debian armhf 34350
Raspbian armhf 26680

User avatar
fjen
Posts: 14
Joined: Mon May 07, 2012 7:15 pm
Location: Germany

Re: Debian Hard Float (armhf) for RPi

Tue May 08, 2012 6:12 am

Thank you for keeping us up to date!

Is it possible to compile things like xbmc directly on the raspi without running out of memory? I just saw that there is only a armhf package in sid of it. Or would you like to include it in an extra repository or even complete sid?

I would also like to ask how you can afford so much build devices, mpthompson? I read you have 7 machines running and ordered some more of these not very cheap boards. You spend much time in this project, what is your motivation?

User avatar
mpthompson
Posts: 620
Joined: Fri Feb 03, 2012 7:18 pm
Location: San Carlos, CA
Contact: Website

Re: Debian Hard Float (armhf) for RPi

Tue May 08, 2012 7:08 am

Fjen, thank you for your interest in Raspbian.

Since neither Plugwash nor I yet have Raspberry Pi hardware we can't really comment on what will or will not compile on the device.  However, we do know that the code we are building will run on the device as others have verified that for us. Also, we expect the performance would be significantly less than the Freescale iMX53 QSB systems we are using to build the software.

There does seem to be a lot of interest in XMBC, but I really don't know much about it.  I notice that there is a Debian Wheezy version of XMBC so I'm pretty sure it will be in our build queue eventually.  Perhaps plugwash can provide a better answer.  Or, there are people working specifically on a port of XBMC for the Raspberry Pi so questions about it may best be directed to those threads.  I would hope that Raspbian would be of use to them, but time will tell what distribution they base their images on.

With regards to my motivation for creating Raspbian, that is probably a pretty complex question to answer.  I have two children that I would love to learn about computers.  However, like many techie dad's, I find it hard to capture their interest the way mine was captured when home personal computers were first introduced in the mid-70's.  I fondly remember my first TRS-80 color computer that I saved up money for a whole year to buy.  So part of my motivation is to figure out if the Raspberry Pi can be used as intended to help me teach my kids about computers.

While investigating the Raspberry Pi a few months ago I noticed that they dropped my favorite distribution of Linux as the preferred distribution and instead chose Fedora.  I was a bit baffled why the Debian community wasn't jumping on the chance to make Debian the premiere distribution on a device that has a good chance of introducing 100,000s of new users to Linux.  Seeing an opportunity, but knowing very little about what it would eventually involve, I jumped into a project that I hope will be appreciated by both the Raspberry Pi and Debian communities.  It's kinda my way to pay back a community that I had always appreciated, but never really participated in prior to this.

Currently, the build cluster consists of five ARM-based build servers and a single intel-based repository server.  I hope to have add three more build servers this week.

So far, the total costs have been about $2000.  It would be great if I could eventually recoup part or all these costs with donations, but if not, I'm still gaining a lot of experience and participating in something I feel is very much worthwhile.  Who knows where it will lead.

grantek
Posts: 4
Joined: Mon Apr 30, 2012 9:03 am

Re: Debian Hard Float (armhf) for RPi

Tue May 08, 2012 9:46 am

just read through the whole thread, I've got a couple of questions...

- Did you end up sorting out your kernel issues with the iMX53s? Are they running at full speed?

- Speaking of kernels, is there any advantage to running a RasPi kernel on your ARMv7 hardware for automated testing once you're free from Debian armhf?

- Do you have a list of problematic packages that you could automate or semi-automate? Something like a bug tracker would be nice, but even just dumping a list on your wiki would let people poke around a bit easier. I've registered on your wiki if you could give me edit access, but I have the same experience with building Debian packages as you did on page 1

- Do you have any plans to do cross-compilation with eg. distcc? If not, do you think you may start hitting swap heavily with "only" 1GB of RAM per node? I know everyone's clamouring for xbmc, I actually thought of that specifically because I've run out of RAM compiling it on an amd64 system with 1GB RAM and no swap...

mdewey
Posts: 37
Joined: Wed Dec 07, 2011 10:47 am
Location: UK

Re: Debian Hard Float (armhf) for RPi

Tue May 08, 2012 12:38 pm

Suggestions for the FAQ with draft answers plus some general feedback.

1 - What is Debian?

Provide link to Wikipedia entry

2- How fast will it be?

Compared to a 10 year old laptop, a five year old laptop, ...

3 - What do I put in my sources.list before I do apt-get?

No idea what the answer here is, but I guess you do

=======================

I would also consider putting some of the pictures from your posts on the site as an archive of how it all happened perhaps with a chronology so that history is recorded somewhere.

A minimal image to write to SD would be helpful in the early stages as the early users can presumably be relied on to install the packages they want.

Michael

User avatar
mpthompson
Posts: 620
Joined: Fri Feb 03, 2012 7:18 pm
Location: San Carlos, CA
Contact: Website

Re: Debian Hard Float (armhf) for RPi

Tue May 08, 2012 3:24 pm

grantek said:

I've registered on your wiki if you could give me edit access, but I have the same experience with building Debian packages as you did on page 1
Done.  Thanks for helping.  Let me know if you have problems adding or editing pages and I'll look into it.

User avatar
mpthompson
Posts: 620
Joined: Fri Feb 03, 2012 7:18 pm
Location: San Carlos, CA
Contact: Website

Re: Debian Hard Float (armhf) for RPi

Tue May 08, 2012 5:22 pm

grantek said:


- Did you end up sorting out your kernel issues with the iMX53s? Are they running at full speed?


I did indeed solve the problem and the solution was staring me in the face the whole time.  I simply needed to install the default Debian Wheezy armhf kernel for iMX51/3 devices.  For some reason, I was under the impression that the kernel didn't work with the iMX53 QSB and wasted a lot of time trying to build my own kernels.  Probably some misinformation I read on a website somewhere.  And yes, the systems are running at their full 1Ghz speed.  Oh well, at lest now I know how to build kernels as Debian packages .


- Speaking of kernels, is there any advantage to running a RasPi kernel on your ARMv7 hardware for automated testing once you're free from Debian armhf?


I'm almost 100% certain that the RPi kernel with Broadcom device drivers will not function on a FreeScale iMX53 based device.  Although they both have ARM based CPU's, all the peripheral devices are very different between the two SOCs (system on chip).  The fact the kernels are different would only impact a small percentage of packages as all kernels pretty much expose standardized Linux device interfaces to the application software.


- Do you have a list of problematic packages that you could automate or semi-automate? Something like a bug tracker would be nice, but even just dumping a list on your wiki would let people poke around a bit easier. I've registered on your wiki if you could give me edit access, but I have the same experience with building Debian packages as you did on page 1


This is something I do need to look into.  Keeping an updated list on a Wiki page with problematic packages is pretty easy.  Providing the logs for those packages in an automated way would be much tougher as custom scripts and such would have to be created.  I would like to see this done, but I can't promise when I'll have time to address in any meaningful way.

To earn to build packages I just started reading the man pages and using Google to learn to use dpkg-buildpackage, debuild, sbuild, etc...  Then learned how to use chroot, then shcroot, buildd and so on.


- Do you have any plans to do cross-compilation with eg. distcc? If not, do you think you may start hitting swap heavily with "only" 1GB of RAM per node? I know everyone's clamouring for xbmc, I actually thought of that specifically because I've run out of RAM compiling it on an amd64 system with 1GB RAM and no swap...


Cross compilation is not in the cards.  Plugwash does seem to think that we'll be hitting swap pretty heavily on some of the larger packages such as Firefox/IceWeasel.  However, it's probably something we'll just need to muddle our way through for those few specific packages that require more than a GB of memory to compile.  I understand that's pretty much what the Debian armhf team did on their original port.  I think it's pretty ridiculous that software requires more than a 1GB of RAM to compile, but I guess it is what it is.

With regards to xbmc, isn't there other project specifically focusing on that software for the Raspberry Pi?  I'll be happy to see what we can do with it on Raspbian as I assume hard float would improve the encoder/decoder situation, but it seems that other groups specifically concentrating on it would be able to produce something better than the very broad distribution we're attempting to create.

User avatar
mpthompson
Posts: 620
Joined: Fri Feb 03, 2012 7:18 pm
Location: San Carlos, CA
Contact: Website

Re: Debian Hard Float (armhf) for RPi

Tue May 08, 2012 5:58 pm

mdewey said:


Suggestions for the FAQ with draft answers plus some general feedback.


Terrific feedback.  I updated the FAQ as you suggested.  I'll try to add historical information regarding the project and pictures at a later time.

User avatar
mpthompson
Posts: 620
Joined: Fri Feb 03, 2012 7:18 pm
Location: San Carlos, CA
Contact: Website

Re: Debian Hard Float (armhf) for RPi

Tue May 08, 2012 7:35 pm

Ugh!!! I see Mr. Murphy is paying me a visit today.  I restarted the Dreamhost server that is hosting http://www.raspbian.org in preparation to install some new software and the server never came back up.  I can"t get ssh or web access to the system.  I have a service request into Dreamhost to fix the problem and examine why it did not come back online.  We shall see how long it takes to get this fixed.

Sorry folks.  I should have tested the server out by rebooting it when I first got the dedicated system.  I hope I don"t have to hold my breath in the future every time I reboot the system.  What a drag.

[Update: It's back up 45 minutes later.  I actually did get a call back in a timely manner which was nice.  The administrator said I had changed ssh ports so no-one could log in, but I hadn't done anything like that.  Anyway, whatever it is they figured it out and the system is back up.  Just wish I knew the problem so I would know what to be weary of.  I wonder if it's the automatic FSCK that periodically happens on ext3 file systems on boot that I had to wait through?]

User avatar
nick.mccloud
Posts: 804
Joined: Sat Feb 04, 2012 4:18 pm

Re: Debian Hard Float (armhf) for RPi

Tue May 08, 2012 7:49 pm

No, it's my fault – I didn't download the qemu package this afternoon when I tried out the image – so Mr Murphy put a spanner in my evenings geek-fest of investigating the qemu version!

I can't profess to understanding much of the detail of this thread but this is awe inspiring work.

How big do you think the final repo may be? I may be able to find some mirror space in the UK.

How close are we to lxde and the noob stuff the rest of us understand?

AND IT'S BACK!!!!!

User avatar
mpthompson
Posts: 620
Joined: Fri Feb 03, 2012 7:18 pm
Location: San Carlos, CA
Contact: Website

Re: Debian Hard Float (armhf) for RPi

Tue May 08, 2012 8:12 pm

nmcc said:

How big do you think the final repo may be? I may be able to find some mirror space in the UK.
The repository is current 62GB is size, but will likely be in the neighborhood of 90GB to 100GB when complete.  That includes all the source code and the binaries.


How close are we to lxde and the noob stuff the rest of us understand?


A little hard to say.  I've been trying out LXDE nightly on the QEMU image, but there are still too many unbuilt dependencies.  It's mostly just a matter of time.  Plugwash has been a great help in identifying dependencies and then clearing them out through manual builds.  We all owe him a big thank you as he's moving this project along much faster than I could have on my own.

User avatar
nick.mccloud
Posts: 804
Joined: Sat Feb 04, 2012 4:18 pm

Re: Debian Hard Float (armhf) for RPi

Tue May 08, 2012 9:57 pm

Got qemu going - has all the parts for xfce4 and after a short delay ....

.... a desktop.

Hurrah! You and Plugwash are stars!

Will try it out on a real Pi in the morning after I've got the unpleasant matter of work out of the way!

User avatar
mpthompson
Posts: 620
Joined: Fri Feb 03, 2012 7:18 pm
Location: San Carlos, CA
Contact: Website

Re: Debian Hard Float (armhf) for RPi

Tue May 08, 2012 10:04 pm

nmcc said:


Got qemu going – has all the parts for xfce4 and after a short delay ….

…. a desktop.

Hurrah! You and Plugwash are stars!

Will try it out on a real Pi in the morning after I've got the unpleasant matter of work out of the way!


Really?!?  I was going after lxde?  I'll try xfce4 later tonight…  Under QEMU for now...

grantek
Posts: 4
Joined: Mon Apr 30, 2012 9:03 am

Re: Debian Hard Float (armhf) for RPi

Tue May 08, 2012 10:39 pm

mpthompson said:


Really?!?  I was going after lxde?  I"ll try xfce4 later tonight…  Under QEMU for now…


I downloaded xserver's various pieces as well as wmaker and it all worked fine

(this is in QEMU, I don't have a RPi board yet)

User avatar
mpthompson
Posts: 620
Joined: Fri Feb 03, 2012 7:18 pm
Location: San Carlos, CA
Contact: Website

Re: Debian Hard Float (armhf) for RPi

Wed May 09, 2012 7:34 am

To better solicit help from others interested in Raspbian, I've been working this evening on scripts that will automate the process of pushing the logs of all package built by the autobuilders into a 'logs' folder that sits parallel to the repository.  A URL to where the logs will be found is here:

http://archive.raspbian.org/ra.....logs/main/

The directory structure is nearly a mirror of how the packages are placed within the repository pool itself.  Logs for specific source packages can be located by indexing on their first letter of the package name in to the directory — or by 'lib' and the next letter for library packages.  Hopefully it's fairly self evident how it works.

Also, within a day or so, I'll start posting on a regular basis a page on the Raspbian wiki with a current list of problem packages that can be examined.  This list is currently nearly 120 packages long so Plugwash and I could certainly use any help others might provide.

If you have the QEMU image or an actual Raspberry Pi hardware running Raspbian and some knowledge of how Debian packages can be built and modified locally, there should be enough of Raspbian functioning that you'll have everything you need to help us debug packages that aren't autobuilding.

Note that the logs will be missing for packages which we manually built outside the autobuilding system or for packages built more than a week and a half ago.  However, for now, the priority is on making sure the logs for packages which didn't build are available.

Let me know if you have any suggestions on improving how things might work.

Also, I realize that it's probably past time to look into a bug base of some kind so that issues with Raspbian can be better identified, prioritized and tracked to a resolution.  I'll try to start putting some cycles towards this over the next few days.

I gotta get some sleep now.  And I didn't get to yet try to run XFCE within Raspbian .

User avatar
nick.mccloud
Posts: 804
Joined: Sat Feb 04, 2012 4:18 pm

Re: Debian Hard Float (armhf) for RPi

Wed May 09, 2012 1:00 pm

Overnight netsurf got built - so now we have a graphical desktop with the ability to read webpages and program and do many educational type things!

Well done!

plugwash
Forum Moderator
Forum Moderator
Posts: 3439
Joined: Wed Dec 28, 2011 11:45 pm

Re: Debian Hard Float (armhf) for RPi

Wed May 09, 2012 1:23 pm

nmcc said:


How close are we to lxde


Pretty close, the lxde-core metapackage is already installable. The lxde metapackage is blocked up on lxterminal which has blocked up on some python stuff but i think i've got it cleared and hopefully we are now just waiting for the buildds to do their stuff.

veryevil
Posts: 30
Joined: Fri Mar 09, 2012 3:38 pm

Re: Debian Hard Float (armhf) for RPi

Wed May 09, 2012 2:19 pm

Managed to get XFCE4 and netsurf on Hexxeh's raspbian image! Seems more responsive than the debian disto although netsurf is very limited and chrome / iceweasel is not yet done.

It is a little unstable and does seem to lock up some times but havent had chance to find out why.

Plugwash: Have you received the PI i sent to you on Saturday? Also check your PMs

Steven

User avatar
mpthompson
Posts: 620
Joined: Fri Feb 03, 2012 7:18 pm
Location: San Carlos, CA
Contact: Website

Re: Debian Hard Float (armhf) for RPi

Wed May 09, 2012 3:29 pm

I put a list of the current packages that are failing to autobuild on the wiki at the following URL:

http://www.raspbian.org/BuildAttempted

It's probably worth a look if you are curious about the range of issues we're typically facing with problematic packages.

I'm not certain how many other packages are dependent upon these issues being cleaned up before the dependent packages can be built.

Return to “Raspbian”