Uncategorized

Long term, deep water, satellite connected ocean monitoring system

Conservation, hackspaces and Raspberry Pis. And sharks. How could this not be the blog of the day? Gary Fletcher of ZSL sent us this report.

Marine Conservation Camera

ZSL have developed low cost cameras to monitor marine biodiversity in large marine protected areas (MPAs) using the $35 Raspberry Pi single board computers and standard webcams and running opensource Motion tracking software. ZSL reached out to UK hackspaces to help design the cameras and achieved unprecedented economy and features.

 Why Raspberry Pi?

Traditionally it has been incredibly difficult to capture events underwater – all of the usual apparatus such as PIR/heat, infrared and ultrasonic sensors simply do not work underwater. The Raspberry Pi literally opened up a new door with its low power consumption and processing power. It allowed us to deploy a solution which really fits the bill and without it would have been very troublesome to achieve!

Hardware

Each camera was deployed on an anchored buoy. Mounted directly onto the buoys were two solar panels for charging two deep cycle 90Ah lead-acid gel batteries, the aerial, and a waterproof box containing the communications system. This was then connected to a 50m SWA cat-5 cable running down to the pressure vessel containing the camera itself.

Pressure Vessel

The cameras are designed to operate at depths between 20 and 50 meters. Rlab’s (Reading Hackspace) Ryan White suggested basing the design around a clear polycarbonate tube, with machined HDPE end caps secured by threaded rods and double o-rings. One end-cap had a threaded hole which SWA cat- 5 cable was run though, anchored to the inside and then potted. This cable runs the power and communications.

BuildBrighton’s Mike Poutney and Paul Strotten machined the endcaps on their lathe and offered some great technical advice which was very well received.

The outer pressure vessels easily survived a 100m pressure test in a hydrostatic chamber. It should go significantly deeper had the internal structure not failed at that point.

Internal Structure

Rlab’s (Reading Hackspace) Barnaby Shearer designed the internal support structure. This was laser cut from 3mm acrylic. The designs were done in 3D in OpenSCAD to check all the components fitted together, then projected into 2d for laser cutting. The acrylic was glued with tensol.

Cable waterproofing

The junction box was 3d printed and then sealed using potting compound and left to dry for some time also forming a mechanical join between the inside and the cable gland.

Communications

Attached to the buoy in a waterproof case was a Raspberry Pi to coordinate the communications. This had an Ethernet link to the Raspberry Pi in the pressure vessel. It also had a WiFi dongle running in access point mode to allow easy monitoring and reconfiguration form the research vessel. The Pi also has a serial connection to an Iridium satellite modem so it can stream pictures of the images captured.

The satellite image transfer software was specially developed by Cambridge Consultants and the equipment and satellite bandwidth for this trip was kindly sponsored by Iridium.

Electronics

Attached to the bottom Pi was an Eve board to provide the Pi a RTC and a temperature sensor. Also attached was Ciseco’s Humble Pi hosting an AVR and a mosfet to to turn the Pi off at night (and critically back on each morning). This Pi wake was developed by Miles and Matt from Ciseco, who make an amazing range of Raspberry Pi and microelectronics and are well worth a look – http://www.ciseco.co.uk/

These boards were slightly modified to handle a HackHD camera via the AVR so we could capture high definition footage as well as stills.

The boards were assembled at Nottingham Hackspace.

Camera

The camera used is the Microsoft LifeCam Cinema HD, a cost effective camera conforming to the UVC specification. The only gotcha proved to be that it seems to only respond to a few ‘magic’ exposure settings (5,10,20,39,78,156,312,625,1250,2500,5000,10000,20000), and you have to wait 100ms and reset the brightness after any exposure change.

Software

Rlab (Reading Hackspace), Gary Fletcher and Doug Snead provided a simple command line program to control the camera, and a slimmed down version of MJPEG-Streamer optimized for this camera and with some additional time stamping.

This stream then fed into Motion which starts saving the frames as JPEGs after it detects an event. The JPEGs are then rsynce’d up to the top Pi (backups are always a good thing). ImageMagick then thumbnails and montages the images for efficient sending over the (slow) satellite link.

Stereo Vision

The project did spur off onto some stereo vision development work with Doug Snead and Gary Fletcher but could not be completed in time for deployment. It was hoped that it would be possible to develop this solution as so it could automatically size the passing fish to add to our conservation data.

Image showing the accurate sizing of a fish tied to the ceiling flapping in front of an oscillating fan.

What did it Look Like

The deepest ever Pi?

At 50 meters deep – could this be the deepest Pi to date?

Where was it Deployed?

The system was tested at ZSL in London Zoo behind the scenes and then went onto to open Ocean tests in the largest marine protected area in the world, the Chagos Archipelago.

Gary Fletcher and Barnaby Shearer the camera at ZSL London Zoo, behind the scenes

Results

Well as you can see the results speak for itself, but there is still quite a lot of development work to do but once these sentient units are complete, it will offer a low-cost monitoring system that, when deployed as a network, will greatly expand ocean areas that can be observed.

For those that would like a little further reading on the actual deployment, please have a look here on the Chagos Trust website http://chagos-trust.org/projects/latest/feb-2013-expedition/battling-heat

Acknowledgements

Gary Fletcher, Barnaby Shearer, Ryan White, Richard Ibbotson, Doug Snead, Paul Strotten, Mike Pountney, Miles Hodkinson, Matt Lloyd, Adam Markwell, Gary Fletcher Senior, Anna Fletcher, Charles Turner, Marty Morriss, David Curnick, Matthew Gollock, Heather Koldewey, Alasdair Davies, Charles and Anne Shepard, Yannick Mandarin, Ronan Roche, Reece Pitts, Richard Traherne, Marion Campbell, Jonathan Pallant, Ant Skelton.


Raspberry Jam at the National STEM Centre York, Saturday 8th June 2013

A post from Mr Raspberry Jam himself, Alan O’Donohoe.  This one promises to huge and fabulous, and the National STEM Centre is an outstanding venue. I am strangley drawn to joining the trans-Pennine convoy from Preston.

“You may have seen that we are holding a big Raspberry Jam in York on Saturday 8th June, 3 weeks today. Tickets are available here.

I’ve been working with the folks at the National STEM Centre there to help establish a presence in Yorkshire for Raspberry Jams. There are a lot of people in Yorkshire who have bought Raspberry Pi computers or who have not bought one but are interested in discovering just what you can do with them and this event is to give them a taste of the Raspberry Pi.

This event is going to be much larger than our regular Raspberry Jam events and we know that people  are travelling from far across the UK to attend. I will be travelling in a convoy of cars from Preston bringing old friends and new friends on the journey across the Pennines. Road-trips like these add an enormous amount to the whole experience.

Our busy programme of talks, demonstrations, stalls and hands-on classes is now filling up. We are delighted to have Pete Lomas of the Raspberry Pi foundation speaking and leading a workshop, he is the gentleman who actually designed the Raspberry Pi. We also have Amy Mather the 14 year old who learnt how to program Conway’s Game of Life on the Raspberry Pi, her film on YouTube has attracted over 27000 views.

If you look at our programme , you will see that we have a wide range of classes from how to switch on a Raspberry Pi to building weather stations, interfacing, networking, robotics, game making for all the family and programming in Python. David Whale will be running a workshop on how you can set up an after school Raspberry Pi club.

‘import random’, the start of great game! A Raspberry Jam programming class.

We hope to attract a diverse range of people to attend this family friendly event:

  • Teachers - who want to know if the Raspberry Pi is something they can use to support the teaching of Computing
  • Families - who together want to develop and nurture an interest in technology and computing
  • Hobbyists - who want to meet other people with a desire to share what they have been doing with their Raspberry Pi
  • Anyone - who has an interest in the Raspberry Pi at any level.

If you have never been to the National STEM centre before, this is a fantastic opportunity to come and discover what is there http://www.nationalstemcentre.org.uk/


Maker Faire Google Hangout with Eben

We’ll be talking more later this week about the New Out Of the Box System (NOOBS) which Eben discusses towards the end of this video. We have a suspicion you’ll like it.


New in the Swag Shop: recycled CD case pencils!

These feel lovely when you sharpen them, have a beauteous Raspberry Pi logo at the end, and are made in solid colours from recycled CD cases. Buy some, look funky when you doodle, help fund computing education, and save the planet too – what could be better?

Amazingly, we have not yet sold out of our first batch of Babbage the Bear. (He has stayed on the shelves longer than the camera boards, which we find shocking and remarkable.) Get him while he’s snuggly!


Raspberry Pi for Dummies: sample chapter

If you’re wondering about introducing your kids to Scratch, but aren’t quite sure where to start, here’s a handy resource for you. Sean McManus, one of the authors of Raspberry Pi for Dummies, has sent me a link to a couple of sample chapters of the book, including the first chapter on Scratch. You’re welcome to download it to find out whether the book’s for you.

Raspberry Pi For Dummies PDF Sampler

 


Frederik and Ernest’s Europe – Middle East – Africa roadtrip

Frederik and Ernest Lotter from Blue Horizon Embedded Systems in South Africa are driving from the UK to South Africa via Russia and the Middle East, taking in seventeen countries on their way.

They are making the journey in a Land Rover Defender which is fitted with a Raspberry Pi-based distributed light control system. The Raspberry Pi, and their lighting rig design, will be put to the test over 22000km of harsh conditions and rough terrain.

The Lotter brothers are experienced electronic engineers and are offering to meet up with groups of potential Raspberry Pi or ARM enthusiasts along the way. There may even be a Pi-themed reward available if you can find them using the live GPS tracking system they have installed.

You can track them live online, and if you want them to come and talk to your school, business or another group about Raspberry Pi and ARM processors while they’re in your country, they’re inviting you to email them - please get in touch as soon as possible if you’d like them to visit. Watch the video to learn more, and to find out what their route looks like. Thanks Fred and Ernest; we’re looking forward to tracking your progress!


Welcome to the Swag Shop!

You may have noticed that a little while ago, we quietly withdrew Raspberry Pi branded t-shirts from sale. Since then, we’ve been working on a reboot of the store. Shirts have been totally redesigned, and are now screen-printed rather than transfer-printed, which gives a much higher-quality and tougher finish; we’ve also listened to your requests for more colours and thicker material. [Edit to add: a few of you have asked about the larger sizes. At the moment they're available in black (up to 3XL), and chocolate (Eben's favourite), red and sport grey (2XL). We'll add more if the demand's there, so please let us know what you want.]

Every purchase you make goes to fund the Raspberry Pi Foundation’s educational activities, so you’re not just making yourself look swanky; you’re directly helping kids.

T-shirts, just calling out for you to buy them so they can escape their cruel imprisonment in plastic crates.

So today we’re relaunching the shop, under a new name, with new management (things are being run by our friends at Pimoroni), and new goodies for you to buy.

Babbage the bear needs a home.

Shirts (for ladies, men and kids) aren’t the only thing we’ve got in stock: you can now buy your very own Babbage the bear, and we also have Raspberry Pi mugs, bags, and travelcard holders for you to sip from, carry things in, and wave at turnstiles proudly.

A holder for your Oyster card (or your library card, bus pass or ID). Thanks to TfL for allowing us to mess with their map, and to Paul Beech for the design.

A tough drawstring bag for your hacking tools, about twelve Babbages or your overnight gear.

It’s a mug. You put coffee in it.

We’ll be introducing more goodies to the store as time goes on, and announcing them here when we do. We hope you like it! Please get buying – every penny of profit we make goes straight back into the Raspberry Pi Foundation, where it’s used directly to help educate kids in computer science.

 

 


Camera board project: time lapse video

Our friends at DesignSpark have produced a really beautiful time-lapse video with one of our new camera boards. It doesn’t start very beautifully, because it was filmed on a day whose start can best be described as “sodden”, but by afternoon the clouds parted and England started to look exceptionally green and pleasant. If you want to skip the rain, fast-forward to 1m46. (There’s a guest appearance from a double rainbow later on, too.)

You can find detailed instructions on how to make your own time-lapse video with your own camera board over at DesignSpark. Big thanks to Andrew Back for the vid and the tutorial!


Camera board available for sale!

The camera boards are now available for order! You can buy one from RS Components or from Premier Farnell/Element14. We’ve been very grateful for your patience as we’ve tweaked and refined things; it’d have been good to get the camera board out to you last month, but we wanted your experience to be as good as possible, and we’ve been working on the software right up until last night. Thank you to Gordon and Rob at Raspberry Pi and to Dom Cobley for their work on the firmware (Rob also worked on the documentation); to JamesH for his work on the software; to the Broadcom Cambridge ISP team, particularly David Plowman and Naush Patuck, for volunteering to help with tuning; to Bruce Gentles at Broadcom for his volunteering to help with some of the initial bring-up; to James Adams at Raspberry Pi for running the hardware project, and everybody at Sony Pencoed for making it happen.

Tehzeeb Gunza at OmniVision coordinated things from their end, and helped us with sensor selection. Thanks also to Gert van Loo and Rob Gwynne for their work on the hardware design. (And thank you to Broadcom for letting us take advantage of your team’s willingness to volunteer for us!) This, for the curious, is the camera lab we’ve been borrowing from Broadcom for testing. The mannequin’s name is Veronica. She’s lousy company. The room gives us a calibrated and fixed target to use during tuning; it’s designed to be filled with examples of the sorts of things people tend to take pictures of. Which makes it a kind of creepy place to hang out. Between this and anechoic chambers, we’re getting the full range of testing chambers that give us the shivers.

Click to enlarge. You might be interested to learn that this was snapped with a Nokia N8, which uses an earlier version of the imaging core that’s in the Raspberry Pi (but a different sensor and optics).

For such a small device, this has been an enormous project, and a year-long effort for everybody involved. We’re pretty proud of it: we hope you like it!

How to set up the camera hardware

Please note that the camera can be damaged by static electricity. Before removing the camera from its grey anti-static bag, please make sure you have discharged yourself by touching an earthed object (e.g. a radiator or water tap).

The flex cable inserts into the connector situated between the Ethernet and HDMI ports, with the silver connectors facing the HDMI port. The flex cable connector should be opened by pulling the tabs on the top of the connector upwards then towards the Ethernet port. The flex cable should be inserted firmly into the connector, with care taken not to bend the flex at too acute an angle. The top part of the connector should then be pushed towards the HDMI connector and down, while the flex cable is held in place. (Please view the video above to watch the cable being inserted.)

The camera may come with a small piece of translucent blue plastic film covering the lens. This is only present to protect the lens while it is being mailed to you, and needs to be removed by gently peeling it off.

How to enable camera support in Raspbian

Boot up the Pi and log in. The default username is pi, and the default password is raspberry. (Note: if you have changed these from the default then you will need to supply your own user/password details).

Run the following commands in a terminal to upgrade the Raspberry Pi firmware to the latest version:

sudo apt-get update

Click to enlarge

sudo apt-get upgrade

Click to enlarge

Access the configuration settings for the Pi by running the following command:

sudo raspi-config

Navigate to “camera” and select “enable”.

Click to enlarge

Click to enlarge

Select “Finish” and reboot.

Click to enlarge

How to use the Raspberry Pi camera software

raspivid is a command line application that allows you to capture video with the camera module, while the application raspistill allows you to capture images.

-o or –output specifies the output filename and -t or –timeout specifies the amount of time that the preview will be displayed in milliseconds. Note that this set to 5s by default and that raspistill will capture the final frame of the preview period.

-d or –demo runs the demo mode that will cycle through the various image effects that are available.

Example commands

Capture an image in jpeg format:

raspistill -o image.jpg

Capture a 5s video in h264 format:

raspivid -o video.h264

Capture a 10s video:

raspivid -o video.h264 -t 10000

Capture a 10s video in demo mode:

raspivid -o video.h264 -t 10000 -d

To see a list of possible options for running raspivid or raspistill, you can run:

raspivid | less

raspistill | less

Use the arrow keys to scroll and type q to exit.

Extended documentation is available.

Note that we recommend that you change SSH password if you are using a camera, in order to prevent unwanted access.

How to stream video from the Raspberry Pi camera over a network

To view the feed on Linux

Install the dependencies by running the following in a terminal:

sudo apt-get install mplayer netcat

Find your IP address by running ifconfig. (Your IP address will be listed in the console output and will probably be of the form 192.168.1.XXX).

Run the following command in a terminal to view the feed using MPlayer:

nc -l -p 5001 | mplayer -fps 31 -cache 1024 -

To view the feed on Windows

Install and run Linux instead.

Find your IP address by running ipconfig. (Your IP address will be listed in the console output and will probably be of the form 192.168.1.XXX).

Download MPlayer.

Download Netcat.

Note that your browser may complain that these files are malicious, as they are unsigned executables.

Press the Windows key and the ‘r’ key simultaneously to bring up the “Run” dialog. Enter cmd.exe into the dialog and press enter/return to open a DOS prompt.

Enter the following command at the prompt to view the feed using MPlayer:

[Path to nc.exe]\nc.exe -L -p 5001 | [Path to mplayer.exe]\mplayer.exe -fps 31 -cache 1024 -

To view the feed on OS X

Download MPlayer.

Alternatively, you can download mplayer using Brew, which we recommend.

Find your IP address by running ifconfig. (Your IP address will be listed in the console output and will probably be of the form 192.168.1.XXX).

Run the following command in Terminal to view the feed using MPlayer:

nc -l -p 5001 | mplayer -fps 31 -cache 1024 -

To view the feed on a Raspberry Pi:

Find your IP address by running ifconfig. (Your IP address will be listed in the console output and will probably be of the form 192.168.1.XXX).

Run the following commands in a terminal on the receiving Pi:

mkfifo buffer

nc -p 5001 -l > buffer | /opt/vc/src/hello_pi/hello_video/hello_video.bin buffer

To transmit the feed from the Pi with camera module attached

After setting up the “receiving” machine as per the instructions above, run the following commands in a terminal on the “transmitting” Pi:

raspivid -t 999999 -o – | nc [insert the IP address of the client] 5001

You can then use the commands listed in the “How to use the Raspberry Pi camera software” section above to change the capture time or to add a video effect.

How to submit bug reports or see the source code

You can see the source code and submit bug reports for raspivid and raspicam here.


Guest post from Mythic Beasts: how we dealt with those DDoS attacks

Do you remember the distributed denial of service (DDoS) attacks this website was undergoing a few months ago? They made the news (partly because it was just so bizarre to see someone attacking an educational computing charity) – if you want to refresh your memory, see this, this, or this.

Pete Stevens, who runs marathons and our hosting company, Mythic Beasts, thought you’d be interested in what he’s been doing to try to ensure this can’t happen again. (Famous last words, Pete.) Here’s what he did. Over to Pete!

In the past we’ve had occasional trouble with denial of service attacks against the Raspberry Pi website. In particular, simply overflowing us with traffic has proved not that difficult – the server only has a 1Gbps uplink. When your admin (me) cocks up, it turns out you can saturate a core calculating syncookies leaving the other cores idle because he should have configured IRQ balancing properly.

Pete, pounding the pavement. Click the image for what is possibly the funniest local news story to come out of Cambridge in the last decade.

We briefly investigated cloud based DDoS protection which we still hold in reserve, but it has a habit of declaring that Liz can’t post things because apparently she’s a spambot. We also had to switch off IPv6 access to the website to use them, which, for an educational project, was unfortunate as there is going to eventually be a large network transition to IPv6 and allowing people to learn about it and use it is desirable.

So we’ve scaled out the hosting infrastructure out to a distributed cluster of machines. We’ve installed four additional little dual core machines, two in our Telecity Sovereign House site, two in our Telecity Harbour Exchange site. Each of these runs a load balancer and forwards connections back to the main webserver. This means the inbound load is now shared over 4 separate 1Gbps links and there’s rather more CPU available to calculate syncookies when required and rather more bandwidth to saturate.

We load balance over the load balancers using DNS round robin, as you can see from our public DNS.

$ dig www.raspberrypi.org AAAA +short
lb.raspberrypi.org.
2a00:1098:0:80:1000:13:0:5
2a00:1098:0:80:1000:13:0:6
2a00:1098:0:82:1000:13:0:5
2a00:1098:0:82:1000:13:0:6

$ dig www.raspberrypi.org A +short
lb.raspberrypi.org.
93.93.130.39
93.93.130.214
93.93.128.211
93.93.128.230

Now, everybody knows that this is a stupid way of load balancing, and you don’t get anything like even usage across your sites. This isn’t even slightly born out by the bandwidth figures for the last few days:

93.93.128.211 347.04 GB
93.93.128.230 341.61 GB
93.93.130.39 349.58 GB
93.93.130.214 347.88 GB

That’s agreement to within 2%, which is a pretty even split. So much for commonly held wisdom, we prefer science.

We’ve set the entire internal network up with IPv6. So when you connect to one of the front end machines, it’ll connect back to the main webserver over IPv6. One of the reasons for this is so we’re now running mission critical services over IPv6 – a move to IPv6 worldwide is happening at a glacial pace, and we want to make sure that our support works well. Having an angry Liz phone you up if it doesn’t is a very effective motivator.

You may have seen odd forum and comment issue while we were setting this up. One of the forum spam filters allowed filtering people based on source IP address. The move to the new setup means that clicking the filter by IP address feature resulted in dropping all new comments from that load balancer – a quarter of our traffic. *Oops*. We had to fix that to read the forwarded-by headers.

Now of course the real question is, why aren’t we fronting the site with a massive cluster of Pis? Testing with hping3 suggests that a Pi starts to struggle at around 2500 syns/sec. The front-ends we have are absolutely fine at at 50,000 syncs/sec (reading roughly 10% cpu), so with four of them we can probably handle around 1,000,000+ syns/second. That’d require 400 Pis to keep up, so it’d be a very very large cluster of Pis, not to mention 5 switches in each site.

Of course a very stern warning has been given out to people who have access to the front end machines – not only can they receive a million syns/sec, they can also send them, and that could seriously upset other internet users if it was directed at them.

Now, a side effect of this scale-out is we’re left with a bunch of machines that have a reasonable amount of excess CPU. Eben has *strong views* about wasting CPU cycles, it makes him very sad. So we’ve put them to use.

Rob Bishop and Gordon Hollingworth at Raspberry Pi spend quite a lot of time building software. Compiling it is time-consuming, and their laptops get hot and make fan noises. So we’ve installed a set of dual core VMs on the five core servers running under KVM. When everything is fully operational the software team can kick off a build from the master VM which will then use distcc to farm out the compile across all five machines. This means there’s effectively 10 cores available most of the time for building software. When the website gets busy, the lower priority VMs slow down and hand the cycles back to the load balancer/Apache/PHP/MySQL.

Now, the Raspberry Pi is an educational project. It’s not just about educating children: adults still need to learn things, and that includes me. We’ve run many dual stack IPv4/IPv6 machines before, but we thought we’d try IPv6 only machines and discover the difficulties in order to improve our support for IPv6. So the distcc VMs are IPv6 only – they can’t access anything on the internet that isn’t accessible over IPv6. In reality this means they can see Google, Facebook, lots of mirror servers and a small fraction of other sites.

In the process of setting this up I discovered that I was unable to get the Debian Squeeze network installer to install from an IPv6 only network, so I had to do the initial install to the VMs from a full install image rather than the cut down one. I then realised that Mythic Beasts still doesn’t have an IPv6 aware resolver yet, which we need to sort out, so I had to use Googles public resolver. This is still on my todo list along with full DNSSEC resolver support.

Happily Debian appears to work fine with IPv6 only. The mirrors are v6 enabled, so the VMs recieve updates and can install packages fine, and so far it appears to be going well.

There’s still some things to do and questions to answer: should we move apache/php processing to the front end nodes? Will WordPress Supercache and the other plugins cope in a distributed environment? Will file uploads still work? Can we solve that with NFS? Does NFS even work over IPv6? Should we install a varnish cache on the front end nodes and disable WordPress Supercache? Should we do both? Will it confuse people if we have two layers of caching that expire at different times? Is that better than what we have now? Instead of having tcp-syn cookies on the whole time we could only enable them when under attack. Have we made a dreadful mistake with the build VMS, and is it all going to go offline when Rob tries to compile OpenOffice? Should we stop worrying about all of these questions and instead work out whose job it is to buy the first round at the Cambridge beer festival?

If this is the sort of thing you’d find interesting, and you would like to be paid to solve exactly these sorts of questions, Mythic Beasts is recruiting.

http://www.mythic-beasts.com/cgi-bin/job.pl

We’re looking for both junior and senior people, we very strongly like bright motivated people who get things done, and we’re not overly impressed by certifications. We’d really like a full time person or two but are not averse to taking on summer or gap year students providing they’re smart and they get things done.