User avatar
Davespice
Raspberry Pi Foundation Employee & Forum Moderator
Raspberry Pi Foundation Employee & Forum Moderator
Posts: 1624
Joined: Fri Oct 14, 2011 8:06 pm
Location: London, United Kingdom
Contact: Website Twitter

STICKY: Volunteers to assist with project: Honey Bee Counter

Sun May 03, 2015 5:17 pm

Hi all!

We're looking for volunteers to assist with this project and I'm also hoping to show it to BBC Spring Watch (which we would involve you with of course).

The goal is to make a bee counter using only visual data captured by the camera module. Specifically to keep a count of bees out and bees in for a particular hive. I don't want this thread to be a discussion of alternative bee counting methods please so no need to post about them, I am aware of them. We're trying to investigate if this can be done using only visual information because a lot of bee keepers don't like to impede the natural flow of bees in the entrance by putting something in their way.

The plan is to have the camera module pointing vertically downwards above the entrance so that the hole is at the bottom of the view. You'll then see bees fly upwards in the view when they leave and downwards when they arrive. The trick is how to identify what is the same bee and how to measure if it's got more up or downward movement. If it's an upward flight then add one to the out counter. If it's a downward flight then add one to the in counter. That's basically all we want.

To give you something to work with I've captured an hour's worth of test video that you can download and try to process. The image below on the left shows my Motorola Lapdock sitting on top of a one of my brothers bee hives with the camera module aiming downwards. The one on the right is a screen shot from the camera viewpoint, you can see some bees at the bottom.

Image Image

I only used the Lapdock for convenience since it had a battery. Each video is 10 minutes in length and was captured using raspivd with both the -o and the -x parameters. This will output a h264 encoded video along with a file containing the encoding motion vectors. It may be that the vectors are helpful for this, I am not sure so I captured them to disk anyway. Use them if you see fit. The vectors are explained in Gordon's blog post here.

I also used a 16 GB class 10 SD card in a 512 MB Pi 1 model B (all I had to hand at the time). From the playback you'll notice that, now and again, there is a jump. I think this is because so much data was being written to the SD card that the Pi was clearing a buffer or something. I am not entirely sure what caused it. Hopefully it won't cause much of an issue.

Download: https://www.raspberrypi.org/files/bees/

The videos were recorded at 1080p at 30 fps. I am considering doing another set recorded at 640 x 480 / 90 fps if people think this would help?

It doesn't need to be super accurate only good enough to give an indication of hive activity. This is really useful information for bee keepers when they're monitoring the health of a hive and could warn against sudden colony collapse disorder or be an early warning system for swarming.

Thanks and kind regards

Dave

guyver2
Posts: 30
Joined: Thu Feb 26, 2015 4:05 pm

Re: Volunteers to assist with this project: Honey Bee Counte

Tue May 12, 2015 7:51 am

well, I'm getting there I think. It's still work in progress and far from perfect but I should be able to achieve decent precision in real time.
Image

guyver2
Posts: 30
Joined: Thu Feb 26, 2015 4:05 pm

Re: Volunteers to assist with this project: Honey Bee Counte

Tue May 12, 2015 2:03 pm

Here is a first shot at computing the INs and OUTs from one of your video. I still get some errors from bees flying away very quickly from the hive entrance but it's a good start :)

Most of the issues I'm having come from the motion and I downgraded images to 640x480 anyway. So I think it would be very beneficial to get 640x480@60fps sequences.

Bees are counted as soon as they show in one image, hence some bees get counted multiple time as IN and OUT as they wander around the entrance. It shouldn't be much of an issue if you only consider the difference between INs and OUTs since such bees effect would sum up to 0.

Also, it makes a very nice videos, but at least 3/4th of the image is useless for counting. As you're only interested in getting ins and outs from the hive. You should consider zooming/geting closer to the entrance. The camera orientation (top view) is very good though.

http://sxbn.org/~antoine/stuff/bees_result_1.mp4

ShiftPlusOne
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 4698
Joined: Fri Jul 29, 2011 5:36 pm
Location: The unfashionable end of the western spiral arm of the Galaxy

Re: Volunteers to assist with this project: Honey Bee Counte

Tue May 12, 2015 2:56 pm

You sir, are a legend. Is this up on github or something similar?

guyver2
Posts: 30
Joined: Thu Feb 26, 2015 4:05 pm

Re: Volunteers to assist with this project: Honey Bee Counte

Tue May 12, 2015 3:31 pm

well, no, as I am currently working on it. Uploading right now

guyver2
Posts: 30
Joined: Thu Feb 26, 2015 4:05 pm

Re: Volunteers to assist with this project: Honey Bee Counte

Tue May 12, 2015 3:48 pm

here it is,

The code is python only.
Depedencies :
  • numpy
  • opencv (cv2)
the code is still under heavy development. You can tweak a few parameters around line 154 :

Code: Select all

# input video file
cap = cv2.VideoCapture('bees1.h264')
# morphological structuring element to clean the image
kernel = cv2.getStructuringElement(cv2.MORPH_ELLIPSE,(5,5))
# background model 
bgm = BGmodel(30) # 30 is the number of frame used for building the BG model
frameid = -1

bees = []
hive = Hive(53,412,582,62) # position of the beehive entrance as a rectangle = top left corner X, Y, width, height
THRESHBEE = 60 # maximum "jump" a bee can make when connecting its trajectory

https://github.com/guyver2/beehive


EDIT :
The code makes use of the Munkres implementation by https://github.com/scikit-learn/scikit- ... gnment_.py

User avatar
blachanc
Posts: 447
Joined: Sat Jan 26, 2013 5:03 am
Location: Quebec,canada(french)

Re: Volunteers to assist with this project: Honey Bee Counte

Tue May 12, 2015 4:03 pm

guyver2 wrote:here it is,

https://github.com/guyver2/beehive
EDIT :
The code makes use of the Munkres implementation by https://github.com/scikit-learn/scikit- ... gnment_.py
you got to be kidding,
1K 600 lines of code to achieve the brilliant results you shared.

Simply amazing.

Ben
PS: now I need to try to understand the code ;)
Autism/Asperger syndrome: what is your score on this quiz?
http://www.raspberrypi.org/forums/viewtopic.php?f=62&t=70191

kcx
Posts: 15
Joined: Tue Apr 21, 2015 10:29 pm
Location: Central Texas

Re: Volunteers to assist with this project: Honey Bee Counte

Thu May 14, 2015 5:07 am

This is absolutely great! I have my first nuc arriving at the end of the month and I'd really like to get this going as well.

Great job!

User avatar
Davespice
Raspberry Pi Foundation Employee & Forum Moderator
Raspberry Pi Foundation Employee & Forum Moderator
Posts: 1624
Joined: Fri Oct 14, 2011 8:06 pm
Location: London, United Kingdom
Contact: Website Twitter

Re: Volunteers to assist with this project: Honey Bee Counte

Fri May 15, 2015 8:34 pm

Hi guyver2

Thank you for doing a great bit of work with these beehive videos. It certainly looks promising. I will see if I can record some new ones at 640x480. I was thinking to record at 90 fps but do you think 60 would be enough? I could do both. I think there is value in doing this and seeing how much of an improvement you see before trying to run with this properly.

How much effort would it be for you to modify the code to take live input from the Raspberry Pi camera module as opposed to processing a h264 file? This is how we would eventually want to use it. I would probably run your code as a daemon with say an open socket that I could connect to, from some data acquisition code being run by cron, to get the count and or reset the count for the next time block. Then keep logging the counts in a MySQL database or something.

Also, were the vector files useful? If not I'll not bother to capture them next time.

Dave

P.S. Sorry I didn't reply sooner, had a hell of week :)

guyver2
Posts: 30
Joined: Thu Feb 26, 2015 4:05 pm

Re: Volunteers to assist with this project: Honey Bee Counte

Mon May 18, 2015 8:02 am

I was thinking to record at 90 fps but do you think 60 would be enough?
the more images the better. If it's too much, it's always possible to drop images while processing.
How much effort would it be for you to modify the code to take live input from the Raspberry Pi camera module as opposed to processing a h264 file?
Almost no effort at all. The Picamera library provides the same interface as reading from a file.

I would probably run your code as a daemon with say an open socket that I could connect to, from some data acquisition code being run by cron, to get the count and or reset the count for the next time block. Then keep logging the counts in a MySQL database or something.
That seems quite easy to implement also. It's just a matter of allowing remote access to some of the variables.

Also, were the vector files useful?
I did not use them.

User avatar
waveform80
Posts: 303
Joined: Mon Sep 23, 2013 1:28 pm
Location: Manchester, UK

Re: Volunteers to assist with this project: Honey Bee Counte

Tue Jul 28, 2015 7:28 am

guyver2 wrote:
I was thinking to record at 90 fps but do you think 60 would be enough?
the more images the better. If it's too much, it's always possible to drop images while processing.
How much effort would it be for you to modify the code to take live input from the Raspberry Pi camera module as opposed to processing a h264 file?
Almost no effort at all. The Picamera library provides the same interface as reading from a file.

I would probably run your code as a daemon with say an open socket that I could connect to, from some data acquisition code being run by cron, to get the count and or reset the count for the next time block. Then keep logging the counts in a MySQL database or something.
That seems quite easy to implement also. It's just a matter of allowing remote access to some of the variables.

Also, were the vector files useful?
I did not use them.
What a cool project :) If anyone needs a hand wringing any more performance out of picamera for this do let me know - I haven't got much time free these days, but I've got quite a bag of tricks which can be quickly thrown at image processing applications (I'm trying to get around to including them in the next version of the docs!).


Dave.

kcx
Posts: 15
Joined: Tue Apr 21, 2015 10:29 pm
Location: Central Texas

Re: Volunteers to assist with this project: Honey Bee Counte

Fri Jul 31, 2015 5:28 am

If somebody could tweak this to run in realtime that would be great. An easy way to mark the entrance would also be a welcome feature.

guyver2
Posts: 30
Joined: Thu Feb 26, 2015 4:05 pm

Re: Volunteers to assist with this project: Honey Bee Counte

Fri Jul 31, 2015 7:14 am

it already runs in real time (on my pi model B).
As for the easy marking of the entrance, that can be done. Today it only requiers you to feed in 4 numbers, if you manage to run the code, I guess you can also get those numbers right :)
One other improvement would be to display the entrance to make sure it's in the correct position.

kcx
Posts: 15
Joined: Tue Apr 21, 2015 10:29 pm
Location: Central Texas

Re: Volunteers to assist with this project: Honey Bee Counte

Fri Jul 31, 2015 3:28 pm

Sorry I wasn't clear. What I meant was take a live feed from a camera and process it in realtime. It's possible it already does this-- when I originally pulled the code down it didn't. Is this a feature now?

guyver2
Posts: 30
Joined: Thu Feb 26, 2015 4:05 pm

Re: Volunteers to assist with this project: Honey Bee Counte

Mon Aug 10, 2015 9:39 am

Sorry for the delay (holidays...)
No, it doesn't take the picamera feed as input, yet. What I meant by real-time, is that the code processes each frame faster than 1/25th of a second, hence it doesn't suffer from lag. I'll make another version of the code that will take the feed from a picamera instead. But since I do not have a beehive myself, I won't be able to test it and I cannot grant that it will behave as well as with the video files provided by OP.

I'll update the code this afternoon and let you know here when it's ready.

Some little things that will improve your results :
- the background should be as static as possible (light wind and slowly moving grass is fine)
- the background should not have the same color as a bee (grass is fine, hay not so much)
- proper illumination (less accurate results should be expected on cloudy days or in the evening)

guyver2
Posts: 30
Joined: Thu Feb 26, 2015 4:05 pm

Re: Volunteers to assist with this project: Honey Bee Counte

Tue Aug 11, 2015 2:30 pm

Ok, I updated the file and made some quick test on my RPI compute module (wich is not as powerful as a regular one).
The bad news is that it doesn't run very fast, using the camera module takes a good part of the cpu time and the framerate dropped to 5-ish using the same parameters as before... I can hardly get more than 15 fps using poor quality parameters.
I'd be interested to know if you guys use it on a more decent RPI.

parameters that can be tweaked to make the processing faster (but less accurate) are :
[*] https://github.com/guyver2/beehive/blob ... ra.py#L150
The downsampling factor on the input stream, bigger is faster but less accurate

[*] https://github.com/guyver2/beehive/blob ... ra.py#L154
The size of the background model, smaller is faster but less accurate


The code has been updated with two more files. To use it with the camera module on a RPI, use

Code: Select all

python bees_picamera.py
https://github.com/guyver2/beehive

hivetool
Posts: 3
Joined: Fri Jun 07, 2013 10:33 am
Contact: Website

Sun Aug 23, 2015 12:07 pm

Hello folks. This is great! We've been working on a bee counter for some time. Here is a sample written in c using openCV:
http://hivetool.org/w/index.php?title=B ... r_software

Background is here:
http://hivetool.org/w/index.php?title=Bee_Counter

Have you seen the Intel Research Pittsburgh's paper Video Monitoring of Honey Bee Colonies at the Hive Entrance, by Campbell, Mummert, and Sukthankar? There is a link to it.

You may also be interested in Tracks, written by Tom Goddard. Totally different approach/camera angle. A link is here:
http://hivetool.org/w/index.php?title=H ... ee_Counter

I'd love to get some collaboration going. It would be fantastic to get a reliable bee counter working.

--Paul

gtoal
Posts: 86
Joined: Sun Nov 18, 2012 12:02 am

Re: Volunteers to assist with this project: Honey Bee Counte

Tue Sep 29, 2015 11:51 pm

I'm volunteering, or more accurately, offering to collaborate. I've been working with a biologist and beekeeper who is starting a project that has a lot in common with yours. (I'm not a bee guy, I'm the techie.) My friend is another David by the way, hope it doesn't get confusing if we end up working together! David Ammons is a university researcher and hobbyist beekeeper, and he has a brother who is a professional bee keeper with a lot of hives. David's focus is on detecting colony collapse; mine is on building a cost-effective system and possibly creating a cooperative network of hive monitors such as exists today in other domains such as weather stations or ADS-B flight receivers - together we have complementary skills and I think a reasonable chance of succeeding in our goals.

David wants to instrument a bunch of hives and gather data on what happens before/during colony collapse and compare it to the various states of a normally-functioning hive, in order to detect colony collapse before the bees are gone so that the beekeeper can take steps to save the hive. In the process of doing so, we expect to gather raw data that will also be of value to colony collapse researchers.

The primary instrumentation will be a Raspberry Pi camera, but we'll throw in as many sensors as we can that are cost effective, given that we'll probably want to install a whole bunch of these in the field to get enough data in order to compare a good hive with a collapsing colony (and as a spin-off if we can build the hardware in bulk, it could be a good tool for beekeepers just to monitor their hives routinely...)

I've started work on the hardware and of course the first problem is the close-up capability of the picam. I tried _very_ hard to break the seal on the lens so it would unscrew a little for macro focus but just could not make it turn. I tried the eraser on the end of a pencil trick and various fine pliers. No joy. I eventually superglued the camera to the back of a watchmaker's glass and that more or less works. (images here: https://www.youtube.com/watch?v=Y8RxCWrUEig - I did turn on edge enhancement though, the natural focus is not quite perfect)

We've only just started this project. We'll be documenting as we go along at http://hive.honeybeegardens.net/

Our basic idea is to build a walkway to place in front of the hive entrance and video the bees going in and out of the walkway, and analyse the video for various parameters (as well as basic counting) - type of bee, amount of pollen being carried, whether mites are visible on their backs, etc. Given bandwidth limitations we definitely need to do the image analysis in realtime in the field - either on the pi itself or on a local server attached by an ethernet. (We're using ethernet rather than wifi to power the pi over PoE - saves all the hassle with solar/batteries/power cables)

One way to add the extra environment sensors may be to use one a weather-station hat for the pi. (The official one if we can get one, otherwise I think I remember seeing someone selling something similar on Tindie that I need to find again)

I want to throw every type of sensor and data gathering that I can at the problem - since this is pure research to find out what is going on rather than to confirm some hypothesis, I won't rule out any kind of sensor. I'm particularly intrigued by the possibility of using the sound of the hive to diagnose its health - adding a microphone to the recordings would not be difficult and the article I linked to below (2nd link) argues convincingly that there is data there to be had, even if critics of that author debate the specific meanings of particular sounds. But if there is something there, and especially if data from multiple sources can be correlated, then we can probably find it, whether by something easy like a Naive Bayesian classifier (eg https://www.uclassify.com/ ), or more complex data mining with a neural net. (We're not looking for an explanation of colony collapse, we're looking for advanced warning that it is about to happen, so that the beekeeper can do something about it in time)

We've hacked up a cardboard prototype walkway/camera housing to do the proof of concept, then as soon as we're happy we'll build something more solid and make the electronics waterproof to put outdoors more long-term.

The pi is not online right now - I brought it home today to add infrared LEDs to the walkway. By having a darkened walkway we control the illumination and make image processing easier. (we're using a Pi-Noir) I'll have it back online in a day or two and will be able to show you some near-realtime video. We don't have the bandwidth to stream video but we can upload a short sample once every few minutes.

(I've also give a little consideration to using FLIR: https://groups.google.com/forum/#!topic ... SoB5PgT_XQ )

Anyway, the first milestone of our project is to start gathering video and seeing if we can do the necessary image analysis. Only if that is a success will we start on making it into a serious research project.

I didn't know about your project until today - with luck we can reuse your work on video processing. David is a great biologist (his last project came up with a new mechanism that can be used as a targeted delivery mechanism for vaccines and possibly anti-cancer treatments), and I'm a reasonable programmer and systems guy (I've written DNA assembly software; I've managed server farms and ran an ISP, and I'm a "Maker" for whatever that's worth), but haven't done any work on video processing - so we had been hoping we could find someone with expertise in that area to collaborate with. We'll volunteer whatever skills we have that you might need in return.

Graham
PS Here are some related links I've found. The first is on video processing, the second on audio signals.

http://www.jacksimpson.co/2014/04/18/id ... -features/
https://web.archive.org/web/20130812065 ... enbees.htm
http://www.instructables.com/id/iphone- ... rm-detect/
http://www.hobos.de/en/beekeepers-farme ... werke.html
https://books.google.com/books?id=amoLB ... g=SA1-PA35
http://journals.plos.org/plosone/articl ... ne.0084656
http://gizmodo.com/5945925/digital-beeh ... s-collapse
http://www.rte.ie/news/2014/1201/663768-bees/
http://www.instructables.com/id/How-to- ... Busy-Bees/
http://www.instructables.com/id/Track-H ... mperature/
http://www.instructables.com/id/Sugar-C ... r-Varroa-/
http://www.instructables.com/id/Build-a ... bee-hives/
http://hivemind.co.nz/
https://web.archive.org/web/20120717001 ... ounter.php
http://hivetool.org/hardware/counters/ (currently offline)

gtoal
Posts: 86
Joined: Sun Nov 18, 2012 12:02 am

Re: Volunteers to assist with this project: Honey Bee Counte

Wed Oct 14, 2015 9:45 pm

Well in the week or two since I posted we've learned quite a lot - there are a *lot* of projects that have done significant parts of what we want to do, but there are also things that still can be done. What interests me most is in understanding why there appear to be a lot of small scale projects and why none of this stuff has taken off on a large scale - is it a lack of interest by the majority of beekeepers or is it just that no-one has tackled the problems of commercial scale assembly and large-scale networking?

Anyway, here's our first experiment in creating a bee walkway entrance to a hive that can be videoed under controlled conditions, both for counting and for detecting pollen and mites.

It's a partial success from the video point of view, but a failure from the beekeeping point of view, as the IR lights we used created too much heat and kept the bees fanning all night trying to keep the hive cool. This may be fixable by using lower power LEDs, and if not there are alternative schemes to consider.

Here's the video:

https://youtu.be/5uvzqcUS4I0

Graham

eyesonhives
Posts: 5
Joined: Wed Nov 18, 2015 11:59 am

Re: Volunteers to assist with this project: Honey Bee Counte

Fri Nov 20, 2015 8:26 am

Wonderful project Dave!

Our team are big fans of the RPi, and have been working on this same problem for the past year since a startup weekend really kicked our project off. After many iterations, we have actually managed to solve many of the problems described in this thread, and many more that will come up later (reliability, OTA updates, analytics platform in the cloud, video playback etc).

I'll save our marketing spin, but let you know we’ve been running a pilot with 15+ hives and so far local beekeepers have told us that our system helped them more quickly manage ant infestations, robbing and even detected queenlessness early enough to do something about it.

One of the most exciting things we found was what we call the "orientation spike" that shows up in the data - a key insight is that it's not just the 'count' of the bees that matters, it's their flight behavior. Not just looking at the bees, but at the hive itself as a superorganism!

We'd love to get mentoring from the RPi Foundation on how to build a business and be open source - it's a real concern for us presently, so we're presently closed until we have a good answer. Our core dev team all contribute to the open source community, but this is our first venture trying to build a longer term business focused on sustainable agriculture. We'd love be able to dedicate ourselves to our project full time, and still support our families too. Thanks for taking the time to read!

-Kelton & the EyesOnHives team.

gtoal
Posts: 86
Joined: Sun Nov 18, 2012 12:02 am

Re: Volunteers to assist with this project: Honey Bee Counte

Wed Nov 25, 2015 1:32 am

Welcome, Kelton. Impressive what you can deduce with just a camera and some statistical processing. We're heading in a similar direction except that we're trying to get a bit closer and just observe the entrance. You can see our current cache of video at http://hive.honeybeegardens.net/hive0001/ (which is liable to change as we work on it).

We know that the video quality is critical path so we've been working on that first. We're using a Pi2 with PiNoir. The first iteration was a camera housed inside an enclosed walkway. On the first night we had a large number of bees fanning overnight and several were dead by the morning. Not good. We concluded it must have been the heat from the two powerful IR LEDs so we switched to a new design with much cooler IR LEDs and an outside camera - no covered walkway to capture rising hot air. We still got a couple of bees fanning at night but not excessive numbers and no deaths, so that's an improvement, but we're still looking at why the bees are fanning. They really should not be able to see a 940nm IR LED and the LEDs I'm using have no visible component. (Some IR leds have a dull red glow)

The Pi camera is very effective. We can easily spot pollen, and we've also caught a couple of mites. (Visually - we haven't started on object recognition software yet - we're getting the mechanics of the camera in place first - the better the input image quality, the better the image recognition will perform)

The project is developing at an acceptable pace. We've hit occasional problems which have all been fixable. The current issue is that our SD card (for the saved videos) is returning disk errors. It's either a bad card or possibly its due to the fact that it is being continuously written to over periods of days. Although the video files get evenly distributed across the disk, the directory/index information will always be written to the same area and I think that may be where the problems are coming from.
Rather than risk another large SD card. I'm going to try a USB flash drive next. (128Gb)

Bandwidth to our hive is our other problem. At the resolution and speed that we're filming at the moment, we can only upload one video out of four to our offsite server. This pretty much necessitates doing all the video processing locally on the Pi2 rather than offloading it to cloud servers. That'll be fun!

Graham

elriri35
Posts: 10
Joined: Wed Dec 02, 2015 10:44 pm

Re: Volunteers to assist with this project: Honey Bee Counte

Thu Dec 03, 2015 8:53 pm

guyver2 wrote:here it is,

The code is python only.
Depedencies :
  • numpy
  • opencv (cv2)
the code is still under heavy development. You can tweak a few parameters around line 154 :

Code: Select all

# input video file
cap = cv2.VideoCapture('bees1.h264')
# morphological structuring element to clean the image
kernel = cv2.getStructuringElement(cv2.MORPH_ELLIPSE,(5,5))
# background model 
bgm = BGmodel(30) # 30 is the number of frame used for building the BG model
frameid = -1

bees = []
hive = Hive(53,412,582,62) # position of the beehive entrance as a rectangle = top left corner X, Y, width, height
THRESHBEE = 60 # maximum "jump" a bee can make when connecting its trajectory

https://github.com/guyver2/beehive


EDIT :
The code makes use of the Munkres implementation by https://github.com/scikit-learn/scikit- ... gnment_.py
I am currently trying to use your work to count flies but sometimes I have strange behaviors.
If a fly appeared in the entrance and disappears immediately, it is counted as an incoming fly.

But regardless, my goal is just to count flies through a virtual door (line) and to determine the direction (left or right). Could you help me to do it?

Sorry for my english

elriri35
Posts: 10
Joined: Wed Dec 02, 2015 10:44 pm

Re: Volunteers to assist with this project: Honey Bee Counte

Fri Dec 04, 2015 9:24 am

Here is my problem: https://youtu.be/kLVhmkU43vM

We notice when "a bee" appears in the rectangle is not counted in OUT but the script count correctly when a bee disappears in the entry.

Here is my code:

Code: Select all

#------------   

# statistical background model, build a model from previous frames
class BGmodel(object):
    def __init__(self, size):
        self.size = size #number of frame used to compute a model
        self.hist = np.zeros((self.size,480,640))
        self.model = None
        self.cpt = 0
        self.ready = False

    # add a frame and update the model
    def add(self, frame, updateModel = True):
        self.hist[self.cpt,:,:] = np.copy(frame)
        self.cpt += 1
        if self.cpt == (self.size - 1) :
            self.ready = True
        self.cpt %= self.size
        if updateModel:
            self.updateModel()
    
    # update the model from the current frame history
    def updateModel(self):
        # np.mean is faster but median yields better results
        self.model = np.median(self.hist, axis=0).astype(np.int32)
    
    def getModel(self):
        return np.copy(self.model)
       
    # substract the background to the current frame
    def apply(self, frame):
        self.add(frame)
        # *2 to enhance the contrast
        res = 2*(np.abs(frame-self.model).astype(np.int32))
        res = np.clip(res, 0, 255)
        return res.astype(np.uint8)



# -----------------

# convininent Bee object to hold a bee's position, motion history and provides
# drawing functions
class Bee(object):
    staticID = 0
    def __init__(self, pos):
        self.pos = [list(pos)]
        self.lastSeen = 0
        self.ID = Bee.staticID
        Bee.staticID += 1
        self.age = 0
        self.color =  tuple(255*(0.2+4*np.random.random((3))/5))
    
    # move the bee to it's new position
    def move(self, pos):
        self.age += 1
        self.pos.append(list(pos))
    
    # remove last position
    def pop(self):
        self.age -= 1
        self.pos.pop()
    
    # compute distance to a point from given previous position
    def dist(self, pt, offset = 0):
        return math.sqrt((pt[0]-self.pos[-(1+offset)][0])**2 + (pt[1]-self.pos[-(1+offset)][1])**2)
    
    # draw the bee's path as a line onto an image
    def draw(self, img):
        #print "drawing bee#%d"%self.ID
        cv2.polylines(img, [np.int32(self.pos)], False, self.color,3)



# -----------------


class Hive(object):
    def __init__(self, x, y, w, h):
        self.IN = 0 # number of bees that went in
        self.OUT = 0 # number of bees that went out
        self.OUTAWAY = 0 # number of bees that went out
        self.x = x # X coordinate of top left corner of the entrance
        self.y = y # Y coordinate of top left corner of the entrance
        self.w = w # width of the entrance
        self.h = h # height of the entrance
   
    
    # new bee, check if it came from the entrance
    def append(self, pt):
        # if a bee pops out from the hive entrance, then count it
        if pt[0] > self.x and pt[0] < self.x+self.w and pt[1] > self.y and pt[1] < self.y+self.h:
            self.OUT += 1
	else :
	    self.OUTAWAY += 1
    
    # lost bee, check if it went into the entrance
    def remove(self, pt):
        # if a bee desapears closer to the hive entrance, then count it
        if pt[0] > self.x and pt[0] < self.x+self.w and pt[1] > self.y and pt[1] < self.y+self.h:
            self.IN += 1
    	    #self.IN = 0
    # draw counters on the image
    def draw(self, img):
        cv2.putText(img,"IN  : %d OUT : %d OUTAWAY : %d"%(self.IN, self.OUT, self.OUTAWAY), (10, 30), cv2.FONT_HERSHEY_SIMPLEX, 0.5, (0,0,255),2)
        





#-------------------------------------------------------------------------------
#
# INPUT and parameters
#
#-------------------------------------------------------------------------------

#------------   DATASET dependent
# input video file
cap = cv2.VideoCapture('video3.3gp')
# position of the hive entrance rectangle with top left corner (X,Y) and 
# size (width, height)
hive = Hive(0,0,50,480)


#------------   GENERIC parameters
# morphological structuring element to clean the image
kernel = cv2.getStructuringElement(cv2.MORPH_ELLIPSE,(5,5))
# background model with history size
bgm = BGmodel(60)
# maximum "jump" a bee can make between two consecutive detections
THRESHBEE = 60


#------------   BLOB detector parametrization

# Setup SimpleBlobDetector parameters.
params = cv2.SimpleBlobDetector_Params()
 
# Change thresholds
params.filterByColor = False;
#params.minThreshold = 150;
#params.maxThreshold = 250;
 
# Filter by Area.
params.filterByArea = True
params.minArea = 400
#params.maxArea = 400
 
# Filter by Circularity
params.filterByCircularity = False
params.minCircularity = 0.70
 
# Filter by Convexity
params.filterByConvexity = False
params.minConvexity = 0.70
 
# Filter by Inertia
params.filterByInertia = False
params.minInertiaRatio = 0.1
params.maxInertiaRatio = 0.5
 
# Create a detector with the parameters
ver = (cv2.__version__).split('.')
if int(ver[0]) < 3 :
    detector = cv2.SimpleBlobDetector(params)
else : 
    detector = cv2.SimpleBlobDetector_create(params)



#-------------------------------------------------------------------------------
#
# MAIN LOOP
#
#-------------------------------------------------------------------------------
frameid = -1
bees = []


try :
    while(cap.isOpened()):
        # get a new frame from the video file
        frameid += 1
        ret, frame = cap.read()
        # downsample to 640x480
        frame = cv2.resize(frame, (640, 480), 0, 0, cv2.INTER_CUBIC);
        # convert to grayscale and apply slight blur
        gray = cv2.cvtColor(frame, cv2.COLOR_BGR2GRAY)
        gray = cv2.GaussianBlur(gray,(5,5),0).astype(np.int32)
       
        
        # update the model and apply it to the current frame
        fgmask = bgm.apply(gray)
        
        # Mathematical morphology to enhance the mask and remove outliers
        #fgmask = cv2.erode(fgmask,kernel,iterations = 1)
        fgmask = cv2.morphologyEx(fgmask, cv2.MORPH_OPEN, kernel)
        fgmask = cv2.morphologyEx(fgmask, cv2.MORPH_CLOSE, kernel)
        #cv2.imshow('gray',fgmask)
        
        im_with_keypoints = None
        
        # blob detection only if we gathered enough images for the background model
        if bgm.ready :
            keypoints = detector.detect(fgmask)
            # Draw detected blobs as red circles.
            im_with_keypoints = cv2.drawKeypoints(fgmask, keypoints, np.array([]), (0,0,255))
	
            if len(bees) == 0: # no bees yet, no matching to do, just add them
                for kp in keypoints:
                    bees.append(Bee(kp.pt))
                    hive.append(kp.pt)
            else :
                # MUNKRES assignment, slightly better
                freeBees = [True for i in xrange(len(bees))]
                freeKP = [True for i in xrange(len(keypoints))]
                # build cost matrix 
                cost = np.zeros((len(keypoints), len(bees)))
                for i,kp in enumerate(keypoints):
                    for j,b in enumerate(bees):
                        cost[i,j] = b.dist(kp.pt)
                # proper assignment
                assignment = linear_assignment(cost)
                for ass in assignment :
                    if cost[ass[0], ass[1]] < THRESHBEE:
                        bees[ass[1]].move(keypoints[ass[0]].pt)
                        freeBees[ass[1]] = False
                        freeKP[ass[0]] = False
                for i in xrange(len(freeBees)): # lost bees
                    if freeBees[i]:
                        bees[i].lastSeen += 1
                for i in xrange(len(freeKP)): # new keypoints
                    if freeKP[i]:
                        bees.append(Bee(keypoints[i].pt))
                        hive.append(kp.pt)                    
            # remove lost bees (not seen for at least 15 frames)
            tmp = []
            for b in bees:
                if b.lastSeen < 15: # bee still "alive"
                    tmp.append(b)
                    b.draw(im_with_keypoints)
                    b.draw(frame)
                else :
                    hive.remove(b.pos[-1])            
            bees = tmp
            hive.draw(im_with_keypoints)
            hive.draw(frame)
            print "frame : %d /  IN : %d /  OUT : %d"%(frameid, hive.IN, hive.OUT)
	    cv2.rectangle(frame, (0, 0), (50, 480), (255,0,0), 2)
            cv2.imwrite("out/frame_%05d.jpg"%frameid, frame)
                    
                

       
        #debug output, uncomment to see each step
        #cv2.imshow('frame',np.concatenate((fgmask, gray2), axis=1))
        #cv2.imshow('model',bgm.model.astype(np.uint8))
        #cv2.imshow('frame',gray.astype(np.uint8))
        #cv2.imshow('mask',fgmask)
        #if im_with_keypoints is not None : 
        #    cv2.imshow("Keypoints", im_with_keypoints)
	
        #cv2.imshow('color',frame)

        # needs to be uncommented for the images to show properly
        #if cv2.waitKey(1) & 0xFF == ord('q'):
        #    break
except :
    # always end with an error, but sequences are too long for me to bother debuging :(
    pass
finally :
    cap.release()
    cv2.destroyAllWindows()

print "IN : ", hive.IN
print "OUT: ", hive.OUT
Would you have an idea of the problem?

thank you in advance

guyver2
Posts: 30
Joined: Thu Feb 26, 2015 4:05 pm

Re: Volunteers to assist with this project: Honey Bee Counte

Fri Dec 04, 2015 9:34 am

Hi,

I haven't touched the code for a while now but I remember this issue. I didn't care too much as it goes both ways.
From my first post :
Bees are counted as soon as they show in one image, hence some bees get counted multiple time as IN and OUT as they wander around the entrance. It shouldn't be much of an issue if you only consider the difference between INs and OUTs since such bees effect would sum up to 0.
It's not too big of an issue if you consider the difference between INs and OUTs. But I understand it is if you need accurate values for these parameters.

As for the direction, I implemented a very coarse trajectory history in the Bee class in my code.

Code: Select all

Bee.pos
is a list of the previous position of a given bee. You can easily use this to detect if a bee crossed your virtual line.


Also I cannot see you video, it's private :(

elriri35
Posts: 10
Joined: Wed Dec 02, 2015 10:44 pm

Re: Volunteers to assist with this project: Honey Bee Counte

Fri Dec 04, 2015 9:36 am

Thanks for reply!

Video is public now.

Return to “Camera board”

Who is online

Users browsing this forum: No registered users and 4 guests