jit
Posts: 33
Joined: Fri Apr 18, 2014 2:52 pm

Re: PiKrellCam: motion vector detect + OSD web interface

Tue Nov 24, 2015 6:25 pm

billw wrote:
jbeale wrote: Outstanding! Everything I was hoping for, and even the csv columns line up nicely. Looking forward to some data analysis shortly.

Here's an example of that stats output for one video "event" this evening.
Image
Nice charts! At some point can you outline your steps for generating them?
Would be pretty cool to integrate those into the UI.

User avatar
jbeale
Posts: 3499
Joined: Tue Nov 22, 2011 11:51 pm
Contact: Website

Re: PiKrellCam: motion vector detect + OSD web interface

Tue Nov 24, 2015 6:37 pm

One curious thing about the stats is that motion in frame from left to right is increasing in (x) value, but the (dx) value is negative.
Motion from right to left is decreasing (x) value, but the (dx) value is positive.

Now that we have the stats data available in a file, there are many possibilities for how to manipulate and visually present that data. I'll be playing around with some of the data- I hope other people do as well. CSV (comma separated values) is a simple common text data format and you can plot these files using Excel or many different programs. I like a free program called KST Plot, and you can set it up so you double-click on the .csv file and the data is immediately shown in a plot. Actually I change one thing in the file first: I have to remove the "#" in front of the first line with the column headers so it is read as a column header and not a comment.

There are several packages that can add a plot to a web page- I haven't used them but maybe I'll look into that at some point. What is most interesting to me at the moment is a fairly simple use of this data: separating out distinct moving objects from one recorded video "event" so I can extract a representative still frame from the .mp4 file for each different object.

billw
Posts: 403
Joined: Tue Sep 18, 2012 8:23 pm

Re: PiKrellCam: motion vector detect + OSD web interface

Tue Nov 24, 2015 8:25 pm

jbeale wrote:One curious thing about the stats is that motion in frame from left to right is increasing in (x) value, but the (dx) value is negative.
Motion from right to left is decreasing (x) value, but the (dx) value is positive.
Right, a camera motion vector x,y value is for a position in the frame that can use pixels from the
previous frame. So the vectors point in the direction the pixels come from. But it makes more sense
for the display to show direction motion is towards, so the sign needs changing.

It would have been better if I had thought to do that before printing the value. I can go ahead
and do that for the next upgrade if you are OK with making the adjustment in your stuff?

jit
Posts: 33
Joined: Fri Apr 18, 2014 2:52 pm

Re: PiKrellCam: motion vector detect + OSD web interface

Tue Nov 24, 2015 9:21 pm

jbeale wrote:One curious thing about the stats is that motion in frame from left to right is increasing in (x) value, but the (dx) value is negative.
Motion from right to left is decreasing (x) value, but the (dx) value is positive.

Now that we have the stats data available in a file, there are many possibilities for how to manipulate and visually present that data. I'll be playing around with some of the data- I hope other people do as well. CSV (comma separated values) is a simple common text data format and you can plot these files using Excel or many different programs. I like a free program called KST Plot, and you can set it up so you double-click on the .csv file and the data is immediately shown in a plot. Actually I change one thing in the file first: I have to remove the "#" in front of the first line with the column headers so it is read as a column header and not a comment.

There are several packages that can add a plot to a web page- I haven't used them but maybe I'll look into that at some point. What is most interesting to me at the moment is a fairly simple use of this data: separating out distinct moving objects from one recorded video "event" so I can extract a representative still frame from the .mp4 file for each different object.
I've used Google Visualizations in the past, they are pretty good: https://developers.google.com/chart/int ... cs/gallery The only downside to them is that you can't download and refer to a local copy, which means you need an active internet connection to use them. Not the end of the world for most, but certainly not ideal.

My favourite is the Timeline chart (https://google-developers.appspot.com/c ... y/timeline). I'm not sure if you can add images onto the time line, but I'm thinking it would be pretty cool to see the thumbnail previews in a timeline.

I would love to spend some time adding charts to the UI, but am a bit snowed under at the moment. I think I may have some free time in a few weeks though, so may have a quick play around then and see what I can put together.

User avatar
jbeale
Posts: 3499
Joined: Tue Nov 22, 2011 11:51 pm
Contact: Website

Re: PiKrellCam: motion vector detect + OSD web interface

Tue Nov 24, 2015 11:24 pm

billw wrote:Right, a camera motion vector x,y value is for a position in the frame that can use pixels from the
previous frame. So the vectors point in the direction the pixels come from. But it makes more sense
for the display to show direction motion is towards, so the sign needs changing.

It would have been better if I had thought to do that before printing the value. I can go ahead
and do that for the next upgrade if you are OK with making the adjustment in your stuff?
Yes, please change the sign of the dx, dy velocities. I haven't written any code around this particular file format yet.

Also, for me there is some convenience to having the first header text line without the initial "#" character, but no big deal.
In other cases I can see having # on all non-numeric lines is convenient.

User avatar
jbeale
Posts: 3499
Joined: Tue Nov 22, 2011 11:51 pm
Contact: Website

"24 fps" isn't quite

Fri Nov 27, 2015 8:13 pm

It probably doesn't matter to most people, but FWIW when PiKrellCam on my Pi is set to 24 fps, the actual recorded frame rate is approximately 23.05 fps. I measured this using an LED driven by a stable pulse generator and noticing the vertical drift rate of the horizontal image bands, due to interaction between the light and the rolling shutter start time on each scanline. I confirmed the pulse timing to within 0.1% with a scope.

User avatar
jbeale
Posts: 3499
Joined: Tue Nov 22, 2011 11:51 pm
Contact: Website

Re: PiKrellCam: some huge motions not detected

Tue Dec 01, 2015 7:40 pm

PiKrellCam is mostly working fine. I am experimenting with a setup using a 3rd party RPi compatible camera board with a longer lens (16mm focal length). In this case, the moving object may fill most or all of the frame, and come and go in about 1 second. In some cases, PiKrellCam fails to trigger a motion event even though, with the vector display on, I can see there are huge motion vectors all over the frame as an object passes by. I guess these are rejected as noise somehow? Is there a way to adjust the strength of the "rejected motion" algorithm?

billw
Posts: 403
Joined: Tue Sep 18, 2012 8:23 pm

Re: PiKrellCam: some huge motions not detected

Wed Dec 02, 2015 12:31 am

jbeale wrote:PiKrellCam is mostly working fine. I am experimenting with a setup using a 3rd party RPi compatible camera board with a longer lens (16mm focal length). In this case, the moving object may fill most or all of the frame, and come and go in about 1 second. In some cases, PiKrellCam fails to trigger a motion event even though, with the vector display on, I can see there are huge motion vectors all over the frame as an object passes by. I guess these are rejected as noise somehow? Is there a way to adjust the strength of the "rejected motion" algorithm?
There's no adjustments yet - it's been on my list but I haven't got to it yet.
Is your confirm gap > 0? If so, for objects in frame for ~ 1 sec a zero confirm gap might help.

If you can post some "pikrellcam -vm" output for some missed events it would help to
figure out why they were missed. If you need to, ^C out of the program when an event happens
so the output is stopped.

tomtgrp
Posts: 41
Joined: Thu Jul 09, 2015 8:35 am

Re: PiKrellCam: motion vector detect + OSD web interface

Wed Dec 02, 2015 6:41 pm

Hello,

regarding live streaming of the h264 full hd stream I did some tests mit more ore less success.
The Idea:
Grab the frames when they are written to the circular buffer and send them via tcp socket.
TCP connect->send H264 header->send h264 buffer until disconnect->wait for next connect.

I put some code inside mmalcam.c, video_h264_encoder_callback

Code: Select all


[b]before vcb_video_write(VideoCircularBuffer *vcb):[/b]
//++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++      
#include <sys/types.h>
#include <sys/socket.h>
#include <sys/poll.h>
#include <sys/ioctl.h>
#include <netinet/in.h>
#include <string.h>
#include <unistd.h>
#include <netdb.h>
 extern int listenfd, connfd, num_sent;
 extern socklen_t clilen;
 extern struct sockaddr_in cliaddr, servaddr;
 extern int h264_conn_status;
 
 void my_send_h264(char * what, void *data, int len)
  {
    if(h264_conn_status != H264_TCP_WAIT_FOR_CONNECT)
    {  
      if(h264_conn_status == H264_TCP_SEND_DATA)
      {
        num_sent=send(connfd, data,len, MSG_NOSIGNAL);
        if (pikrellcam.debug)
          printf("write tcp %s:%d \n",what, len);
        if (num_sent < 0) 
        {
          perror("Conn closed\n");
          shutdown(connfd, SHUT_RDWR); 
          close(connfd);
          h264_conn_status=H264_TCP_WAIT_FOR_CONNECT;
          return;
        }
      }        
  
    }
  }
 //++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++      
 
[b]in video_h264_encoder_callback:[/b]
.....
		if (vcb->state == VCB_STATE_MANUAL_RECORD_START)
			{
			/* Write mp4 header and set tail to most recent keyframe.
			|  So manual records may have up to about a sec pre_capture.
			*/
			fwrite(vcb->h264_header, 1, vcb->h264_header_position, vcb->file);
			pikrellcam.video_header_size = vcb->h264_header_position;
			pikrellcam.video_size = vcb->h264_header_position;
			vcb->tail = vcb->key_frame[vcb->cur_frame_index].position;
			vcb->record_start = t_cur;
			vcb->state = VCB_STATE_MANUAL_RECORD;
			event |= EVENT_PREVIEW_SAVE;
			}

//++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++      
    if(h264_conn_status != H264_TCP_WAIT_FOR_CONNECT)
      {        
        if(h264_conn_status == H264_TCP_SEND_HEADER) 
        {
          num_sent=send(connfd, vcb->h264_header, vcb->h264_header_position, MSG_NOSIGNAL);
   //       fwrite(vcb->h264_header, 1, vcb->h264_header_position, vcb->file);
   //       vcb->tail = vcb->key_frame[vcb->cur_frame_index].position;
          h264_conn_status=H264_TCP_SEND_DATA;
          if (pikrellcam.debug)
            printf("write h264 header:%d \n",vcb->h264_header_position);
          if (num_sent < 0) 
          {
            perror("Conn closed\n");
            shutdown(connfd, SHUT_RDWR); 
            close(connfd);
            h264_conn_status=H264_TCP_WAIT_FOR_CONNECT;
          }
        }
      }
//++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
      
		/* Save video data into the circular buffer.
		*/
		mmal_buffer_header_mem_lock(mmalbuf);
		end_space = vcb->size - vcb->head;
		if (mmalbuf->length <= end_space)
		{
			memcpy(vcb->data + vcb->head, mmalbuf->data, mmalbuf->length);
//++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
my_send_h264("data 1",vcb->data + vcb->head, mmalbuf->length);
//++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
		}
		else
		{
			memcpy(vcb->data + vcb->head, mmalbuf->data, end_space);
//++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
my_send_h264("data 2",vcb->data + vcb->head, end_space);
//++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
			memcpy(vcb->data, mmalbuf->data + end_space, mmalbuf->length - end_space);
//++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
my_send_h264("data 3",vcb->data, mmalbuf->length - end_space);
//++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
		}
    
		vcb->head = (vcb->head + mmalbuf->length) % vcb->size;

		mmal_buffer_header_mem_unlock(mmalbuf);



and in pikrellcam.c:

Code: Select all

//++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
#include <stdlib.h>
#include <stdio.h>
#include <sys/types.h>
#include <sys/socket.h>
#include <sys/poll.h>
#include <sys/ioctl.h>
#include <netinet/in.h>
#include <string.h>
#include <unistd.h>
#include <netdb.h>
#include <signal.h>
#include <fcntl.h>
#include <arpa/inet.h>  

#define SERV_PORT 3000  /*port*/
#define LISTENQ 1       /*maximum number of client connections */

int listenfd, connfd, num_sent=0;
socklen_t clilen;
struct sockaddr_in cliaddr, servaddr;
long save_fd;
int h264_conn_status=H264_TCP_WAIT_FOR_CONNECT;
  
void setup_h264_tcp_server(void)
{
 //creation of the socket
 listenfd = socket (AF_INET, SOCK_STREAM, 0);

 //preparation of the socket address
 servaddr.sin_family = AF_INET;
 servaddr.sin_addr.s_addr = htonl(INADDR_ANY);
 servaddr.sin_port = htons(SERV_PORT);
 
 int reuse = 1;
 if (setsockopt(listenfd, SOL_SOCKET, SO_REUSEADDR, (const char*)&reuse, sizeof(reuse)) < 0)
        perror("setsockopt(SO_REUSEADDR) failed");

 #ifdef SO_REUSEPORT
 if (setsockopt(listenfd, SOL_SOCKET, SO_REUSEPORT, (const char*)&reuse, sizeof(reuse)) < 0) 
        perror("setsockopt(SO_REUSEPORT) failed");
 #endif
 
 if(bind (listenfd, (struct sockaddr *) &servaddr, sizeof(servaddr))<0);
  perror("binding");
 
 listen (listenfd, LISTENQ);

 save_fd = fcntl( listenfd, F_GETFL );
 save_fd |= O_NONBLOCK;
 fcntl( listenfd, F_SETFL, save_fd );
 
 fprintf(stderr,"%s\n","Server running...waiting for connections.");  
}
//++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++


[b]and in main:
[/b]
	for (i = 1; i < argc; i++)
		get_arg_pass1(argv[i]);

//++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
  setup_h264_tcp_server();
//++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++

  
	if (!config_load(pikrellcam.config_file))
		config_save(pikrellcam.config_file);
pikrellcam.h:

Code: Select all


#define	PIKRELLCAM_VERSION	"2.0.4"

#define H264_TCP_WAIT_FOR_CONNECT 0
#define H264_TCP_SEND_HEADER      1
#define H264_TCP_SEND_DATA        2


With some tricks I get a video stream:

nc localhost 3000 > tst.h264

gives a (vlc) playable file

Connecting with vlc to
tcp://pikam:3000
shows nothing, somethimes a black screen.

using cvlc:
cvlc tcp://127.0.0.1:3000 --sout '#standard{access=http,mux=ts,dst=:8160}' :demux=h264
and accessing with vlc to http://pikam-ip:8160
plays a stream for a few seconds, then artefacts, then again some frames and so on.

Any ideas?

Or would be there a better way,such like a fourth port path ,
CAMERA_STREAM_PORT 3?



Br
Tom

User avatar
jbeale
Posts: 3499
Joined: Tue Nov 22, 2011 11:51 pm
Contact: Website

Re: PiKrellCam: some huge motions not detected

Fri Dec 04, 2015 4:28 pm

billw wrote:Is your confirm gap > 0? If so, for objects in frame for ~ 1 sec a zero confirm gap might help.
If you can post some "pikrellcam -vm" output for some missed events it would help to
figure out why they were missed. If you need to, ^C out of the program when an event happens
so the output is stopped.
With confirm gap = 0, the pi (very occasionally) misses a large motion event. Below is an example from pikrellcam -vm of such an event at 07:44:25 today. From this debug info, it looks to me like the object was detected in five separate frames, but rejected as a valid motion in each case for some reason. In this case the subject is a dark blue car that passes from left to right , mostly filling the entire frame because this camera has a telephoto lens with 10 degree horizontal angle of view, and is set close to the road. I understand that currently, left to right motion (increasing x coordinate) shows as negative (dx) values. Here, in some regions I see a positive (dx) value; not sure why that would be. The camera is at roughly a 45 degree angle to the road, so geometric perspective causes the motion vectors at different (x,y) locations in the frame to point in somewhat different directions, is that a problem? Is it not valid motion when (for each detection region) the "reject" value is larger than the "count" value? What drives the "reject" count?

Code: Select all

cvec[0]: x,y(14,32) dx,dy(-52,3) mag2,count(2713,149) reject:231 box:18x18
   in_box[count:78 rej:51] motion:0 vetical:0 sparkle:2 limit_count:40
cvec[1]: x,y(30,27) dx,dy(-59,-38) mag2,count(4925,127) reject:542 box:16x16
   in_box[count:55 rej:218] motion:0 vetical:0 sparkle:1 limit_count:40
cvec[2]: x,y(50,28) dx,dy(11,-44) mag2,count(2057,121) reject:482 box:16x16
   in_box[count:51 rej:157] motion:0 vetical:0 sparkle:0 limit_count:40
cvec[3]: x,y(69,31) dx,dy(51,-4) mag2,count(2617,169) reject:376 box:18x20
   in_box[count:107 rej:191] motion:0 vetical:0 sparkle:0 limit_count:40
any:2200 reject:1631 sparkle:3 sparkle_expma:0.1
07:44:25 motion count:0 fail:4 window:0  

cvec[0]: x,y(10,33) dx,dy(-61,23) mag2,count(4250,97) reject:536 box:14x14
   in_box[count:47 rej:172] motion:0 vetical:0 sparkle:0 limit_count:40
cvec[1]: x,y(30,21) dx,dy(-42,-39) mag2,count(3285,65) reject:571 box:12x12
   in_box[count:32 rej:91] motion:0 vetical:0 sparkle:0 limit_count:40
cvec[2]: x,y(49,25) dx,dy(-56,-12) mag2,count(3280,95) reject:399 box:14x14
   in_box[count:22 rej:137] motion:0 vetical:0 sparkle:2 limit_count:40
cvec[3]: x,y(67,31) dx,dy(-62,-27) mag2,count(4573,177) reject:383 box:18x20
   in_box[count:132 rej:235] motion:2 vetical:0 sparkle:1 limit_count:40
any:2326 reject:1889 sparkle:3 sparkle_expma:0.1
07:44:25 motion count:1 fail:3 window:0  

cvec[0]: x,y(11,30) dx,dy(-62,-2) mag2,count(3848,179) reject:477 box:18x20
   in_box[count:125 rej:227] motion:0 vetical:0 sparkle:0 limit_count:40
cvec[1]: x,y(32,28) dx,dy(-61,-8) mag2,count(3785,168) reject:494 box:18x20
   in_box[count:66 rej:264] motion:0 vetical:0 sparkle:0 limit_count:40
cvec[2]: x,y(48,28) dx,dy(-58,-7) mag2,count(3413,160) reject:509 box:18x18
   in_box[count:94 rej:259] motion:0 vetical:0 sparkle:1 limit_count:40
cvec[3]: x,y(68,30) dx,dy(4,-44) mag2,count(1952,64) reject:417 box:12x12
   in_box[count:29 rej:101] motion:0 vetical:1 sparkle:1 limit_count:40
any:2470 reject:1897 sparkle:2 sparkle_expma:0.1
07:44:25 motion count:0 fail:4 window:0  

cvec[0]: x,y(10,35) dx,dy(53,13) mag2,count(2978,145) reject:223 box:18x18
   in_box[count:65 rej:72] motion:0 vetical:0 sparkle:3 limit_count:40
cvec[2]: x,y(49,32) dx,dy(-59,10) mag2,count(3581,142) reject:359 box:16x18
   in_box[count:62 rej:129] motion:0 vetical:0 sparkle:2 limit_count:40
cvec[3]: x,y(68,31) dx,dy(-60,-3) mag2,count(3609,183) reject:329 box:20x20
   in_box[count:100 rej:257] motion:0 vetical:0 sparkle:1 limit_count:40
any:1906 reject:1399 sparkle:7 sparkle_expma:0.2
07:44:25 motion count:0 fail:3 window:0  

cvec[0]: x,y(11,34) dx,dy(50,29) mag2,count(3341,109) reject:363 box:14x16
   in_box[count:43 rej:96] motion:0 vetical:0 sparkle:3 limit_count:40
cvec[1]: x,y(29,33) dx,dy(53,1) mag2,count(2810,164) reject:385 box:18x20
   in_box[count:77 rej:165] motion:0 vetical:0 sparkle:1 limit_count:40
cvec[2]: x,y(51,30) dx,dy(-50,-1) mag2,count(2501,161) reject:422 box:18x18
   in_box[count:89 rej:139] motion:0 vetical:0 sparkle:2 limit_count:40
cvec[3]: x,y(68,32) dx,dy(-57,-7) mag2,count(3298,148) reject:386 box:18x18
   in_box[count:107 rej:184] motion:2 vetical:0 sparkle:1 limit_count:40
any:2145 reject:1556 sparkle:7 sparkle_expma:0.3
07:44:25 motion count:1 fail:3 window:0 
Here is debug data from another missed event at 12:24:04 where the motion counts are big, but the reject count is higher

Code: Select all

cvec[1]: x,y(30,24) dx,dy(-38,49) mag2,count(3845,44) reject:579 box:10x10
   in_box[count:14 rej:91] motion:0 vetical:0 sparkle:1 limit_count:40
cvec[2]: x,y(48,31) dx,dy(-59,0) mag2,count(3481,203) reject:464 box:20x22
   in_box[count:114 rej:291] motion:0 vetical:0 sparkle:0 limit_count:40
cvec[3]: x,y(68,34) dx,dy(-67,2) mag2,count(4493,179) reject:440 box:18x20
   in_box[count:92 rej:263] motion:0 vetical:0 sparkle:0 limit_count:40
any:2440 reject:1985 sparkle:1 sparkle_expma:0.1
12:24:04 motion count:0 fail:3 window:0  

cvec[0]: x,y(14,30) dx,dy(-57,-5) mag2,count(3274,139) reject:248 box:16x18
   in_box[count:77 rej:108] motion:0 vetical:0 sparkle:1 limit_count:40
cvec[1]: x,y(30,27) dx,dy(-54,-49) mag2,count(5317,61) reject:497 box:12x12
   in_box[count:7 rej:123] motion:0 vetical:0 sparkle:5 limit_count:40
cvec[2]: x,y(46,30) dx,dy(-64,40) mag2,count(5696,91) reject:314 box:14x14
   in_box[count:27 rej:156] motion:0 vetical:0 sparkle:4 limit_count:40
cvec[3]: x,y(68,32) dx,dy(-54,0) mag2,count(2916,97) reject:169 box:14x14
   in_box[count:38 rej:78] motion:0 vetical:0 sparkle:10 limit_count:40
any:1636 reject:1228 sparkle:20 sparkle_expma:0.3
12:24:04 motion count:0 fail:4 window:0  

cvec[0]: x,y(14,23) dx,dy(-51,-17) mag2,count(2890,59) reject:147 box:10x12
   in_box[count:32 rej:86] motion:0 vetical:0 sparkle:6 limit_count:40
cvec[1]: x,y(29,25) dx,dy(-58,-11) mag2,count(3485,133) reject:357 box:16x18
   in_box[count:72 rej:166] motion:0 vetical:0 sparkle:0 limit_count:40
cvec[2]: x,y(50,28) dx,dy(-62,-3) mag2,count(3853,145) reject:434 box:18x18
   in_box[count:53 rej:205] motion:0 vetical:0 sparkle:0 limit_count:40
cvec[3]: x,y(72,33) dx,dy(-54,0) mag2,count(2916,134) reject:275 box:16x18
   in_box[count:81 rej:80] motion:0 vetical:0 sparkle:1 limit_count:40
any:1691 reject:1213 sparkle:7 sparkle_expma:0.3
12:24:04 motion count:0 fail:4 window:0  

cvec[3]: x,y(67,25) dx,dy(-55,-8) mag2,count(3089,138) reject:196 box:16x18
   in_box[count:95 rej:149] motion:0 vetical:0 sparkle:0 limit_count:40
any:357 reject:196 sparkle:4 sparkle_expma:0.4
12:24:04 motion count:0 fail:1 window:0  
Last edited by jbeale on Fri Dec 04, 2015 9:16 pm, edited 1 time in total.

User avatar
jbeale
Posts: 3499
Joined: Tue Nov 22, 2011 11:51 pm
Contact: Website

Re: PiKrellCam: motion vector detect + OSD web interface

Fri Dec 04, 2015 8:55 pm

Trying to answer my question about how the reject count is formed... from comments in https://github.com/billw2/pikrellcam/bl ... c/motion.c it seems that reject_count increments if individual motion vectors differ too much in direction?

Code: Select all

	/* If we are left with enough counts for a composite vector, filter out
	|  motion vectors not pointing in the composite directon.
	|  Vectors of sufficient mag2 but not pointing in the right direction
	|  will be rejects and their count can be large for noisy frames.
	|  Dot product to allow a spread, (avoiding sqrt() to get magnitude and scale by 100 for integer math):
	|    cos(a) = (v1 dot v2) / mag(v1) * mag(v2))	# cos(25) = 0.906
	|    100 * cos(25)^2 = 82 = 100 * (v1 dot v2)^2 / mag(v1)^2 * mag(v2)^2)
	*/
...

	if (   dot > 0		/* angle at least < 90 deg */
	   && tvec.mag2 > 0
   	   && (cos2 = 100 * dot * dot / (tvec.mag2 * mvmag2)) >= 82
				   )
	{ ... }
		else
	{
	  *(mf->trigger + mf->width * y + x) = 2;	/* reject flag */
	  mf->reject_count += 1;
	  mreg->reject_count += 1;
	}
If this is the problem (valid motion vectors from fast, large, close-by moving objects look too random in direction) I'm guessing that my existing setup may not work, without relaxing the "reject noise" filter so much that I get a lot of spurious detects on rain, moving branches, blowing leaves, etc. and I will have to settle for using a wider-angle field of view (another camera with the standard RPi lens seems to work fine.)

billw
Posts: 403
Joined: Tue Sep 18, 2012 8:23 pm

Re: PiKrellCam: motion vector detect + OSD web interface

Sat Dec 05, 2015 4:54 pm

jbeale wrote:Trying to answer my question about how the reject count is formed... from comments in https://github.com/billw2/pikrellcam/bl ... c/motion.c it seems that reject_count increments if individual motion vectors differ too much in direction?

If this is the problem (valid motion vectors from fast, large, close-by moving objects look too random in direction) I'm guessing that my existing setup may not work, without relaxing the "reject noise" filter so much that I get a lot of spurious detects on rain, moving branches, blowing leaves, etc. and I will have to settle for using a wider-angle field of view (another camera with the standard RPi lens seems to work fine.)
Yes, vector direction filtering is key in rejecting frames with noise and to lower the thresholds would
be a tradeoff with reduced noise immunity. All of the cvecs in your example had pretty high
reject counts and I'm pretty sure that lowering the threshold enough for that event to have passed as
motion would cause increased numbers of false detects of noise.

That event is particularly hard to classify as valid motion because not only were the overall
frame rejects large, the ratio of in_box rejects was also very high. So a suspicion I'm starting to
have is when there's a large object filling the frame and areas of the object are uniform, the
camera can generate vectors pointing in odd directions not related to the overall object direction
of motion. It's probably like what I found for rain. I thought I could use a vertical filter to detect
that, but with a screen full of rain drops, any given vector could be pointing at a screen patch
to the left, right or up. The individual vectors don't have to necessarily point down just because
the drops are falling down. A drop could match in any direction.

So this is something I'll think about.

Edit: Also, if you look at dx,dy for cvecs within a frame and cvec to cvec over consecutive
frames, the directions don't match up well and it makes it harder to override the reject counts
and call the event valid motion. This reinforces my idea that for this event, the vectors are
difficult to distinguish from noise... but I'll still think it over...
Last edited by billw on Sat Dec 05, 2015 6:38 pm, edited 1 time in total.

billw
Posts: 403
Joined: Tue Sep 18, 2012 8:23 pm

Re: PiKrellCam: motion vector detect + OSD web interface

Sat Dec 05, 2015 6:29 pm

tomtgrp wrote:Hello,

regarding live streaming of the h264 full hd stream I did some tests mit more ore less success.
The Idea:
Grab the frames when they are written to the circular buffer and send them via tcp socket.
TCP connect->send H264 header->send h264 buffer until disconnect->wait for next connect.
...
With some tricks I get a video stream:

nc localhost 3000 > tst.h264

gives a (vlc) playable file

Connecting with vlc to
tcp://pikam:3000
shows nothing, somethimes a black screen.

using cvlc:
cvlc tcp://127.0.0.1:3000 --sout '#standard{access=http,mux=ts,dst=:8160}' :demux=h264
and accessing with vlc to http://pikam-ip:8160
plays a stream for a few seconds, then artefacts, then again some frames and so on.

Any ideas?

Or would be there a better way,such like a fourth port path ,
CAMERA_STREAM_PORT 3?



Br
Tom
I've not had time yet to try this, but I hope to get to it. It would be a nice feature so if you
come up with code changes, please continue to post them.

I think tapping off the existing h264 stream would be better. There are conditiions
where the camera is pushing it's limits and another camera stream could overload it for
the 24fps 4 mjpeg_divider defaults. But I say that without testing...

User avatar
jbeale
Posts: 3499
Joined: Tue Nov 22, 2011 11:51 pm
Contact: Website

Re: PiKrellCam: motion vector detect + OSD web interface

Sun Dec 06, 2015 1:58 am

billw wrote: if you look at dx,dy for cvecs within a frame and cvec to cvec over consecutive
frames, the directions don't match up well and it makes it harder to override the reject counts
and call the event valid motion. This reinforces my idea that for this event, the vectors are
difficult to distinguish from noise... but I'll still think it over...
I agree with all your points. For my specific case, I think it would be nice to at least have the option to allow a sudden short-term drastic increase in motion vectors (even in random directions) to be a valid motion trigger. The background activity that I want to reject (like rain or tree branches moving) I believe usually has a longer period of activity, and a more gradual onset and decay. For this specific case, I see between 3 and 6 consecutive frames with large motion vectors in two, three or all four regions, preceded by and followed by hundreds of frames with no motion over threshold in any region.

billw
Posts: 403
Joined: Tue Sep 18, 2012 8:23 pm

Re: PiKrellCam: motion vector detect + OSD web interface

Sun Dec 06, 2015 5:10 pm

jbeale wrote: For my specific case, I think it would be nice to at least have the option to allow a sudden short-term drastic increase in motion vectors (even in random directions) to be a valid motion trigger.
I'll add in this option. It will probably at least at first be labeled experimental because I would like
to see some longer term feedback on if it can have a good signal to noise.

jit
Posts: 33
Joined: Fri Apr 18, 2014 2:52 pm

Re: PiKrellCam: motion vector detect + OSD web interface

Tue Dec 08, 2015 8:38 am

billw wrote:
jbeale wrote: For my specific case, I think it would be nice to at least have the option to allow a sudden short-term drastic increase in motion vectors (even in random directions) to be a valid motion trigger.
I'll add in this option. It will probably at least at first be labeled experimental because I would like
to see some longer term feedback on if it can have a good signal to noise.
I'd be happy to test this and provide feedback. I've noticed a couple of occasions where someone passing across the field of view that's close (within 1.5m) to the camera gets missed.

jit
Posts: 33
Joined: Fri Apr 18, 2014 2:52 pm

Re: PiKrellCam: motion vector detect + OSD web interface

Tue Dec 08, 2015 8:44 am

With regards to motion regions, I noticed that the default is four regions placed across horizontally. Is there any advantage to this compared to say a single region spanning the entire view?

I have a camera where I only want the lower third of the view to detect motion. So wondering what the optimal setup for this is? Single region vs multiple.

tomtgrp
Posts: 41
Joined: Thu Jul 09, 2015 8:35 am

Re: PiKrellCam: motion vector detect + OSD web interface

Tue Dec 08, 2015 9:09 am

Hello,

I noticed a crash when I was adjusting the motion regions. The crash is reproducible,
Moving the start of this region to the left screen border and resizing it
crashes prikrellcam, when the region reaches the right border.

add_region 0.100 0.321 0.835 0.679

This seems only to happen when left border=0.
Vertical movements of the region didn't show a crash.

assertion failure:/home/dc4/projects/staging/userland/interface/vcos/generic/vcos_generic_blockpool.c:356:vcos_generic_blockpool_free():(subpool) && (subpool)->magic == VCOS_BLOCKPOOL_SUBPOOL_MAGIC && subpool->start >= subpool->mem
Abgebrochen


Br
Tom

tomtgrp
Posts: 41
Joined: Thu Jul 09, 2015 8:35 am

Re: PiKrellCam: motion vector detect + OSD web interface

Tue Dec 08, 2015 12:20 pm

billw wrote: I've not had time yet to try this, but I hope to get to it. It would be a nice feature so if you
come up with code changes, please continue to post them.

I think tapping off the existing h264 stream would be better. There are conditiions
where the camera is pushing it's limits and another camera stream could overload it for
the 24fps 4 mjpeg_divider defaults. But I say that without testing...

Hello,
got it running, attached the patches against actual version 2.0.4.
patches.zip
(4.25 KiB) Downloaded 249 times
Stream runs now for several hours in full hd, a real live stream :-) , delay ~ 1 sec, cpu load ~10% on a 4 core pi2.
No side effects seen on live preview, motion and all the other features.

Short description inside tcpserver.c

Code: Select all

// Goal:
// Live preview of the h264 Full HD Camera Stream
//
// Idea:
// Grab the data being written to circular buffer and send them via tcp server to a listener.
// Maybe there is a better solution, like a further Pipeline which gets the h264 live stream and sends it.
// Maybe also multiple streams possible with different resolution (full, medium, mobile, ...)
//
// use with gst-rtsp-server-1.4.4 on Raspbian Jessie (on Wheezy I didn't get gst-rtsp-server built) or 
// gst-variable-rtsp-server (look for gst-gateworks-apps-master)
// and the following pipeline (!! do-timestamp=true is important !!, blocksize=262144 optional, can be lower)
//
// ./gst-variable-rtsp-server -d 99 -p 8555 -m /stream  
//   -u "(tcpclientsrc port=3000 do-timestamp=true blocksize=262144 
//   ! video/x-h264,stream-format=byte-stream,profile=high 
//   ! h264parse ! rtph264pay name=pay0 pt=96 )" 
//
// then open the stream, e.g. vlc rtsp://your_pi_addr:8555/stream
//
// delay between live and stream is ~ 1 sec.
// CPU load with 1 active full hd stream is ~10% on a PI2 (4 Cores, 100% load is 400% displayed with 'top')
//

Br
Tom

billw
Posts: 403
Joined: Tue Sep 18, 2012 8:23 pm

Re: PiKrellCam: motion vector detect + OSD web interface

Tue Dec 08, 2015 7:25 pm

jit wrote:
billw wrote:
jbeale wrote: For my specific case, I think it would be nice to at least have the option to allow a sudden short-term drastic increase in motion vectors (even in random directions) to be a valid motion trigger.
I'll add in this option. It will probably at least at first be labeled experimental because I would like
to see some longer term feedback on if it can have a good signal to noise.
I'd be happy to test this and provide feedback. I've noticed a couple of occasions where someone passing across the field of view that's close (within 1.5m) to the camera gets missed.
As I've been looking at this it seems to be developing into a large/close object motion detect case
that pikrellcam does need so it may not end up being "experimental". But getting feedback on
testing it will be good.

billw
Posts: 403
Joined: Tue Sep 18, 2012 8:23 pm

Re: PiKrellCam: motion vector detect + OSD web interface

Tue Dec 08, 2015 7:50 pm

jit wrote:With regards to motion regions, I noticed that the default is four regions placed across horizontally. Is there any advantage to this compared to say a single region spanning the entire view?

I have a camera where I only want the lower third of the view to detect motion. So wondering what the optimal setup for this is? Single region vs multiple.
Multiple regions are primarily for laying out motion sensitive regions. If one region covers
you then that's fine with one caveat. If the region is large then motion detection can possibly be
compromised for the case where two small objects are moving simultaneously in separated areas
of the region. This is because a composite vector for each region is built and the vector density
in a box at the vector center determines detection.

So two small objects separated makes a composite vector centered between
the objects with the individual motion vectors located away from this center. But this is
theoretical. Practically there will be motion bursts from each object and motion will get detected.
But if you are worried about it, split your bottom region into two regions.

On the flip side, motion regions should not get so small (relatively) that they cannot hold the
number off vectors you have set as the vector limit count. This also should not be an issue
for most cases because, say you have a limit count of 20, then a region should be able to contain
maybe at least twice those vectors which would be about a 7x6 region. That's pretty small compared
to a 1080p full screen motion vector array of 120x68. (motion vectors span 16x16 screen pixels)

billw
Posts: 403
Joined: Tue Sep 18, 2012 8:23 pm

Re: PiKrellCam: motion vector detect + OSD web interface

Tue Dec 08, 2015 7:58 pm

tomtgrp wrote: Hello,
got it running, attached the patches against actual version 2.0.4.
patches.zip
Stream runs now for several hours in full hd, a real live stream :-) , delay ~ 1 sec, cpu load ~10% on a 4 core pi2.
No side effects seen on live preview, motion and all the other features.

Br
Tom
Thanks for this! I'll apply the patches to pikrellcam for the next update.

lassiko
Posts: 10
Joined: Tue May 12, 2015 7:04 pm

Re: PiKrellCam: motion vector detect + OSD web interface

Wed Dec 09, 2015 8:40 pm

Hi all,

Next question regarding commands from shell: how can i start/stop motion detection from a shell?
Log gives me this info but that leads to nothing command_process: motion_enable toggle

Thnx

User avatar
jbeale
Posts: 3499
Joined: Tue Nov 22, 2011 11:51 pm
Contact: Website

Re: PiKrellCam: motion vector detect + OSD web interface

Wed Dec 09, 2015 8:57 pm

lassiko wrote:Next question regarding commands from shell: how can i start/stop motion detection from a shell?
I haven't tried it, but does this work?

Code: Select all

echo 'motion_enable on' >> /home/pi/pikrellcam/www/FIFO
echo 'motion_enable off' >> /home/pi/pikrellcam/www/FIFO

Ongelma
Posts: 113
Joined: Wed Dec 03, 2014 6:46 am

Re: PiKrellCam: motion vector detect + OSD web interface

Thu Dec 10, 2015 1:06 pm

i have maged to install this to three raspi's one works fine, two of them not. The ones not working let me go to the webpage but the system / start wont work. Any ideas?

Return to “Camera board”