jeanot1314
Posts: 27
Joined: Mon Feb 24, 2014 2:01 pm

Pi3D

Sat Oct 08, 2016 7:45 pm

I have been playing with pi3D for few days now, it work very well! But I have no idea if what I actually want to do is possible or not...

I added a character as an object from blender. On the ForestWalk Exemple, it worked very good and I can find the guy in the middle of the woods.

But now I would like to make this little fellow moving. I can, of course, change it's position on the map, but if I want him to rise an arm, would it be possible? Blender is very nice to make an object move. But can I do it with Pi3D?

I quiet new on all of this, and maybe my Raspberry Pi is actually not powerful enough for this, but it's worth a try :)

Thanks for your help

User avatar
paddyg
Posts: 2566
Joined: Sat Jan 28, 2012 11:57 am
Location: UK

Re: Pi3D

Sat Oct 08, 2016 9:30 pm

@jeanot1314 pi3d doesn't have rigged mesh distortion built in. It would be possible to do using numpy but it would be a major project and might not be very quick when finished. However there is a facility to make a Shape into a child of another Shape so you could construct an articulated character along the lines of a LEGO man.
also https://groups.google.com/forum/?hl=en-GB&fromgroups=#!forum/pi3d

jeanot1314
Posts: 27
Joined: Mon Feb 24, 2014 2:01 pm

Re: Pi3D transform object

Sat Oct 08, 2016 9:58 pm

Thanks!

That's what I was thinking...

I will try to build a lego man, it could be very helpful and fun for my project :lol:

User avatar
paddyg
Posts: 2566
Joined: Sat Jan 28, 2012 11:57 am
Location: UK

Re: Pi3D

Sat Oct 08, 2016 11:26 pm

The offset system (cx,cy,cz) might be a bit tricky to get the hang of - so here's a v. simple sketch to give you the idea:

Code: Select all

import pi3d
import math

DISPLAY = pi3d.Display.create()
shader = pi3d.Shader("uv_light")
tex = pi3d.Texture('textures/PATRN.PNG')

body = pi3d.Cylinder(radius=0.5, height=2.0, z=5.0)
body.set_draw_details(shader, [tex])

arm_1 = pi3d.Cylinder(radius=0.1, height=0.5, x=0.6, y=0.5, cy=-0.25)
arm_1.set_draw_details(shader, [tex])

arm_2 = pi3d.Cylinder(radius=0.1, height=0.4, y=-0.45, cy=-0.2)
arm_2.set_draw_details(shader, [tex])

body.add_child(arm_1)
arm_1.add_child(arm_2)
t = 0
keys = pi3d.Keyboard()
while DISPLAY.loop_running():
  t += 0.01
  body.draw()
  arm_1.rotateToX(20.0 * math.sin(t))
  arm_1.rotateToY(15.0 * math.sin(1.7 * t))
  arm_2.rotateToX(30.0 + 30.0 * math.sin(t))
  arm_2.rotateToZ(10.0 * math.sin(2.7 * t))
  body.rotateIncY(0.05)
  if keys.read() == 27:
    keys.close()
    DISPLAY.destroy()
    break
PS It wouldn't be too hard to write something that would parse a bvh (or other motion capture) file - that way you could get any number of ready made animations for your character (not sure about the order of applying rotations pi3d is fixed zxy)
also https://groups.google.com/forum/?hl=en-GB&fromgroups=#!forum/pi3d

jeanot1314
Posts: 27
Joined: Mon Feb 24, 2014 2:01 pm

Re: Pi3D

Sun Oct 09, 2016 12:21 pm

Great I made a nice little Lego Man!!

I just have 2 more questions, i am really sorry to disturb you that much :(

1. How to set a "normal" color to a part of the body. The set_draw_details allow me to put a shader, but what about setting just e blue color for exemple, do y have to find some kind of shader library for it?

2. Probably more tricky.. Now I have my lego man moving on the forestWalk map with a camera behind him. I would like to set the Man as the central point of camera rotating. Right now, the camera move around itself.
I am not sure if it's what you explained here : viewtopic.php?f=32&t=153180
On the video, it looks like the rotation you make are still from the camera itself.

To try to be a bit more clear, lets say I want to record a tree on the forestWalk. I need to be few step away from the tree, and the Camera always point to the tree, but turn around if I move my mouse.

thanks again for your great help, I can create a new forum topic if you want :)

User avatar
paddyg
Posts: 2566
Joined: Sat Jan 28, 2012 11:57 am
Location: UK

Re: Pi3D

Sun Oct 09, 2016 12:56 pm

No prob.
1. You need to use the 'mat_..' shaders and set the materal properties.

Code: Select all

shader = pi3d.Shader("mat_bump")
shader2 = pi3d.Shader("mat_light")
tex = pi3d.Texture('textures/floor_nm.jpg')

body = pi3d.Cylinder(radius=0.5, height=2.0, z=5.0)
body.set_draw_details(shader, [tex], 4.9)
body.set_material((1.0, 0.0, 0.0))

arm_1 = pi3d.Cylinder(radius=0.1, height=0.5, x=0.6, y=0.5, cy=-0.25)
arm_1.set_draw_details(shader, [tex], 2.0)
arm_1.set_material((1.0, 1.0, 0.0))

arm_2 = pi3d.Cylinder(radius=0.1, height=0.4, y=-0.45, cy=-0.2)
arm_2.set_shader(shader2)
arm_2.set_material((0.5, 0.0, 1.0))
2, The Camera.relocate() method can do that - look at the Orbit demo where it points towards the Sun
also https://groups.google.com/forum/?hl=en-GB&fromgroups=#!forum/pi3d

jeanot1314
Posts: 27
Joined: Mon Feb 24, 2014 2:01 pm

Re: Pi3D

Tue Oct 11, 2016 9:33 pm

IMG_5149.JPG
IMG_5149.JPG (33.81 KiB) Viewed 5449 times
It's going very well,

I attached my lego man ;-)

User avatar
paddyg
Posts: 2566
Joined: Sat Jan 28, 2012 11:57 am
Location: UK

Re: Pi3D

Tue Oct 11, 2016 10:19 pm

Nice. And exposes a flaw in this forum code - it doesn't seem to check exif info on image rotation!
also https://groups.google.com/forum/?hl=en-GB&fromgroups=#!forum/pi3d

jeanot1314
Posts: 27
Joined: Mon Feb 24, 2014 2:01 pm

Re: Pi3D

Sun Nov 13, 2016 10:08 am

Hello!!

My PI3D project is going very well, thanks to you :)

I succeed tu use python-vrzero code, which is based on pi3d to use the stereocam. I also added my lego guy.

Resizing the project to a 7inch 800x480p screen, I am now running at 60FPS, which is the exact speed I want (but it actually seems stocked at this maximum speed, I don't know why...).

My little bad surprise, is adding several lego guy seems to slow down drastically the general speed (with three I dropped at around 20FPS). He is only build with 20 or 25 shapes like spheres and cylinders. I use this method to create a body part :
shaderGuy=pi3d.Shader("mat_bump")
colorshirt=(0.2,0.3,0.2)
self.body = pi3d.Cylinder(radius, height, z)
self.body = set_shader(shaderGuy)
self.body=set_material(colorshirt)

I do this for all the part and fix them together with
self.body.add_child(self.head)

and then I use
roger.body.position(x,y,z)
roger.body.draw() on the loop

There is probably another way to do this, or it mean that i am not gonna be able to create a more complex and design lego man. I want to have several of them in my map, to maybe create a simple game.

I was also wondering how do you find the .egg files to create the cathedral or the conference room. They look amazing and don't consume much FPS.

Thanks a lot :)

User avatar
paddyg
Posts: 2566
Joined: Sat Jan 28, 2012 11:57 am
Location: UK

Re: Pi3D

Sun Nov 13, 2016 12:14 pm

jeanot,

I'm glad your project is coming together. There is an issue when you have lots of objects as each Shape has to have a projection matrix recalculated it either it has moved or the camera has moved and that calculation requires up to 5 4x4 dot products. As a rule of thumb 50 Shapes seems to run ok but 100 Shapes sometimes starts to slow down.

There are various ways round this.
The easiest is probably to merge together the parts that don't need to be articulated - see how the robot is constructed in RobotWalkabout.py, so you could make each lego figure out of 6 shapes with each shape being merged from three or four others. The disadvantage is that it merges into the same Buffer object and pi3d only has functionality to define one material for the whole Buffer.
An alternative would be to design more complicated shapes for head, arm, body, leg in blender and create a UV mapping to allow a texture to be used. This would allow more colour variation and sophisticated shapes with a smaller number of vertices. (This is probably the route I would take.)

Do you use the bump shader normal mapping - in your example you don't look to load a texture? This shader will take quite a bit more processing (of the GPU) than the mat_light shader.

The egg files are an alternative format used by panda3d but I don't think it can do anything that's not available in the obj files so I would suggest using that. There should be some info in the FAQ about exporting from blender to obj.

Paddy
also https://groups.google.com/forum/?hl=en-GB&fromgroups=#!forum/pi3d

jeanot1314
Posts: 27
Joined: Mon Feb 24, 2014 2:01 pm

Re: Pi3D

Sun Nov 13, 2016 6:36 pm

All right, I understand better now...

The speed seems really dependent on the number of shapes. So my Lego guy is really heavy on the program.

The complexity of the shape seem to matter less. So I started on a blender file with a guy that I cut it in several pieces (sounds weird to say it like this).

I load the object files on pi3d and I have a way more complex guy, plus I am still running above 55fps! I can move the different part between them.

Thank you so much, I ll keep you in touch on how the project 's going ;)

User avatar
paddyg
Posts: 2566
Joined: Sat Jan 28, 2012 11:57 am
Location: UK

Re: Pi3D

Sun Nov 13, 2016 7:44 pm

Great, yes I'd like to know how it progresses. Also, I forgot to mention: It is possible in blender to make a high resolution model with lots of fine detail then generate a normal map from it (look up render, bake to normal) You can then use a really low-poly mesh with the uv_bump or uv_reflect shader. Alternatively (and possibly easier) you can just 'draw' the bump map on to of the unwrapped surface (in blender uv view export uv layout) so that white is maximum bump size and black is no bump. When you generate a Texture using that image you can specify the argument normal_map=1.0 (or any other non-zero value ,-ve will stick in) and the image will be converted to a normal map.

Paddy
also https://groups.google.com/forum/?hl=en-GB&fromgroups=#!forum/pi3d

jeanot1314
Posts: 27
Joined: Mon Feb 24, 2014 2:01 pm

Re: Pi3D

Mon Nov 14, 2016 2:53 pm

Hi,

Another little question, I feel really sorry to bother you that much... :?

I was wondering how complex it would be to run pi3d on another machine? Could be embedded like an odroid C2 on ubuntu, or even on mac/windows.

I am building some sensors to recognize the moving part of the body and move my blender guy. I made it run on a Rpi with a 7 inch screen and stereoscopic for VR. It would be also very nice to be able to build a map and play on a computer with the body sensors with more speed and more complex code.

I guess some low layers of the code are calling the GPU acceleration from the Pi, so it's not gonna be as easy as "of course, python run on every machine!"

Thanks

jeanot1314
Posts: 27
Joined: Mon Feb 24, 2014 2:01 pm

Re: Pi3D

Mon Nov 14, 2016 3:20 pm

My bad... my question is explained in the documentation!

https://pi3d.github.io/html/ReadMe.html

User avatar
paddyg
Posts: 2566
Joined: Sat Jan 28, 2012 11:57 am
Location: UK

Re: Pi3D

Mon Nov 14, 2016 3:25 pm

jeanot, pi3d runs ok on other machines provided they have something installed to translate the OpenGLES 2.0 functionality to OpenGL - and of course there has to be hardware which runs OpenGL, which includes normal computers that aren't too old. (and obviously python, numpy and, usually, Pillow) Linux is easiest but windows isn't too bad. OSX should be OK too but I don't have a mac handy so it's a struggle to sort out what the issues are. You can also port pi3d apps to Android if you really want, such as this.
peter hess managed to get pi3d running on cubitruck and odroid after a fashion and I have a CHIP on its way (which also uses the mali400 gpu) so hopefully I will see what's involved getting that working.

EDIT, crossed in post but will leave here as references to running on ODroid, which you specifically mentioned.
also https://groups.google.com/forum/?hl=en-GB&fromgroups=#!forum/pi3d

jeanot1314
Posts: 27
Joined: Mon Feb 24, 2014 2:01 pm

Re: Pi3D

Sat Dec 17, 2016 6:21 pm

Hi,

My project is almost done, and my Instructable almost ready. Should be posted next week :)

I am just really struggling on something right now. The avatars parts are assembled with the add_child method. It work great but I have a little problem If I play with two parts like a arm and a hand :

When I move my arm, the hand (attached to the arm with add_child) automatically turn in the same direction. When I have one DOF sensor on my arm and one on my hand, I don't want it to take the same direction as the arm (the position is given by it's own sensor).
So I started compensating my angles, so the hand move with "HAND_EULER - ARM_EULER"
But in fact, my Euler library isn't symmetrical! (Yep, I don't understand why ether...) so -euler0 isn't the opposite of euler0
So I tried to find function to manipulate Euler, but I really don't understand how to just have the opposite. And I don't want to change this Arduino Euler library as this one is the most precise I could find.

Do you have any idea how to make the hand from add_child not rotating when the arm is? something not implying to negative my angles?
Or maybe you have an idea on how to manipulate Euler angle, that could work too :roll:

Thanks,
I send you the project link when it's posted :)

User avatar
paddyg
Posts: 2566
Joined: Sat Jan 28, 2012 11:57 am
Location: UK

Re: Pi3D

Sun Dec 18, 2016 1:38 am

Hmm, I take it you mean you want to be able to rotate the arm but the hand remain un-rotated in the 'world' frame of reference. When the shape is made an child of another shape all of its rotations are defined relative to the parent's frame of reference.

The problem is basically that matrix multiplication is not commutative which means that if X means rotation about x axis and if XYZ means apply Z to a vector then apply Y to the result then apply X to the result this will be a different rotation to YXZ, YZX, ZYX, XZY and ZXY if you use the (non-standard) axis orientation of pi3d (and the opengl screen) of z in, y up and x rightwards and take the vector 0,0,1 (i.e. straight into the screen) and apply 90 degree rotations you will see that XYZ is pointing left -1,0,0 but YXZ gives 0,1,0 and YZX gives 0,0,1 i.e. doesn't change it!

pi3d always applies the rotations z then x then y ie YXZ

So, to cut a long story short to get the reversed rotation for the hand you really need to apply the negative rotations also in the reverse order i.e. -Z-X-Y(YXZ) will bring you back to where you started but -Y-X-Z(YXZ) will not do. But you can't easily do this in pi3d. So, in order to reverse the rotation of the hand to exactly compensate for the rotation of the arm you need to find three new rotations P about the x axis Q about the y and R about the z such that QPR == -Z-X-Y which is tricky, however there are two methods in Camera which might be used in some way: Camera.matrix_from_two_vecors() and Camera.euler_angles()
OR attempt to solve the nine simultaneous equations you can derive from here https://en.wikipedia.org/wiki/Euler_ang ... ion_matrix something like

Code: Select all

        cR.cP-sR.sP.sQ    cQ.sR+cR.sP.sQ    -cP.sQ
        -cP.sR               cR.cP           sP
        cR.sQ+Qy.sR.sP   sR.sQ-cR.cQ.sP      cP.cQ
matches
        cY.cZ-sY.sX.sZ     -cX.sZ           cY.sX.sZ
        cZ.sY.sX+cY.sZ     cX.cZ          -cY.cZ.sX+sY.sZ
       -cX.sY               sX                cY.cX
where RPQ are the xyz axis rotations to apply to the hand to compensate for the XYZ rotations applied to the arm.

NB you will have to check through these as I have almost certainly made several mistakes AND you need to be aware that, as well as using different axes pi3d also uses C style arrays for its matrices (which are laid out like reading a book x is axis 0 and y is axis 1 i.e. x is the line number and y is the character along each line) whereas standard mathematical notation uses fortran arrays (x is left to right and z is up and down) AND I have to warn you that I spent quite a bit of time fiddling around getting the matrices to work with pi3d!

After reading through what I have written I think my approach might be to create the reverse rotation matrix for the arm using numpy to save time doing lots of trig (see pi3d.Shape.draw()) something like:

Code: Select all

rox_neg = arm.rox.copy()
rox_neg[1,2] *= -1
rox_neg[2,1] *= -1
roy_neg = arm.roy.copy()
roy_neg[0,2] *= -1
roy_neg[2,0] *= -1
roz_neg = arm.roz.copy()
roz_neg[0,1] *= -1
roz_neg[1,0] *= -1
p, q, r = camera.euler_angles(np.dot(roz_neg, np.dot(rox_neg, roy_neg)))
also https://groups.google.com/forum/?hl=en-GB&fromgroups=#!forum/pi3d

jeanot1314
Posts: 27
Joined: Mon Feb 24, 2014 2:01 pm

Re: Pi3D

Mon Dec 19, 2016 8:13 pm

Hi,
Thanks for the help!

So, if I understand correctly, using camera.euler_angles() I can calculate the three "inversed angles".
When I have those three angles ready, I just need to do :

Hand_angleX - Inverse_arm_angleX = New_Hand_angleX
Hand_angleY - Inverse_arm_angleY = New_Hand_angleY
Hand_angleZ - Inverse_arm_angleZ = New_Hand_angleZ

Am I correct?

PS: I wanted to add a little gif to show you the Arduino Bluetooth sensors working with an Avatar, but the website seems limited to 64kb.. I guess I will have to wait to show you when I publish :p

User avatar
paddyg
Posts: 2566
Joined: Sat Jan 28, 2012 11:57 am
Location: UK

Re: Pi3D

Mon Dec 19, 2016 9:50 pm

jeanot, your project sounds amazing and I am really looking forward to seeing it in full. However I'm a bit worried that the process of subtracting the correction rotations and adding back the absolute rotations will still not give the right answer, it's not that I can see a specific problem, just that in my experience the behavior of linear algebra constantly diverges from the expected! But I think that BVH and other motion capture files use a system of euler angle rotations of each bone relative to its parent but (presumably) the raw info is captured as absolute rotations so there must be some standard code to do the conversion.

Is the info you start with a series of vectors in world coordinates representing the direction of each of the bones? You can probably generate a set of euler angles to apply to the child bone relative to the parent using something like the Camera.matrix_from_two_vectors()
also https://groups.google.com/forum/?hl=en-GB&fromgroups=#!forum/pi3d

jeanot1314
Posts: 27
Joined: Mon Feb 24, 2014 2:01 pm

Re: Pi3D

Wed Dec 21, 2016 12:35 am

I probably found another solution. As my sensors give me quaternions coordinates, I compensate the hand sensor before calculate the Eulers... On Free_IMU arduino library it's called "home position" it seems to work, but I had to change a part of my code as the Euler couldn't be calculated in the nodes anymore.

I have a last question before posting tomorrow :

I use "tilt, orientation = CAMERA.point_at()" to get the orientation of the camera. I want to adapt the camera with the head of my avatar.
I am receiving my Joystick data on Serial because it's better for me to don't have a HID joystick.
So I want to use my two Joystick position like this: (it's inspired from vr-zero pi3d code)

if math.fabs(joystick_v_axis_pos) > 0.1:
xm -= math.sin(orientation)*-joystick_v_axis_pos
zm -= math.cos(orientation)*-joystick_v_axis_pos
if math.fabs(joystick_h_axis_pos) > 0.1:
xm -= math.cos(-orientation)*-joystick_v_axis_pos
zm -= math.sin(-orientation)*-joystick_v_axis_pos

But "point_at" seems to send me a very small number (between -0.086 and 0.086) when I move my camera all around the avatar.
I use :

CAMERA.relocate(rot, tilt, point=[xm, ym, zm], distance[15, 15, 15])
CAMERA.rotateZ(roll)

and also

CAMERA = pi3d.Camera(absolute=False)

Any idea why is "point at" so small? I tried to multiply it by 20 to compensate, but its hard to calibrate when you don't understand what's happening :p

Thanks!

jeanot1314
Posts: 27
Joined: Mon Feb 24, 2014 2:01 pm

Re: Pi3D

Wed Dec 21, 2016 7:56 am

Just reading again my post, I forgot to say that it's working if I set the orientation to 0. :lol:

jeanot1314
Posts: 27
Joined: Mon Feb 24, 2014 2:01 pm

Re: Pi3D

Wed Dec 21, 2016 8:23 am

It seems that CAMERA.point_at() is also creating some interference. I have some "blynk" on the screen like if the position was changing for few microseconds.

I retested the joystick this morning, and this code work well with "camera_orientation=0".

if math.fabs(joystick_v_axis_pos) > 0.1:
xm -= math.sin(math.pi/2+orientation)*-joystick_v_axis_pos
zm -= math.cos(-math.pi/2+orientation)*-joystick_v_axis_pos
if math.fabs(joystick_h_axis_pos) > 0.1:
xm -= math.cos(-math.pi/2+orientation)*-joystick_v_axis_pos
zm -= math.sin(math.pi/2+orientation)*-joystick_v_axis_pos

jeanot1314
Posts: 27
Joined: Mon Feb 24, 2014 2:01 pm

Re: Pi3D

Wed Dec 21, 2016 10:47 pm

All right, real pictures and video are here!

I added you to the list of people to thank at the end ;)

Hope you will like the project!

User avatar
paddyg
Posts: 2566
Joined: Sat Jan 28, 2012 11:57 am
Location: UK

Re: Pi3D

Wed Dec 21, 2016 11:13 pm

@jeanot, I will have a look at point_at() it's quite a while since that method was introduced and there might well be bugs that need ironing out.
Camera.get_direction() will return a unit vector in the direction the Camera is pointing (numpy 1D array) and

Code: Select all

v1 = myshape.unif[0:3] - CAMERA.eye
will return a numpy vector from the Camera to the location of myshape (again as numpy 1D array) you can convert to unit vector by

Code: Select all

v1 /= (v1 ** 2).sum() ** 0.5
also https://groups.google.com/forum/?hl=en-GB&fromgroups=#!forum/pi3d

jeanot1314
Posts: 27
Joined: Mon Feb 24, 2014 2:01 pm

Re: Pi3D

Wed Dec 21, 2016 11:43 pm

oups, the link didn't work....

The project is "BeYourHero" on Instructables.com

instructables.com/id/BeYourHero-DIY-VR-Immersive-Games/

Return to “Python”