User avatar
paddyg
Posts: 2337
Joined: Sat Jan 28, 2012 11:57 am
Location: UK

Re: Pi3D

Thu Mar 27, 2014 9:26 pm

Jim

Don't worry about the deleting messages; they happen because the Buffer and Texture classes try to keep track of the graphics objects (buffers) they create, they have __del__ methods but 'belt and braces' style there is also tidying up exercise done by the Display as it closes down. Sometimes the __del__ methods manages to delete the buffers (they are notoriously non-specific as destructors go) but doesn't manage to tell the Display object before it closes down and also tries to delete them too.

The error message looks to happen because the concatenated 'path' + mtllib this uses the supposedly clever os.path.join but it might be being caught out. The error message looks like the string parsed out as mtllib is "" but I'm not sure why. Could you post the obj and the mtl files that give this problem?
also https://groups.google.com/forum/?hl=en-GB&fromgroups=#!forum/pi3d

User avatar
Jim Manley
Posts: 1600
Joined: Thu Feb 23, 2012 8:41 pm
Location: SillyCon Valley, California, and Powell, Wyoming, USA, plus The Universe
Contact: Website

Re: Pi3D

Thu Mar 27, 2014 9:59 pm

Hi Paddy,

I kinda thought it might be a path thing. Is it possible to specify the absolute path until this gets resolved (e.g., if the path doesn't begin with a dot)? The small .objs (cube, diamond, pyramid, tetrahedron, etc.) don't have .mtl files - are they required? The large .objs are, well, large and some of each are stashed at [url]http://raspberry_office.byethost11.com/Objects[/url] which will list them in a browser.

I should note that some objects are identified in the .obj files with a leading "g" before the name, while others are identified with a leading "o".

I also realized that at least some of the files were written on a DOS/Windoze box somewhere along the line and so have DOS line endings, not Unix style, if that makes a difference.

Thanks for the most rapid/rabid reply! :lol:
Jim
The best things in life aren't things ... but, a Pi comes pretty darned close! :D
"Education is not the filling of a pail, but the lighting of a fire." -- W.B. Yeats
In theory, theory & practice are the same - in practice, they aren't!!!

User avatar
Jim Manley
Posts: 1600
Joined: Thu Feb 23, 2012 8:41 pm
Location: SillyCon Valley, California, and Powell, Wyoming, USA, plus The Universe
Contact: Website

Re: Pi3D

Thu Mar 27, 2014 10:22 pm

OK, so an absolute path works with a .obj file that refers to a .mtl file, although nothing is displayed and the CPU meter is around 50% load. So, "works" means no errors and the OGLES layer is black, possibly because of some issue with the Ka, Kd, Ks, and Tr values.
The best things in life aren't things ... but, a Pi comes pretty darned close! :D
"Education is not the filling of a pail, but the lighting of a fire." -- W.B. Yeats
In theory, theory & practice are the same - in practice, they aren't!!!

User avatar
Jim Manley
Posts: 1600
Joined: Thu Feb 23, 2012 8:41 pm
Location: SillyCon Valley, California, and Powell, Wyoming, USA, plus The Universe
Contact: Website

Re: Pi3D

Thu Mar 27, 2014 11:18 pm

It definitely looks like it's assumed that there will be a .mtl file associated with each .obj file, as any .obj file that doesn't have an mtllib line causes an error as parse_mtl() is called whether there is an mtllib line or not.
The best things in life aren't things ... but, a Pi comes pretty darned close! :D
"Education is not the filling of a pail, but the lighting of a fire." -- W.B. Yeats
In theory, theory & practice are the same - in practice, they aren't!!!

User avatar
paddyg
Posts: 2337
Joined: Sat Jan 28, 2012 11:57 am
Location: UK

Re: Pi3D

Fri Mar 28, 2014 12:12 am

Jim, I've played around with this a little bit:

A while ago I changed pi3d to cope with obj files that didn't have texture mapping (vt lines) or normals (vn lines) however I assumed that some kind of surface would need to be defined. Either it's going to be rgb or a mapped image texture. I suppose I could put something in that makes it default to raspberry colour or something? Let me know if you think that would be a good idea.

The pyramid.obj also seems to gratuitously start it's vertex numbering at 2. I can't think what the virtue is in doing this, it means the whole of the faces (f lines) would have to be parsed to find what the max and min vertex references are! I changed it to this and it works ok.

Code: Select all

# OBJ file created by ply_to_obj.c
#
mtllib skull.mtl #added this line
g Object001
v  0  0  0 #added this line
v  0  0  0
v  1  0  0
v  1  1  0
v  0  1  0
v  0.5  0.5  1.6
usemtl Dflt0 #added this line
f  5  2  3
f  4  5  3
f  6  3  2
f  5  6  2
f  4  6  5
f  6  4  3
However I did struggle for a while getting the models to render because I forgot to change the shader to 'mat_light'. All the uv_ shaders and mat_bump and mat_reflect need to have texture mapping in the obj file

Generally speaking it's much better to get blender (or whatever) to calculate the normals as it will be much better and quicker than pi3d at doing it and you can control the smoothness or angularity of edges. It only needs to be done once rather than every time the model is loaded.

Paddy

ps Skull is really excellent, I might utilise that elsewhere!
Screenshot27mar14.jpg
Screenshot27mar14.jpg (33.43 KiB) Viewed 4643 times
pps I didn't need to change the file paths or anything, they were fine as they were (just need to have a material file), though I am running on ubuntu laptop not RPi, can't see that making a difference.
also https://groups.google.com/forum/?hl=en-GB&fromgroups=#!forum/pi3d

User avatar
Jim Manley
Posts: 1600
Joined: Thu Feb 23, 2012 8:41 pm
Location: SillyCon Valley, California, and Powell, Wyoming, USA, plus The Universe
Contact: Website

Re: Pi3D

Fri Mar 28, 2014 4:30 am

Hi Paddy,

Well, I figured some assumption(s) had been made that weren't intuitively obvious to this uncomfortable observer, and I just needed to ask the stupid questions that others might never get around to asking. I was perplexed that Blender and Wavefront were able to open and render a random sampling of the files, including that boffo skull, which is one of the key things I need to render via Pi3D. I was also mystified by the ersatz pyramid.obj face numbering - it wasn't my idea, I was just downloading stuff to try out and that seemed like the simplest case to get working first.

I saw Davide's and your entry in the Google DevArt competition just yesterday, and if you hadn't already seen, I also entered the competition, but just yesterday. I somehow need to get my prototype code working before 6 PM GMT Friday (T-minus 13 hours, 17 minutes, and 14 seconds, but who's counting? :lol: ). I figure I've got about a 1% chance of winning since I only became aware of the competition last weekend, but I did get to spend Saturday in the Google Garage ... ohhhh, mannnnn, do I ever now have a shopping list if I ever get a cost-is-no-object grant to fund building a real STEM lab! It's where Googlers can go to work on their 20% projects, particularly those needing physical work with tools, although, in addition to top-end 3-D printers and several workbenches outfitted with the best hand tools and hand-held power tools available on the planet, there are plenty of top-end desktop systems with large displays to take care of the software end of projects. I'm using Pi3D on the Pi to implement my prototype and would use Pii extensively in the installation in the unlikely event that anyone would be crazy enough to declare me a digital artist.

Thanks and good luck on your entry which actually has much artistic merit and effort behind it, as opposed to my too-little-too-late, laughable, feeble attempt.

All the Best,
Jim
The best things in life aren't things ... but, a Pi comes pretty darned close! :D
"Education is not the filling of a pail, but the lighting of a fire." -- W.B. Yeats
In theory, theory & practice are the same - in practice, they aren't!!!

User avatar
Jim Manley
Posts: 1600
Joined: Thu Feb 23, 2012 8:41 pm
Location: SillyCon Valley, California, and Powell, Wyoming, USA, plus The Universe
Contact: Website

Re: Pi3D

Fri Mar 28, 2014 6:21 am

Hi Paddy,

OK, per your wise and all-knowing realization that it really helps to use the right shader (DUH!) and to have enough vertices for the faces to refer to the correct number of indices (Double-DUH!!), as well as including a .mtl file and referring to a material contained therein (Triple-DUH!!!), pyramid.obj showed up, spinnin' like a top! Using the right shader (Quadruple-DUH!!!!) with skull.obj resulted in a very nice brother of Mr. Crossbones also making his spinnin'/grinnin' debut in gleaming bone white!

Thanks very much for the troubleshooting. I seem to recall that someone(s) is(are) writing a book on Pi3D, but I now feel a little more qualified to remember my old, musty 3-D principles enough to at least write some footnotes for such a publication. I'm really good at doing stupid things that no expert would ever attempt, but naive users would do immediately. Much more footnote-worthy material to come! :lol:

All the Best,
Jim
The best things in life aren't things ... but, a Pi comes pretty darned close! :D
"Education is not the filling of a pail, but the lighting of a fire." -- W.B. Yeats
In theory, theory & practice are the same - in practice, they aren't!!!

User avatar
Jim Manley
Posts: 1600
Joined: Thu Feb 23, 2012 8:41 pm
Location: SillyCon Valley, California, and Powell, Wyoming, USA, plus The Universe
Contact: Website

Re: Pi3D

Fri Mar 28, 2014 8:36 pm

Everyone, say hello to mah li'l fren! :lol:

Image
The best things in life aren't things ... but, a Pi comes pretty darned close! :D
"Education is not the filling of a pail, but the lighting of a fire." -- W.B. Yeats
In theory, theory & practice are the same - in practice, they aren't!!!

User avatar
paddyg
Posts: 2337
Joined: Sat Jan 28, 2012 11:57 am
Location: UK

Re: Pi3D

Sat Mar 29, 2014 10:12 pm

Couple of minor fixes to the model loading (pushed to github.com/tipam/pi3d=>develop branch). Now copes with most of the models in Jim's list, even where no material specified (but not made it cope with re-zeroing vertex index). As you can just about see the normals generated by pi3d are unsmoothed and much better if added with blender.
scrcap.jpg
scrcap.jpg (24.05 KiB) Viewed 4508 times
scrcap2.jpg
scrcap2.jpg (22.77 KiB) Viewed 4508 times
also https://groups.google.com/forum/?hl=en-GB&fromgroups=#!forum/pi3d

User avatar
Jim Manley
Posts: 1600
Joined: Thu Feb 23, 2012 8:41 pm
Location: SillyCon Valley, California, and Powell, Wyoming, USA, plus The Universe
Contact: Website

Re: Pi3D

Mon Mar 31, 2014 5:34 pm

Out-freaking-standing, Paddy! Now I'm glad that I've been stumbling around finding such corner cases (although a corner large enough to park an airplane in, in this case ;) ), although I apologize for the extra work this has meant for you. One of these days, I'll have time to actually make such fixes myself, but until then, I'll be pushing the existing code to see what other wrinkles need to be worked out.

I hope to have the distant spherical starfield for Pi-finity! (the stars for which parallax doesn't provide distance) texture-mapped within a sphere as soon as I get a chance today, and with any luck by next weekend, we might have the stars for which parallax can show distance represented as Points. The databases containing locating data for these provide right ascension (angle above the horizon, for readers of this not familiar), declination (angle from the North Pole), and distance in parsecs (parallax in arcseconds of angle). For some reason, there doesn't seem to be a Pi3D API for plotting a Point using those units (yet 8-) ).

Thanks again, and keep up the great work!
The best things in life aren't things ... but, a Pi comes pretty darned close! :D
"Education is not the filling of a pail, but the lighting of a fire." -- W.B. Yeats
In theory, theory & practice are the same - in practice, they aren't!!!

User avatar
paddyg
Posts: 2337
Joined: Sat Jan 28, 2012 11:57 am
Location: UK

Re: Pi3D

Mon Mar 31, 2014 7:06 pm

Jim, I don't know how many 'nearby' stars are in your catalogue (almost certainly more than the Pi gpu can cope with) but you can do calculations on big arrays in the twinkle (!) of an eye using numpy. Things like:

Code: Select all

xyz = np.zeros(rad.shape) #x,y,z radius,ascension,declination
xyz[:,1] = np.sin(rad[:,2])
xyz[:,2] = rad[:,0] * np.cos(rad[:,2]) # r in y=0 plane
xyz[:,0] = xyz[:,2] * np.sin(rad[:,1])
xyz[:,2] = xyz[:,2] * np.cos(rad[:,1])
also https://groups.google.com/forum/?hl=en-GB&fromgroups=#!forum/pi3d

simon8383
Posts: 3
Joined: Thu May 22, 2014 5:26 pm

Re: Pi3D

Thu May 22, 2014 5:41 pm

I hope that my issue fits to this topic.

I am trying to program a diashow with pi3d. Displaying the photos and make some smooth "cover-flow effects" works fine so far. But when loading the textures the GPU memory seems to be full after a while - and I am not able to load new textures anymore (the textures are just black...). I did not manage to get any errors which could trigger the unloading of "older" textures. At the moment I could only use a rule of thumb ("when n textures are loaded, unload the oldest one..."). I would have to identify this n manually (which might depend on various factors...)

My questions are:
- Is there a possiblity to get an information about the currently used (or free) GPU memory via python?
- Is there a solution to which automatically removes unused textures when the GPU memory is full?
- How many textures fit in the GPU memory? (At the moment the issue already appears at the fifth texture (!), size 800 x 800, gpu mem 128)

Thank you for any hints or ideas. Maybe I just missed something very basic?

User avatar
paddyg
Posts: 2337
Joined: Sat Jan 28, 2012 11:57 am
Location: UK

Re: Pi3D

Thu May 22, 2014 6:51 pm

I'm surprised you've run out of memory with only 5 images at 800x800, as a rough and ready estimate, for RGBA textures you will need 4 bytes per pixel which works out at 2.4MB so it does seem odd. Are you basing your project on one of the Slideshow demos? I think they use a rotating 'carousel' of 8 slides and I've run them on the RPi with full screen pictures.

Texture class does keep a note of the width and height of the image (Texture.ix and Texture.iy) so you could use those to keep an estimate of the space used.

I think there is a limit of around 5 or 6 textures for the gpu to use simultaneously. These represent lookup samplers all passed to the shaders to use at the same time. In pi3d I've only got round to using three: ambient colour, normal map and reflection. So unless you have rolled your own shader it's probably not that!

Is it possible to see your code (on github or could you post it here)?
also https://groups.google.com/forum/?hl=en-GB&fromgroups=#!forum/pi3d

simon8383
Posts: 3
Joined: Thu May 22, 2014 5:26 pm

Re: Pi3D

Thu May 22, 2014 7:15 pm

My View does something like to following during init (some lines are left out because they are not important for this issue):

Code: Select all

        
def __init__(self, controller, model):  
   self.controller = controller
   self.model = model
   self.display = pi3d.Display.create(background = (255, 255, 255, 1))      

   stringCamera = pi3d.Camera(is_3d = False)
        
   self.shader = pi3d.Shader("uv_flat")        
   self.image = pi3d.Sprite(w = 800, h = 800, camera = stringCamera) 
   self.image.set_draw_details(self.shader, [self.model.getImage().getTexture()])
During the update loop the position of the image is updated and the Texture is updated as well (the controller listens for keyboard inputs and changes the model accordingly - i'm aware that this is not the perfect way to solve this, but it should be ok for a first proof of concept - later i will implement an observer, which informs the view about changes of the model)

Code: Select all

        
def update(self):
    self.image.position(self.x, self.y, self.image.z())
    self.image.set_draw_details(self.shader, [self.model.getImage().getTexture()])
    self.image.draw()
The model contains a list with all image objects and holds a reference on the current image. A method of my own image class returns the texture (this method loads the texture if it is not loaded yet):

Code: Select all

    
def getTexture(self):
if self.__texture__ == None:
    self.__texture__ = pi3d.Texture(self.__getPreviewPath__(), blend=True, mipmap=False)
return self.__texture__
Maybe simultaneously is already the right keyword here? However, if I understand the concept right only one Texture is used by my self.shader simultaneously - self.image.set_draw_details should replace the old texture, shouldn't it?

User avatar
paddyg
Posts: 2337
Joined: Sat Jan 28, 2012 11:57 am
Location: UK

Re: Pi3D

Thu May 22, 2014 8:10 pm

Yes you are only using one Texture at once. I can't see anything definite but I'm guessing a bit as to what methods belong to what class etc. I would say, however, that I don't think you should be resetting the shader and texture every loop. If you want to draw different images each loop then it would be much more efficient to have different Sprite (image) objects each with their own Texture and Shader rather than swapping them over several times each loop. You only need to call set_draw_details if the Shader or Texture for that Sprite have changed.

But I can't really see why that would cause an issue. I would suspect some kind of leakage or circular reference here's a handy bit of code to help quantify and pin down memory leaks. If that doesn't show anything then it points to something happening in the graphics memory.
also https://groups.google.com/forum/?hl=en-GB&fromgroups=#!forum/pi3d

simon8383
Posts: 3
Joined: Thu May 22, 2014 5:26 pm

Re: Pi3D

Thu May 22, 2014 8:43 pm

Thank you for your help so far.

I will modify / simplify my code according to your tips and see if this will help.

npsy
Posts: 4
Joined: Wed Dec 30, 2015 8:18 pm

Re: Pi3D

Wed Dec 30, 2015 10:51 pm

Hello,

is it possible to get the texture of a PostProcess object as a numpy array? I tried it with this code, but failed. Unfortunately I can't use screenshots in this application. Thanks for your help!

post.start_capture()##<<<<<<<<<<<<<<
plane.draw(flatsh, [texture])
post.end_capture()##>>>>>>>>>>>>>>


#result=np.array(post.im.getdata()).reshape((post.im.size[0], post.im.size[1], 4))
result=post.image
cv2.imwrite("filterResult.png", result[:, :, :3])

User avatar
paddyg
Posts: 2337
Joined: Sat Jan 28, 2012 11:57 am
Location: UK

Re: Pi3D

Sat Jan 02, 2016 12:33 am

@npsy, Sorry, I've just seen this question. I will have a look at this as there could be a few applications this would help. It's certainly possible to load OffScreenTextures into ctypes arrays as done in

Code: Select all

here in Clashtest
but there's probably a way of using numpy.ctypes to do the process in one step, as I switched to in Texture.
also https://groups.google.com/forum/?hl=en-GB&fromgroups=#!forum/pi3d

User avatar
paddyg
Posts: 2337
Joined: Sat Jan 28, 2012 11:57 am
Location: UK

Re: Pi3D

Wed Jan 06, 2016 2:05 pm

Sorry to take so long to get back on this question. As a by-product I have fixed a long standing bug in the Clashtest and Screenshot classes (when running on 'big' computers, neither worked). I will post up an example of how to pipe the screen capture to a numpy array using glReadPixels.

I'm not sure if it's worth using the OffScreenTexture system as I think the only way to get the pixel into into CPU space is using glReadPixels as is used in Screenshot. (The only reason would be if you wanted to render different things to a numpy array from whatever showed on the screen)

Paddy
also https://groups.google.com/forum/?hl=en-GB&fromgroups=#!forum/pi3d

User avatar
paddyg
Posts: 2337
Joined: Sat Jan 28, 2012 11:57 am
Location: UK

Re: Pi3D

Wed Jan 06, 2016 5:30 pm

OK this is the way to get the framebuffer into a numpy array:

Code: Select all

import numpy as np
import ctypes
...
ntex = np.zeros((post.iy, post.ix, 4), dtype=np.uint8) # make an empty array of the correct size
...
  # inside the drawing loop. If you are offscreen rendering then you need to do this
  #  BEFORE you switch back to the normal view with end_capture
  post.start_capture() #<<<<<<<<<<<<<<
  plane.draw(flatsh, [texture])
  pi3d.opengles.glReadPixels(0, 0, post.ix, post.iy, pi3d.GL_RGBA, pi3d.GL_UNSIGNED_BYTE, 
                                    ntex.ctypes.data_as(ctypes.POINTER(ctypes.c_short)))
  # ntex is now a numpy array that can be used with opencv or whatever
  post.end_capture() #>>>>>>>>>>>>>>
HOWEVER I have to ask why? You look to be just rendering a flat texture, couldn't you simply put that into a numpy array - in fact it already is a numpy array so you could

Code: Select all

cv2.imwrite("filterResult.png", texture.image[:,:,:3])
Maybe there's some more code missing from your post that would explain what you are trying to do.
also https://groups.google.com/forum/?hl=en-GB&fromgroups=#!forum/pi3d

npsy
Posts: 4
Joined: Wed Dec 30, 2015 8:18 pm

Re: Pi3D

Wed Feb 10, 2016 11:21 pm

Sorry about my extremely late reply. Thanks for looking into this. I'll try your solution asap.
HOWEVER I have to ask why? You look to be just rendering a flat texture, couldn't you simply put that into a numpy array - in fact it already is a numpy array so you could [...]
The idea is to apply a shader to the rendered texture. Also a homography transform can be calculated on the GPU by changing the vertices of the textured rectangle. Which leads me to the next question: Can Pi3D be used without X or framebuffer?

User avatar
paddyg
Posts: 2337
Joined: Sat Jan 28, 2012 11:57 am
Location: UK

Re: Pi3D

Thu Feb 11, 2016 8:43 am

Yes, pi3d can be used without the X11 server. On the Raspberry Pi it gets the render surface directly using the bcm_host shared library.

I'm also intrigued by what you're trying to do, and a bit more info might save you some trouble. With GPU shaders you don't need to render a texture first in order to apply a transformation matrix to it - you can 'simply' pass the texture to the GPU. It's only if you want to generate a scene using miscellaneous 3D objects and apply a transformation to the resultant 2D image that you would need a two stage process. BUT for simple transformations, that can be defined by matrix operation or small amounts of trigonometry, it is MUCH faster (even than numpy) to do this inside the GPU using special shaders - this is what is being done in the FilterDemo.py. For processing that requires the whole texture to be analysed, such as fft or finding the average colour value, then extracting the render buffer to a numpy array as described above would be the best way.

Paddy
also https://groups.google.com/forum/?hl=en-GB&fromgroups=#!forum/pi3d

npsy
Posts: 4
Joined: Wed Dec 30, 2015 8:18 pm

Re: Pi3D

Sun Feb 21, 2016 10:48 pm

Thanks paddyg! Now it works perfectly both on the PC and on the raspi.
I'm currently using the Conway.py demo with a custom shader for things like median filter and gaussian blur.

User avatar
paddyg
Posts: 2337
Joined: Sat Jan 28, 2012 11:57 am
Location: UK

Re: Pi3D

Mon Feb 22, 2016 7:34 am

Ah, that sounds really interesting. It's a pity that the shader language is rather inflexible in ES2.0 it would be really handy to have variable ranges for convolutions etc. is the median taken over a block or between frames?
also https://groups.google.com/forum/?hl=en-GB&fromgroups=#!forum/pi3d

npsy
Posts: 4
Joined: Wed Dec 30, 2015 8:18 pm

Re: Pi3D

Mon Feb 22, 2016 10:13 pm

The median is calculated for the local 5x5 region as a filter. Implementing non-linear filters on this GPU is rather unpleasant. ;)

Return to “Python”