Inkblot
Posts: 21
Joined: Wed Oct 24, 2018 5:13 pm
Location: UK

Pi3D - Embedding Display in Tkinter Window

Tue Jun 23, 2020 9:04 pm

I'm trying to place a Pi3d Display in a tkinter Frame

Looking over the source code, I can see that the TkWin class inherits from Tk so I could (in theory) use:

Code: Select all

from pi3D import Display
from tkinter import Tk

def config(parent, win):
    win.overrideredirect(True)
    win.transient(parent)

def run():
    disp.loop_running()
    # animation stuff here
    root.after(100, run)

root = Tk()
disp = Display.create(x=0, y=0, w=520, h=372, tk=True)
disp.resize(562, 411, 520, 372)
config(root, disp.tkwin)
run()
root.mainloop()
   
This should remove the titlebar from the pi3D window and group it with the tkinter window, essentially making it behave like a tkinter Toplevel widget, giving the impression that it is embedded in the root window.

However, this class is only used on Android and Raspberry pi (I'm on Ubuntu) and the class used on other systems is one that does not inherit from Tk and hence this solution won't work for it.

I've taken a look at the use_pygame option but it is stated in the source that it does not work for Linux and also means the display must be fullscreen anyways - a restriction I'd prefer not to have.

Is there any other way to achieve the intended solution? I have taken a look at the PyOpenGL library but I (obviously) prefer the abstractions that pi3D offer.

EDIT:
I have come across the display_config option and found the following options:

Code: Select all

DISPLAY_CONFIG_DEFAULT = 0
DISPLAY_CONFIG_NO_RESIZE = 1
DISPLAY_CONFIG_NO_FRAME = 2
DISPLAY_CONFIG_FULLSCREEN = 4
DISPLAY_CONFIG_MAXIMIZED = 8
DISPLAY_CONFIG_HIDE_CURSOR = 16
However, using any of them seems to have no effect.

Inkblot
Posts: 21
Joined: Wed Oct 24, 2018 5:13 pm
Location: UK

Re: Pi3D - Embedding Display in Tkinter Window

Fri Jun 26, 2020 6:28 pm

Just an update on some issues I've run into

I'm trying to read mouse inputs from the Pi3D window but have run into some difficulties.

Assuming a mouse attribute inside in a class:

Code: Select all

def loop(self):
        mx, my = self.mouse.position()
        self.mouse_clicked(self.mouse.button_status(), mx, my)
def mouse_clicked(self, btn, mx, my):
        print(btn)  
        
My system's (Ubuntu 18.04) touchpad has no distinct buttons, i.e. bottom left is lmb, bottom right is rmb and I think because of this, no matter where I click the touchpad it is always interpreted as the left mouse button.

To fix this, I tried to use the InputEvents class but it doesn't seem to work, simply trying to instantiate it, like so:

Code: Select all

self.input_events = InputEvents()
Freezes my laptop to the point where I have to hold the power button until it shuts down completely.

Is there any way I can read mouse inputs with Pi3D on Linux?

User avatar
paddyg
Posts: 2541
Joined: Sat Jan 28, 2012 11:57 am
Location: UK

Re: Pi3D - Embedding Display in Tkinter Window

Tue Jun 30, 2020 10:16 pm

@Inkblot, sorry not to respond before. I do all the development of pi3d on ubuntu 18.04 so it ought to work ok. Embedding the display surface that pi3d (OpenGL) writes to in another GUI toolkit is a bit messy - I've done a couple of examples in pi3d_demos using GTK and Qt (I think the Qt one was more satisfactory but can't remember why) You have to let the toolkit app do the event loop and make it call the Display.loop_running() like in this https://github.com/pi3d/pi3d_demos/blob ... be.py#L100. I've just tried running that demo PyQtCube.py on this laptop and there are a couple of issues: 1. I get an error message ``Failed to load module "canberra-gtk-module"`` which seems odd 2. The drawing surface window that pi3d uses and from where the image is copied isn't being moved to be behind the the x window used by Qt (or just hidden completely). I think the methods for doing that might have been fixed after the demo was made so I will review the code and make it nicer.

On the mouse button question I will also have a look at what my laptop does and if there is some info available from the xserver that pi3d could pass on.

Paddy
also https://groups.google.com/forum/?hl=en-GB&fromgroups=#!forum/pi3d

Inkblot
Posts: 21
Joined: Wed Oct 24, 2018 5:13 pm
Location: UK

Re: Pi3D - Embedding Display in Tkinter Window

Wed Jul 01, 2020 2:11 pm

Hi @paddyg I think you're onto something here with those demos

In the demos, the 'embedding' is just taking a screenshot and pasting it into the GUI toolkit so I tried the same with tkinter:

Code: Select all

from tkinter import Tk, Canvas

import pi3d
from PIL import ImageTk, Image


class TkWin(Tk):
    def __init__(self, title):
        super().__init__(className=title, baseName=title)
        self.DISPLAY = pi3d.Display.create(w=500, h=500, layer=-128)
        self.cube = pi3d.Cuboid(z=2)
        self.display_canvas = None
        self.canvas_image = None
        self.initUI()
        self.pi3d_loop()

    def initUI(self):
        self.rowconfigure(0, weight=1)
        self.columnconfigure(0, weight=1)
        self.display_canvas = Canvas(self, bg='blue')
        self.display_canvas.grid(row=0, column=0, sticky='nesw')
        self.canvas_image = ImageTk.PhotoImage(Image.open('blank.jpg'))
        self.display_canvas.create_image(0, 0, anchor='nw', image=self.canvas_image, tags='image')

    def pi3d_loop(self):
        self.DISPLAY.loop_running()
        self.cube.draw()
        self.cube.rotateIncY(360 / 86400 * 250)
        self.swap_image(pi3d.screenshot())
        self.after(10, self.pi3d_loop)

    def swap_image(self, new_image):
        self.canvas_image = ImageTk.PhotoImage(Image.fromarray(new_image))
        self.display_canvas.itemconfig('image', image=self.canvas_image)


win = TkWin('test')
win.mainloop()
This works perfectly fine except for the actual pi3d window still being there. Performance isn't the greatest but the difference is only noticeable to me because I've run (too) many pi3d simulations these past few days. Blindly slapping the canvas and swap_image function into my actual application means It now uses 20% of my CPU (previously 9-12%), maybe a little optimisation is needed on my part...

Now that I can show the content in a tkinter canvas I will have a go at trying to read mouse inputs on the canvas instead, rather than the pi3d display and post my findings.

Thanks.

User avatar
paddyg
Posts: 2541
Joined: Sat Jan 28, 2012 11:57 am
Location: UK

Re: Pi3D - Embedding Display in Tkinter Window

Wed Jul 01, 2020 3:08 pm

@Inkblot, sounds hopeful, and I would be keen to add the code, once it's fixed, so there's a tk alternative to qt and gtk in pi3d_demos.

I will revisit the issue of hiding the screen as well possibly more efficient ways of doing this later tonight if I get a chance.

Paddy
also https://groups.google.com/forum/?hl=en-GB&fromgroups=#!forum/pi3d

Inkblot
Posts: 21
Joined: Wed Oct 24, 2018 5:13 pm
Location: UK

Re: Pi3D - Embedding Display in Tkinter Window

Wed Jul 01, 2020 3:24 pm

@paddyg Great!

So I've had a little play around and I can confirm the solution does work. I just need to setup the scene and then do all input reading via tkinter and then run the appropriate functions to alter the scene.

Only issue is that the inputs are a little slow but for now it's perfectly fine - I'll see what's causing the delays later.

All that's left now is hiding that display for good!

User avatar
paddyg
Posts: 2541
Joined: Sat Jan 28, 2012 11:57 am
Location: UK

Re: Pi3D - Embedding Display in Tkinter Window

Wed Jul 01, 2020 7:13 pm

Hiding the pi3d window is easy enough now. Use the Display.resize() method to move it off to one side (or off the bottom). I also created the blank canvas from the code rather than having an image the right size.

Code: Select all

from tkinter import Tk, Canvas
import numpy as np
import demo
import pi3d
from PIL import ImageTk, Image

(W, H) = (500, 500)

class TkWin(Tk):
    def __init__(self, title):
        super().__init__(className=title, baseName=title)
        self.display = pi3d.Display.create(w=W, h=H)
        self.cube = pi3d.Cuboid(z=2)
        self.display_canvas = None
        self.canvas_image = None
        self.initUI()
        self.display.resize(x=0, y=self.display.max_height, w=W, h=H) ##<<<<<<<<<<<<
        self.pi3d_loop()

    def initUI(self): # I don't really understand the virtue of splitting __init__ into two functions..
        self.rowconfigure(0, weight=1)
        self.columnconfigure(0, weight=1)
        self.display_canvas = Canvas(self, bg='blue', width=W, height=H) ##<<<<<<<<<<<<<<<<<<
        self.display_canvas.grid(row=0, column=0, sticky='nesw')
        self.canvas_image = ImageTk.PhotoImage(Image.fromarray(np.zeros((W,H,3), dtype=np.uint8))) ##<<<<<<<<<<<<<<<
        self.display_canvas.create_image(0, 0, anchor='nw', image=self.canvas_image, tags='image')
...
My poor laptop runs at 50% of one CPU with this and up to 65% if I increase the pi3d window to 750x750 so saving and copying the pixels in this way is quite an overhead. I will have another look at getting hold of a drawing surface from tk and rendering directly into that - so long ago now I can't remember what the problems were exactly.

Paddy

PS pyOpenGL seems to use GLX so maybe I should look at that. I did a bit of work with that to allow transparent window backgrounds not long ago so I might have some code in there that I can tweak!
Last edited by paddyg on Wed Jul 01, 2020 8:46 pm, edited 1 time in total.
also https://groups.google.com/forum/?hl=en-GB&fromgroups=#!forum/pi3d

Inkblot
Posts: 21
Joined: Wed Oct 24, 2018 5:13 pm
Location: UK

Re: Pi3D - Embedding Display in Tkinter Window

Wed Jul 01, 2020 8:46 pm

Thanks again paddy, I'll be here waiting for a response ;) in the meantime, I'll see if I can optimise my end a bit to reduce cpu usage

Inkblot
Posts: 21
Joined: Wed Oct 24, 2018 5:13 pm
Location: UK

Re: Pi3D - Embedding Display in Tkinter Window

Mon Jul 27, 2020 6:51 pm

*Bump*

Just checking in ... but I should also add that calling DISPLAY.destroy(), DISPLAY.stop(), setting DISPLAY.loop_running to False all do not actually close the pi3d window. I would like to close the window but keep the program alive if possible

User avatar
paddyg
Posts: 2541
Joined: Sat Jan 28, 2012 11:57 am
Location: UK

Re: Pi3D - Embedding Display in Tkinter Window

Mon Jul 27, 2020 6:57 pm

Sorry @Inkblot, diverted onto other things but hopefully get back to the tk display surface eventually. But I will checkout why you can't kill off the pi3d display - that ought be be possible.

Paddy
also https://groups.google.com/forum/?hl=en-GB&fromgroups=#!forum/pi3d

User avatar
paddyg
Posts: 2541
Joined: Sat Jan 28, 2012 11:57 am
Location: UK

Re: Pi3D - Embedding Display in Tkinter Window

Tue Jul 28, 2020 9:57 am

@Inkblot, that took a bit of digging out. The opengles function calls (via ctypes) fail without trace so the only way to pin them down is to put in print statements every other line! And there were several issues. see https://github.com/tipam/pi3d/commit/a6 ... d33f337ea4 It's only in the develop branch at the moment so if you want to try it you will have to `git clone https://github.com/tipam/pi3d` then, in the pi3d directory `git checkout develop` and before you import pi3d in your script `import demo` which simply puts the local git repository in the path (i.e. `import sys` `sys.path.insert(1, '/home/pi/pi3d')`)

The last image rendered by pi3d obviously stays on the tk canvas. Also, more importantly, this seems to be a one way process. After destroying the Display the creation of a new one seems to be impeded - just producing a blank X window with no graphical drawing surface. Need to look further into that.

Paddy
also https://groups.google.com/forum/?hl=en-GB&fromgroups=#!forum/pi3d

Inkblot
Posts: 21
Joined: Wed Oct 24, 2018 5:13 pm
Location: UK

Re: Pi3D - Embedding Display in Tkinter Window

Tue Jul 28, 2020 9:19 pm

Hi paddy,

Just cloned the repository and still the same issue ... I'm just not sure why the window won't close whether it be programmatically or using the 'x' button

The tkinter canvas I can simply close by calling destroy() and it removes the canvas

Perhaps there is a way to get the pi3d process pi3d and then forcefully kill it using a module like psutil?

Calling the destroy method of the display resets the instance (so it should be able to reopened) so then killing the pi3d process alone should then close the window (just spitballing here). That's if there exists a way of actually accessing the pi3d process pi3d.

User avatar
paddyg
Posts: 2541
Joined: Sat Jan 28, 2012 11:57 am
Location: UK

Re: Pi3D - Embedding Display in Tkinter Window

Wed Jul 29, 2020 7:36 am

@Inkblot, It's possible to kill the window (not completely trivial if you look at the code https://github.com/tipam/pi3d/blob/deve ... GL.py#L212 you will see that the X11 controls are a bit unweildly called from python) However the code wasn't getting to the critical

Code: Select all

        xlib.XCloseDisplay(self.d)
because of errors in the code trying to tidy up the GPU buffer, texture, shader and program objects. Those errors should have been fixed in the revised code, certainly here (running ubuntu laptop) I can close the pi3d window fine either with display.destroy() or clicking the cross. Just to check where your code is getting to could you edit the pi3d/pi3d/util/DisplayOpenGL.py code lines 360 to 417 and put in a few print("here01") statements to ensure that the code is running. (You did run `git checkout develop` in the local pi3d repository?)

Paddy
also https://groups.google.com/forum/?hl=en-GB&fromgroups=#!forum/pi3d

Inkblot
Posts: 21
Joined: Wed Oct 24, 2018 5:13 pm
Location: UK

Re: Pi3D - Embedding Display in Tkinter Window

Wed Jul 29, 2020 1:35 pm

I started to add print statements and none showed up and so decided to put one as the first line of the destroy method - it wasn't printed either.

I looked around the code and saw a lot of x11 talk and literally just remembered that my Ubuntu version (18.04) uses gdm instead of x11 - could that possibly be the reason I'm facing this issue? (more spitballing)

Running this command in the terminal:

Code: Select all

grep '/usr/s\?bin' /etc/systemd/system/display-manager.service
Gives the output

Code: Select all

ExecStart=/usr/sbin/gdm3


Telling me I'm using gdm as my display manager. I did try using x11 (installing Xorg) a while back but if I remember correctly, it nearly made my laptop unusable :lol:

User avatar
paddyg
Posts: 2541
Joined: Sat Jan 28, 2012 11:57 am
Location: UK

Re: Pi3D - Embedding Display in Tkinter Window

Wed Jul 29, 2020 2:12 pm

It seems odd that *anything* works if x11 isn't running the windows etc. Also lines 361 to 405 in destroy() are common to all platforms so not likely to be x11 v Wayland issue. It would be nice if pi3d just worked under Wayland but I'm not holding my breath... What do you get if
you

Code: Select all

$ loginctl
then take the session number under SESSION - in my case "c2" and run
$ loginctl show-session c2 -p Type
I get
Type=x11
The most likely thing is that you are not loading the pi3d from the downloaded repository. If you are not running on a Raspberry Pi then the code in pi3d_demos/demo.py will be wrong, you need to change it or just put the relevant code at the top of your test script before the import pi3d line. For me I have:

Code: Select all

import sys
sys.path.insert(1, "/home/patrick/python/pi3d")
import pi3d
Are you trying to close the window by clicking on the cross or by calling display.destroy() from the python code? Either works OK for me with the patched DisplayOpenGL code.
also https://groups.google.com/forum/?hl=en-GB&fromgroups=#!forum/pi3d

Inkblot
Posts: 21
Joined: Wed Oct 24, 2018 5:13 pm
Location: UK

Re: Pi3D - Embedding Display in Tkinter Window

Wed Jul 29, 2020 4:23 pm

Output from the terminal command is 'wayland' (I have tried both using the destroy method and clicking the x button)

I've tried to clone the repository and checkout the develop branch again (git clone ... then git checkout develop) and my code is:

Code: Select all

import sys
sys.path.insert(1, '/home/user/Desktop/pi3d_new')
import pi3d_new.pi3d as pi

s = pi.Sphere()
     
DISPLAY = pi.Display.create(w=500, h=500)
while DISPLAY.loop_running():
        s.draw()
Then gives an attribute error:

Code: Select all

Traceback (most recent call last):
  File "/home/user/Desktop/pi3d test.py", line 11, in <module>
    s.draw()
  File "/home/user/Desktop/pi3d_new/pi3d/Shape.py", line 208, in draw
    b.draw(self, self.M, self.unif, shader, txtrs, ntl, shny)
  File "/home/user/Desktop/pi3d_new/pi3d/Buffer.py", line 273, in draw
    self.load_opengl()
  File "/home/user/Desktop/pi3d_new/pi3d/util/Loadable.py", line 42, in load_opengl
    self._load_opengl()
  File "/home/user/Desktop/pi3d_new/pi3d/Buffer.py", line 184, in _load_opengl
    self.disp.vbufs_dict[str(self.vbuf)] = [self.vbuf, 0]
AttributeError: 'NoneType' object has no attribute 'vbufs_dict'
As a side note, I had to import the module under a different name ('pi') otherwise it would use the previously installed one

Printing out the value of DISPLAY shows that it is not None but a Display object. However, trying to print out the value of self.disp at the end of the constructor of the Buffer class returns None.

Now, trying to print Display.INSTANCE in the constructor of the Display class shows that it has been set. However, leaving both print statements in gives:

Code: Select all

>>> None
>>> <pi3d.Display.Display object at 0x7f4ce56b8fd0>
Which immediately suggests to me that the drawing of the sphere is happening before the display is created which I'm not sure why

EDIT
I should add that checking for an instance in Buffer()._load_opengl() has fixed the issue, the first few lines of the function now read:

Code: Select all

  
  def _load_opengl(self):
    self.vbuf = GLuint()
    opengles.glGenBuffers(GLsizei(1), ctypes.byref(self.vbuf))
    self.ebuf = GLuint()
    opengles.glGenBuffers(GLsizei(1), ctypes.byref(self.ebuf))
    from pi3d.Display import Display  # new
    self.disp = Display.INSTANCE  # check one last time for a display instance before accessing attrs
    self.disp.vbufs_dict[str(self.vbuf)] = [self.vbuf, 0]
And now, I am finally able to close the window using either the destroy method or the x button :D

I see in the Display class you have a todo for making fullscreen tkinter windows, if you still need that, it can be done with:

Code: Select all

window.attributes("-fullscreen", flag)  # with window being the Tk object and flag being True or False

User avatar
paddyg
Posts: 2541
Joined: Sat Jan 28, 2012 11:57 am
Location: UK

Re: Pi3D - Embedding Display in Tkinter Window

Wed Jul 29, 2020 10:09 pm

You don't need to worry about using the default centrally installed pi3d, adding the sys.path.insert before import pi3d works fine (That's why it's inserted at position 1 so it comes before anything else that might have pi3d in it too (the first entry in the list has to be left for reasons I can't recall)). If that wasn't the case it would be a monumental job for me making demos and developing pi3d! Try changing it to

Code: Select all

import sys
sys.path.insert(1, '/home/user/Desktop/pi3d_new')
import pi3d
and check if it works, ideally without the other changes you needed to make, you were using pi3d before OK so it would be interesting to know what had caused the new problem, if it was something else in the code from the git clone.

Anyway it sounds like progress (though have you managed to restart pi3d after destroying the display? I still can't). Thanks for the full screen pointer - I really need to find a way of drawing directly to a tk surface.

It's also good, though slightly bizarre that all the X11 commands work with Wayland. Apparently it has an XWayland layer that allow X11 apps to mainly work transparently! Presumably there is slight processing cost.

Paddy
also https://groups.google.com/forum/?hl=en-GB&fromgroups=#!forum/pi3d

Inkblot
Posts: 21
Joined: Wed Oct 24, 2018 5:13 pm
Location: UK

Re: Pi3D - Embedding Display in Tkinter Window

Thu Jul 30, 2020 11:38 am

Wow this is really embarrassing ... :lol: throughout all that testing yesterday I didn't notice that I was creating the sphere before the display ... silly me.

I can now confirm that after removing my changes I can still close the display properly. However, just like before trying to reopen a display gives a blank screen. I tested this by putting the display setup and drawing in the Orbit.py demo into a function called draw() and then put at the bottom of the script:

Code: Select all

for i in range(2):
    draw()
On the second iteration, I get a blank black screen (maybe something to do with the camera?). I've with and without setting up a camera instance and also only creating a single camera instance and they all produce the same effect.

I also had to comment out starting the mouse because I get the following error:

Code: Select all

Traceback (most recent call last):
  File "/home/user/Desktop/pi3d_test.py", line 188, in <module>
    draw()
  File "/home/user/Desktop/pi3d_test.py", line 139, in draw
    mymouse.start()
  File "/home/user/Desktop/pi3d_new/pi3d/Mouse.py", line 88, in start
    super(_nixMouse, self).start()
RuntimeError: threads can only be started once
Despite the fact that at the end of the new draw() function (outside the drawing loop) I have:

Code: Select all

    mykeys.close()
    mymouse.stop()
    DISPLAY.destroy()
Upon inspection into the Mouse class (specifically _nixMouse) I can see the stop method is just:

Code: Select all

  def stop(self):
    self.running = False
I'm going to assume the intended mouse usage is only instantiated one instance of it? Given that the Mouse function is

Code: Select all

def Mouse(*args, **kwds):
  if pi3d.USE_PYGAME:
    if not _pygameMouse.INSTANCE:
      _pygameMouse.INSTANCE = _pygameMouse(*args, **kwds)
    return _pygameMouse.INSTANCE
  else:
    if not _nixMouse.INSTANCE:
      _nixMouse.INSTANCE = _nixMouse(*args, **kwds)
    return _nixMouse.INSTANCE
It only ever instantiates once...

User avatar
paddyg
Posts: 2541
Joined: Sat Jan 28, 2012 11:57 am
Location: UK

Re: Pi3D - Embedding Display in Tkinter Window

Thu Jul 30, 2020 12:12 pm

;) but you are right there probably ought to be an explicit warning if code looks for an instance of Display and there isn't one! Like "you are running this before a Display has been created, are you sure?" Hopefully the issue trying to re-run pi3d within the same python process is relatively simple, a matter of deleting or not deleting something critical, as you noticed, an awful lot of the X11 stuff isn't tidied up on closed down, basically only XCloseDisplay(). Just that I've never tried to do it before.
also https://groups.google.com/forum/?hl=en-GB&fromgroups=#!forum/pi3d

Inkblot
Posts: 21
Joined: Wed Oct 24, 2018 5:13 pm
Location: UK

Re: Pi3D - Embedding Display in Tkinter Window

Fri Jul 31, 2020 8:05 pm

I think a warning and then the attribute error is a bit overkill, how about a custom exception/assertion that signifies the warning? Similar to the single instance display assertion.

Return to “Graphics programming”