Has anyone put together drivers for a VR output for any flavor of the Raspberry pi OSes?
Seems like:
android phone
+
cheap VR headset
+
raspberry pi with vr display capability
+
VNC app
=
cheap head-mounted display for raspberry pi
...right?
Re: VR display drivers for Raspberry pi OS?
Well.... not really.
VR works by letting the image follow your head movements. The delay introduced by VNC makes that totally useless even if you were able to use the phones compass and gyro to control the Pi image.
There is a few android apps like TrinusVR and StreamTheater that uses NVIDIA streaming or special graphics drivers to be able to do this with a powerful PC and require really low-latency video transfer to the phone.
You could use it as a simple stereo image viewer, like the good old Viewmaster. I would say that this is pretty cool but you dont really need to Pi to do this, the android phone is probably capable of doing it itself.
Of the two main VR headsets for PC I think the Oculus Rift has the best Linux support but dont know is anyone have tried a Pi with it.
VR works by letting the image follow your head movements. The delay introduced by VNC makes that totally useless even if you were able to use the phones compass and gyro to control the Pi image.
There is a few android apps like TrinusVR and StreamTheater that uses NVIDIA streaming or special graphics drivers to be able to do this with a powerful PC and require really low-latency video transfer to the phone.
You could use it as a simple stereo image viewer, like the good old Viewmaster. I would say that this is pretty cool but you dont really need to Pi to do this, the android phone is probably capable of doing it itself.
Of the two main VR headsets for PC I think the Oculus Rift has the best Linux support but dont know is anyone have tried a Pi with it.
Re: VR display drivers for Raspberry pi OS?
Oh, yes- you are right about that, but, that's not what I was meaning- "cheap head-mounted display for raspberry pi", maybe mentioning VR was not needed...
Getting a pi to show its own desktop (not vr content) in a stereoscopic view format is all I am looking for, I think a minor lag on mouse movements and keyboard would probably be acceptable. A pi is going to be so low horsepower, I doubt it could do anything other than VirtualBoy gaming.
Putting it in a vr headset for an easy and cheap HMD solution is what I am looking for.
Getting a pi to show its own desktop (not vr content) in a stereoscopic view format is all I am looking for, I think a minor lag on mouse movements and keyboard would probably be acceptable. A pi is going to be so low horsepower, I doubt it could do anything other than VirtualBoy gaming.
Putting it in a vr headset for an easy and cheap HMD solution is what I am looking for.
Re: VR display drivers for Raspberry pi OS?
It looks like I am going to do something like this-
FPV OTG app for phone to achieve the stereoscopic view - easy
Some kind of hack to make the VNC session show in both sides of FPV - harder
Add in vufine+ to VR headset for external camera view (rear) - easy
Work on app to incorporate cell phone front camera into FPV view as well - harder
FPV OTG app for phone to achieve the stereoscopic view - easy
Some kind of hack to make the VNC session show in both sides of FPV - harder
Add in vufine+ to VR headset for external camera view (rear) - easy
Work on app to incorporate cell phone front camera into FPV view as well - harder
Re: VR display drivers for Raspberry pi OS?
This will be difficult to do with good results. How would input work?
Also, if you are in VR, you probably don't want to just view your desktop as it is shown on the monitor, but rather a new VR desktop that is more spatial in nature. Probably you want a new VR window manager.
I would probably start by using Unity to make a simple VR app that is an empty space with a few VR windows floating. Then do something simple, like try to send terminal output (e.g. top command) from the Raspberry Pi to one VR window, maybe a single dynamic web page to another.
I'd also figure out first what kind of data rate you can sustain from the Raspberry Pi to the Android phone. That will dictate how much content you can send.
You're going to find that even with a 2560x1440 phone, the screen door effect makes text not super pleasant. Additionally, VR is very intensive, so quality tends to suffer somewhat (you really don't want to drop frame rate).
Post back if you get something working.
Also, if you are in VR, you probably don't want to just view your desktop as it is shown on the monitor, but rather a new VR desktop that is more spatial in nature. Probably you want a new VR window manager.
I would probably start by using Unity to make a simple VR app that is an empty space with a few VR windows floating. Then do something simple, like try to send terminal output (e.g. top command) from the Raspberry Pi to one VR window, maybe a single dynamic web page to another.
I'd also figure out first what kind of data rate you can sustain from the Raspberry Pi to the Android phone. That will dictate how much content you can send.
You're going to find that even with a 2560x1440 phone, the screen door effect makes text not super pleasant. Additionally, VR is very intensive, so quality tends to suffer somewhat (you really don't want to drop frame rate).
Post back if you get something working.
Re: VR display drivers for Raspberry pi OS?
I am actually not gunning for any VR effect at all, at least initially.
Having a HMD for a raspberry pi using an android phone and a $15 vr headset is the initial goal. Of course, something like that won't be super useful for people who can't touch-type. Seems to me being able to drop an app on an android phone, put on a cheap VR headset and then VNC into a pi might prove handy to more than a few people.
Anyway, where it goes beyond that is not important presently. If interacting with the desktop is simply untenable, time to bail- if it's acceptable, then I will expand upon it.
I have connected the phone to the PI using VNC, and then dropped the phone into the VR headset (sans stereoscopic capability)- the view of the desktop is actually pretty decent over wireless.
I have also connected he phone to the with VNC, started up the camera on the pi and held it up, then held it against the phone and moved around to emulate vr headset use, and see how laggy the view felt in this fashion. I think it would do as a pass-through, though there's definitely a lag.
Having a HMD for a raspberry pi using an android phone and a $15 vr headset is the initial goal. Of course, something like that won't be super useful for people who can't touch-type. Seems to me being able to drop an app on an android phone, put on a cheap VR headset and then VNC into a pi might prove handy to more than a few people.
Anyway, where it goes beyond that is not important presently. If interacting with the desktop is simply untenable, time to bail- if it's acceptable, then I will expand upon it.
I have connected the phone to the PI using VNC, and then dropped the phone into the VR headset (sans stereoscopic capability)- the view of the desktop is actually pretty decent over wireless.
I have also connected he phone to the with VNC, started up the camera on the pi and held it up, then held it against the phone and moved around to emulate vr headset use, and see how laggy the view felt in this fashion. I think it would do as a pass-through, though there's definitely a lag.
Re: VR display drivers for Raspberry pi OS?
I touch type too, I know what you mean.
Search for VNC clients for Unity, see if anyone has one that works on your phone (i.e. not just Windows). If so, then it's easy, just use Unity to make the app.
If not, find a VNC client that has an API you can use to grab the remote desktop image. Then use the Google Cardboard/Daydream SDK to use that API to grab the remote desktop image and feed it into a stereoscopic view. You may have to use raw OpenGL (from Java) to do this, lots of tutorials.
Good luck.
Search for VNC clients for Unity, see if anyone has one that works on your phone (i.e. not just Windows). If so, then it's easy, just use Unity to make the app.
If not, find a VNC client that has an API you can use to grab the remote desktop image. Then use the Google Cardboard/Daydream SDK to use that API to grab the remote desktop image and feed it into a stereoscopic view. You may have to use raw OpenGL (from Java) to do this, lots of tutorials.
Good luck.