TheMegaTB
Posts: 4
Joined: Tue Feb 11, 2020 11:35 am

Re: Apple Carplay

Tue Feb 11, 2020 8:19 pm

Benzelite124 wrote:
Tue Feb 11, 2020 5:34 pm
Woah this is crazy what exactly are you trying to do? I already got carplay working on the raspberry pi with the dongle and running emteria.
My general aim with this is actually to just use the Dongle with a cheap Amazon Tablet, mount that in the car with some 3D printed parts and call it a day. Nonetheless it would be interesting to figure out the inner workings of CarPlay and maybe even get hands on the protocol either it or the China dongles use. This knowledge could be used to make "native" CarPlay on Raspbian or any other Linux machine that could do H264 decoding possible which would be pretty damn cool in terms of hackability.

But right now it looks kinda grim as the Firmware without the required (after some research seemingly non-public) libraries is somewhat useless and the Dongle App has been obfuscated pretty thoroughly which leaves USB sniffing or similar methods open. Honestly I don't think CarPlay without additional hardware is going to be possible in the near future because of the certificate requirements which you only get from Apple or by "stealing" it from Firmware images. This makes the intermediate black-box dongle that does all this and apparently has a certificate a way more attractive target.


EDIT: I can say one thing for sure though, almost all of the Apps out there for those dongles are from one source with minor modifications. Seems like either a white label solution or a *lot* of different versions out in the wild. ZBox seems to be slightly different though, still investigating.
Last edited by TheMegaTB on Tue Feb 11, 2020 8:37 pm, edited 1 time in total.

Benzelite124
Posts: 18
Joined: Sun Oct 20, 2019 6:49 pm

Re: Apple Carplay

Tue Feb 11, 2020 8:37 pm

TheMegaTB wrote:
Tue Feb 11, 2020 8:19 pm
Benzelite124 wrote:
Tue Feb 11, 2020 5:34 pm
Woah this is crazy what exactly are you trying to do? I already got carplay working on the raspberry pi with the dongle and running emteria.
My general aim with this is actually to just use the Dongle with a cheap Amazon Tablet, mount that in the car with some 3D printed parts and call it a day. Nonetheless it would be interesting to figure out the inner workings of CarPlay and maybe even get hands on the protocol either it or the China dongles use. This knowledge could be used to make "native" CarPlay on Raspbian or any other Linux machine that could do H264 decoding possible which would be pretty damn cool in terms of hackability.

But right now it looks kinda grim as the Firmware without the required (after some research seemingly non-public) libraries is somewhat useless and the Dongle App has been obfuscated pretty thoroughly which leaves USB sniffing or similar methods open. Honestly I don't think CarPlay without additional hardware is going to be possible in the near future because of the certificate requirements which you only get from Apple or by "stealing" it from Firmware images. This makes the intermediate black-box dongle that does all this and apparently has a certificate a way more attractive target.


EDIT: I can say one thing for sure though, almost all of the Apps out there for those dongles are from one source with minor modifications. Seems like either a white label solution or a *lot* of different versions out in the wild. ZBox seems to be slightly different though, still investigating.

EDIT2: Seems like the ZBox App hasn't been obfuscated in any way, could be an entry point for figuring out the protocol. Definitely has to be something that Apps can do with the limited USB peripheral capabilities of Android so I doubt it is something completely custom.
Cool good luck, my biggest issue is that the raspberry pi car-play system is a little laggy do you know any ways to make it faster?

TheMegaTB
Posts: 4
Joined: Tue Feb 11, 2020 11:35 am

Re: Apple Carplay

Tue Feb 11, 2020 8:51 pm

That depends, generally CarPlay supports multiple resolutions as seen in this image (1):
Image

Those resolutions are to my current knowledge available in 24, 30 and 60 FPS (2) which seemingly corresponds to "Compatibility", "Speed" and "Fluency" mode in the app. Depending on what type of lag you experience you can try multiple things:

Input lag / Scrolling is behind / Call response is delayed
This is most likely a generic issue due to the nature of this setup where the stream is forwarded through not just one but two devices which makes things inevitably slow. You might be able to improve the decoding speed though by using a more up-to-date RPi, lowering your FPS (compatibility mode) or using a more powerful tablet.

Laggy video
If the UI seems to be lagging (not delayed!) you may increase the FPS if the power supply and RPi CPU allow. Make sure though that you supply enough current to each component in this setup. The Pi needs all it can get for the decoding, while the dongle needs it for the communication with the phone. If any part is starved of power it may break down under sustained/peak load (as seen in the various "black screen" posts which is basically one of the two stopping the streaming pipeline due to lack of resources). You can improve the situation by using a powered USB hub (or building one of your own), a Y splitter cable (although these might require some Diodes so that the power doesn't flow into the RPi) between the RPi and the dongle or some other crazy means of injecting power. If you are so inclined you could check with a USB amp meter (just search for it there are plenty out there). Also make sure you don't over-demand from the Pi - driving a 1080p display, decoding an H.264 stream and processing touch inputs and audio at the same time can put some heavy load on the chipset and additional drain from the USB ports does not help the problem.


(1) Taken from the carlinkit manual: http://carlinkit.com/cpzsk
(2) Found in the CarPlay configuration file from the firmware posted some pages back in this thread: https://pastebin.com/iN5jUrfJ


EDIT: Regarding the power situation, if there simply isn't enough power available (not quite sure what the absolute upper limit of these USB 12V plugs is) you can try attaching a battery bank directly to the RPi power feed GPIOs instead of the regular USB port so that it can draw more current (research that on your own :P). Alternatively or maybe in addition, use some capacitors across the 5V and GND pins (5.5V 1F should be plenty) to prevent power spikes from nuking the system.

huisinro
Posts: 123
Joined: Thu Jul 11, 2013 7:39 pm

Re: Apple Carplay

Sat Feb 15, 2020 3:01 pm

What I can say, it’s totally possible to do it on RPi, or even on Windows.

I did something even more powerful than CarPlay, it full iOS mirroring and control, this video shows a windows version:

https://youtu.be/Y1TY5Px1718

CarPlay is pretty much airplay, plus some handshaking at the beginning. If you want to implement usb, NCM network over usb must be implemented.

testfr
Posts: 2
Joined: Fri Feb 28, 2020 10:00 am

Re: Apple Carplay

Fri Feb 28, 2020 10:03 am

huisinro wrote:
Sat Feb 15, 2020 3:01 pm
What I can say, it’s totally possible to do it on RPi, or even on Windows.

I did something even more powerful than CarPlay, it full iOS mirroring and control, this video shows a windows version:

https://youtu.be/Y1TY5Px1718

CarPlay is pretty much airplay, plus some handshaking at the beginning. If you want to implement usb, NCM network over usb must be implemented.
Hi, how are you controlling the device over usb without being jailbroken ?

TheMegaTB
Posts: 4
Joined: Tue Feb 11, 2020 11:35 am

Re: Apple Carplay

Fri Feb 28, 2020 10:49 am

You can mirror the screen of a device. It is a built in feature used for various applications. If you connect the phone to a Mac and open quick time for example you can view the same stream (it just doesn’t pass touch inputs back).

testfr
Posts: 2
Joined: Fri Feb 28, 2020 10:00 am

Re: Apple Carplay

Fri Feb 28, 2020 4:50 pm

TheMegaTB wrote:
Fri Feb 28, 2020 10:49 am
You can mirror the screen of a device. It is a built in feature used for various applications. If you connect the phone to a Mac and open quick time for example you can view the same stream (it just doesn’t pass touch inputs back).
Yup I know that, it's just in his youtube video, it seems he can control the device too, so it's a game changer.

I'm interesting too in reverse engineering about carplay so we can use it without any dongle

lodi12
Posts: 1
Joined: Sun Mar 22, 2020 7:02 pm

Re: Apple Carplay

Sun Mar 22, 2020 7:05 pm

Here (https://github.com/45clouds/WirelessCarPlay/ there are some documents about CarPlay protocol. I hope it can be helpful.

PiLabs
Posts: 26
Joined: Sat Jan 18, 2020 10:34 pm
Contact: YouTube

Re: Apple Carplay

Fri Mar 27, 2020 8:28 pm

I have heard of USB CarPlay Dongles and HDMI carplay dongles if that helps.
5 Raspberry Pis at home

Benzelite124
Posts: 18
Joined: Sun Oct 20, 2019 6:49 pm

Re: Apple Carplay

Fri Apr 03, 2020 9:41 pm

PiLabs wrote:
Fri Mar 27, 2020 8:28 pm
I have heard of USB CarPlay Dongles and HDMI carplay dongles if that helps.
Yes I have explained this, we are just going in circles now, these people above you are trying to turn the carplay usb interface dongles into a software.

Benzelite124
Posts: 18
Joined: Sun Oct 20, 2019 6:49 pm

Re: Apple Carplay

Fri Apr 03, 2020 9:43 pm

I am trying to go back to this forum and get my car play working better, every time I try to put my carplay software in speed or fluency mode it eventually crashes to a black screen telling me to put it in compatible mode.

Benzelite124
Posts: 18
Joined: Sun Oct 20, 2019 6:49 pm

Re: Apple Carplay

Sat Apr 04, 2020 7:24 pm

Anyone made progress with wireless carplay on raspberry pi without dongle?

Tripp396
Posts: 12
Joined: Sat Apr 04, 2020 11:20 pm

Re: Apple Carplay

Mon Apr 13, 2020 1:44 am

Ive got a pi working with android, the usb dongle and autokit. Anyone with a similar setup also have a reverse camera? Would love to get one setup that turns on when I put my car in reverse and returns to autokit when I go out of reverse.

themlruts
Posts: 4
Joined: Mon Sep 14, 2015 2:38 am

Re: Apple Carplay

Wed Apr 15, 2020 3:54 am

Glad I found this forum. I am doing the same as others. I have Emteria running on a B+. Seems to work great. Apple play dongle runs fine no issue there. Its now the sound and Bluetooth. Has anyone found a fix fort his yet? Any 3rd party BT dongles work?
Thanks
Mike

Kake88
Posts: 1
Joined: Tue Apr 28, 2020 10:31 pm

Re: Apple Carplay

Tue Apr 28, 2020 10:37 pm

Any progress on this?

I'm planning to connect my Rasp to my Audi MMI system in following way:

- Using CAN hat map the the commands from control knob to main unit and use them to control the Rasp
- Using the Rasp's TV out, connect it to ether a) Composite AUX video in or b) Reverse camera composite video in
- Some how have wired CarPlay working on it.

But it appears that the key mapping would be easily done in Rasbian, but apparently there is no CarPlay support for linux. I could purchase one of those USB dongles, throw Android on the Pi but then I would have issue with the CAN hat and the translation of CAN messages from the knob to actually controlling the UI.

Any ideas?

kurusadeli
Posts: 1
Joined: Sun May 03, 2020 9:32 pm

Re: Apple Carplay

Sun May 03, 2020 10:10 pm

Hi Kake88 and all,

I also intend to do almost the same.

In my case, I would like to integrate a RP to the existing OEM receiver in my car. The receiver is touch enabled and has all the features (satellite, HD-FM, GPS, Amp, etc) except Apply Car Play/Android Auto. It only integrates with my iphone in iPod mode over USB and takes the touch commands.

In order to achieve cellphone mirroring, the RP should be integrated with the OEM touch screen. I suppose working with the video & audio signal will not be challenging. the only unknown piece is receiving the "touch commands" from the OEM receiver by the RP, and process the commands. Are the touch commands on the receiver's screen are also sent out through the CAN Bus interface at the back of the receiver? Or should I use the USB port (the way that the iPod mode works)? I intend to use the AUX video/audio input of the receiver for RP HDMI output.

I know there are devices like "Beat-Sonic" that does the exact job. However the kit would cost +$500 plus installation. I would rather spend $500 on this project and learn something than spending on a product. That sets my budget $500.

Project Summary:
- Integrate a RP to existing car infotainment system (the OEM receiver is touch enabled)
- Using the AUX-IN port of the OEM receiver to display RP output.
- Mirror my cellphone on the RP display over Car Play or Android Auto protocol (to be displayed on the OEM receiver display)
- No additional display, tablet, GPS receiver, FM receiver, antenna, etc. The car already has them all.

Challenge:
- RP reading the touch commands from the OEM receiver touch screen

Any idea/ hint will be appreciated

Cdell1
Posts: 10
Joined: Thu Jun 25, 2020 7:53 am

Re: Apple Carplay

Thu Jun 25, 2020 8:06 am

Tripp396 wrote:
Mon Apr 13, 2020 1:44 am
Ive got a pi working with android, the usb dongle and autokit. Anyone with a similar setup also have a reverse camera? Would love to get one setup that turns on when I put my car in reverse and returns to autokit when I go out of reverse.
I've got the same issues you had with being asked to enable all permissions when they were already on, how did you end up fixing the issue?

ekstepj
Posts: 1
Joined: Sun Oct 04, 2020 7:05 am

Re: Apple Carplay

Sun Oct 04, 2020 7:16 am

Automotive grade Linux automotivelinux.org might have some assets. Has anyone looked at it?

Riccard0o0o
Posts: 2
Joined: Fri Jan 29, 2021 7:27 pm

Re: Apple Carplay

Fri Jan 29, 2021 7:42 pm

Hi, I have Raspberry Pi 4b 8gb model. I bought Carlinkit to make this project but if I plug the dongle to the raspberry after some times it reboot everything and if the dongle stay plugged while the raspberry is booting up, it immediately reboot.

I’m using LineageOS 16.0 because I have official 7inch display.

I tried power the Pi with a power supply and a powerbank, but the result is the same. How can I fix this? I really need help , because I need to complete this project asap.

Thanks

Riccard0o0o
Posts: 2
Joined: Fri Jan 29, 2021 7:27 pm

Re: Apple Carplay

Fri Jan 29, 2021 8:16 pm

Hi, I have Raspberry Pi 4b 8gb model. I bought Carlinkit to make this project but if I plug the dongle to the raspberry after some times it reboot everything and if the dongle stay plugged while the raspberry is booting up, it immediately reboot.

I’m using LineageOS 16.0 because I have official 7inch display.

I tried power the Pi with a power supply and a powerbank, but the result is the same. How can I fix this? I really need help , because I need to complete this project asap.

Thanks

ZephyrInTheOsiers
Posts: 2
Joined: Thu Feb 25, 2021 3:56 pm

Re: Apple Carplay

Fri Feb 26, 2021 4:02 pm

Did you ever find a solution for this problem? I am having the same issue now with lineagos as well as emteria. Thanks!

Amro
Posts: 1
Joined: Fri May 07, 2021 2:25 am

Re: Apple Carplay

Fri May 07, 2021 2:36 am

Dears

I have my car Opel Insignia Sports 2020, my head unit OEM is equipped with Android Auto and Apple Carplay.
I am going to buy Raspberry pi 4 as I need to program it via Android to connect it directly to head unit USB without any dongles and to have google maps, youtube vlc and many other apps from google play store. I want to start my project but I need your help, recommendation and advise. Is it possible that project come to life? Knowing that in China there is a box is called MMB is doing this job but for some reason I can't get it in my country

I need your help

Best Regards

Amro

jeph
Posts: 1
Joined: Sat May 15, 2021 4:53 pm

Re: Apple Carplay

Sat May 15, 2021 4:56 pm

ZephyrInTheOsiers wrote:
Fri Feb 26, 2021 4:02 pm
Did you ever find a solution for this problem? I am having the same issue now with lineagos as well as emteria. Thanks!
I'm Using a pi4 with lineagos and carlinkit with no issues. I did have a crashing issue when I first went to boot, however that was tied to the lack of a gps module, so I disabled location and it fixed it. I don't know if that is your issue, but it fixed mine...

Return to “Media centres”