To extend Phil's comment, if the VPU has the I2C peripheral open then it will have registered an interrupt handler for it. That is the point where you'll get issues if the ARM also tries to use the peripheral, as the VPU typically has lower interrupt latency.
With the camera subsystem it pretty much holds the I2C peripheral open the whole time that the camera component is enabled/executing, mainly as it is normally adjusting exposure and gain settings every frew frames. If you could arrange to shut the camera down at the point where you want to talk to other I2C devices then you should be safe.
You will also need to be aware of pinmuxing though. The VPU will switch the I2C peripheral muxing to the configured GPIO pins whenever it wants to talk to the camera and doesn't restore it afterwards (it sees no reason to). If your extra peripherals aren't on the same pins the ARM isn't likely to check the muxing and therefore you'll see no devices.
If you're running 2 camera modules then you'll also need to reserve two sets of I2C pins for them (0/1, 28&29, or 44&45) as the camera modules have the same address. (I guess one OV5647 and one IMX219 could run at the same time on the same pins as they have different addresses, but that is unlikely to be that useful).
If you really need to split the bus up due to other I2C address conflicts then there are various I2C mulitplexer chips available that are supported by Linux. The TCA/PCA9548a 8 way I2C controlled mux works well, and can be cascaded if you really need to go that far.
Software Engineer at Raspberry Pi Trading. Views expressed are still personal views.
Please don't send PMs asking for support - use the forum.
I'm not interested in doing contracts for bespoke functionality - please don't ask.