so here is a simple formula for forwarding a video stream from the pi camera to another computer with relatively low latency (around half a second latency on a pi zero w) I tried another method that only involved vlc but not netcat and the latency for that was >2 seconds....
on viewing side... netcat -l -p 5000 | vlc stream:///dev/stdin :demux=h264 --h264-fps=24
on camera side... raspivid -vf -o - -t 0 -n -w 320 -h 240 -fps 24 | nc 192.168.1.<whatever> 5000
Hopefully this helps someone, with this my robot is drive-able.
If anyone knows a good way to get the latency down to the 0-<not noticeable> range, please feel free to chime in