OpenCV and Python.
Task will be significantly easier if you use brightly coloured balls that have a high contract from each other and the background. Silver Balls on sand won't be a good contrast.
OpenCV will be able to filter the image by colours and then detect circles. Once you've detected the circles you can measure how many pixels apart they are.
Getting the camera in a good position might be challenging though.
You might want to start your project by testing the positioning of the camera and take some test photos which you can then use to develop your code. Then if you get stuck you can share the code and the photos with others to ask for help.
Electronic and Computer Engineer
Pi Interests: Home Automation, IOT, Python and Tkinter