gyanezfeliu
Posts: 1
Joined: Sat Jul 14, 2018 1:14 am

Sending images to a web server

Sat Jul 14, 2018 2:45 pm

Hello!!

I'm working on an educational project, which consists of taking timelapses of the growth of bacteria expressing fluorescent protein. We already have the device up and running and we are working on the next step: sending the pictures to a web server, in which also will be hosted a web app that wil be used to see the pictures and making analysis.
My questions is:
Because of this architecture, we decided to use AWS that offers storage, data base and processor, so will be ideal for our whole app. Amazon has its S3 SDK for python and it's pretty easy to send the images using the mqtt protocol.
Do you think a better way for doing this?

Thanks a lot!!
G

Frollo
Posts: 20
Joined: Thu Aug 27, 2015 8:44 pm

Re: Sending images to a web server

Sun Jul 15, 2018 9:43 am

As far as i am concerned, i would have simply done that with FTP and a shell script...

Code: Select all

sudo apt-get install ftp
and the script executed at convenience, kind of :

Code: Select all

#!/bin/sh

USER1="user"
PASS1="pass"
SERVER1="192.168.0.99"
# Directory where file is located / destination
DIR_R="/www/html/snapshots/"
#  name of the file is the date and time
FILE=`date +%Y-%m-%d.%H%M%S.jpg`

cd "/home/pi/source_images/"
fswebcam $FILE # if generation of the snapshot needed, else comment this line...

ftp -n -p -v -i <<EOF
open $SERVER1
user $USER1 $PASS1
binary
cd $DIR_R
put $FILE
cd /var/www/html/images
put $FILE
put $FILE lastsnap.jpg # if you want to have the last snap
quit
EOF

rm $FILE # if needed

#Fin de Script

and the crontab -e :

Code: Select all

*/5 * * * * sudo sh /path/to/script

tpyo kingg
Posts: 148
Joined: Mon Apr 09, 2018 5:26 pm
Location: N. Finland

Re: Sending images to a web server

Sun Jul 15, 2018 11:02 am

Frollo wrote:
Sun Jul 15, 2018 9:43 am
As far as i am concerned, i would have simply done that with FTP and a shell script...
The script and the crontab are definitely good advice but to the idea that FTP is acceptable in 2018 I vigorously and vehemently (but hopefully politely) object. AWS costs real money and when unwanted guests get in via FTP, that will be felt economically in addition to the trouble it causes everyone else. So instead of using FTP to send the user name, password, and data over the wild net without any encryption whatsoever it would be much better to use SFTP or Rsync.

From a script perspective, SFTP can work approximately the same way as deprecated, old FTP. However, it has several advantages such as 1) being secure, 2) being able to use SSH keys for authentication.
gyanezfeliu wrote:
Sat Jul 14, 2018 2:45 pm
Amazon has its S3 SDK for python and it's pretty easy to send the images using the mqtt protocol.
Do you think a better way for doing this?
I'd say SFTP or Rsync, probably the latter.

For MQTT to be used safely, it should be tunneled over SSL. With less work it could be tunneled over SSH but if you are going to use SSH anyway then you might as well just use SFTP or Rsync. Rsync might be the least trouble since it only sends the new data and won't resend the old, meaning that you won't also need a separate mechanism to find the newest files to transfer. That is important when you have metered bandwidth:

Code: Select all

rsync -a -e 'ssh -i /home/foo/.ssh/aws_rsa_key' /some/local/directory/*.jpeg yourserver.example.org:/var/www/pictures/

User avatar
DougieLawson
Posts: 33393
Joined: Sun Jun 16, 2013 11:19 pm
Location: Basingstoke, UK
Contact: Website

Re: Sending images to a web server

Sun Jul 15, 2018 11:30 am

MQTT doesn't need external tunneling as it natively supports TLS on port 8883 if you have the certificates for that set up.
Microprocessor, Raspberry Pi & Arduino Hacker
Mainframe database troubleshooter
MQTT Evangelist
Twitter: @DougieLawson

2012-18: 1B*5, 2B*2, B+, A+, Z, ZW, 3Bs*3, 3B+

Any DMs sent on Twitter will be answered next month.

tpyo kingg
Posts: 148
Joined: Mon Apr 09, 2018 5:26 pm
Location: N. Finland

Re: Sending images to a web server

Sun Jul 29, 2018 12:12 pm

DougieLawson wrote:
Sun Jul 15, 2018 11:30 am
MQTT doesn't need external tunneling as it natively supports TLS on port 8883 if you have the certificates for that set up.
I've looked and it seems that TLS tunneling is provided (or not) by the various MQTT implementations. The protocol itself does not address it. Would you have a recommended MQTT implementation?

User avatar
DougieLawson
Posts: 33393
Joined: Sun Jun 16, 2013 11:19 pm
Location: Basingstoke, UK
Contact: Website

Re: Sending images to a web server

Sun Jul 29, 2018 12:32 pm

Mosquitto or HiveMQ or RabbitMQ, pick your poison.
Microprocessor, Raspberry Pi & Arduino Hacker
Mainframe database troubleshooter
MQTT Evangelist
Twitter: @DougieLawson

2012-18: 1B*5, 2B*2, B+, A+, Z, ZW, 3Bs*3, 3B+

Any DMs sent on Twitter will be answered next month.

Return to “Camera board”

Who is online

Users browsing this forum: No registered users and 13 guests