Susdru5
Posts: 6
Joined: Wed Mar 29, 2017 6:23 am

Import Json data from uart then post string to http

Postby Susdru5 » Wed Mar 29, 2017 6:49 am

looking for some help as I have been struggling to understand what I need to do to complete my task.

I have a stream of json data coming from a sensor which is connected to my raspberry pi. I essentially need to post this data to another sensor that is connected via http.

First, I am having a hard time even reading the data. I am trying to use this script

Code: Select all

import serial

ser = serial.Serial('/dev/ttyACM0', 115200)

ser.write(b'\x4c\xff\x46')
Here is my goal, import the Json data from sensor 1 which includes gps information and orientation data from an Imu, parse the data to the expected format of sensor 2, then post this feed via http to sensor 2.

Any guidance would be greatly appreciated, thanks.
scotty101
Posts: 2185
Joined: Fri Jun 08, 2012 6:03 pm

Re: Import Json data from uart then post string to http

Postby scotty101 » Wed Mar 29, 2017 4:31 pm

The script example you have provided writes data. It doesn't read it.

You mention two sensors. How are they both connected? You can't have two devices connected to the same UART.
Electronic and Computer Engineer
Pi Interests: Home Automation, IOT, Python and Tkinter
Susdru5
Posts: 6
Joined: Wed Mar 29, 2017 6:23 am

Re: Import Json data from uart then post string to http

Postby Susdru5 » Wed Mar 29, 2017 10:57 pm

Thanks for your reply.

Sensor 1 which is sending the Odata is connected via UART. Sensor 2 will be connected via ethernet I believe, or at least that is my expectation.
ghp
Posts: 1141
Joined: Wed Jun 12, 2013 12:41 pm
Location: Stuttgart Germany
Contact:

Re: Import Json data from uart then post string to http

Postby ghp » Thu Mar 30, 2017 5:51 am

Hello,
you provide not that information which allows others to assist.
scotty101 already asked for the type of sensors.

The needed information could look like, fill in the <NEEDTOKNOW> fields.
I have a sensor1 of type <TYPE> from company <COMPANY> and a description is found in <DESCRIPTION>.
This sensor is connected by <SERIAL / USBSERIAL> to raspberry.
The sensor provides json data, a sample is <SAMPLE>; the field <FIELD> needs to be forwarded
to sensor2 of type <TYPE> from company <COMPANY> and a description is found in <DESCRIPTION>.
This sensor is connected by ... There is data conversion needed...
If you have a setup already available, provide a picture.

Regards,
Gerhard
Susdru5
Posts: 6
Joined: Wed Mar 29, 2017 6:23 am

Re: Import Json data from uart then post string to http

Postby Susdru5 » Thu Mar 30, 2017 8:41 pm

I have a sensor1 which is a DJI A3 flight controller
Link to documentation, I only need the broadcast feed which is streaming from the API plug.
https://developer.dji.com/onboard-sdk/d ... guide.html
This sensor is connected by to RPI's GPIO uart. I'm not sure what the json tables look like yet. I have not been able to view any data.

sensor2 is a red edge from MicaSense and a description is found in here http://micasense.github.io/rededge-api/
I assume this sensor will be connected to ethernet for HTTP.
This is the broadcast type from DJI's stream on the api plug:

Code: Select all

typedef struct BroadcastData
{
    unsigned short dataFlag;
    TimeStampData timeStamp;
    QuaternionData q;
    CommonData a;
    VelocityData v;
    CommonData w;
    PossitionData pos;
    MagnetData mag;
    RadioData rc;
    GimbalData gimbal;
    FlightStatus status;
    BatteryData battery;
    CtrlInfoData ctrlInfo;
    //! @note these variables are not send from FMU,
    //! just a record for user.
    uint8_t controlStatus;
    uint8_t activation;
} BroadcastData;
This is what the camera expects for the feed being pushed to the camera.

Code: Select all

{
   "latitude" : 0.1363053,
   "longitude" : -2.399923234,
   "altitude" : 332.503,
   "vel_n" : 0.73,
   "vel_e" : 1.2,
   "vel_d" : 0.49,
   "p_acc" : 2.456,
   "v_acc" : 1.54,
   "fix3d" : true,
   "utc_time" : "2014-10-08T20:27:23.321Z"
}
Sorry about that, I should have explained that a little better. I need to get gimbal data and Position data to the camera.
ghp
Posts: 1141
Joined: Wed Jun 12, 2013 12:41 pm
Location: Stuttgart Germany
Contact:

Re: Import Json data from uart then post string to http

Postby ghp » Fri Mar 31, 2017 6:02 am

Hello,
the camera has a rest api.
You can access this with a python library "requests".

Code: Select all

import requests
pyload = {
   "latitude" : 0.1363053,
   "longitude" : -2.399923234,
   "altitude" : 332.503,
   "vel_n" : 0.73,
   "vel_e" : 1.2,
   "vel_d" : 0.49,
   "p_acc" : 2.456,
   "v_acc" : 1.54,
   "fix3d" : True,
   "utc_time" : "2014-10-08T20:27:23.321Z"
}  

r = requests.get('http://localhost:8080/jetty_servlet/gps', params=pyload)
# success ?
print ( r.url ) 
print ( r.text )

r = requests.post('http://localhost:8080/jetty_servlet/gps', params=pyload)
# success ?
print ( r.url ) 
print ( r.text )
I have a local jetty_servlet which just receives data to verify this works.
When you look to the data, the true wen to True as booleans are thsi way in python. You need to change this to a string if lowercase 'true' is needed.

For the serial connection, there are many examples around.

Hope this helps
Gerhard
Susdru5
Posts: 6
Joined: Wed Mar 29, 2017 6:23 am

Re: Import Json data from uart then post string to http

Postby Susdru5 » Mon Apr 03, 2017 6:59 am

Thank you, I will try this when I return from travels.
Susdru5
Posts: 6
Joined: Wed Mar 29, 2017 6:23 am

Re: Import Json data from uart then post string to http

Postby Susdru5 » Mon May 15, 2017 6:02 pm

Ok, this was delayed, but I am finally back now.

Here is the response I received after running your code on my camera. Please note, the camera and the RPI are connected via the cameras hotspot.

Code: Select all

Python 3.4.2 (default, Oct 19 2014, 13:31:11) 
[GCC 4.9.1] on linux
Type "copyright", "credits" or "license()" for more information.
>>> ================================ RESTART ================================
>>> 
Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/urllib3/connectionpool.py", line 516, in urlopen
    body=body, headers=headers)
  File "/usr/lib/python3/dist-packages/urllib3/connectionpool.py", line 308, in _make_request
    conn.request(method, url, **httplib_request_kw)
  File "/usr/lib/python3.4/http/client.py", line 1090, in request
    self._send_request(method, url, body, headers)
  File "/usr/lib/python3.4/http/client.py", line 1128, in _send_request
    self.endheaders(body)
  File "/usr/lib/python3.4/http/client.py", line 1086, in endheaders
    self._send_output(message_body)
  File "/usr/lib/python3.4/http/client.py", line 924, in _send_output
    self.send(msg)
  File "/usr/lib/python3.4/http/client.py", line 859, in send
    self.connect()
  File "/usr/lib/python3/dist-packages/urllib3/connection.py", line 154, in connect
    conn = self._new_conn()
  File "/usr/lib/python3/dist-packages/urllib3/connection.py", line 133, in _new_conn
    (self.host, self.port), self.timeout, **extra_kw)
  File "/usr/lib/python3/dist-packages/urllib3/util/connection.py", line 87, in create_connection
    raise err
  File "/usr/lib/python3/dist-packages/urllib3/util/connection.py", line 78, in create_connection
    sock.connect(sa)
ConnectionRefusedError: [Errno 111] Connection refused

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/requests/adapters.py", line 362, in send
    timeout=timeout
  File "/usr/lib/python3/dist-packages/urllib3/connectionpool.py", line 559, in urlopen
    _pool=self, _stacktrace=stacktrace)
  File "/usr/lib/python3/dist-packages/urllib3/util/retry.py", line 245, in increment
    raise six.reraise(type(error), error, _stacktrace)
  File "/usr/lib/python3/dist-packages/six.py", line 624, in reraise
    raise value.with_traceback(tb)
  File "/usr/lib/python3/dist-packages/urllib3/connectionpool.py", line 516, in urlopen
    body=body, headers=headers)
  File "/usr/lib/python3/dist-packages/urllib3/connectionpool.py", line 308, in _make_request
    conn.request(method, url, **httplib_request_kw)
  File "/usr/lib/python3.4/http/client.py", line 1090, in request
    self._send_request(method, url, body, headers)
  File "/usr/lib/python3.4/http/client.py", line 1128, in _send_request
    self.endheaders(body)
  File "/usr/lib/python3.4/http/client.py", line 1086, in endheaders
    self._send_output(message_body)
  File "/usr/lib/python3.4/http/client.py", line 924, in _send_output
    self.send(msg)
  File "/usr/lib/python3.4/http/client.py", line 859, in send
    self.connect()
  File "/usr/lib/python3/dist-packages/urllib3/connection.py", line 154, in connect
    conn = self._new_conn()
  File "/usr/lib/python3/dist-packages/urllib3/connection.py", line 133, in _new_conn
    (self.host, self.port), self.timeout, **extra_kw)
  File "/usr/lib/python3/dist-packages/urllib3/util/connection.py", line 87, in create_connection
    raise err
  File "/usr/lib/python3/dist-packages/urllib3/util/connection.py", line 78, in create_connection
    sock.connect(sa)
urllib3.exceptions.ProtocolError: ('Connection aborted.', ConnectionRefusedError(111, 'Connection refused'))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/pi/Desktop/Scripts for API's/import_data_from_camera.py", line 15, in <module>
    r = requests.get('http://localhost:8080/jetty_servlet/gps', params=pyload)
  File "/usr/lib/python3/dist-packages/requests/api.py", line 60, in get
    return request('get', url, **kwargs)
  File "/usr/lib/python3/dist-packages/requests/api.py", line 49, in request
    return session.request(method=method, url=url, **kwargs)
  File "/usr/lib/python3/dist-packages/requests/sessions.py", line 457, in request
    resp = self.send(prep, **send_kwargs)
  File "/usr/lib/python3/dist-packages/requests/sessions.py", line 569, in send
    r = adapter.send(request, **kwargs)
  File "/usr/lib/python3/dist-packages/requests/adapters.py", line 407, in send
    raise ConnectionError(err, request=request)
requests.exceptions.ConnectionError: ('Connection aborted.', ConnectionRefusedError(111, 'Connection refused'))
>>> 
Susdru5
Posts: 6
Joined: Wed Mar 29, 2017 6:23 am

Re: Import Json data from uart then post string to http

Postby Susdru5 » Tue May 16, 2017 5:31 pm

I've managed to spend a little more time with getting a positive connection with the camera. Here is how I changed the code to at least get a response from the camera.

Code: Select all

import requests
pyload = {
   "latitude" : 0.1363053,
   "longitude" : -2.399923234,
   "altitude" : 332.503,
   "vel_n" : 0.73,
   "vel_e" : 1.2,
   "vel_d" : 0.49,
   "p_acc" : 2.456,
   "v_acc" : 1.54,
   "fix3d" : True,
   "utc_time" : "2014-10-08T20:27:23.321Z"
}  

r = requests.get('http://192.168.10.254/gps', params=pyload)
# success ?
print ( r.url ) 
print ( r.text )

r = requests.post('http://192.168.10.254:/gps', params=pyload)
# success ?
print ( r.url ) 
print ( r.text )
I then received this response.


Code: Select all

http://192.168.10.254/gps?vel_e=1.2&longitude=-2.399923234&utc_time=2014-10-08T20%3A27%3A23.321Z&vel_d=0.49&fix3d=True&altitude=332.503&vel_n=0.73&latitude=0.1363053&p_acc=2.456&v_acc=1.54
{"altitude":0,"fix3d":false,"latitude":0,"longitude":0,"p_acc":0,"utc_time":"1970-01-01T00:00:00.000Z","v_acc":0,"vel_d":0,"vel_e":0,"vel_n":0}

http://192.168.10.254/gps?vel_e=1.2&longitude=-2.399923234&utc_time=2014-10-08T20%3A27%3A23.321Z&vel_d=0.49&fix3d=True&altitude=332.503&vel_n=0.73&latitude=0.1363053&p_acc=2.456&v_acc=1.54
{"message":"Invalid format on utc_time field"}

>>> 

My question is, would the /jetty_Servlet change the response?, or did I re configure the code properly to receive the response from the camera?
ghp
Posts: 1141
Joined: Wed Jun 12, 2013 12:41 pm
Location: Stuttgart Germany
Contact:

Re: Import Json data from uart then post string to http

Postby ghp » Tue May 16, 2017 7:10 pm

Hello,
the json string looks ok. If I run the following code, there are noerrors.
[code]x = '{"altitude":0,"fix3d":false,"latitude":0,"longitude":0,"p_acc":0,"utc_time":"1970-01-01T00:00:00.000Z","v_acc":0,"vel_d":0,"vel_e":0,"vel_n":0}'
import json
d = json.loads(x)
print (d)[/code]
In your code is a flaw: remove the ':' in "254:/gps"
r = requests.post('http://192.168.10.254:/gps', params=pyload)
Try removing the Time zone info from the end of the time string and perhaps shorten it to 1970-01-01T00:00. Could also be that the target systems does not like this 'default time string', the 1970-01-01 is beginning of time on unix systems. REplace by a meaningful setting.
Regards,
Gerhard