Swannph
Posts: 7
Joined: Wed May 16, 2018 11:35 am
Location: Madrid, Spain

USB serial stop sending data

Wed May 16, 2018 12:05 pm

Hi,

First of all, sorry for my English, I'm a french student.

I'm new in this forum, I'm novice in python language and I have a little problem to read and save datas from a Serial USB. In the while loop at the begining all is good, that's to say the raspberry receives and saves all the data from my serial but it stops reveid datas (after 8 kb every time) for a reason that I don't know.

Thank you

Code: Select all

op = open("test.txt", "w") 
serial.Serial('/dev/ttyUSB1',baudrate=115200,parity=serial.PARITY_NONE,stopbits=serial.STOPBITS_ONE,bytesize=serial.EIGHTBITS,timeout=1)

while(1):
    line = ser.readline().decode()
    op.write(line)
    print (line)
I attached an image of what I have.
Attachments
Capture.PNG
You can see that it stoped, there is anything after the 34A
Capture.PNG (15.89 KiB) Viewed 426 times

User avatar
lmarmisa
Posts: 1219
Joined: Thu Feb 14, 2013 2:22 am
Location: Jávea, Spain

Re: USB serial stop sending data

Wed May 16, 2018 1:25 pm

Welcome to the forum.

Maybe this is a buffer overrun problem. You have not defined any flow control protocol (for example xon/xoff).

So, the receiver maybe is losing the chars received once its buffer is full. The receiver is unable to say to the sender: "please, wait until I will be able to receive more chars".

You have to define the same flow control protocol on both sides of the serial line.

NOTE: xon/xoff can not be used if you are sending binary data. This protocol is oriented to text. According to your capture, you are receiving ascii text.

Swannph
Posts: 7
Joined: Wed May 16, 2018 11:35 am
Location: Madrid, Spain

Re: USB serial stop sending data

Wed May 16, 2018 2:32 pm

Thank you for your answer but it doesn't work, when I print ser.inWaiting() it seems that the buffer isn't full. Furthermore, when I use the Terminal of the raspberry ("minicom") it works very well, it means that the problem come from python maybe.

Thank you, if you or anyone else have an other idea don't hesitate.

User avatar
lmarmisa
Posts: 1219
Joined: Thu Feb 14, 2013 2:22 am
Location: Jávea, Spain

Re: USB serial stop sending data

Wed May 16, 2018 2:57 pm

You could explain a little bit how is the behavior of the sender.

Try adding a couple of flush statements to your code too:

Code: Select all

import sys
...
op = open("test.txt", "w") 
serial.Serial('/dev/ttyUSB1',baudrate=115200,parity=serial.PARITY_NONE,stopbits=serial.STOPBITS_ONE,bytesize=serial.EIGHTBITS,timeout=1)
while(1):
    line = ser.readline().decode()
    op.write(line)
    op.flush()
    print (line)
    sys.stdout.flush()
NOTE: the file op is never closed. Consider using a solution based on "with open(..., "a"):"

https://stackoverflow.com/questions/615 ... ne-to-file

Swannph
Posts: 7
Joined: Wed May 16, 2018 11:35 am
Location: Madrid, Spain

Re: USB serial stop sending data

Wed May 16, 2018 3:25 pm

The sender is a Zynq 7000 development board which is programed to send this type of data and the entire size of this data is approximately 30 kb.

I tried to use flush for the serial, sys and op in the while loop but it doesn't change anything.

The thing that seems to me weird is that the program works on windows but no on a raspberry. Furthermore the Terminal can read all the datas sent by the Zynq without problem.

User avatar
lmarmisa
Posts: 1219
Joined: Thu Feb 14, 2013 2:22 am
Location: Jávea, Spain

Re: USB serial stop sending data

Wed May 16, 2018 3:38 pm

Swannph wrote:
Wed May 16, 2018 3:25 pm
The sender is a Zynq 7000 development board which is programed to send this type of data and the entire size of this data is approximately 30 kb.

I tried to use flush for the serial, sys and op in the while loop but it doesn't change anything.

The thing that seems to me weird is that the program works on windows but no on a raspberry. Furthermore the Terminal can read all the datas sent by the Zynq without problem.
If the same program works in Windows (I suppose a PC) but not in Raspbian, this is a problem related to performance. Remember that a PC is a faster machine than a RPi. Therefore, I believe that the overrun theory makes sense. Does the Zynq 7000 support xon/xoff protocol?.

Swannph
Posts: 7
Joined: Wed May 16, 2018 11:35 am
Location: Madrid, Spain

Re: USB serial stop sending data

Thu May 17, 2018 11:13 am

Does the Zynq 7000 support xon/xoff protocol?.
I don't know and if it doesn't support xonxoff protocol I can't change anything on it. I have to do my project without changing anything on the Zynq.

Thank you for your help

User avatar
lmarmisa
Posts: 1219
Joined: Thu Feb 14, 2013 2:22 am
Location: Jávea, Spain

Re: USB serial stop sending data

Thu May 17, 2018 11:35 am

Try enabling software flow control (xonxoff) in the statement serial.Serial().

Other point to consider is why timeout should be 1 second. According to your code, you could use a blocking mode too.

http://pyserial.readthedocs.io/en/lates ... l_api.html

Code: Select all

serial.Serial('/dev/ttyUSB1',baudrate=115200,parity=serial.PARITY_NONE,stopbits=serial.STOPBITS_ONE,bytesize=serial.EIGHTBITS,xonxoff=true)
There are other alternatives too. For example, you could read lines or chars into an array and finally dump them to a file when the entire block of data is completed. Write operations to SD are very slow and they require long times.

I have a question: do you receive data from a FPGA hardware or from the develpment board?.
Last edited by lmarmisa on Thu May 17, 2018 12:18 pm, edited 1 time in total.

User avatar
lmarmisa
Posts: 1219
Joined: Thu Feb 14, 2013 2:22 am
Location: Jávea, Spain

Re: USB serial stop sending data

Thu May 17, 2018 12:09 pm

Other people have detected similar problems to yours, but the speed of your data seems not too high. Apparently there is no way to change the rx buffer size in Linux.

https://github.com/pyserial/pyserial/issues/102

Swannph
Posts: 7
Joined: Wed May 16, 2018 11:35 am
Location: Madrid, Spain

Re: USB serial stop sending data

Thu May 17, 2018 12:59 pm

The timeout is 1s because the sender of data can sometime stop sending data because of an intern error but it's normal.
May be I will try with the array method.

I receive the data from a FPGA Hardware, why do you want to know that?

User avatar
lmarmisa
Posts: 1219
Joined: Thu Feb 14, 2013 2:22 am
Location: Jávea, Spain

Re: USB serial stop sending data

Thu May 17, 2018 1:29 pm

Swannph wrote:
Thu May 17, 2018 12:59 pm
I receive the data from a FPGA Hardware, why do you want to know that?
I have no experience with FPGA but I know that there are very powerful development tools supporting high level modules and libraries. So, I believe that xon/xoff flow control could be included in projects where serial line comm is used.

Data flow control should be considered in designs. The rule is clear: the emitter should adapt its transmission speed to the receiver capabilities in such a way that no data is lost.

Hardware flow control methods (RTS/CTS) require additional wires. Xon-xoff is simple (the receiver sends "orders" to the sender using its tx serial line: xoff = wait; xoff = continue) but it needs a bidirectional channel and imposes some limitations about data.

I do not know how many wires connect the FPGA and your RPi. If the serial comms are bidirectional, you could test if xonxoff works. Very little effort and some probability of success.

Swannph
Posts: 7
Joined: Wed May 16, 2018 11:35 am
Location: Madrid, Spain

Re: USB serial stop sending data

Thu May 17, 2018 1:35 pm

Ok thank you, I will inform my supervisor (who managed the project) to see if their is a solution in this way. If I find one, I will tell you.

User avatar
lmarmisa
Posts: 1219
Joined: Thu Feb 14, 2013 2:22 am
Location: Jávea, Spain

Re: USB serial stop sending data

Thu May 17, 2018 1:38 pm

A question: just the same python script works fine in Windows but loses data in Raspbian. Is that so?.

Swannph
Posts: 7
Joined: Wed May 16, 2018 11:35 am
Location: Madrid, Spain

Re: USB serial stop sending data

Thu May 17, 2018 2:18 pm

lmarmisa wrote:
Thu May 17, 2018 1:38 pm
A question: just the same python script works fine in Windows but loses data in Raspbian. Is that so?.
Yes is that exept that it's not Raspbian but Debian.

User avatar
lmarmisa
Posts: 1219
Joined: Thu Feb 14, 2013 2:22 am
Location: Jávea, Spain

Re: USB serial stop sending data

Thu May 17, 2018 2:28 pm

Swannph wrote:
Thu May 17, 2018 2:18 pm
lmarmisa wrote:
Thu May 17, 2018 1:38 pm
A question: just the same python script works fine in Windows but loses data in Raspbian. Is that so?.
Yes is that exept that it's not Raspbian but Debian.
Raspbian is Debian too :D

Raspbian is very well supported. They have a very good technical team.

Not sure if your specific distro Debian is so well tested and maintained.

Return to “Python”

Who is online

Users browsing this forum: No registered users and 17 guests