obabarin
Posts: 12
Joined: Tue Oct 09, 2018 10:25 am

RPi losing bytes from custom sensor(Atmel microcontroller) over UART

Tue Oct 09, 2018 11:03 am

I am looking for help/insight into a situation with my data transfer setup between a custom board and my Raspberry Pi3 setup. The custom board has a ATmel Microcontroller (AT32UC3A1512) which was programmed to configure a PLL/VCO, A/D conversion and transmit samples over UART to the Pi3 which saves the data to file.
A/D conversion is 10 bit in two channels at 250 samples/sec (250Hz). The 20 bits from both channels are mapped into 3 bytes. First, it was observed that Pi3 will not read the uint_8 format of the bytes, and I had to 'convert' to char first. Please see what I did in code below:

Code: Select all

	uint8_t IQsamples[78]; 
	uint8_t IQbuff; //buffer to store CW before transmit on UART
	int j;
	int k;
	int singletone;
	while(1){//while 1
		
		TxBits(0x5,0x00000218); // VCO Subsystem 3 - ManualRFO mode
		
		//sensor1
		IQsamples[1] = 0x01;
		IQsamples[2] = 0x4B;  //number of bytes to expect


for(singletone=0;singletone<20;singletone++){//for1
			
			cpu_delay_us(1000, 3*BOARD_OSC0_HZ);
			
			k=3;
			
			for(j=0;j<25;j++){//for2
			
			// launch conversion on all enabled channels
			adc_start(adc);
			
			// Acquire the I/Q channels
			Isample = adc_get_value(adc, I_CHANNEL);
			Qsample = adc_get_value(adc, Q_CHANNEL);
			
			// Creating byte 1
			temp1=0x8000;
			temp2 =((Isample & 0x00000300) << 5 );
			temp1=(temp1 | temp2);
			temp1= ((temp1 | 0x1000) >> 8  );
			temp2=((Isample & 0x000000F0) >> 4);
			IQsamples[k] = (uint8_t)(temp1 | temp2);
			
			// Creating byte 2
			temp1 =((Isample & 0x0000000F) << 4 );
			temp1= (temp1 | 0x08);
			temp2= ((Qsample & 0x380) >> 7);
			IQsamples[k+1] = (uint8_t)(temp1 | temp2);
			
			// Creating byte 3
			temp1=0x80;
			temp2= ((Qsample & 0x7F));
			IQsamples[k+2] = (uint8_t)(temp1 | temp2);
			
			
			if (k<75) {//specifies sampling frequency
			cpu_delay_us(4000, 3*BOARD_OSC0_HZ);
			
			}//if
			
			k=k+3;
					
			}//for2
			
			for (int a = 1; a < 78; a++){//for3
			
			////convert content of IQsamples to decimals and 'print' to IQbuff
			sprintf(IQbuff, "%d;", IQsamples[a]);
			
			usart_write_line(USART, IQbuff);
			
			}//for3
			
			
			usart_write_line(USART, "\r\n");
			
		}//for1
		
	}//while
			

On the receiving end at the pi3, I am trying to read 770 bytes per second, since this is the amount of data available to me every second from the custom board, but I am always short of bytes which could be up to 154 or more at every instance. I have a high baudrate of 115200 which seems the highest my pi3 could go.

Here code on the pi3 is as follows. Kindly help- am I missing something? What can I do to rectify this?

Code: Select all

/////source code- http://www.raspberry-projects.com/pi/programming-in-c/uart-serial-port/using-the-uart/////////	

#include <stdio.h>
#include <unistd.h>			//Used for UART
#include <fcntl.h>			//Used for UART
#include <termios.h>		//Used for UART
#include <stdlib.h>
#include <string.h>
#include <stdint.h>			//intx_t and uintx_t defined in this header

#include <wiringPi.h>
#include <time.h>

#include <sys/time.h>

int main(int argc, char **argv){
	
	
	
	
////-------------------------
//----- SETUP USART 0 -----
//-------------------------
//At bootup, pins 8 and 10 are already set to UART0_TXD, UART0_RXD (ie the alt0 function) respectively
int uart0_filestream = -1;

//OPEN THE UART
//The flags (defined in fcntl.h):
//	Access modes (use 1 of these):
//		O_RDONLY - Open for reading only.
//		O_RDWR - Open for reading and writing.
//		O_WRONLY - Open for writing only.
//
//	O_NDELAY / O_NONBLOCK (same function) - Enables nonblocking mode. When set read requests on the file can return immediately with a failure status
//											if there is no input immediately available (instead of blocking). Likewise, write requests can also return
//											immediately with a failure status if the output can't be written immediately.
//
//	O_NOCTTY - When set and path identifies a terminal device, open() shall not cause the terminal device to become the controlling terminal for the process.
//uart0_filestream = open("/dev/ttyAMA0", O_RDWR | O_NOCTTY | O_NDELAY);		//Open in non blocking read/write mode, default
uart0_filestream = open("/dev/ttyS0", O_RDWR | O_NOCTTY | O_NDELAY);		//Open in non blocking read/write mode
if (uart0_filestream == -1)
{
	//ERROR - CAN'T OPEN SERIAL PORT
	printf("Error - Unable to open UART.  Ensure it is not in use by another application\n");
}

//CONFIGURE THE UART
//The flags (defined in /usr/include/termios.h - see http://pubs.opengroup.org/onlinepubs/007908799/xsh/termios.h.html):
//	Baud rate:- B1200, B2400, B4800, B9600, B19200, B38400, B57600, B115200, B230400, B460800, B500000, B576000, B921600, B1000000, B1152000, B1500000, B2000000, B2500000, B3000000, B3500000, B4000000
//	CSIZE:- CS5, CS6, CS7, CS8
//	CLOCAL - Ignore modem status lines
//	CREAD - Enable receiver
//	IGNPAR = Ignore characters with parity errors
//	ICRNL - Map CR to NL on input (Use for ASCII comms where you want to auto correct end of line characters - don't use for bianry comms!)
//	PARENB - Parity enable
//	PARODD - Odd parity (else even)
struct termios options;
tcgetattr(uart0_filestream, &options);
//options.c_cflag = B9600 | CS8 | CLOCAL | CREAD;		
//options.c_cflag = B115200 | CS8 | CLOCAL | CREAD | PARENB | PARODD;		//<Set baud rate	
options.c_cflag = B115200 | CS8 | CLOCAL | CREAD | PARENB | PARODD;		//<Set baud rate

//options.c_cflag = B57600 | CS8 | CLOCAL | CREAD | PARENB | PARODD;		//<Set baud rate
//options.c_cflag = B1152000 | CS8 | CLOCAL | CREAD | PARENB | PARODD;
options.c_iflag = IGNPAR;
options.c_oflag = 0;
options.c_lflag = 0;
tcflush(uart0_filestream, TCIFLUSH);
tcsetattr(uart0_filestream, TCSANOW, &options);	
	
//delay(300*1000); //allow PLL to settle
	
FILE*file1 = fopen("/home/pi/Documents/myC/Data/tictoc0910.txt","w");
FILE*file2 = fopen("/home/pi/Documents/myC/Data/dataStart0910_.txt","w");

//FILE*file1 = fopen("/media/pi/E27F-7413/PiData/timedata.txt","w");
//FILE*file2 = fopen("/media/pi/E27F-7413/PiData/testdata.txt","w");


///declare time variables
unsigned int t; //t being in seconds
time_t start, end;//
struct timeval timecheck;
double timespent;
    
    
///initialise start
gettimeofday(&timecheck, NULL);
start = (long)timecheck.tv_sec;
printf("%ld start time is\n", start);
fprintf(file1, "%ld \n", start); 
	

for (t=0;1;t++){
	if (uart0_filestream != -1){
		
		//unsigned char rx_buffer[77];
		uint8_t rx_buffer[77];
				
		int rx_length = read(uart0_filestream, (void*)rx_buffer, 77);
		
		if (rx_length < 0)
		{
			////An error occured (will occur if there are no bytes)
		}
		else if (rx_length == 0)
		{
			////No data waiting
		}
		else
		{
			//Bytes received
			rx_buffer[rx_length] = '\0';
			
			printf("%s", rx_buffer);
			//printf("%d", rx_buffer);
			
			fprintf(file2, "%s", rx_buffer); //stores data to file (data.txt)
			//fprintf(file2, "%d", rx_buffer); //stores data to file (data.txt)
							
			}//else
		
	}//if
	



///check time of prog run
gettimeofday(&timecheck, NULL);
end = (long)timecheck.tv_sec;

///break after set time
timespent = (double)(end- start); 
if (timespent >= 1) //(time count in seconds)
	break;

}//for 


fprintf(file1, "%ld", end); 

fclose(file1);	
fclose(file2);//close files
	
////closing uart
////----- CLOSE THE UART -----
close(uart0_filestream);
	
return 0;
	
}//main


User avatar
scruss
Posts: 1821
Joined: Sat Jun 09, 2012 12:25 pm
Location: Toronto, ON
Contact: Website

Re: RPi losing bytes from custom sensor(Atmel microcontroller) over UART

Tue Oct 09, 2018 12:49 pm

What kind of flow control are you using?
Can you get all the bytes using grabserial? https://github.com/tbird20d/grabserial
‘Remember the Golden Rule of Selling: “Do not resort to violence.”’ — McGlashan.

Heater
Posts: 9960
Joined: Tue Jul 17, 2012 3:02 pm

Re: RPi losing bytes from custom sensor(Atmel microcontroller) over UART

Tue Oct 09, 2018 1:17 pm

In the world of serial port communications there is no concept of "packet" of data or message. It's just a stream of bytes arriving over the line. When you do a read of the serial port you will get back whatever number of bytes are currently available. Which may start at any position within the data you are sending.

So the first problem is to impose some kind of "packet" structure on your transmissions so that: a) You know when the start of a new data transmission is. b) You can check that the bytes you have received are correct.

One way to do this is to convert your data values into ASCII character strings and send a coherent data update as a series of values separated by commas and terminated with a new line. For example:

22.4, 100.3, 60.1\n

This way the receiver can read one line at a time and parse the string for the data values.

Converting to text makes the messages a bit longer but I think you have time for that.

If you want to continue with binary transmissions then you need to put some kind of "frame" around the data.

1) Transmit some special, fixed, byte or bytes that indicates to the receiver that a new data update is coming.

2) Transmit the length of the incoming data packet. Perhaps a single byte will do, perhaps two.

3) Transmit the data bytes.

4) Transmit a checksum. Perhaps just a simple sum of all the bytes in the message.

With that in place the receiving end can detect when a new message is coming in, read the required length of the message, read length number of data bytes, calculating the check sum as it goes. Read the checksum at the end of the message a verify it against the one it calculated.

I would suggest sending the 10 bit data as two bytes. Saves all that fiddling around trying to pack bits into bytes.

Of course there are many other schemes for doing reliable serial data transmission, of various degrees of complexity.

Andyroo
Posts: 433
Joined: Sat Jun 16, 2018 12:49 am
Location: Side of the hill in Lincolnshire

Re: RPi losing bytes from custom sensor(Atmel microcontroller) over UART

Tue Oct 09, 2018 1:47 pm

A couple of quick things you could try:

1) Better / sorter cables inc a decent ground link
2) Drop the speed

Low voltage serial is prone to dropping bits (literally) it seems and with no software flow control (XON / XOFF for example) you may not have time to read data and process it before a buffer overrun occurs.
Need Pi spray - these things are breeding in my house...

obabarin
Posts: 12
Joined: Tue Oct 09, 2018 10:25 am

Re: RPi losing bytes from custom sensor(Atmel microcontroller) over UART

Wed Oct 10, 2018 8:13 am

Thanks for the contribution everyone. Since I am relatively new to programming and embedded systems (in fact, my codes are snippets from the internet), I will need more time to understand your proffered solutions and humbly request for simpler approaches and if possible code blocks examples to understand your suggestions.

obabarin
Posts: 12
Joined: Tue Oct 09, 2018 10:25 am

Re: RPi losing bytes from custom sensor(Atmel microcontroller) over UART

Wed Oct 10, 2018 8:15 am

Andyroo wrote:
Tue Oct 09, 2018 1:47 pm
A couple of quick things you could try:

1) Better / sorter cables inc a decent ground link
2) Drop the speed

Low voltage serial is prone to dropping bits (literally) it seems and with no software flow control (XON / XOFF for example) you may not have time to read data and process it before a buffer overrun occurs.

I have relatively short cables, and speeds lower than 115200buad/sec even gives smaller values of acquired/received data.

User avatar
bensimmo
Posts: 3309
Joined: Sun Dec 28, 2014 3:02 pm
Location: East Yorkshire

Re: RPi losing bytes from custom sensor(Atmel microcontroller) over UART

Wed Oct 10, 2018 8:21 am

As you are using a Pi3, I'll add this link, since by default you'll be using the mini-UART. If it is significant or not (see what you don't get with miniUART at the bottom) I don't know.
But just for reference https://www.raspberrypi.org/documentati ... on/uart.md

obabarin
Posts: 12
Joined: Tue Oct 09, 2018 10:25 am

Re: RPi losing bytes from custom sensor(Atmel microcontroller) over UART

Fri Oct 12, 2018 9:21 am

bensimmo wrote:
Wed Oct 10, 2018 8:21 am
As you are using a Pi3, I'll add this link, since by default you'll be using the mini-UART. If it is significant or not (see what you don't get with miniUART at the bottom) I don't know.
But just for reference https://www.raspberrypi.org/documentati ... on/uart.md
I tried swapping the UART from ttyS0 to ttyAMA0 as the link provided suggested. No much improvement.

Running cat /sys/devices/system/cpu/cpu0/cpufreq/scaling_cur_freq indicates that my cpufreq is 600MHz, which I think should be sufficient for a baudrate of 115200.

User avatar
bensimmo
Posts: 3309
Joined: Sun Dec 28, 2014 3:02 pm
Location: East Yorkshire

Re: RPi losing bytes from custom sensor(Atmel microcontroller) over UART

Fri Oct 12, 2018 2:18 pm

obabarin wrote:
Fri Oct 12, 2018 9:21 am
bensimmo wrote:
Wed Oct 10, 2018 8:21 am
As you are using a Pi3, I'll add this link, since by default you'll be using the mini-UART. If it is significant or not (see what you don't get with miniUART at the bottom) I don't know.
But just for reference https://www.raspberrypi.org/documentati ... on/uart.md
I tried swapping the UART from ttyS0 to ttyAMA0 as the link provided suggested. No much improvement.

Running cat /sys/devices/system/cpu/cpu0/cpufreq/scaling_cur_freq indicates that my cpufreq is 600MHz, which I think should be sufficient for a baudrate of 115200.
That's the resting speed for the Pi.
It was more the flow control etc that you gain, just in case that was a problem and if you needed it.

So try what they say above.

Return to “C/C++”