LeMury
Posts: 33
Joined: Mon Jun 11, 2018 3:06 pm

FT5406 Touch coordinate corruption

Thu Sep 27, 2018 2:47 pm

Hello,

I am using the official Pi's FT5406 Touchscreen connected to a Pi 3 in a bare-metal situation
and got it to work by gleaning info from the linux TS driver.
Everything seems to work fine, however I experience occasional glitches/jumps in touch coordinates during multi-touch.

More precisely, I am only interested in Single Touch.
So in my driver code I only handle touch_id 0 coordinates and events.
This works reliable as long as I do not multi-touch the screen.
If I do, there's an occasional jump/corruption of coordinates.
Any ideas anyone?

Also I wonder?
Whilst polling/copying the FT5406 register buffer struct as done in all TS code i have seen,
shouldn't there be a locking mechanism to prevent simultaneous access on it by GPU and CPU?

jamesh
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 27715
Joined: Sat Jul 30, 2011 7:41 pm

Re: FT5406 Touch coordinate corruption

Thu Sep 27, 2018 8:19 pm

The way our TS linux driver works is not to acces the TS, but to grab the coordinates from the GPU via a mailbox call, where there is a driver that talks to the TS. So no locking needed. If you are using a different TS driver, then you may be conflicting.
Principal Software Engineer at Raspberry Pi (Trading) Ltd.
Contrary to popular belief, humorous signatures are allowed.
I've been saying "Mucho" to my Spanish friend a lot more lately. It means a lot to him.

LeMury
Posts: 33
Joined: Mon Jun 11, 2018 3:06 pm

Re: FT5406 Touch coordinate corruption

Thu Sep 27, 2018 9:14 pm

The way our TS linux driver works is not to acces the TS, but to grab the coordinates from the GPU via a mailbox call, where there is a driver that talks to the TS. So no locking needed.
Yes, I think I do exactly the same as your linux driver.

1. Firstly Get or Set the FT5406 Registers buffer via a mailbox call.
VC_TAG_GET_TOUCHBUF 0x0004000f
VC_TAG_SET_TOUCHBUF 0x0004801f
(only setting the buffer seems to work for me)

2. Poll that buffer for number of touches, coordinates etc.

The mailbox call is used only once to establish the buffer.
Not during polling the FT5406 Registers buffer content!
So how is that synced?

jamesh
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 27715
Joined: Sat Jul 30, 2011 7:41 pm

Re: FT5406 Touch coordinate corruption

Fri Sep 28, 2018 9:11 am

Not quite sure of your question.

The driver code is here

https://github.com/raspberrypi/linux/bl ... i-ft5406.c

ft5406_thread is the polling loop, read memory buffer from the GPU which contains coordinates. The GPU is stuff data in at one end, we read it out at the other. Are you asking how that is synced to ensure there is no corruption? Answer is not sure, I'll need to check.
Principal Software Engineer at Raspberry Pi (Trading) Ltd.
Contrary to popular belief, humorous signatures are allowed.
I've been saying "Mucho" to my Spanish friend a lot more lately. It means a lot to him.

LeMury
Posts: 33
Joined: Mon Jun 11, 2018 3:06 pm

Re: FT5406 Touch coordinate corruption

Fri Sep 28, 2018 9:37 am

ft5406_thread is the polling loop, read memory buffer from the GPU which contains coordinates. The GPU is stuff data in at one end, we read it out at the other. Are you asking how that is synced to ensure there is no corruption?
Yes, that is exactly what I'm asking. Thank you.

In my bare metal code I do pretty much the same as in the above mentioned Linux driver code.
However I get occasional coordinate jumps and even out of bound coordinate values when multi-touched.

I am trying to track down the cause of these glitches and wondered if there might be a syncing issue.
I mean, how does the GPU firmware know not to fill the shared coordinate buffer while the CPU copies (memcpy) that buffer ?

jamesh
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 27715
Joined: Sat Jul 30, 2011 7:41 pm

Re: FT5406 Touch coordinate corruption

Fri Sep 28, 2018 10:08 am

OK, the only syncing I can see is the number of coordinates received is set to 99 by the GPU whilst the buffer is being updated, and then to the number of coordinates.

So, if the number of coords is 99, you should not be reading the buffer as the GPU is writing to it.

BUT, the GPU doesn't do a similar check in the reverse direction, so its possible that you might be reading the buffer when the GPU suddenly sets the number of coords to 99 and starts writing.

This is a bug, so I'll need to take a look at that.

One workaround is to wait for the 99 to be set, then quickly read the buffer. The GPU polls at 60Hz (17ms ish), so that should be enough time to read the buffer before the GPU updates it again. Or on every buffer item read, check the number of coords hasn't changed to 99. Or at the end of your buffer read, if the num coords has changed to 99, then the buffer read might be unreliable.
Principal Software Engineer at Raspberry Pi (Trading) Ltd.
Contrary to popular belief, humorous signatures are allowed.
I've been saying "Mucho" to my Spanish friend a lot more lately. It means a lot to him.

LeMury
Posts: 33
Joined: Mon Jun 11, 2018 3:06 pm

Re: FT5406 Touch coordinate corruption

Fri Sep 28, 2018 10:23 am

Aha Ok. I see.

However, looking at the code https://github.com/raspberrypi/linux/bl ... i-ft5406.c
at line 83:

Code: Select all

iowrite8(99, ts->ts_base +offsetof(struct ft5406_regs, num_points));
we see the CPU writing 99 to the buffer as well.
Why is that then?

LeMury
Posts: 33
Joined: Mon Jun 11, 2018 3:06 pm

Re: FT5406 Touch coordinate corruption

Fri Sep 28, 2018 11:16 am

Ok, below is what I basically do.
I call Poll_TS() at 1ms-ish interval.
Unfortunately the occasional glitches are still there.

Code: Select all

// buffer shared by GPU and CPU
static struct ft5406_regs TSBuffer;

int Poll_TS()
{
	struct ft5406_regs bufcopy;

	if( TSBuffer->num_points == 99 )
	{
		// GPU is filling the buffer
		return 0;
	}

	// copy the buffer content
	memcpy_fromio((char*)&bufcopy, (char*)TSBuffer, sizeof(struct ft5406_regs) );

	// invalidate buffer content?
	TSBuffer->num_points = 99;

	// ... do stuff with the coordinates

	return 1;
}

jamesh
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 27715
Joined: Sat Jul 30, 2011 7:41 pm

Re: FT5406 Touch coordinate corruption

Fri Sep 28, 2018 12:14 pm

LeMury wrote:
Fri Sep 28, 2018 10:23 am
Aha Ok. I see.

However, looking at the code https://github.com/raspberrypi/linux/bl ... i-ft5406.c
at line 83:

Code: Select all

iowrite8(99, ts->ts_base +offsetof(struct ft5406_regs, num_points));
we see the CPU writing 99 to the buffer as well.
Why is that then?
Ah, not seen that. I think that is to ensure you don't get the same set of coordinates twice. It could be used to singnal that the buffer is being read though.
Principal Software Engineer at Raspberry Pi (Trading) Ltd.
Contrary to popular belief, humorous signatures are allowed.
I've been saying "Mucho" to my Spanish friend a lot more lately. It means a lot to him.

jamesh
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 27715
Joined: Sat Jul 30, 2011 7:41 pm

Re: FT5406 Touch coordinate corruption

Fri Sep 28, 2018 12:15 pm

LeMury wrote:
Fri Sep 28, 2018 11:16 am
Ok, below is what I basically do.
I call Poll_TS() at 1ms-ish interval.
Unfortunately the occasional glitches are still there.

Code: Select all

// buffer shared by GPU and CPU
static struct ft5406_regs TSBuffer;

int Poll_TS()
{
	struct ft5406_regs bufcopy;

	if( TSBuffer->num_points == 99 )
	{
		// GPU is filling the buffer
		return 0;
	}

	// copy the buffer content
	memcpy_fromio((char*)&bufcopy, (char*)TSBuffer, sizeof(struct ft5406_regs) );

	// invalidate buffer content?
	TSBuffer->num_points = 99;

	// ... do stuff with the coordinates

	return 1;
}
Hmm, not sure what to suggest then. I pressumee you are up to date with Raspbian - there was a change in the firmware a year ago or so that fixed a corruption issue IIRC.
Principal Software Engineer at Raspberry Pi (Trading) Ltd.
Contrary to popular belief, humorous signatures are allowed.
I've been saying "Mucho" to my Spanish friend a lot more lately. It means a lot to him.


jamesh
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 27715
Joined: Sat Jul 30, 2011 7:41 pm

Re: FT5406 Touch coordinate corruption

Fri Sep 28, 2018 1:26 pm

LeMury wrote:
Fri Sep 28, 2018 12:28 pm
I use the latest greatest from https://github.com/raspberrypi/firmware ... aster/boot
That should be fine.

Not sure what to suggest. Does it only happen when you have multiple points pressed?
Principal Software Engineer at Raspberry Pi (Trading) Ltd.
Contrary to popular belief, humorous signatures are allowed.
I've been saying "Mucho" to my Spanish friend a lot more lately. It means a lot to him.

LeMury
Posts: 33
Joined: Mon Jun 11, 2018 3:06 pm

Re: FT5406 Touch coordinate corruption

Fri Sep 28, 2018 1:56 pm

Does it only happen when you have multiple points pressed?
It happens occasionally but yes, mostly when there are multiple touch points detected and released.

Thanks for helping btw!

LeMury
Posts: 33
Joined: Mon Jun 11, 2018 3:06 pm

Re: FT5406 Touch coordinate corruption

Fri Sep 28, 2018 2:41 pm

This is the actual polling code I use.
(Just to rule out some huge mess-up on my part)

Code: Select all

volatile static struct FT5406_Registers* TSBuffer=0;
volatile static int ReleasePending=0;

/*
 * Polled at ~1msec interval
 */
int Poll_TS_Simple(TouchEvent* ev)
{
	struct FT5406_Registers ts;

	if( TSBuffer->ntouch == 99 )
	{
		// GPU is filling the buffer
		return 0;
	}

	// grab the buffer content
	memcpy_fromio( &ts, TSBuffer, sizeof(struct FT5406_Registers) );

	if( TSBuffer->ntouch == 99 )
	{
		// GPU might have changed the buffer during copy
		DBG("invalid?\n");
		return 0;
	}

	// signal back we read it?
	TSBuffer->ntouch = 99;

	// go through the touch events
	if( ts.ntouch  )
	{
		for(int i=0; i<ts.ntouch; i++)
		{
			u8 id = (ts.point[i].yh >> 4) & 0xf;

			// We are only interested in touch with id 0
			if(id==0)
			{
				int x = (((int) ts.point[i].xh & 0xf) << 8) + ts.point[i].xl;
				int y = (((int) ts.point[i].yh & 0xf) << 8) + ts.point[i].yl;
				ev->x = x;
				ev->y = y;

				if(!ReleasePending)
				{
					ReleasePending=1;
					ev->event = FTS_TOUCH;
					return 1;
				}
				else
				{
					ev->event = FTS_MOVE;
					return 1;
				}
			}
		}
	}

	// if we come here, no id0 touch is detected
	if(ReleasePending)
	{
		ReleasePending=0;
		ev->event = FTS_RELEASE;
		return 1;
	}

	return 0;
}

jamesh
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 27715
Joined: Sat Jul 30, 2011 7:41 pm

Re: FT5406 Touch coordinate corruption

Fri Sep 28, 2018 3:38 pm

I think you might be using the touchid not the eventtype in the if().

Check out the Linux driver, event_type = (regs.point.xh >> 6) & 0x03 and you should compare that against the FTS_TOUCH_DOWN and FTS_TOUCH_CONTACT to see what happened for that touchid

Unless I have misunderstood your code.
Principal Software Engineer at Raspberry Pi (Trading) Ltd.
Contrary to popular belief, humorous signatures are allowed.
I've been saying "Mucho" to my Spanish friend a lot more lately. It means a lot to him.

LeMury
Posts: 33
Joined: Mon Jun 11, 2018 3:06 pm

Re: FT5406 Touch coordinate corruption

Fri Sep 28, 2018 4:06 pm

I think you might be using the touchid not the eventtype in the if().
Yes, that is correct.
Since I implement a Single Touch GUI, I only look at touch points with ID=0.
I then keep track of it's touch state.

But your remark about the event_type makes me realize now that the Firmware driver
already keeps track of the touch state (touch, hold and release).
No need for doing that twice!
Perhaps that's the culprit somehow.

I'll adapt my code and report back.

LeMury
Posts: 33
Joined: Mon Jun 11, 2018 3:06 pm

Re: FT5406 Touch coordinate corruption

Mon Oct 01, 2018 8:39 am

@jamesh

Hello again,

For testing purposes I have literally copied the polling loop code from the Linux driver
at https://github.com/raspberrypi/linux/bl ... i-ft5406.c
That got rid of the "out of bound values" however the occasional glitches during rapid multi-touches still occur.

I think I know what's going wrong.

The polling code simply misses touch events!
As you yourself explained, the firmware doesn't wait for the CPU grabbing the touch data.
The reason it is hardly noticed is because missing a FTS_TOUCH_DOWN or FTS_TOUCH_CONTACT event isn't very time critical because it is a long event and will be caught on the next poll.
A release event however can be very short especially during multi-touch.
Missing a release event trows off any touch cycle logic and, in my case, manifests as coordinate jumps.

Also, if I lower my polling frequency I get more glitches.
Which makes sense since I miss more touch events.

Does this make any sense?

okenido
Posts: 78
Joined: Thu Aug 02, 2018 11:47 am

Re: FT5406 Touch coordinate corruption

Fri Oct 12, 2018 1:06 pm

Maybe my problem is related to yours. I'm using the Circle framework to access the touch screen, it's using this code : https://github.com/rsta2/circle/blob/ma ... screen.cpp

I call Update() 60 times per sec as suggested, and it's working well except sometimes it misses a Touch Release event. I never missed a Move or Press event, but like 1 of 20 times the Release event isn't triggered.

rst
Posts: 494
Joined: Sat Apr 20, 2013 6:42 pm
Location: Germany

Re: FT5406 Touch coordinate corruption

Fri Oct 12, 2018 1:38 pm

okenido wrote:
Fri Oct 12, 2018 1:06 pm
Maybe my problem is related to yours. I'm using the Circle framework to access the touch screen, it's using this code : https://github.com/rsta2/circle/blob/ma ... screen.cpp

I call Update() 60 times per sec as suggested, and it's working well except sometimes it misses a Touch Release event. I never missed a Move or Press event, but like 1 of 20 times the Release event isn't triggered.
Circle is using the original algorithm for detecting touch screen events from the Linux driver. There has been an update to the Linux driver later, which fixes a spurious event issue from the touch screen. This fix has not been applied to the Circle driver yet. I will check this soon.

LeMury
Posts: 33
Joined: Mon Jun 11, 2018 3:06 pm

Re: FT5406 Touch coordinate corruption

Fri Oct 12, 2018 3:38 pm

okenido wrote:
Fri Oct 12, 2018 1:06 pm
Maybe my problem is related to yours. I'm using the Circle framework to access the touch screen, it's using this code : https://github.com/rsta2/circle/blob/ma ... screen.cpp

I call Update() 60 times per sec as suggested, and it's working well except sometimes it misses a Touch Release event. I never missed a Move or Press event, but like 1 of 20 times the Release event isn't triggered.
I don't use circle, but I think the problem is related. The handling of the release event in the firmware is weird.
The FTS_TOUCH_UP is never used by any linux driver version i have seen.
Instead the release event must be deducted by the absence of touch and previous states.
Since we can only poll the firmware, which doesn't wait for us, it's easy to miss such a short absence of touch, which then throws off any logic depending on it.

jamesh
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 27715
Joined: Sat Jul 30, 2011 7:41 pm

Re: FT5406 Touch coordinate corruption

Fri Oct 12, 2018 3:50 pm

IIRC, all the firmware does is pass on the data it get from the touchscreen. No real processing done on it.
Principal Software Engineer at Raspberry Pi (Trading) Ltd.
Contrary to popular belief, humorous signatures are allowed.
I've been saying "Mucho" to my Spanish friend a lot more lately. It means a lot to him.

rst
Posts: 494
Joined: Sat Apr 20, 2013 6:42 pm
Location: Germany

Re: FT5406 Touch coordinate corruption

Fri Oct 12, 2018 5:00 pm

rst wrote:
Fri Oct 12, 2018 1:38 pm
Circle is using the original algorithm for detecting touch screen events from the Linux driver. There has been an update to the Linux driver later, which fixes a spurious event issue from the touch screen. This fix has not been applied to the Circle driver yet. I will check this soon.
I did some tests using the sample/28-touchscreen of Circle. If CTouchScreenDevice::Update() is called 60 times per second, as it is normally done, I cannot notice any missed touch events. I reduced this update rate to 10 times per second and there are often missed touch events, which should not wonder. The only thing which can be done from my point-of-view is to ensure the requested update rate is met. This is definitely critical.

In the Linux driver this update process runs in a kernel thread, which sleeps 17 milliseconds between two updates. Unfortunately I cannot tell, how accurate this timing is for a kernel thread under high system load. I compared the original FT5406 Linux touch screen driver with the one after the update mentioned above has been applied. I cannot see any relevant modification in logic related to this "touch event missed" issue. Even the current driver on the rpi-4.14.y branch has the same function in this respect.

jamesh
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 27715
Joined: Sat Jul 30, 2011 7:41 pm

Re: FT5406 Touch coordinate corruption

Sat Oct 13, 2018 6:55 am

rst wrote:
Fri Oct 12, 2018 5:00 pm
rst wrote:
Fri Oct 12, 2018 1:38 pm
Circle is using the original algorithm for detecting touch screen events from the Linux driver. There has been an update to the Linux driver later, which fixes a spurious event issue from the touch screen. This fix has not been applied to the Circle driver yet. I will check this soon.
I did some tests using the sample/28-touchscreen of Circle. If CTouchScreenDevice::Update() is called 60 times per second, as it is normally done, I cannot notice any missed touch events. I reduced this update rate to 10 times per second and there are often missed touch events, which should not wonder. The only thing which can be done from my point-of-view is to ensure the requested update rate is met. This is definitely critical.

In the Linux driver this update process runs in a kernel thread, which sleeps 17 milliseconds between two updates. Unfortunately I cannot tell, how accurate this timing is for a kernel thread under high system load. I compared the original FT5406 Linux touch screen driver with the one after the update mentioned above has been applied. I cannot see any relevant modification in logic related to this "touch event missed" issue. Even the current driver on the rpi-4.14.y branch has the same function in this respect.
Ah, that makes sense. The buffer is a FIFO, so if you read at less than the speed the data is being put in, then you will lose entries.
Principal Software Engineer at Raspberry Pi (Trading) Ltd.
Contrary to popular belief, humorous signatures are allowed.
I've been saying "Mucho" to my Spanish friend a lot more lately. It means a lot to him.

okenido
Posts: 78
Joined: Thu Aug 02, 2018 11:47 am

Re: FT5406 Touch coordinate corruption

Sat Oct 13, 2018 9:13 am

Okay I found what happens.

All the Release events are catch, but sometimes a duplicated Move event is also generated just before, and it has swapped X/Y coordinates. Here is the example #28 from Circle's framework running :

Image

During the example I was just pressing and releasing my finger on the same zone on the display.

In my app, the incorrect move coords made my button think it lost focus, not triggering its release action.


It seems to happen more in my real program than in your test #28 (had to keep pressing/releasing for about 1 minute), so it may be someting very timing-sensitive.

okenido
Posts: 78
Joined: Thu Aug 02, 2018 11:47 am

Re: FT5406 Touch coordinate corruption

Sat Oct 13, 2018 9:46 am

rst wrote:
Fri Oct 12, 2018 1:38 pm

Circle is using the original algorithm for detecting touch screen events from the Linux driver. There has been an update to the Linux driver later, which fixes a spurious event issue from the touch screen. This fix has not been applied to the Circle driver yet. I will check this soon.
Applying those changes fixed the problem ! I can do a pull request if you want

Return to “Bare metal, Assembly language”