jamesh
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 24168
Joined: Sat Jul 30, 2011 7:41 pm

Re: Raspberry Pi 3 B+ lockups

Mon Jun 18, 2018 9:40 am

wybielacz wrote:
Sun Jun 17, 2018 4:01 pm
jamesh wrote:
Sun Jun 17, 2018 2:49 pm
Sometimes, bugs are just REALLY difficult to fix. Simply as that. I have a laptop with a Intel Wireless chip in. It took Intel (Yes Intel, with all their thousands of engineers), 18 MONTHS to get that wireless chip driver stable, and tbh, I still don;t think it's particular good.Since the 3B+ was launched we have had to deal with some really quite complex issues, most of which were caused by other peoples chips and software, none of which came up in pre release testing. Annoying, but not uncommon with a new release.

I believe your fault is not the one mainly being talked about in this thread, but possibly related the network driver TCP segmentation offload issue. Have you tried turning that off (as described in the ethernet/NTFS problems thread)?

Yes, i did. I already wrote that on the previous page.

Ethernet is always killing my 3B+ when i mount a network drive, the commands didn't help. WiFi works fine and the 3B+ runs stable without any crashes, but i bought the 3B+ specially because of the faster ethernet so this is not acceptable for me...
Sadly, I don't have time to dig through pages of posts to find one line of information. Can you give specific instructions on what you do to make this go wrong, and any errors that may be display (e.g. syslogs). Since this does NOT appear to be the tcp segmentation offload issue (although symptoms appear the same - welcome to the wonderful world of ethernet bugs), it would be worth adding a very detailed issue in github (if you haven't already done so) which is the official way of reporting bugs, it also means they do not get lost. Post a link here to the github issue to cross reference.

In the meantime, does forcing it to 100bT make the problem go away (yes, I know you want Gig ethernet, this is for testing purposes to narrow down the issue)
Principal Software Engineer at Raspberry Pi (Trading) Ltd.
Contrary to popular belief, humorous signatures are allowed. Here's an example...
“I think it’s wrong that only one company makes the game Monopoly.” – Steven Wright

wybielacz
Posts: 29
Joined: Wed Nov 06, 2013 9:37 pm

Re: Raspberry Pi 3 B+ lockups

Mon Jun 18, 2018 11:31 am

jamesh wrote:
Mon Jun 18, 2018 9:40 am

Sadly, I don't have time to dig through pages of posts to find one line of information. Can you give specific instructions on what you do to make this go wrong, and any errors that may be display (e.g. syslogs). Since this does NOT appear to be the tcp segmentation offload issue (although symptoms appear the same - welcome to the wonderful world of ethernet bugs), it would be worth adding a very detailed issue in github (if you haven't already done so) which is the official way of reporting bugs, it also means they do not get lost. Post a link here to the github issue to cross reference.

In the meantime, does forcing it to 100bT make the problem go away (yes, I know you want Gig ethernet, this is for testing purposes to narrow down the issue)
You already have a 3B+ with an sd card with this issue in your office. Anyway we can safely assume that every 3B+ has this issue, just the right steps needs to be performed to make it crash.

Steps to reproduce are as follows:

1. Install a fresh image of Raspbian
2. Update firmware
3. Mount a cifs network drive
4. 3B+ is dead overnight when using Ethernet, there are no errors, LAN chip is burning hot

Sadly, i do not have time anymore to invest in this, i literally setup my multiple RPi's 3B+ in the past months hundreds of times new to help you debug and narrow down the issue, even that i am not responsible to do that...

All the 3B+ problems seems to me like this is a faulty ethernet hardware which you trying to cover with software fixes. If that is really the case then please at least do an official statement about it. I mean come on, it's been 3 months...

jamesh
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 24168
Joined: Sat Jul 30, 2011 7:41 pm

Re: Raspberry Pi 3 B+ lockups

Mon Jun 18, 2018 12:28 pm

wybielacz wrote:
Mon Jun 18, 2018 11:31 am
jamesh wrote:
Mon Jun 18, 2018 9:40 am

Sadly, I don't have time to dig through pages of posts to find one line of information. Can you give specific instructions on what you do to make this go wrong, and any errors that may be display (e.g. syslogs). Since this does NOT appear to be the tcp segmentation offload issue (although symptoms appear the same - welcome to the wonderful world of ethernet bugs), it would be worth adding a very detailed issue in github (if you haven't already done so) which is the official way of reporting bugs, it also means they do not get lost. Post a link here to the github issue to cross reference.

In the meantime, does forcing it to 100bT make the problem go away (yes, I know you want Gig ethernet, this is for testing purposes to narrow down the issue)
You already have a 3B+ with an sd card with this issue in your office. Anyway we can safely assume that every 3B+ has this issue, just the right steps needs to be performed to make it crash.

Steps to reproduce are as follows:

1. Install a fresh image of Raspbian
2. Update firmware
3. Mount a cifs network drive
4. 3B+ is dead overnight when using Ethernet, there are no errors, LAN chip is burning hot

Sadly, i do not have time anymore to invest in this, i literally setup my multiple RPi's 3B+ in the past months hundreds of times new to help you debug and narrow down the issue, even that i am not responsible to do that...

All the 3B+ problems seems to me like this is a faulty ethernet hardware which you trying to cover with software fixes. If that is really the case then please at least do an official statement about it. I mean come on, it's been 3 months...
I have already tested that sequence overnight and seen no issues, so No, your assumption that this is a problem with every 3B+ appears to be wrong. So there is probably an extra stage of usage or perhaps something about your particular network router that needs to be determined. For example, what is you network doing overnight? Is it in use? Is a lot of data being transferred? Do you have flow control turned on on your router? What software is running on the Pi that might have some impact?

As I said, 3 months is NOTHING when compared to the complexity of trying to figure out ethernet issues that some people see and others don't. I've personally spent months working on ethernet or wifi issues on the Pi over the last years and a half. And what is annoying NONE of them were Raspberry Pi's fault - they are either in the firmware of the device or in the driver. Neither of which we have much control over, apart from reporting them to the firmware writers, or mailing the Linux netdev group, or trying to figure them out ourselves.

Semi-Official statement. All software has bugs. All HW has bugs, Usually we can get round HW problems with software. In this case, we do not yet know where the problem arises. We have determined at least one cause so far, the tcp segmentation offloading, which has fixed networking problems for most people. There appears to still be something going on, so we will continue to investigate.

I would prefer the investigation to be done via github, because more engineers see it there.
Principal Software Engineer at Raspberry Pi (Trading) Ltd.
Contrary to popular belief, humorous signatures are allowed. Here's an example...
“I think it’s wrong that only one company makes the game Monopoly.” – Steven Wright

wybielacz
Posts: 29
Joined: Wed Nov 06, 2013 9:37 pm

Re: Raspberry Pi 3 B+ lockups

Mon Jun 18, 2018 3:00 pm

jamesh wrote:
Mon Jun 18, 2018 12:28 pm

I have already tested that sequence overnight and seen no issues, so No, your assumption that this is a problem with every 3B+ appears to be wrong. So there is probably an extra stage of usage or perhaps something about your particular network router that needs to be determined. For example, what is you network doing overnight? Is it in use? Is a lot of data being transferred? Do you have flow control turned on on your router? What software is running on the Pi that might have some impact?

As I said, 3 months is NOTHING when compared to the complexity of trying to figure out ethernet issues that some people see and others don't. I've personally spent months working on ethernet or wifi issues on the Pi over the last years and a half. And what is annoying NONE of them were Raspberry Pi's fault - they are either in the firmware of the device or in the driver. Neither of which we have much control over, apart from reporting them to the firmware writers, or mailing the Linux netdev group, or trying to figure them out ourselves.

Semi-Official statement. All software has bugs. All HW has bugs, Usually we can get round HW problems with software. In this case, we do not yet know where the problem arises. We have determined at least one cause so far, the tcp segmentation offloading, which has fixed networking problems for most people. There appears to still be something going on, so we will continue to investigate.

I would prefer the investigation to be done via github, because more engineers see it there.

Once more, i got multiple 3B+ all suffering the same issue, so either i can start playing lottery or every 3B+ has this problem under certain circumstances.
My network is idling overnight, there is nothing happening. There is no data transferred. I have no flow control on my router. And again with the same setup a 3B runs fine with no crashes overnight using ethernet.

jamesh
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 24168
Joined: Sat Jul 30, 2011 7:41 pm

Re: Raspberry Pi 3 B+ lockups

Mon Jun 18, 2018 4:06 pm

wybielacz wrote:
Mon Jun 18, 2018 3:00 pm
jamesh wrote:
Mon Jun 18, 2018 12:28 pm

I have already tested that sequence overnight and seen no issues, so No, your assumption that this is a problem with every 3B+ appears to be wrong. So there is probably an extra stage of usage or perhaps something about your particular network router that needs to be determined. For example, what is you network doing overnight? Is it in use? Is a lot of data being transferred? Do you have flow control turned on on your router? What software is running on the Pi that might have some impact?

As I said, 3 months is NOTHING when compared to the complexity of trying to figure out ethernet issues that some people see and others don't. I've personally spent months working on ethernet or wifi issues on the Pi over the last years and a half. And what is annoying NONE of them were Raspberry Pi's fault - they are either in the firmware of the device or in the driver. Neither of which we have much control over, apart from reporting them to the firmware writers, or mailing the Linux netdev group, or trying to figure them out ourselves.

Semi-Official statement. All software has bugs. All HW has bugs, Usually we can get round HW problems with software. In this case, we do not yet know where the problem arises. We have determined at least one cause so far, the tcp segmentation offloading, which has fixed networking problems for most people. There appears to still be something going on, so we will continue to investigate.

I would prefer the investigation to be done via github, because more engineers see it there.

Once more, i got multiple 3B+ all suffering the same issue, so either i can start playing lottery or every 3B+ has this problem under certain circumstances.
My network is idling overnight, there is nothing happening. There is no data transferred. I have no flow control on my router. And again with the same setup a 3B runs fine with no crashes overnight using ethernet.
"Certain circumstances" is the key. We need to find out what they are. This is the main reason it takes so long to find this stuff - narrowing down the specific circumstances that cause the issue.

The 3B+ uses a different ethernet chip which has a different driver (smsc95xx vs lan78xx), so really very different in that area, so certainly possible there is another bug in there somewhere.
Principal Software Engineer at Raspberry Pi (Trading) Ltd.
Contrary to popular belief, humorous signatures are allowed. Here's an example...
“I think it’s wrong that only one company makes the game Monopoly.” – Steven Wright

wybielacz
Posts: 29
Joined: Wed Nov 06, 2013 9:37 pm

Re: Raspberry Pi 3 B+ lockups

Mon Jun 18, 2018 4:39 pm

jamesh wrote:
Mon Jun 18, 2018 4:06 pm

"Certain circumstances" is the key. We need to find out what they are. This is the main reason it takes so long to find this stuff - narrowing down the specific circumstances that cause the issue.

The 3B+ uses a different ethernet chip which has a different driver (smsc95xx vs lan78xx), so really very different in that area, so certainly possible there is another bug in there somewhere.

Yes, that is what i am trying to say here. The ethernet chip has a bug, i dunno if it software or hardware but it has still an issue which is not resolved.

And i already provided you with the exact steps to reproduce this issue, i even sent you my sd card which has this issue. So you already have the "Certain circumstances"...

gkreidl
Posts: 6139
Joined: Thu Jan 26, 2012 1:07 pm
Location: Germany

Re: Raspberry Pi 3 B+ lockups

Mon Jun 18, 2018 5:01 pm

wybielacz wrote:
Mon Jun 18, 2018 4:39 pm
jamesh wrote:
Mon Jun 18, 2018 4:06 pm

"Certain circumstances" is the key. We need to find out what they are. This is the main reason it takes so long to find this stuff - narrowing down the specific circumstances that cause the issue.

The 3B+ uses a different ethernet chip which has a different driver (smsc95xx vs lan78xx), so really very different in that area, so certainly possible there is another bug in there somewhere.

Yes, that is what i am trying to say here. The ethernet chip has a bug, i dunno if it software or hardware but it has still an issue which is not resolved.

And i already provided you with the exact steps to reproduce this issue, i even sent you my sd card which has this issue. So you already have the "Certain circumstances"...
No, they don't. It may be related to your router, for example.
Minimal Kiosk Browser (kweb)
Slim, fast webkit browser with support for audio+video+playlists+youtube+pdf+download
Optional fullscreen kiosk mode and command interface for embedded applications
Includes omxplayerGUI, an X front end for omxplayer

jamesh
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 24168
Joined: Sat Jul 30, 2011 7:41 pm

Re: Raspberry Pi 3 B+ lockups

Mon Jun 18, 2018 7:37 pm

wybielacz wrote:
Mon Jun 18, 2018 4:39 pm
jamesh wrote:
Mon Jun 18, 2018 4:06 pm

"Certain circumstances" is the key. We need to find out what they are. This is the main reason it takes so long to find this stuff - narrowing down the specific circumstances that cause the issue.

The 3B+ uses a different ethernet chip which has a different driver (smsc95xx vs lan78xx), so really very different in that area, so certainly possible there is another bug in there somewhere.

Yes, that is what i am trying to say here. The ethernet chip has a bug, i dunno if it software or hardware but it has still an issue which is not resolved.

And i already provided you with the exact steps to reproduce this issue, i even sent you my sd card which has this issue. So you already have the "Certain circumstances"...
If we had the exact circumstances and could reliably replicate the issue, do you think I'd still be asking for more information?
Principal Software Engineer at Raspberry Pi (Trading) Ltd.
Contrary to popular belief, humorous signatures are allowed. Here's an example...
“I think it’s wrong that only one company makes the game Monopoly.” – Steven Wright

User avatar
bensimmo
Posts: 4187
Joined: Sun Dec 28, 2014 3:02 pm
Location: East Yorkshire

Re: Raspberry Pi 3 B+ lockups

Mon Jun 18, 2018 8:00 pm

I've just had a look through your posts and cannot see a mention of the router etc you are using, or if you are running through other switches/powerlines or a direct connection.
Is the router at default setting, does it have some other option been enabled.
Somebody else may be able to recreate your setup and try it out.



It may/may not give another reference for them.
I do know if it'll help but your other option is to send the 3B+'s back as not working and use you 3Bs or something else.

Have you tried turning EEE off?
Adding
dtparam=eee=off
to /boot/config.txt
?

wybielacz
Posts: 29
Joined: Wed Nov 06, 2013 9:37 pm

Re: Raspberry Pi 3 B+ lockups

Mon Jun 18, 2018 8:10 pm

bensimmo wrote:
Mon Jun 18, 2018 8:00 pm
I've just had a look through your posts and cannot see a mention of the router etc you are using, or if you are running through other switches/powerlines or a direct connection.
Is the router at default setting, does it have some other option been enabled.
Somebody else may be able to recreate your setup and try it out.



It may/may not give another reference for them.
I do know if it'll help but your other option is to send the 3B+'s back as not working and use you 3Bs or something else.

Have you tried turning EEE off?
Adding
dtparam=eee=off
to /boot/config.txt
?

So i have 12 devices connected to my router (LAN and WiFi) which is a netgear x4s with default settings, none of those devices ever made problems or experienced overnight crashes. It is a direct connection, no switches or powerlines. Also my 3B which i connect to that router using the same sd card from the 3B+ survives the night. The 3B+ survives the night when i connect it using WiFi. I can safely assume that the router is perfectly ok and the only variable here is the ethernet chip from the 3B+.

jamesh wrote:
Mon Jun 18, 2018 7:37 pm
If we had the exact circumstances and could reliably replicate the issue, do you think I'd still be asking for more information?

I cannot serve you the issue any better then sending you the sd card with the circumstances ...
If there would not be any issue like you say, do you think i would still waste my time here?

User avatar
bensimmo
Posts: 4187
Joined: Sun Dec 28, 2014 3:02 pm
Location: East Yorkshire

Re: Raspberry Pi 3 B+ lockups

Mon Jun 18, 2018 8:26 pm

I'm assuming this github thread seems to be the main ethernet debugging thread similar to this?
https://github.com/raspberrypi/linux/issues/2482

jamesh
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 24168
Joined: Sat Jul 30, 2011 7:41 pm

Re: Raspberry Pi 3 B+ lockups

Tue Jun 19, 2018 7:32 am

bensimmo wrote:
Mon Jun 18, 2018 8:26 pm
I'm assuming this github thread seems to be the main ethernet debugging thread similar to this?
https://github.com/raspberrypi/linux/issues/2482
Closest one I think.
Principal Software Engineer at Raspberry Pi (Trading) Ltd.
Contrary to popular belief, humorous signatures are allowed. Here's an example...
“I think it’s wrong that only one company makes the game Monopoly.” – Steven Wright

jamesh
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 24168
Joined: Sat Jul 30, 2011 7:41 pm

Re: Raspberry Pi 3 B+ lockups

Tue Jun 19, 2018 7:33 am

wybielacz wrote:
Mon Jun 18, 2018 8:10 pm
bensimmo wrote:
Mon Jun 18, 2018 8:00 pm
I've just had a look through your posts and cannot see a mention of the router etc you are using, or if you are running through other switches/powerlines or a direct connection.
Is the router at default setting, does it have some other option been enabled.
Somebody else may be able to recreate your setup and try it out.



It may/may not give another reference for them.
I do know if it'll help but your other option is to send the 3B+'s back as not working and use you 3Bs or something else.

Have you tried turning EEE off?
Adding
dtparam=eee=off
to /boot/config.txt
?

So i have 12 devices connected to my router (LAN and WiFi) which is a netgear x4s with default settings, none of those devices ever made problems or experienced overnight crashes. It is a direct connection, no switches or powerlines. Also my 3B which i connect to that router using the same sd card from the 3B+ survives the night. The 3B+ survives the night when i connect it using WiFi. I can safely assume that the router is perfectly ok and the only variable here is the ethernet chip from the 3B+.

jamesh wrote:
Mon Jun 18, 2018 7:37 pm
If we had the exact circumstances and could reliably replicate the issue, do you think I'd still be asking for more information?

I cannot serve you the issue any better then sending you the sd card with the circumstances ...
If there would not be any issue like you say, do you think i would still waste my time here?
It won't be the SD card, there's probably an issue in the driver somewhere, so the priority is to replicate the issue in house so we can debug it.
Principal Software Engineer at Raspberry Pi (Trading) Ltd.
Contrary to popular belief, humorous signatures are allowed. Here's an example...
“I think it’s wrong that only one company makes the game Monopoly.” – Steven Wright

User avatar
bensimmo
Posts: 4187
Joined: Sun Dec 28, 2014 3:02 pm
Location: East Yorkshire

Re: Raspberry Pi 3 B+ lockups

Tue Jun 19, 2018 8:04 am

James, I think you do have one of his SD cards in a Pi3B+ he sent you (he wasn't saying it was the SD card)

It's obviously the 3B+ Ethernet chipsets having problems (soft or hard), but it may also be the way it interacts with some methods from the routers ethernet chipset, which can be found here (V1 and V2 use the same https://wikidevi.com/wiki/Netgear_R7500 ) or it's software.
Of course, that also means you should be able to check the router logs and not just the Pi's, it might need to be made more descriptive or good old ssh into it and see what the router says is happening before, at and after the connection/Pi3B+ freezes.


Anyways, just trying to help, but github is where the debugging and main team will see the information.

jamesh
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 24168
Joined: Sat Jul 30, 2011 7:41 pm

Re: Raspberry Pi 3 B+ lockups

Tue Jun 19, 2018 8:53 am

bensimmo wrote:
Tue Jun 19, 2018 8:04 am
James, I think you do have one of his SD cards in a Pi3B+ he sent you (he wasn't saying it was the SD card)

It's obviously the 3B+ Ethernet chipsets having problems (soft or hard), but it may also be the way it interacts with some methods from the routers ethernet chipset, which can be found here (V1 and V2 use the same https://wikidevi.com/wiki/Netgear_R7500 ) or it's software.
Of course, that also means you should be able to check the router logs and not just the Pi's, it might need to be made more descriptive or good old ssh into it and see what the router says is happening before, at and after the connection/Pi3B+ freezes.


Anyways, just trying to help, but github is where the debugging and main team will see the information.
I doubt the SD card will help - I believe it's pretty standard Raspbian. We are aware it could be environmental (i.e. what its plugged in to), it's how the EEE bug occurred (long cables), and the TCP segmentation offload (required a dropped packet at a particular time, so a slightly less that perfect network). Basically, finding Ethernet issues is an utter PITA unless we can narrow down the exact circumstances where they occur.
Principal Software Engineer at Raspberry Pi (Trading) Ltd.
Contrary to popular belief, humorous signatures are allowed. Here's an example...
“I think it’s wrong that only one company makes the game Monopoly.” – Steven Wright

6by9
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 7570
Joined: Wed Dec 04, 2013 11:27 am
Location: ZZ9 Plural Z Alpha, aka just outside Cambridge.

Re: Raspberry Pi 3 B+ lockups

Tue Jun 19, 2018 11:07 am

jamesh wrote:
Tue Jun 19, 2018 7:32 am
bensimmo wrote:
Mon Jun 18, 2018 8:26 pm
I'm assuming this github thread seems to be the main ethernet debugging thread similar to this?
https://github.com/raspberrypi/linux/issues/2482
Closest one I think.
TBH If it's rebooting the Pi then it's a new issue. #2482 and #2449 were NFS and Samba (and anything else using TCP) stalling due to TCP never resending a dropped packet. Neither resulted in a reboot.

There is certainly something in the general setup of wybielacz's Pis/network that is causing issue. For all we know there may be another device on the network with a cron job that SSHs into a fixed IP address and reboots it. Switching to Wifi results in a different IP address and is therefore stable. From the information we have been given it fits all the conditions, but is unlikely to be the cause.
(I would suggest trying changing the password or disabling SSH to confirm that it isn't external over SSH).

@wybielacz You say it's a Netgear X4S router - which one of the 3 models that Netgear call X4S? C7800, EX7500, or R7800?
If you have an expectation of us fixing an issue without giving us all the information to be able to reproduce it, then your expectations are seriously unrealistic.

Very few software events will trigger a full reboot. A kernel panic normally locks the device up so that the debug is still present when you come back to look at the device.
Software Engineer at Raspberry Pi Trading. Views expressed are still personal views.
I'm not interested in doing contracts for bespoke functionality - please don't ask.

User avatar
RaTTuS
Posts: 10504
Joined: Tue Nov 29, 2011 11:12 am
Location: North West UK
Contact: Twitter YouTube

Re: Raspberry Pi 3 B+ lockups

Tue Jun 19, 2018 12:06 pm

if things are getting rebooted then I'd look to a brown out situation on the PSU,
if the Ethernet chip is getting hot then it may be drawing lots of power because of reasons and causing a brown out ....

[anecdote]
13years back when we moved offices we took all our equipment with us PC's servers switches and set them up in the new place
all the equipment was the same
the wiring was different new office was wired to Cat 6 that was the only change

our Dell Dimension 9200's where all in place as workstations - everyone was happy

at the end the first full day off new office work , people left and shut down their machines

the network ground to a halt [near halt , everything was soooooo slooooow]

I went and power cycled a few machines and it all came back OK , I left for the night

...
next day everything was fine until home time and it happened again, I though it must of been one of a couple of machines , when I switched them back on the network comeback

the next day this did not work - it was another machine that caused the problem

this went on for a couple of weeks until we just got everyone to leave their machines on , we got dell to come and investigate as they where not able to reproduce the effect in their offices. they went away with some info , we got add in NIC's and disabled the on board ones ..
after 8months ? dell confirmed an issue with the onboard NIC's but was unable to offer a BIOS fix it was problem low down .... and very dependant of various things [possibly in the shutdown sequence of the NIC]

debugging of stuff is harder than you think sometimes - especially with hardware

[we also got hit by the Fdiv bug in 1994 late on in November / December had a couple of our programmers going mental for ages]

* some details may be miss remembered
[/anecdote]
How To ask Questions :- http://www.catb.org/esr/faqs/smart-questions.html
WARNING - some parts of this post may be erroneous YMMV

1QC43qbL5FySu2Pi51vGqKqxy3UiJgukSX
Covfefe

5ft24dave
Posts: 41
Joined: Tue Jun 12, 2018 3:38 am

Re: Raspberry Pi 3 B+ lockups

Tue Jun 19, 2018 1:37 pm

The situation with mine wasn't the power supply for the rebooting. Running a 5.2v 2.4A supply (raspberry pi recommended supply from adafruit). Pi also was tried with and without the UPS-pico+ HV450 hat. Voltage at the GPIO 5v pins was monitored, was seeing a solid 5.1-5.2v. only time it dropped before that was when running the CPU stress a53 program. It would drop to between 4.8 and 4.9 v, but never crashed or rebooted. I had a couple reboots with sdram_freq=450. Never anything in any logs. Everything would be normal then the next line would be the normal boot messages. Mine was plugged into my Cisco DPC3941T gateway (supplied by Comcast/Xfinity) v1. Gigabit port. I also have a pi 2B+ and my PC plugged into the gigabit ports. Everything else is WiFi. What cured mine was to go to WiFi and not use Ethernet. I tried the EEE off, offloading the tx and rx, and added over voltage =3 for sdram and core. Never had a crash/lockup, always just reboots. Everything in config.txt is back to defaults, and it's been up 4 days without issues, and has been printing for most of that time (octoprint running my Anet A8). So I can see that this is definitely an Ethernet issue in one way or another. I'd love to get back to using Ethernet, but until we can find a fix, I'm going to have to stay on wifi.

jamesh
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 24168
Joined: Sat Jul 30, 2011 7:41 pm

Re: Raspberry Pi 3 B+ lockups

Tue Jun 19, 2018 2:20 pm

5ft24dave wrote:
Tue Jun 19, 2018 1:37 pm
The situation with mine wasn't the power supply for the rebooting. Running a 5.2v 2.4A supply (raspberry pi recommended supply from adafruit). Pi also was tried with and without the UPS-pico+ HV450 hat. Voltage at the GPIO 5v pins was monitored, was seeing a solid 5.1-5.2v. only time it dropped before that was when running the CPU stress a53 program. It would drop to between 4.8 and 4.9 v, but never crashed or rebooted. I had a couple reboots with sdram_freq=450. Never anything in any logs. Everything would be normal then the next line would be the normal boot messages. Mine was plugged into my Cisco DPC3941T gateway (supplied by Comcast/Xfinity) v1. Gigabit port. I also have a pi 2B+ and my PC plugged into the gigabit ports. Everything else is WiFi. What cured mine was to go to WiFi and not use Ethernet. I tried the EEE off, offloading the tx and rx, and added over voltage =3 for sdram and core. Never had a crash/lockup, always just reboots. Everything in config.txt is back to defaults, and it's been up 4 days without issues, and has been printing for most of that time (octoprint running my Anet A8). So I can see that this is definitely an Ethernet issue in one way or another. I'd love to get back to using Ethernet, but until we can find a fix, I'm going to have to stay on wifi.
I don't understand the reboot thing. As stated elsewhere, if there is a kernel problem, it will lock up and not reboot. The only things I can think of that could cause reboot is a power brownout/failure, or some sort of perhaps malicious software command, or faulty HW. Why this would be related to ethernet is currently rather inexplicable.
Principal Software Engineer at Raspberry Pi (Trading) Ltd.
Contrary to popular belief, humorous signatures are allowed. Here's an example...
“I think it’s wrong that only one company makes the game Monopoly.” – Steven Wright

User avatar
RaTTuS
Posts: 10504
Joined: Tue Nov 29, 2011 11:12 am
Location: North West UK
Contact: Twitter YouTube

Re: Raspberry Pi 3 B+ lockups

Tue Jun 19, 2018 2:31 pm

5ft24dave wrote:
Tue Jun 19, 2018 1:37 pm
... Cisco DPC3941T gateway....
my betting it is this that is causing issues and even a NIC switch may help in between
How To ask Questions :- http://www.catb.org/esr/faqs/smart-questions.html
WARNING - some parts of this post may be erroneous YMMV

1QC43qbL5FySu2Pi51vGqKqxy3UiJgukSX
Covfefe

ejolson
Posts: 3825
Joined: Tue Mar 18, 2014 11:47 am

Re: Raspberry Pi 3 B+ lockups

Tue Jun 19, 2018 3:18 pm

RaTTuS wrote:
Tue Jun 19, 2018 12:06 pm
debugging of stuff is harder than you think sometimes - especially with hardware
I had an issue with Kingston networking cards: The first generation worked great but they revised the design without changing the product number. The revised cards caused system crashes and spontaneous reboots. It took about a week to catch on to what was happening; I remember seeing similar complaints by Google employees who were also building clusters on the mailing lists--those were early days. The loss of reputation these faulty cards produced are likely one of the reasons Kingston got out of the networking business.

I don't think people would deny that debugging hardware and networking issues can be complex. The point appears to be that a computer as simple as the Pi from a company as reputable as the Raspberry Pi Foundation should already be debugged before shipping. Moreover, even if it is not compatible with all networking equipment, it shouldn't just crash or reboot.

So far my Pi 3B+ is stable. The performance of the WiFi access point to WiFi masquerading firewall is disappointing, but I don't have anything to compare it to at this location. Ethernet to WiFi seems fine. I'll be doing some stuff with Samba soon and will report back if anything strange happens.

jamesh
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 24168
Joined: Sat Jul 30, 2011 7:41 pm

Re: Raspberry Pi 3 B+ lockups

Tue Jun 19, 2018 3:41 pm

ejolson wrote:
Tue Jun 19, 2018 3:18 pm
RaTTuS wrote:
Tue Jun 19, 2018 12:06 pm
debugging of stuff is harder than you think sometimes - especially with hardware
I had an issue with Kingston networking cards: The first generation worked great but they revised the design without changing the product number. The revised cards caused system crashes and spontaneous reboots. It took about a week to catch on to what was happening; I remember seeing similar complaints by Google employees who were also building clusters on the mailing lists--those were early days. The loss of reputation these faulty cards produced are likely one of the reasons Kingston got out of the networking business.

I don't think people would deny that debugging hardware and networking issues can be complex. The point appears to be that a computer as simple as the Pi from a company as reputable as the Raspberry Pi Foundation should already be debugged before shipping. Moreover, even if it is not compatible with all networking equipment, it shouldn't just crash or reboot.

So far my Pi 3B+ is stable. The performance of the WiFi access point to WiFi masquerading firewall is disappointing, but I don't have anything to compare it to at this location. Ethernet to WiFi seems fine. I'll be doing some stuff with Samba soon and will report back if anything strange happens.
We generally have some sort of prototypes up to a year in advance, although I think it was 6 months in the 3b+ case. We test them in that time. And yet we still had not encountered these issues. Not only that but we are not the only users of these drivers, or the HW block used in the chip (the chip is new, but uses existing HW blocks, so a repackage job) so one would have hoped that after being available for some years all the bugs would have been weeded out by previous users. Seems to not be the case. Like I said, this reboot thing is odd, we've not seen it in any testing ever.

And the Pi is not a simple computer! Look at the feature set. Low cost does not mean simple.

As for wifi performance, what sort of problems are you seeing? Might be worth starting a new thread for that.
Principal Software Engineer at Raspberry Pi (Trading) Ltd.
Contrary to popular belief, humorous signatures are allowed. Here's an example...
“I think it’s wrong that only one company makes the game Monopoly.” – Steven Wright

5ft24dave
Posts: 41
Joined: Tue Jun 12, 2018 3:38 am

Re: Raspberry Pi 3 B+ lockups

Tue Jun 19, 2018 4:12 pm

RaTTuS wrote:
Tue Jun 19, 2018 2:31 pm
5ft24dave wrote:
Tue Jun 19, 2018 1:37 pm
... Cisco DPC3941T gateway....
my betting it is this that is causing issues and even a NIC switch may help in between
I do have a little 8 port Netgear Gigabit switch I can put in there and try.
I'll do that tonight after work and see what happens

ejolson
Posts: 3825
Joined: Tue Mar 18, 2014 11:47 am

Re: Raspberry Pi 3 B+ lockups

Tue Jun 19, 2018 8:04 pm

ejolson wrote:
Tue Jun 19, 2018 3:18 pm
I'll be doing some stuff with Samba soon and will report back if anything strange happens.
Instead of Samba I setup an NFS server on the Pi 3B+ and mounted the export on the Linux PC that is connected to the Pi through wired networking. Performing the command

$ find . -print0 | xargs -0 cat >/dev/null

on the PC for the NFS mount freezes in seconds but not always at the same place. Note that the Pi didn't freeze, just the NFS mount. After I turned off transmission TCP segmentation offload on the Pi with the command

# ethtool -K eth0 tx-tcp-segmentation off

then the command

$ find . -print0 | xargs -0 cat >/dev/null

runs fine. While this is probably well known, I'm posting my results as another data point, in case anyone else is experiencing similar problems.

Proxx
Posts: 6
Joined: Thu Jun 07, 2018 6:22 pm

Re: Raspberry Pi 3 B+ lockups

Tue Jun 26, 2018 10:21 pm

Woohoo 12days uptime after setting ram to 450

my question now is, can i get a replacement?

This board is out of spec isnt it?

Return to “Troubleshooting”