FIXED: man <command> exits with code 1

[ZPMMaker]

I have a dozen RPi's set up. I updated the software on them yesterday using apt-get update & apt-get upgrade.

Since then, whenever I try to view a command's man page, I see the first page/screen, and then at the bottom of that first page, I see an following error and the man page exits (and I'm presented with the bash command prompt)... this happens on all 12 of the RPi's.

for man date:

Code: Select all

man: command exited with status 1: sed -e '/^[[:space:]]*$/{ N; /^[[:space:]]*\n[[:space:]]*$/D; }' | (cd <fd 17> && LESS=-ix8RmPm Manual page date(1) ?ltline %lt?L/%L.:byte %bB?s/%s..?e (END):?pB %pB\%.. (press h for help or q to quit)$PM M
anual page date(1) ?ltline %lt?L/%L.:byte %bB?s/%s..?e (END):?pB %pB\%.. (press h for help or q to quit)$ MAN_PN=date(1) pager)

for man grep:

man: command exited with status 1: sed -e '/^[[:space:]]*$/{ N; /^[[:space:]]*\n[[:space:]]*$/D; }' | LESS=-ix8RmPm Manual page date(1) ?ltline %lt?L/%L.:byte %bB?s/%s..?e (END):?pB %pB\%.. (press h for help or q to quit)$PM Manual page date
(1) ?ltline %lt?L/%L.:byte %bB?s/%s..?e (END):?pB %pB\%.. (press h for help or q to quit)$ MAN_PN=date(1) pager

for man top:
man: command exited with status 1: (cd /usr/share/man && /usr/lib/man-db/zsoelim) | (cd /usr/share/man && /usr/lib/man-db/manconv -f UTF-8:ISO-8859-1 -t UTF-8//IGNORE) | (cd /usr/share/man && preconv -e UTF-8) | (cd /usr/share/man && tbl) |
(cd /usr/share/man && nroff -mandoc -rLL=234n -rLT=234n -Tutf8)

I tried uninstalling and reinstalling less (as per https://stackoverflow.com/questions/122 ... -1-pager-s) and man-db and manpages, without success.

Code: Select all

$ uname -a
Linux v1iOTS3 4.19.95+ #1292 Thu Jan 16 15:39:51 GMT 2020 armv6l GNU/Linux

Code: Select all

$ cat /etc/*-release
PRETTY_NAME="Raspbian GNU/Linux 9 (stretch)"
NAME="Raspbian GNU/Linux"
VERSION_ID="9"
VERSION="9 (stretch)"
VERSION_CODENAME=stretch
ID=raspbian
ID_LIKE=debian
HOME_URL="http://www.raspbian.org/"
SUPPORT_URL="http://www.raspbian.org/RaspbianForums"
BUG_REPORT_URL="http://www.raspbian.org/RaspbianBugs"

Does anyone know how to fix this issue please?

Thanks,
David

[BeauSlim]

does

Code: Select all

zcat /usr/share/man/man1/top.1.gz | groff -man -T ascii | less

work?

[ZPMMaker]

does

Code: Select all

zcat /usr/share/man/man1/top.1.gz | groff -man -T ascii | less

work?

Interesting.

So I have two separate ways of SSH'ing into the Pi's.

If I SSH into a Pi one way, the above command works perfectly, as do all man pages. Problem is, this option for SSH connection is costly, so I'd prefer to use method 2.

If I use method 2, when I run the above command, I see part of the output, but after one screen-ful, I get the following error:

Code: Select all

grotty:<standard input>:31144:fatal error: output error

I'll follow-up with the developer of "method 2", of course, as the issue seems to only occur when I control the Pi via that method.
That said, if you have any further thoughts on diagnosing the exact problem (or even how to fix it), I'd greatly appreciate it.

Thanks for your help!
Dave

[ZPMMaker]

Just some more potentially-useful information for diagnosing this issue...

Using method 1, I run an SSH command on my computer's command prompt to control the Pi. Controlling via this method, the man pages work fine. This is logged in as user "pi".

Using method 2, I go to a website which presents me with a command prompt displayed within the web browser that controls the Pi. I don't need to run anything - the website sets the connection up for me. However, when I first access the Pi, I have to run a command to switch to user "Pi".

Code: Select all

su -c '/bin/bash -i' pi

Once switched to user 'pi', if I try to access any man page, the error occurs. However, if I access a man page before running the above command to switch to user "pi", then the man page works perfectly - I can use the arrow keys to scroll up/down the man page and I can see the entire thing without any error.

Prior to switching to user "pi", the command prompt reads "root@hostname", so it's logged in as the root user at that point.

Something appears to be causing the error only to occur when logged in as user "Pi", but only through this method of controlling the Pi; if I just use "method 1" (an SSH command I run in my own terminal on my Mac, not through a web browser), it works fine.

[BeauSlim]

A web terminal that logs you in as root sounds really risky. I hope this is only on your local network or you are at least connecting over https://

I would assume that the $TERM variable isn't being set correctly. Do an 'env' or 'echo $TERM' and make sure it matches whatever the web terminal is most compatible with.

It could also be that the web terminal software simply doesn't support "full screen" curses-based programs. "more" might work instead of less. Editing will be a nightmare.

[ZPMMaker]

A web terminal that logs you in as root sounds really risky. I hope this is only on your local network or you are at least connecting over https://

I would assume that the $TERM variable isn't being set correctly. Do an 'env' or 'echo $TERM' and make sure it matches whatever the web terminal is most compatible with.

It could also be that the web terminal software simply doesn't support "full screen" curses-based programs. "more" might work instead of less. Editing will be a nightmare.

Via HTTPS. The web GUI requires a login with username and password, followed by entering a 2FA code, so it's not that bad...

The output of echo $TERM is simply xterm. I have no idea what the software is compatible with but I'll ask the developer.

It could also be that the web terminal software simply doesn't support "full screen" curses-based programs. "more" might work instead of less.

Sorry, what did you mean by "more" working instead of "less". I've never used those commands before...

I was able to run man more and got the gist of what more does, although I could see all the documentation due to this error.
I tried opening one of my scripts: more path/to/script.sh - that output the first page to stdout, but it then presented me with the BASH prompt so I wasn't able to scroll further into the script. Is there a way of scrolling through or will more only show the first page of a file?

Editing will be a nightmare.

Thankfully, I edit all my scripts on my Mac and then upload them to a server. The Pi routinely checks for changes and downloads updates as needed. So I rarely need to edit anything on the Pi directly (usually only when something is completely stuffed ).
That said, I'm able to nano path/to/script.sh and that works fine - I'm able to scroll up and down the script without any issues. This seems isolated to only man pages.

Thanks for your help!

[jojopi]

su -c '/bin/bash -i' pi

In another thread I said that a "tcsetattr" error meant that bash thought it was interactive but had no terminal. Here, you are passing the -i flag to bash, forcing it to be interactive when it presumably does not want to be.

According to the man page, bash should be interactive by default if both standard input and standard error are terminals. This is the obvious thing to check:

Code: Select all

ls -l /proc/$$/fd

Ideally, all four of descriptors 0, 1, 2, and 255 should be the same device file, with a name pts/* or tty*.

I do not see how anything interactive could work without a terminal, though, even as root. Is it possible that the mysterious "developer of option 2" made significant modifications to the system to support this method of login, and the recent "apt upgrade" has undone some of those?

[ZPMMaker]

su -c '/bin/bash -i' pi

In another thread I said that a "tcsetattr" error meant that bash thought it was interactive but had no terminal. Here, you are passing the -i flag to bash, forcing it to be interactive when it presumably does not want to be.

According to the man page, bash should be interactive by default if both standard input and standard error are terminals. This is the obvious thing to check:

Code: Select all

ls -l /proc/$$/fd

Ideally, all four of descriptors 0, 1, 2, and 255 should be the same device file, with a name pts/* or tty*.

I do not see how anything interactive could work without a terminal, though, even as root. Is it possible that the mysterious "developer of option 2" made significant modifications to the system to support this method of login, and the recent "apt upgrade" has undone some of those?

Here's the output of that command:

Code: Select all

:~ $ ls -l /proc/$$/fd                                                                                                                                                                                                                
total 0                                                                                                                                                                                                                                          
lrwx------ 1 pi pi 64 Jan 25 09:58 0 -> /dev/pts/1                                                                                                                                                                                               
lrwx------ 1 pi pi 64 Jan 25 09:59 1 -> /dev/pts/1                                                                                                                                                                                               
lrwx------ 1 pi pi 64 Jan 25 09:59 10 -> 'socket:[5880174]'                                                                                                                                                                                      
lrwx------ 1 pi pi 64 Jan 25 09:59 11 -> 'socket:[13742]'                                                                                                                                                                                        
lrwx------ 1 pi pi 64 Jan 25 09:59 12 -> 'socket:[17513]'                                                                                                                                                                                        
lr-x------ 1 pi pi 64 Jan 25 09:59 13 -> 'pipe:[2498087]'                                                                                                                                                                                        
lrwx------ 1 pi pi 64 Jan 25 09:59 15 -> /dev/ptmx                                                                                                                                                                                               
l-wx------ 1 pi pi 64 Jan 25 09:59 2 -> 'pipe:[5880175]'                                                                                                                                                                                         
l-wx------ 1 pi pi 64 Jan 25 09:59 255 -> 'pipe:[5880175]'                                                                                                                                                                                       
lr-x------ 1 pi pi 64 Jan 25 09:59 3 -> 'pipe:[13252]'                                                                                                                                                                                           
l-wx------ 1 pi pi 64 Jan 25 09:59 4 -> 'pipe:[13252]'                                                                                                                                                                                           
lr-x------ 1 pi pi 64 Jan 25 09:59 5 -> 'pipe:[13253]'                                                                                                                                                                                           
l-wx------ 1 pi pi 64 Jan 25 09:59 6 -> 'pipe:[13253]'                                                                                                                                                                                           
lrwx------ 1 pi pi 64 Jan 25 09:59 7 -> /usr/local/mesh/meshagent.db                                                                                                                                                                             
lrwx------ 1 pi pi 64 Jan 25 09:59 8 -> 'socket:[17501]'                                                                                                                                                                                         
lrwx------ 1 pi pi 64 Jan 25 09:59 9 -> 'socket:[17509]'   

Noting that the path to those numbers is not identical.

I guess it is possible the developer made alterations, but - to be honest - this is all beyond my understanding. I have sent the developer a message and asked them to look at this discussion, hopefully it'll help them determine the issue (unless you have any other ideas based on the above output of the ls command?).

That said, I don't understand how you could possibly run a terminal without it being interactive... the whole point of the terminal - as far as I am aware, anyway - is to be able to enter commands in and see the results, i.e. to interact with it. (?) Could you please explain this to me?

Thanks for your help.
Dave

[scruss]

That said, I don't understand how you could possibly run a terminal without it being interactive... the whole point of the terminal - as far as I am aware, anyway - is to be able to enter commands in and see the results, i.e. to interact with it. (?) Could you please explain this to me?

This is an ancient history convention: a terminal was something that you got output on: a printer, basically. Early ones (like the Model 33 ASR, or slightly later DECwriter LA36) had input keyboards in front of their printers, but there were receive-only variants like the Model 33 RO and the LA35 that lacked keyboards.So when adding terminals to a Unix machine, it really mattered whether the system could expect input from it or not. Hence the need for termcap and terminfo to define what a terminal is and can do, both of which have knocking around for 40 years or so.

The other place that terminal output becomes non-interactive is when you do command | othercommand …. The second command can't send anything back to the first, so it's classed as non-interactive.

I suspect your web terminal isn't setting the interactive modes all the way through its pipeline to your screen. TBH a root shell exposed to the web even over https and 2fa scares the out of me. There's so much ancient stuff layered into terminal handling, plus all the more recent arcana painted on top to support Unicode, that trying to get it into a web page reliably and safely brings the nope.

[ZPMMaker]

That said, I don't understand how you could possibly run a terminal without it being interactive... the whole point of the terminal - as far as I am aware, anyway - is to be able to enter commands in and see the results, i.e. to interact with it. (?) Could you please explain this to me?

This is an ancient history convention: a terminal was something that you got output on: a printer, basically. Early ones (like the Model 33 ASR, or slightly later DECwriter LA36) had input keyboards in front of their printers, but there were receive-only variants like the Model 33 RO and the LA35 that lacked keyboards.So when adding terminals to a Unix machine, it really mattered whether the system could expect input from it or not. Hence the need for termcap and terminfo to define what a terminal is and can do, both of which have knocking around for 40 years or so.

The other place that terminal output becomes non-interactive is when you do command | othercommand …. The second command can't send anything back to the first, so it's classed as non-interactive.

I suspect your web terminal isn't setting the interactive modes all the way through its pipeline to your screen. TBH a root shell exposed to the web even over https and 2fa scares the out of me. There's so much ancient stuff layered into terminal handling, plus all the more recent arcana painted on top to support Unicode, that trying to get it into a web page reliably and safely brings the nope.

Thanks for the explanation

[jojopi]

Code: Select all

:~ $ ls -l /proc/$$/fd
lrwx------ 1 pi pi 64 Jan 25 09:58 0 -> /dev/pts/1
lrwx------ 1 pi pi 64 Jan 25 09:59 1 -> /dev/pts/1
l-wx------ 1 pi pi 64 Jan 25 09:59 2 -> 'pipe:[5880175]'
l-wx------ 1 pi pi 64 Jan 25 09:59 255 -> 'pipe:[5880175]'

So, standard input and output (0 and 1) are connected to a (pseudo)terminal(slave), but standard error (2) is a pipe to some other program. That explains why bash needs to be forced to be interactive. The fact that it has duplicated the pipe as its internal descriptor for the terminal, 255, also suggests that it was unable to open /dev/tty. This causes the tcsetattr error.

In summary, there is a terminal but it is not open on all the right file descriptors, and it may not be a controlling terminal. I also think it is erroneous that so many other descriptors have been left open. At login you have should have just those four.

I can create a similar situation using the command:

Code: Select all

setsid bash -i 2>&1 1>&0 |cat

This reproduces your results: nano still appears to work fine, but man/less crash at the end of the first page.

I think you may see some improvement if you change your command to:

Code: Select all

su -c '/bin/bash -i 2>&1' pi

In fact, I wonder if you might be able to simplify to just:

Code: Select all

su pi 2>&1

When I have been talking about "terminals" I am not really concerned whether the user is sitting at a physical terminal, but whether the programs are communicating with the user via a kernel device driver that implements terminal line discipline.

A user on a physical terminal needs to be able to see what she is typing, so the Linux kernel echoes input back, but programs need to be able to turn echo off when reading passwords or doing full-screen editing. Similarly, the user needs to be able to correct typos by hitting backspace. That is normally handled in-kernel, but can be turned off by programs that do their own editing. Job control, the ability to switch process groups between foreground and background (deciding which program gets the user's input at a particular time) is also a feature of the kernel's terminal driver. The Ctrl+C character is turned into an interrupt signal to the foreground process, in the kernel, again this can be turned off by programs that want to use control characters in their own way.

In short, much of the bevaviour that we expect from interactive programs requires support from the kernel via the terminal device driver. Interactive programs cannot work correctly on pipes or other non-terminal devices.

[ZPMMaker]

Code: Select all

:~ $ ls -l /proc/$$/fd
lrwx------ 1 pi pi 64 Jan 25 09:58 0 -> /dev/pts/1
lrwx------ 1 pi pi 64 Jan 25 09:59 1 -> /dev/pts/1
l-wx------ 1 pi pi 64 Jan 25 09:59 2 -> 'pipe:[5880175]'
l-wx------ 1 pi pi 64 Jan 25 09:59 255 -> 'pipe:[5880175]'

So, standard input and output (0 and 1) are connected to a (pseudo)terminal(slave), but standard error (2) is a pipe to some other program. That explains why bash needs to be forced to be interactive. The fact that it has duplicated the pipe as its internal descriptor for the terminal, 255, also suggests that it was unable to open /dev/tty. This causes the tcsetattr error.

In summary, there is a terminal but it is not open on all the right file descriptors, and it may not be a controlling terminal. I also think it is erroneous that so many other descriptors have been left open. At login you have should have just those four.

I can create a similar situation using the command:

Code: Select all

setsid bash -i 2>&1 1>&0 |cat

This reproduces your results: nano still appears to work fine, but man/less crash at the end of the first page.

I think you may see some improvement if you change your command to:

Code: Select all

su -c '/bin/bash -i 2>&1' pi

In fact, I wonder if you might be able to simplify to just:

Code: Select all

su pi 2>&1

When I have been talking about "terminals" I am not really concerned whether the user is sitting at a physical terminal, but whether the programs are communicating with the user via a kernel device driver that implements terminal line discipline.

A user on a physical terminal needs to be able to see what she is typing, so the Linux kernel echoes input back, but programs need to be able to turn echo off when reading passwords or doing full-screen editing. Similarly, the user needs to be able to correct typos by hitting backspace. That is normally handled in-kernel, but can be turned off by programs that do their own editing. Job control, the ability to switch process groups between foreground and background (deciding which program gets the user's input at a particular time) is also a feature of the kernel's terminal driver. The Ctrl+C character is turned into an interrupt signal to the foreground process, in the kernel, again this can be turned off by programs that want to use control characters in their own way.

In short, much of the bevaviour that we expect from interactive programs requires support from the kernel via the terminal device driver. Interactive programs cannot work correctly on pipes or other non-terminal devices.

You're a genius. Those two options:

Code: Select all

su -c '/bin/bash -i 2>&1' pi

Code: Select all

su pi 2>&1

...both solve the issue.

Thanks heaps for your help! I really appreciate it.
Dave