Page 1 of 2
Scripting woes, cannot parse inut values properly..
Posted: Wed Apr 15, 2020 9:22 am
by Bosse_B
I am trying to multi-purpose a script I have used to install a development IDE such that I can command it to select parts of the install depending on an input argument.
I have studied a lot of tutorials on bash scripting but I simply cannot get this to work..
Please tell me what I am doing wrong in this simple example:
Code: Select all
#!/bin/sh
a=$1
b="help"
c="fpc"
d="laz"
if [ $a == $b ]
then
echo "Arg1 is equal to $a"
elif [ $a == $c ]
then
echo "Arg1 is equal to $a"
elif [ $a == $d ]
then
echo "Arg1 is equal to $a"
else
echo "None of the condition met"
fi
exit
I have tested all valid and invalid arguments but the same response is output:
Code: Select all
./testscript.sh fpc
./testscript.sh: 8: [: fpc: unexpected operator
./testscript.sh: 11: [: fpc: unexpected operator
./testscript.sh: 14: [: fpc: unexpected operator
None of the condition met
The code is an adaptation of the bottom example on
this tutorial page.
I have stared at this for a long time now, not understanding what I am doing wrong...
Must be something simple.
Re: Scripting woes, cannot parse inut values properly..
Posted: Wed Apr 15, 2020 9:52 am
by scotty101
I copy/pasted your example and it worked as expected.
Hoping someone else can confirm this.
Re: Scripting woes, cannot parse inut values properly..
Posted: Wed Apr 15, 2020 10:17 am
by epoch1970
On Raspbian:
Code: Select all
./foo.sh tada
None of the condition met
pi@red:~ $ cat foo.sh
#!/bin/sh
a=$1
b="help"
c="fpc"
d="laz"
if [ $a = $b ]
then
echo "Arg1 is equal to $a"
elif [ $a = $c ]
then
echo "Arg1 is equal to $a"
elif [ $a = $d ]
then
echo "Arg1 is equal to $a"
else
echo "None of the condition met"
fi
exit
Note the string equality operator: "=" not "==".
You should use a case statement anyway I think.
EDIT. Case statement version
Code: Select all
$ ./foo.sh tada
Arg1 [tada] does not match [help]
Arg1 [tada] does not match [fpc]
Arg1 [tada] does not match [laz]
At the end [match=0]
pi@red:~ $ ./foo.sh help
Arg1 [help] is equal to [help]
At the end [match=1]
pi@red:~ $ cat foo.sh
#!/bin/sh
options='help fpc laz'
in="${1:-}"
seen=0
for option in ${options}; do
case "${in}" in
"${option}")
echo "Arg1 [${in}] is equal to [${option}]"
seen=1
break
;;
*)
echo "Arg1 [${in}] does not match [${option}]"
;;
esac
done
echo "At the end [match=${seen}]"
exit 0
Re: Scripting woes, cannot parse inut values properly..
Posted: Wed Apr 15, 2020 11:15 am
by rpdom
As epoch1970 said, you should replace the "==" with "=" in Bash.
Also, I would put quotes around the test arguments like
Code: Select all
if [ "$a" = "$b" ]
then
echo "Arg1 is equal to $a"
(etc...)
That will prevent the script dropping out with an error if there are no arguments.
As suggested, the "case" method might be a better way of handling the tests.
Re: Scripting woes, cannot parse inut values properly..
Posted: Wed Apr 15, 2020 12:11 pm
by jbudd
Linux comes with many different shells, I think you may be running your script with dash, which has less functionality than bash.
On my Pi running raspbian
ls -l /bin/sh gives
lrwxrwxrwx 1 root root 4 Sep 26 2019 /bin/sh -> dash
I suggest you change your first line to
#! /bin/bash
Your script works for me with bash.
Re: Scripting woes, cannot parse inut values properly..
Posted: Wed Apr 15, 2020 1:10 pm
by Bosse_B
Thanks all!
I think I will use the case structure to handle the various commands. Did not know it was available.
It is very confusing when to use == and when not to...
As well as how to nest if, else, elif, fi etc...
Question regarding case usage (I have looked up instructions on the net):
I want to convert an installation script that handles 3 different tasks plus system updates via apt to install a development IDE (Freepascal + Lazarus). The script has been in use since 4 years or so on all of my RPi units.
But whenever I want to update a part of the system I need to manually execute the commands needed.
So I figured I could modify the script to take arguments, which will make possible to install all, fpc only, lazarus only, help system, install/update dependencies only etc.
So if I use a command line argument to differentiate these functions it seems like I also have to put the command sections into functions callable from the case structure, right?
I hope variables are globally available so they can be used by these functions when they are called?
Structure:
- function definitions for all of the processes
- Prepare all of the needed variables and constants
- Read input argument, if none print usage and exit
- case structure where the input argument value causes a specific set of function calls to be executed
- finish
Does this look at all reasonable?
Do I have to put all of the function definitions on top before the actual code execution starts or can they be located further down after the initializations that have to occur every time?
Re: Scripting woes, cannot parse inut values properly..
Posted: Wed Apr 15, 2020 2:09 pm
by epoch1970
Something like that?
Code: Select all
pi@red:~ $ ./foo.sh help ; echo $?
Arg1 [help] is equal to [help] : call do_help
Arg1 [help] -> running do_help
Normal end [executed=1]
0
pi@red:~ $ ./foo.sh bar ; echo $?
Arg1 [bar] does not match [help]
Arg1 [bar] does not match [fpc]
Arg1 [bar] does not match [laz]
Usage: ./foo.sh [help] [fpc] [laz] | [all]
1
pi@red:~ $ ./foo.sh all ; echo $?
Arg1 [all] -> running do_help
Arg1 [all] -> running do_fpc
Arg1 [all] -> running do_laz
Normal end [executed=3]
0
pi@red:~ $ cat ./foo.sh
#!/bin/sh
options='help fpc laz'
in="${1:-}"
task_done=0
do_usage(){
local pretty="[$(echo ${options} | sed -s 's/ /] [/g')]"
echo "Usage: ${0} ${pretty} | [all]"
exit 1
}
do_help(){
echo "Arg1 [${in}] -> running do_help"
}
do_fpc(){
echo "Arg1 [${in}] -> running do_fpc"
}
do_laz(){
echo "Arg1 [${in}] -> running do_laz"
}
for option in ${options}; do
case "${in}" in
"${option}")
echo "Arg1 [${in}] is equal to [${option}] : call do_${option}"
do_${option}
task_done=$(( task_done + 1 ))
break
;;
'all')
do_${option}
task_done=$(( task_done + 1 ))
;;
*)
echo "Arg1 [${in}] does not match [${option}]"
;;
esac
done
[ ${task_done} -eq 0 ] && do_usage
echo "Normal end [executed=${task_done}]"
exit 0
Re: Scripting woes, cannot parse inut values properly..
Posted: Wed Apr 15, 2020 2:10 pm
by jbudd
You have to define functions before you can call them but they can be below your code to handle input.
EDIT - Epoch1970 already did similar!
Here is a way for your script to install perform one or more functions by passing command line options:
Bash includes getopt to handle command line arguments.
The following script can be called with -f (install fpc) , -l (install lazarus) , -fl (both) or -l -f (also both)
Code: Select all
#!/bin/bash
USAGE="Usage: $0 [OPTION]
Options: -f Install fpc
-l Install lazarus"
fpc=false
lazarus=false
while getopts "fl" opt; do
case "$opt" in
f) fpc=true
echo "Will install fpc"
;;
l) lazarus=true
echo "Will install lazarus"
;;
*)
echo $USAGE
exit 1
;;
esac
done
function install_lazarus {
echo "installing lazarus..."
}
function install_fpc {
echo "installing fpc..."
}
if $lazarus
then
install_lazarus
fi
if $fpc
then
install_fpc
fi
Re: Scripting woes, cannot parse inut values properly..
Posted: Wed Apr 15, 2020 3:08 pm
by Bosse_B
Thanks, I will look over the existing script, currently it is 231 lines (lots of comments).
Good ideas here!
Re: Scripting woes, cannot parse inut values properly..
Posted: Wed Apr 15, 2020 4:55 pm
by epoch1970
I give you the Boredom Edition.
This one uses -h and -m switches, it can handle multiple operations and keep them unique and in order.
EDIT: doesn’t require getopts.
Code: Select all
pi@red:~ $ ./foo.sh ; echo "$?"
Nothing to do.
Usage: ./foo.sh -m <module|all> | -h
-h: prints this message
-m <module|all>: install module <module> or all modules
This switch can be repeated. Ordering is handled by the script
Known modules:
help : The help module. Use it often.
fpc : The FreePascal module.
laz : The Lazarus module.
1
pi@red:~ $ ./foo.sh -m foo bar -m all -m help; echo "$?"
Tasks to execute: [help fpc laz]
Running do_help
Running do_fpc
Running do_laz
At the end
0
pi@red:~ $ ./foo.sh -m laz -m laz -h; echo "$?"
Usage: ./foo.sh -m <module|all> | -h
-h: prints this message
-m <module|all>: install module <module> or all modules
This switch can be repeated. Ordering is handled by the script
Known modules:
help : The help module. Use it often.
fpc : The FreePascal module.
laz : The Lazarus module.
0
pi@red:~ $ ./foo.sh -m laz -m laz -m laz -m help; echo "$?"
Tasks to execute: [help laz]
Running do_help
Running do_laz
At the end
0
pi@red:~ $ ./foo.sh -m alleluia -m lazy -m fpcwhat?-m-m ; echo "$?"
Nothing to do.
Usage: ./foo.sh -m <module|all> | -h
-h: prints this message
-m <module|all>: install module <module> or all modules
This switch can be repeated. Ordering is handled by the script
Known modules:
help : The help module. Use it often.
fpc : The FreePascal module.
laz : The Lazarus module.
1
pi@red:~ $ cat foo.sh
#!/bin/sh -u
args="${@:-}"; tasks_todo=
ordered_tasks='help fpc laz'
print_usage(){
cat << EOF
Usage: ${0} -m <module|all> | -h
-h: prints this message
-m <module|all>: install module <module> or all modules
This switch can be repeated. Ordering is handled by the script
Known modules:
EOF
local help_usage='The help module. Use it often.'
local fpc_usage='The FreePascal module.'
local laz_usage='The Lazarus module.'
for task in $ordered_tasks; do
printf "%-5s: %s\n" ${task} "$(eval echo \$${task}_usage)"
done
}
order_tasks(){
for task in ${ordered_tasks} 'all'; do
case " ${args} " in # Padding space is important! :(
*' -m all '*) tasks_todo="${ordered_tasks}" && break ;;
*" -m ${task} "*) tasks_todo="${tasks_todo} ${task}" ;;
esac
done
tasks_todo="$(echo ${tasks_todo} | xargs)" # remove redundant space characters
}
# Process args
case " ${args} " in # Padding space is important! :(
*' -h '*) print_usage && exit 0 ;;
*' -m '*) order_tasks ;;
esac
if [ -z "${tasks_todo}" ]; then
echo "Nothing to do."
print_usage
exit 1
fi
# Exec tasks
do_help(){
echo " Running do_help"
}
do_fpc(){
echo " Running do_fpc"
}
do_laz(){
echo " Running do_laz"
}
echo "Tasks to execute: [${tasks_todo}]"
for task in ${tasks_todo}; do do_${task}; done
echo "At the end"
exit 0
pi@red:~ $
Re: Scripting woes, cannot parse inut values properly..
Posted: Wed Apr 15, 2020 10:41 pm
by Bosse_B
It seems like I am hunted by ghosts when writing scripts...
I cannot get the getopts function to work properly.
So I have shifted to a test script just in order to get the argument parsing with getopts working.
But it is not. Here is my parsetest.sh script:
Code: Select all
#!/bin/bash
function showhelp
{
echo "Valid arguments:"
echo " -a Install all below from scratch"
echo " -f Install Free Pascal"
echo " -l Install Lazarus"
echo " -h Install help files"
echo " -u Update system and needed packages"
echo "Arguments can be combined like so: -lhu"
}
function checkarguments
{
#Now parse argument list
echo "Parsing arguments"
while getopts "aflhu" c; do
case "${c}" in
a ) cmdall="all"
;;
f ) cmdfpc="fpc"
;;
l ) cmdlaz="laz"
;;
h ) cmdhlp="hlp"
showhelp
;;
u ) cmdupd="upd"
;;
esac
done
}
cmdall="a"
cmdfpc="f"
cmdlaz="l"
cmdhlp="h"
cmdupd="u"
checkarguments
echo "$cmdall $cmdfpc $cmdlaz $cmdhlp $cmdupd"
And here is the response:
Code: Select all
$ ./parsetest.sh -h
Parsing arguments
a f l h u
As you can see the call to checkarguments does not find the argument -h being present...
What am I doing wrong here?
I have tried both of these but they give the same result (none):
Code: Select all
while getopts "aflhu" c; do
case "${c}" in
and
while getopts "aflhu" c; do
case "$c" in
No difference in result...
Re: Scripting woes, cannot parse inut values properly..
Posted: Wed Apr 15, 2020 10:58 pm
by jbudd
It seems not to like getopts being in a function. Possibly the arguments list is reset for a function call, though I'm not aware of that.
Try this:
Code: Select all
#!/bin/bash
function showhelp
{
echo "Valid arguments:"
echo " -a Install all below from scratch"
echo " -f Install Free Pascal"
echo " -l Install Lazarus"
echo " -h Install help files"
echo " -u Update system and needed packages"
echo "Arguments can be combined like so: -lhu"
}
cmdall="a"
cmdfpc="f"
cmdlaz="l"
cmdhlp="h"
cmdupd="u"
#Now parse argument list
echo "Parsing arguments"
while getopts "aflhu" c; do
case "${c}" in
a ) cmdall="all"
;;
f ) cmdfpc="fpc"
;;
l ) cmdlaz="laz"
;;
h ) cmdhlp="hlp"
showhelp
;;
u ) cmdupd="upd"
;;
*) showhelp
exit 1
;;
esac
done
echo "$cmdall $cmdfpc $cmdlaz $cmdhlp $cmdupd"
Edit - you can use getopts in a function, see
https://stackoverflow.com/questions/166 ... h-function but it's a much more complicated example than mine.
ps Even if you get it to work, it's horribly impenetrable code. Just parse the arguments in the main script flow as above.
Re: Scripting woes, cannot parse inut values properly..
Posted: Wed Apr 15, 2020 11:18 pm
by Greg Erskine
I think a ) should be a)
c doesn't equal "a " but "a"
Re: Scripting woes, cannot parse inut values properly..
Posted: Wed Apr 15, 2020 11:23 pm
by jbudd
Greg Erskine wrote: ↑Wed Apr 15, 2020 11:18 pm
I think a ) should be a)
c doesn't equal "a " but "a"
Good thought but I believe whitespace is ignored.
Re: Scripting woes, cannot parse inut values properly..
Posted: Thu Apr 16, 2020 12:18 am
by Greg Erskine
You are correct! Thanks.
Re: Scripting woes, cannot parse inut values properly..
Posted: Thu Apr 16, 2020 12:37 am
by Greg Erskine
Your code works fine in piCore using sh.
I just cut and pasted and changed bash to sh (shouldn't matter)
Code: Select all
tc@pCPJustBoom:~$ ./parsetest.sh -h
Parsing arguments
Valid arguments:
-a Install all below from scratch
-f Install Free Pascal
-l Install Lazarus
-h Install help files
-u Update system and needed packages
Arguments can be combined like so: -lhu
a f l hlp u
tc@pCPJustBoom:~$ ./parsetest.sh -al
Parsing arguments
all f laz h u
tc@pCPJustBoom:~$ ./parsetest.sh -alu
Parsing arguments
all f laz h upd
tc@pCPJustBoom:~$ ./parsetest.sh -aluh
Parsing arguments
Valid arguments:
-a Install all below from scratch
-f Install Free Pascal
-l Install Lazarus
-h Install help files
-u Update system and needed packages
Arguments can be combined like so: -lhu
all f laz hlp upd
tc@pCPJustBoom:~$ ./parsetest.sh -af
Parsing arguments
all fpc l h u
Re: Scripting woes, cannot parse inut values properly..
Posted: Thu Apr 16, 2020 5:43 am
by Bosse_B
Greg Erskine wrote: ↑Wed Apr 15, 2020 11:18 pm
I think a ) should be a)
c doesn't equal "a " but "a"
I have tried both a ) and a), no difference.
I started out without the space, then I saw the space in one on-line example and tried that, to no avail.
Re: Scripting woes, cannot parse inut values properly..
Posted: Thu Apr 16, 2020 6:34 am
by Bosse_B
jbudd wrote: ↑Wed Apr 15, 2020 10:58 pm
It seems not to like getopts being in a function. Possibly the arguments list is reset for a function call, though I'm not aware of that.
....
Edit - you can use getopts in a function, see
https://stackoverflow.com/questions/166 ... h-function but it's a much more complicated example than mine.
ps Even if you get it to work, it's horribly impenetrable code. Just parse the arguments in the main script flow as above.
Well, it seems you are correct in that getopts does not work from inside a function!
I have tried all other suggestions, for example changing the shebang from bash to sh and other as well.
But no deal.
Then I tried to put the parser in mainline code rather than in a function and it worked!
Here is the now working test example:
Code: Select all
#!/bin/bash
showhelp ()
{
echo "Valid arguments:"
echo " -a Install all below from scratch"
echo " -f Install Free Pascal"
echo " -l Install Lazarus"
echo " -h Install help files"
echo " -u Update system and needed packages"
echo "Arguments can be combined like so: -lhu"
}
cmdall="a"
cmdfpc="f"
cmdlaz="l"
cmdhlp="h"
cmdupd="u"
#Now parse argument list
while getopts ":aflhu" c; do
case "${c}" in
a) cmdall="all"
;;
f) cmdfpc="fpc"
;;
l) cmdlaz="laz"
;;
h) cmdhlp="hlp"
showhelp
exit 0
;;
u) cmdupd="upd"
;;
?) echo "Unknown option -$OPTARG"
showhelp
;;
esac
done
echo "$cmdall $cmdfpc $cmdlaz $cmdhlp $cmdupd"
The : at the beginning of the options list will silence the output from getopts on an unknown option.
The ?) case at the end takes care of the unknown option instead.
Here is some sample output:
Code: Select all
$ ./parsetest.sh -lf
a fpc laz h u
$ ./parsetest.sh -lfx
Unknown option -x
Valid arguments:
-a Install all below from scratch
-f Install Free Pascal
-l Install Lazarus
-h Install help files
-u Update system and needed packages
Arguments can be combined like so: -lhu
a fpc laz h u
Re: Scripting woes, cannot parse inut values properly..
Posted: Thu Apr 16, 2020 9:52 am
by Bosse_B
So I think I nailed it now!
The problem seems to be the way getopts gets its input.
The last argument to the getopts call is
optional and if missing getopts reverts to use the full command line arguments as given by $@.
But this seems
not to work inside a function, so what I did was this:
Code: Select all
#!/bin/bash
showhelp ()
{
echo "Valid arguments:"
echo " -a Install all below from scratch"
echo " -f Install Free Pascal"
echo " -l Install Lazarus"
echo " -h Install help files"
echo " -u Update system and needed packages"
echo "Arguments can be combined like so: -lhu"
}
checkarguments ()
{
#Now parse argument list
while getopts ":aflhu" c $appargs; do
case "${c}" in
a) cmdall="all"
;;
f) cmdfpc="fpc"
;;
l) cmdlaz="laz"
;;
h) cmdhlp="hlp"
showhelp
exit 0
;;
u) cmdupd="upd"
;;
?) echo "Unknown option -$OPTARG"
showhelp
exit 0
;;
esac
done
}
cmdall="a"
cmdfpc="f"
cmdlaz="l"
cmdhlp="h"
cmdupd="u"
appargs=$@
checkarguments
echo "$cmdall $cmdfpc $cmdlaz $cmdhlp $cmdupd"
And now after adding the variable $appargs it is working also in a function!
Seems like if inside a function it is unable to get the input arguments from the command line!
But reading these into a variable in mainline code and add that variable at the end of the call to getopts makes it work!
Re: Scripting woes, cannot parse inut values properly..
Posted: Thu Apr 16, 2020 3:11 pm
by Bosse_B
I have a general scripting question:
Say that I have a variable appargs which is set by
appargs=$@
So now it holds all of the command line options entered when the script started.
When I want to test this (not using getopts) for example like this:
Code: Select all
if [ -z $appargs ]
then
echo "Missing options!"
showhelp
exit 2
fi
But in some howto pages on the net they show quotes like this:
Code: Select all
if [ -z "$appargs" ]
then
echo "Missing option!"
showhelp
exit 2
fi
My question is really
when are quotes needed, accepted or forbidden?
For example in connection with getopts it seems like putting quotes around is forbidden, but the code above works both ways.
So sometimes needed and sometimes not or forbidden, what is the rule here?
Very confused about this...
Re: Scripting woes, cannot parse inut values properly..
Posted: Thu Apr 16, 2020 3:22 pm
by rpdom
Quotes are essential when the contents of the variable may contain a space, might be empty or just not defined.
Code: Select all
a="hello bob"
if [ $a = hello bob ]
won't work and you will get "bash: [: too many arguments", but
Code: Select all
a="hello bob"
if [ "$a" = "hello bob" ]
will work. Also there is
Code: Select all
a="hello bob"
if [ $b = something ]
won't work as it will expand to
and you will get "bash: [: =: unary operator expected" because it expects something before the "=". While
will expand to
which makes sense.
You can leave out the quotes when you are comparing anything without a space in and isn't empty, and when comparing numerics.
Re: Scripting woes, cannot parse inut values properly..
Posted: Thu Apr 16, 2020 4:09 pm
by epoch1970
Bosse_B wrote: ↑Thu Apr 16, 2020 3:11 pm
My question is really
when are quotes needed, accepted or forbidden?
Short answer, quote the hell out of your variables by default, e.g. "${FOO}", and if that breaks try removing the double quotes.
Re: Scripting woes, cannot parse inut values properly..
Posted: Thu Apr 16, 2020 4:18 pm
by jbudd
My question is really when are quotes needed, accepted or forbidden?
For example in connection with getopts it seems like putting quotes around is forbidden, but the code above works both ways.
So sometimes needed and sometimes not or forbidden, what is the rule here?
I'm not clear what you mean by quotes forbidden with getopts.
In general it's good practice to put quotes round a variable as rpdom explains. Sometimes an unquoted but empty variable is fine, sometimes it trips you up.
I confess I often leave them out because they can make it harder to read and understand the line of code
And sometimes you will see squiggly brackets as well $var "$var" ${var}
Re: Scripting woes, cannot parse inut values properly..
Posted: Thu Apr 16, 2020 5:03 pm
by Paeryn
Bosse_B wrote: ↑Thu Apr 16, 2020 9:52 am
The last argument to the getopts call is
optional and if missing getopts reverts to use the full command line arguments as given by $@.
But this seems
not to work inside a function,
...
Seems like if inside a function it is unable to get the input arguments from the command line!
But reading these into a variable in mainline code and add that variable at the end of the call to getopts makes it work!
The reason that
$@ doesn't work for you when used in a function is because inside a function the positional variables
$@,
$#,
$*,
$0,
$1 etc. refer to the
function's arguments rather than the script's.
Re: Scripting woes, cannot parse inut values properly..
Posted: Thu Apr 16, 2020 5:20 pm
by Bosse_B
jbudd wrote: ↑Thu Apr 16, 2020 4:18 pm
My question is really when are quotes needed, accepted or forbidden?
For example in connection with getopts it seems like putting quotes around is forbidden, but the code above works both ways.
So sometimes needed and sometimes not or forbidden, what is the rule here?
I'm not clear what you mean by quotes forbidden with getopts.
In general it's good practice to put quotes round a variable as rpdom explains. Sometimes an unquoted but empty variable is fine, sometimes it trips you up.
I confess I often leave them out because they can make it harder to read and understand the line of code
Well, what I mean is this:
When I had this in my code it did not work:
Code: Select all
checkarguments ()
{
#Now parse argument list
while getopts ":aflhu" c "$appargs"; do
case "${c}" in
a) cmdall="all"
But when I removed the quotes around $appargs it started working:
Code: Select all
checkarguments ()
{
#Now parse argument list
while getopts ":aflhu" c $appargs; do
case "${c}" in
a) cmdall="all"
And sometimes you will see squiggly brackets as well $var "$var" ${var}
That is another thing that has me confused. What do these braces mean?