Bash Scripting

13. Bash Scripting#

So far we have used bash commands to navigate our file system as a way to learn about the file system itself. To do this we used commands like:

mv
cd
pwd
ls

Bash is a unix shell for the GNU operating system and it has been adopted in other contexts as well. It is the default shell on Ubuntu linux as well for example (and many others). This is why we teach it.

A Unix shell is both a command interpreter and a programming language. As a command interpreter, the shell provides the user interface to the rich set of GNU utilities. The programming language features allow these utilities to be combined.

Read the official definition of bash and a shell in the bash manual

Today we will start from the main course directory

cd systems
ls

example				test
github-in-class-brownsarahm-1	testobj.md
kwl				tiny-book

13.1. Variables in Bash#

We can create variables

NAME='Sarah'

Important

notice that there are no spaces around the =. spaces in bash separate commands and options, so they cannot be in variable declarations.

and use them with a $ before the variable name.

echo $NAME

Sarah

This variable is local, in memory, to the current terminal window, so if we open a separate window and try echo $NAME it will not work. We can also see that it does not create any file changes.

ls

example				test
github-in-class-brownsarahm-1	testobj.md
kwl				tiny-book

We can, however use the variable at differnt working directories. So if we move

cd github-in-class-brownsarahm-1/
echo $NAME

Sarah

it still works.

The $ is essential syntax for recalling variables in bash. If we forget it, it treats it as a literal

echo NAME

NAME

so we get the variable name out instead of the varialbe value

A common mistake is to put a space around the = sign, this is actually considered good style in many other languages.

:class: cell_input
NAME ='Sarah'

:class: cell_output
bash: NAME: command not found

In bash, however, this creates an error. When there is a space after NAME, bash tried to interpet NAME as a bash command, but then it does not find it, so it gives an error.

Removing the space works again:

NAME='Sarah'

13.2. Bash Loops#

We can also make loops:

for name in Sarah Amoy Scott Bri
> do
> echo $name
> done

A few important things, to make note of:

loop variable does not need to be an iterator. the loop variable here name takes each value from a list (Sarah Amoy Scott Bri)
lists is bash are defined with no brackets and no commas Sarah Amoy Scott Bri is a list
we start the loop body with do and close it with done these are like the { and } in some langauges.

Sarah
Amoy
Scott
Bri

When we get the command back with the up arrow key, it puts it all on one line, because it was one command. The ; (semicolon) separates the “lines”

for name in Sarah Amoy Scott Bri Aiden; do echo $name; done

Sarah
Amoy
Scott
Bri
Aiden

ls

API.md			about.md		helper_functions.py
CONTRIBUTING.md		abstract_base_class.py	important_classes.py
LICENSE.md		alternative_classes.py	setup.py
README.md		docs			tests

13.3. Nesting commands#

We can run a command to generate the list:

for file in $(ls)
> do 
> echo $file
> done
API.md
CONTRIBUTING.md
LICENSE.md
README.md
about.md
abstract_base_class.py
alternative_classes.py
docs
helper_functions.py
important_classes.py
setup.py
tests

the $() tells bash to run that command first and then hold its output as a variable for use elsewhere

Important

I use this to make the date in the filename of your experience reports and the titles of your badge issues

Find & Comment

find where I use this in one of your action files and add a comment explaining the line. Put this commit in its own PR and request a review from @brownsarahm for a community badge!

We can modify what is in the $():

for file in $(ls -a); do  echo $file; done

.
..
.git
.github
API.md
CONTRIBUTING.md
LICENSE.md
README.md
about.md
abstract_base_class.py
alternative_classes.py
docs
helper_functions.py
important_classes.py
setup.py
tests

13.4. Conditionals in bash#

We can also do conditional statements

if test -f docs
> then
> echo "file"
> fi 

the key parts of this:

test checks if a file or directory exists
the -f option makes it check if the item is a file
what to do if the condition is met goes after a then keyword
the fi (backwardsif) closes the if statment

This outputs nothing because docs is a directory not a file.

If we switch it, we get output:

if test -f API.md; then echo "file"; fi 

file

We can put the if inside of the loop.

for file in $(ls); do if test -f $file;  echo $file; fi; done

bash: syntax error near unexpected token `fi'

I forgot the then so bash said it had bad syntax around fi this is because it was treating everything after the if as the condition and looking for a then but when it got to fi it knew it was too late and the then was not coming because the fi is supposed to be after then.

Once we put then in, it works:

for file in $(ls); do if test -f $file;then  echo $file; fi; done

API.md
CONTRIBUTING.md
LICENSE.md
README.md
about.md
abstract_base_class.py
alternative_classes.py
helper_functions.py
important_classes.py
setup.py

13.5. Script files#

We can put our script into a file

nano filecheck.sh 

So that the file

for file in $(ls)
do 
  if test -f $file
  then
     echo $file
  fi
done

and run it with bash <filename>

bash filecheck.sh 

API.md
CONTRIBUTING.md
LICENSE.md
README.md
about.md
abstract_base_class.py
alternative_classes.py
filecheck.sh
helper_functions.py
important_classes.py
setup.py

cat filecheck.sh 

for file in $(ls)
do 
  if test -f $file
  then
     echo $file
  fi
done

13.6. `gh` CLI operations#

When you are working sometimes it is helpful to be able to manipulate (or create) issues, pull requests or even releases from the command line.

gh issue list

Showing 1 of 1 open issue in introcompsys/github-in-class-brownsarahm-1

#3  Create a Add a classmate    about 1 month ago

We can also get options

gh issue list --help

List issues in a GitHub repository.

The search query syntax is documented here:
<https://docs.github.com/en/search-github/searching-on-github/searching-issues-and-pull-requests>

For more information about output formatting flags, see `gh help formatting`.

USAGE
  gh issue list [flags]

FLAGS
      --app string         Filter by GitHub App author
  -a, --assignee string    Filter by assignee
  -A, --author string      Filter by author
  -q, --jq expression      Filter JSON output using a jq expression
      --json fields        Output JSON with the specified fields
  -l, --label strings      Filter by label
  -L, --limit int          Maximum number of issues to fetch (default 30)
      --mention string     Filter by mention
  -m, --milestone string   Filter by milestone number or title
  -S, --search query       Search issues with query
  -s, --state string       Filter by state: {open|closed|all} (default "open")
  -t, --template string    Format JSON output using a Go template; see "gh help formatting"
  -w, --web                List issues in the web browser

INHERITED FLAGS
      --help                     Show help for command
  -R, --repo [HOST/]OWNER/REPO   Select another repository using the [HOST/]OWNER/REPO format

EXAMPLES
  $ gh issue list --label "bug" --label "help wanted"
  $ gh issue list --author monalisa
  $ gh issue list --assignee "@me"
  $ gh issue list --milestone "The big 1.0"
  $ gh issue list --search "error no:assignee sort:created-asc"

LEARN MORE
  Use 'gh <command> <subcommand> --help' for more information about a command.
  Read the manual at https://cli.github.com/manual

We can filter them to only the closed ones

gh issue list -s closed

Showing 2 of 2 issues in introcompsys/github-in-class-brownsarahm-1 that match your search

#2  Create an about file    about 1 month ago
#1  Create a README         about 1 month ago

It is simlar for PRs:

gh pr list 

no open pull requests in introcompsys/github-in-class-brownsarahm-1

When we use create is is interactive (or you can specify the options if you know them).

gh issue create 

Creating issue in introcompsys/github-in-class-brownsarahm-1

? Title bug
? Body <Received>
? What's next? Submit
https://github.com/introcompsys/github-in-class-brownsarahm-1/issues/7

Now we can see our list of issues

gh issue list -s all

Showing 4 of 4 issues in introcompsys/github-in-class-brownsarahm-1 that match your search

#7  bug                         less than a minute ago
#3  Create a Add a classmate    about 1 month ago
#2  Create an about file        about 1 month ago
#1  Create a README             about 1 month ago

We can also search and filter them by piping the output to grep which searches the contents of a file (including stdin). We previously searched the file names with find. So find searches the paths that exist and grep actually reads the contents of the files, it does so faster than many other languages would be.

gh issue list -s all | grep "Create"

OPEN	Create a Add a classmate		2023-01-31 18:11:59 +0000 UTC
CLOSED	Create an about file		2023-02-02 18:14:19 +0000 UTC
CLOSED	Create a README		2023-01-31 18:18:20 +0000 UTC

grep can be used with pattern matching as well

Tip

Learning more about grep is a good explore badge topic

Here are the options

gh issue create --hlp

unknown flag: --hlp

Usage:  gh issue create [flags]

Flags:
  -a, --assignee login   Assign people by their login. Use "@me" to self-assign.
  -b, --body string      Supply a body. Will prompt for one otherwise.
  -F, --body-file file   Read body text from file (use "-" to read from standard input)
  -l, --label name       Add labels by name
  -m, --milestone name   Add the issue to a milestone by name
  -p, --project name     Add the issue to projects by name
      --recover string   Recover input from a failed run of create
  -t, --title string     Supply a title. Will prompt for one otherwise.
  -w, --web              Open the browser to create an issue
  

13.7. Badge Hints#

Important

The goal is to put files in your kwl repo for us to review, that summarizes the contributions you have done in your group repo

See the options for gh pr list.
Use two strategies for what you are the author and when you are the reviewer
Try the json option on gh pr list and see how it can help you
I use bash in the your experience badge action to make a file with a date in the file name
You can run a file from different locations

cd ../test
bash ../github-in-class-brownsarahm-1/filecheck.sh 

new.txt
test.txt

cd ../../
bash systems/github-in-class-brownsarahm-1/filecheck.sh 

ls

fa22		prog4dssp23	systems

13.8. Review today’s class#

Update your KWL Chart learned column with what you’ve learned
Write a bash script that can generate a file in your KWL repo with a list of all of your contributed PRs. Save the script as groupcontributions.sh and its output as group_contributions-YYYY-MM-DD.md

13.9. Prepare for Next Class#

install putty on windows
spend 10 minutes after break before class, thinking about what you know about networking and how it works, what examples you have of it.

13.10. More Practice#

Update your KWL Chart learned column with what you’ve learned
Write a bash script that can generate a file in your KWL repo with a list of all of your PRs and PR reviews. Save the script as groupcontributions.sh and its output as group_contributions-YYYY-MM-DD.md

13.11. Experience Report Evidence#

13.12. Questions After Today’s Class#

13.12.1. Is there any structure/tools to reverse the order of if and do in a bash loop? Writing “do if” doesn’t feel natural to me.#

The do is a part of the for. We put the if in the loop body, as the first thing in there. It could have been like

for file in $(ls)
do 
  echo checking $file
  if test -f $file
  then
     echo $file is a file
  fi
done

13.12.2. Would the script that we wrote into the file still work with semicolons or would adding them ruin the syntax?#

This is easily testable.

13.12.3. How come we have been using git commands in stead of gh commands?#

git commands are the core things we have been learning about so that you can use git no matter what host you use. gh commands only work with GitHub. They will not work on BitBucket, Gitlab, or any other git host. GitHub is popular, but since Microsoft purcahsed it, some people have left because they do not trust Microsoft.

13.12.4. What else can we use this gh command for?#

Its most important uses are for issues and pull requests

13.12.5. What does the $ mean since it’s used in so many places?#

It is the syntax for using a value from a variable.

13.12.6. Are the API.MD, CONTRIBUTING.MD …etc files important? They have disappeared from my main branch#

No, but I do encourage you to try to get them back. Try using git log to see what might have happened.

13.12.7. Can the script be in any type of file or does it have to be in a certain type of file?#

They must be in plain text but the file extension does not matter.

13.12.8. How is this you said fi is the oppisite if but in a contect in a if else statement what do it do? (like else if).#

13.12.9. Is there any major benefit in creating an issue in a repo directly within a terminal, or is it just a matter of preference?#

I use the terminal within vscode to make issues sometimes while I am working on onet thing but I get an idea for a future feature. Using the terminal is helpful because I don’t always have the GitHub web page for thta repo open while I am working, but I do alwasy have a terminal in the folder where I’m working. So I can quickly take note of an idea and then get back to what I was doing. Less switching contexts helps stay focused and complete tasks faster.

13.12.10. How often would I use bash programming as a software engineer?#

For small bits here and there. You might not write bash scripts often but when you do it will be thing sthat could save you a ton of time.
A lot of build processes are bash scripts, as we will see soon.

You will most likely never write a large program in bash, but you will almost definitely write or update small bash scripts on a semi-regular basis.

13.12.11. can you create classes and objects in bash?#

No bash is not an object oriented language

13.12.12. what are the benefits, if any, to bash scripting in the terminal rather then in an IDE?#

knowing how to do to it is important becuse sometimes an IDE is not available. On a regular basis though no difference.

13.12.13. What exactly does grep do, can it only be used with piping?#

grep searches files. We’ll come back to it. It can search the contents of any file, not only stdin

Bash Scripting

Contents

13. Bash Scripting#

13.1. Variables in Bash#

13.2. Bash Loops#

13.3. Nesting commands#

13.4. Conditionals in bash#

13.5. Script files#

13.6. gh CLI operations#

13.7. Badge Hints#

13.8. Review today’s class#

13.9. Prepare for Next Class#

13.10. More Practice#

13.11. Experience Report Evidence#

13.12. Questions After Today’s Class#

13.12.1. Is there any structure/tools to reverse the order of if and do in a bash loop? Writing “do if” doesn’t feel natural to me.#

13.12.2. Would the script that we wrote into the file still work with semicolons or would adding them ruin the syntax?#

13.12.3. How come we have been using git commands in stead of gh commands?#

13.12.4. What else can we use this gh command for?#

13.12.5. What does the $ mean since it’s used in so many places?#

13.12.6. Are the API.MD, CONTRIBUTING.MD …etc files important? They have disappeared from my main branch#

13.12.7. Can the script be in any type of file or does it have to be in a certain type of file?#

13.12.8. How is this you said fi is the oppisite if but in a contect in a if else statement what do it do? (like else if).#

13.12.9. Is there any major benefit in creating an issue in a repo directly within a terminal, or is it just a matter of preference?#

13.12.10. How often would I use bash programming as a software engineer?#

13.12.11. can you create classes and objects in bash?#

13.12.12. what are the benefits, if any, to bash scripting in the terminal rather then in an IDE?#

13.12.13. What exactly does grep do, can it only be used with piping?#

13.6. `gh` CLI operations#