Lab 2: Shell Scripting & Git
Overview
In this lab, you will learn how to use a linux terminal, practice some git commands, and write regular expressions.
To receive credit for this lab, you MUST show your work to the TA during the lab, and push your completed work to the github repositories by the deadline. Please note that submissions will be due right before your respective lab sessions in the following week. For Example, If your lab is on this Friday 10 AM, the submission deadline will be next Friday 10 AM. There is a "NO LATE SUBMISSIONS" policy for labs.
Please DO NOT close or clear your terminals as you are working through the lab as you may need to export some commands and their outputs for your submission.
Learning Objectives
LO1. Get familiar with git branching, merge conflicts and resolutions.L02. Learn about pull requests and the difference between `git pull` and a pull request.
L03. Hands on practice with frequently used Shell commands
L04. Learn enough about regular expressions to be able to create basic ones
LO5. Learn advanced but commonly used git commands
Part A
There is no pre-lab quiz this week.
Part B
1. Practicing Git commands and learning to deal with merge conflicts
Clone your GitHub repository
Github Classroom Assignment
- Using an SSH key
- Using a Personal Access Token (PAT)
git clone git@github.com:CU-CSCI3308-Fall2024/lab-2-git-shell-regex-<YOUR_USER_NAME>.git
git clone https://github.com/CU-CSCI3308-Fall2024/lab-2-git-shell-regex-<YOUR_USER_NAME>.git
Navigate to the repository on your system.
cd lab-2-git-shell-regex-<YOUR_USER_NAME>
You will be simulating both persons of a pair for this part of the lab.
I. Managing code as a team - Person1's perspective
When working in a team, conflicts usually happen when two working Git branches diverge and the files are not consistent when the two branches are going to merge together. In the next few steps we will be simulating a conflict and learn how to fix it before merging the branches successfully.
i. Making code changes
As Person1: Update the README.md with you name and Github profile.
Format :<Firstname> <Lastname> - <Github username>
Run the following commands inside your local git directory. Make sure you have made the appropriate edits to the README.md file.
git add .
git commit -m "updated README.md with team details"
git push
ii. Pushing changes
1. As Person1: Create a new branch, with a sensible name of your choice.
git checkout -b "<Person1_branch_name>"
2. As Person1: Make changes to the file - main.cpp
We are updating the return value in main only for demonstration purposes.
Change the return value to "1" in main.cpp
3. As Person1: Stage your changes
The git add command will stage any files you choose to be staged to commit (update) in your local Git repository. The .(period) lets you select all of the files that have been added or modified since the last commit. Here, we are specifying a file name to ensure that we are staging only the main.cpp file for commit.
git add main.cpp
4. As Person1: Commit your changes
Running the git commit command will now update your local git repository with the changes you staged. The -m flag is required and should be followed with whatever message you wish to state for your commit. This is part of your documentation, so try to make commit messages informative! Ideally, your commit message should indicate what changes are being made in this commit.
git commit -m "Version - <Person1> : <return value>"
5. As Person1: Push the branch to the remote repository
The git push command will upload your changes from your local repository to your remote Git repository which is hosted on GitHub. origin is the name of your remote whereas <Person1> is the name of the branch created by you as Person1. --set-upstream flag is used to push this new branch and set it as the default remote branch.
git push --set-upstream origin <Person1_branch_name>
The next time you choose to push to this branch you do not need to use the --set-upstream flag. You could simply use git push origin <Person1_branch_name>.
You will now be Person 2 in this scenario. Switch to the main branch using the following command:
git checkout main
6. As Person2: Make some edits on the main branch
Person2 will be in main branch. Before you start making edits you should pull the latest changes in remote.
git pull
Go ahead and change the return value to "2" in main.cpp
7. As Person2: Stage your changes
git add main.cpp
8. As Person2: Commit your changes
git commit -m "Version - main : <return value>"
9. As Person2: Push the changes to the main branch
git push origin main
iii. Merging Main
You will now switch to being Person 1. Since you are simulating both persons, you already have the updated main. When working with other developers, always remember to run git pull frequently to ensure that your main is updated with the remote repository.
1. As Person1: Switch back to your branch
The following command helps you to toggle between branches.
git checkout <Person1_branch_name>
2. As Person1: Merge
Merge main on to your branch. You will come across a conflict.
git merge main
Do NOT resolve conflicts through the GUIs!!
Before we go ahead and resolve merge conflict, let's understand how to inspect changes with the help of an example.
Following is an example showing merge conflict in a 'main.cpp' file.
By using your favorite editor, for instance VS Code, you can easily view the merge conflict on the UI. In the above example 'main.cpp' file, lines (12-14) after <<<< HEAD represents the current changes that you have, while the lines (14-16) represents the incoming changes from the remote repository i.e., changes made by some other person on the same section of code. You can accept either of the changes to resolve the conflict.
When working on an actual project, it is necessary that you contact that author of the code that is conflicting with yours and come to an agreement about how to resolve the conflict. It could mean accepting either solutions or re-writing a solution that includes both pieces of code.
iv. Resolving Conflicts
Now, that you have an understanding of how to view merge conflict(s), let's go ahead and resolve it!
1. View conflicts
Using your favorite editor, edit the "main.cpp" file so that it reflects either of the two changes (i.e. keep return 1 or return 2). Then, stage, commit, and push the "main.cpp" file.
2. Push the final version
git add main.cpp
git commit -m "Merging changes with return value <selected value>"
git push
The output from your terminal showing the git conflict and then the resolution.
Name this file git_merge_conflict.txt.
v. Create a pull request
1. As Person1: Create a pull request
Go the repository using the GitHub UI. Click on the "Compare & Pull request" button.
2. As Person1: Write a Description
Write a brief description for the pull request. Then, assign yourself as the assignee and for this example, assign the reviewer as yourself. In the industry, the reviewer would be a dev lead, team lead etc. Create the pull request.
3. As Person2: Review a pull request
Go to the "Pull Requests" tab and choose the pull request you were assigned to review.
Click on "Review changes" and add a comment to your review. Select "Approve" and submit the review.
4. As Person1: Merge your branch
Finally, merge the changes by clicking "Merge pull request".
Select "Create a merge commit."
Confirm the merge action.
Your PR is now merged. Nice work!
A screenshot of GitHub on your browser showing the log of the git pull
request. Name this file git_pull_request.png.
2. Shell scripting and regex tutorial
We will now be learning and practicing Shell commands
I. Connecting to a linux terminal
We'll be using Docker Compose to mount the repository into an Ubuntu Image. Create a docker-compose.yaml file in the root of the repository and copy the contents below.
version: '3.9'
services:
devel:
image: ubuntu:20.04
working_dir: /repository
volumes:
- './:/repository'
command: bash
You'll probably notice that this looks very similar to the docker-compose.yaml file you wrote in lab 1. The main differences are that we are using version 20.04 of the ubuntu image, and running bash at startup.
We also changed the name to devel to better represent how we will be using the container.
You can run the devel service.
docker-compose run devel
Creating lab2-git-scripting_devel_run ... done
Now, as you are connected to the ubuntu terminal, you should see your working directory is /repository and your files match those on the host machine.
pwd
/repository
ls
README.md course-catalog.txt docker-compose.yaml
II. Navigating the file system
Let's practice some linux commands that we can use to navigate the file system.
Earlier, we used the pwd command to show our location in the filesystem. We could also use echo to show the PWD environment variable.
echo $PWD
/repository
Create a file named tmp.sh.
touch tmp.sh
Install the vim editor. Enter "yes" on the prompt that appears.
apt-get update && apt-get install vim
Open the file with the vim editor.
vim tmp.sh
Type "i" to enter insert mode, then write the following line into the file.
echo $HOME
Save and close the file. In the editor press escape (esc) and type: :wq
Execute the file. Did it work? Why or why not?
./tmp.sh
List the contents of the current directory:
ls -l
Change the permissions of the file 'tmp.sh' to make it executable:
chmod +x tmp.sh
List the contents of the current directory again:
ls -l
Execute the file again:
./tmp.sh
Check the contents of 'tmp.sh'.
cat tmp.sh
Delete the tmp.sh file.
rm tmp.sh
List the contents of the current directory.
ls
README.md course-catalog.txt docker-compose.yaml
Export the commands executed and their outputs into a file called intro_to_navigating_filesystem.txt
III. Introduction to Regex
Often when faced with a tricky regex problem it's easiest to start in a dedicated regex editor, such as RegExr and regex101 to develop a working regular expression. This section will walk you through this process with a simple example extracting information from the CU Boulder Computer Science Course Catalog.
Developing regular expressions
This file is somewhat large, so you should only copy the first 10-20 lines into RegExr. Just like any other testing, you'll want positive test cases (examples that match) and negative test cases (examples that don't match). The file is named course-catalog.txt in your repository. This is a slightly cleaned up version of the course catalog in text format, which should make things a bit easier for us.
Before testing your expression, please enable the "Multiline" flag on RegExr as shown below.
To Do:
Our goal will be to count the number of courses which have an odd course number. This course, CSCI 3308, is an even course. Any ideas on how to get started? Here are some tips.
Find a "location" in each example that contains a reliable indicator of course number for all courses. How about the first line?
CSCI 1000 (1) Computer Science as a Field of Work and Study
Introduces curriculum, learning techniques, time management and career opportunities in Computer Science. Includes presentations from alumni and others with relevant educational and professional experience.
Equivalent - Duplicate Degree Credit Not Granted: CSPB 1000
Requisites: Restricted to students with 0-26 credits (Freshmen) Computer Science (CSEN-BSCS or CSEN-BA) or Engineering Open Option (XXEN) majors only.
Additional Information: Departmental Category: General Computer Science
This seems promising. We can use ^ to only match the start of each line, but would this rule out all false positives? What if the description started with the course number? These are all concerns your regex should address.
The 5 following letters are always the same CSCI . Got any ideas on the numbers? As another hint, look at RegExr's cheatsheet on the left-hand side. Some numbers can match any digit, while one must be limited. You may also want to look at [ ] for enclosing a list or characters to match. You can also refer the regex cheatsheet below.
Moving to the terminal
Once you are confident in your solution, you can move on to the terminal. It is okay to skip this recording output for now and return once you are more confident in your solution, but make sure to take a screenshot before closing the window.
To test your regular expression we will use grep. You need to replace the contents of the string with your regular expression from above.
For Linux or WSL2 terminals, to enable Posix, add -p option to grep command.
grep -E "<YOUR_REGULAR_EXPRESSION>" course-catalog.txt
CSCI 2275 (4) Programming and Data Structures
CSCI 2897 (3) Calculating Biological Quantities
CSCI 3155 (4) Principles of Programming Languages
...
Now you could count these manually, but regular expressions are particularly powerful when combined with simple linux tools such as wc -l, which counts the number of lines. You can use a pipe operator to pipe the output of one command to another.
grep -E "<YOUR_REGULAR_EXPRESSION>" course-catalog.txt | wc -l
51
Did you get the right answer? If not, go back to RegExr and verify that the explanation given under "Tools" matches your intuition.
Save the grep commands executed in terminal with their outputs to a file intro_regex_terminal.txt
Regex Cheatsheet
| Metacharacter | Usage |
|---|---|
| . | Any one character |
| [ ] | Any enclosed character |
| * | Zero or more of the preceding character |
| ? | Zero or one of the preceding character |
| + | One or more of the preceding character |
| ^ | Anchor - the beginning of a string |
| $ | Anchor - the end of a string |
| \ | Escape character |
| | | Boolean OR |
| {m, n} | The preceding character appears m to n times |
| \b | Anchor - the beginning of a word |
| [[:blank:]] | Space or tab |
| Regular Expression | Output |
|---|---|
| ^The | Matches any string that starts with "The" |
| of despair$ | Matches any string that ends with "of despair" |
| ^abc$ | A string that starts and ends with "abc" (effectively an exact match) |
| notice | A string that has the text "notice" in it |
| ab* | Matches a string that has an "a" followed by zero or more "b" (e.g., "ac", "abc", "abbc", etc.) |
| ab+ | Same as above BUT the string has at least one "b" (e.g., "abc", "abbc", etc. BUT NOT "ac") |
| ab? | Similar as above, but there might be a single "b" or not (e.g., "ac", "abc" BUT NOT "abbc") |
| a?b+$ | A possible "a" followed by one or more "b" at the end of a string (e.g., a string ending with "ab", or "abb", or "abbb", etc., or "b", or "bb", etc., BUT NOT "aab", or "aabb", etc.) |
Part C
In this section, we will learn additional git commands to cover some workflows you are likely to encounter working on your projects. We provide a brief summary of different commands and links to related materials. For a more thorough and interactive introduction to git, we recommend the Git branching tutorial. At the end of this section, you will be taking a quiz on Canvas.
Git Branching tutorial
While the git tutorial is optional, it is highly recommended!
Git Commands
git rebase
Git rebase is used when there are changes in main that we want to reflect and bring our branch to the same state as main. Rebase helps us create new commits for every commit on the original branch. Rebasing is useful because we have less conflicts and a cleaner structure.
Image Credit: Atlassian Rebase Tutorial
git fetch
Pulls all newly added branches from remote so we can have all the new branches on our local tracking as well!
git rm
We've already seen how to add files to be committed and pushed to a remote repository. But what if we need to remove any files from our repository that we no longer require? In this case, we can use the git rm command.
The git rm command is used to delete files from the git index. This command can be used to remove files from both the staging index and the working directory, but not only from the working directory.
git stash
While working in a project team, you might run into a situation where, for instance, you're writing your code for a feature Ain your branch, but you are suddenly called by your coworker to give a hand in pushing a bug fix for another feature "B". In this situation, you'd want to save your current code and resume it from where you've left off after completing the bug fix. This is where git stash comes into play. Stash means to store/save your current changes safely in one place (stash stack). git stash temporarily shelves (or stashes) changes you've made to your working copy so you can work on something else, and then come back and re-apply them later on. Once you've stashed your changes, you're free to switch branches, commit new changes, and come back later and resume your saved or stashed work with git stash pop. Popping takes your changes from the stash stack and reapplies them to your current working directory.
By default, git stash does not stash the following: Files that have been ignored New files in your working copy that are not staged yet
So, in cases where you need to stash untracked files as well, you will need to execute the git stash command with the --include-untracked or -u flag at the end of the command.
git stash --include-untracked
# OR: git stash -u
git log
The git log command is used to view the history of committed changes within a git repository.
The git log command only works on the committed history. Git log offers a number of options to customize your log output through filtering, ordering and formatting options.
git revert
Git revert is commonly known as a way to "undo" commits in Git. Consider git revert as a "get out of jail free card" whenever you make a mistake while working on a project with others. You use git revert to create new commits that undo the effects of previous commits. For doing this it creates a new commit for undoing the changes, keeping the previous log as it is.
Following is the link that will help you to understand git revert better:
Canvas Quiz
For Part B, take the Lab 2 Part C - Quiz on Canvas to test your understanding.
Submission Guidelines
Expected files:
git_merge_conflict.txt
git_pull_request.png
intro_to_navigating_filesystem.txt
intro_regex_terminal.txt
You need to submit all your files to the repository within a submission folder.
Commit and upload your changes
1. Make sure you copy your files for submission into your local git directory for this lab, within a "submission" folder that you need to create.
2. Run the following commands at the root of your local git directory, i.e., outside of the "submission" folder.
git add .
git commit -m "add all screenshots and changes for lab 2"
git push
Once you have run the commands given above, please navigate to your GitHub remote repository (on the browser) and check if all the files have been updated/uploaded. If you see the files/changes there, you have successfully submitted your work for this lab.
You will be graded on the files that were present before the deadline. If the files are missing/not updated by the deadline, you could receive a grade as low as 0. This will not be replaced as any new pushes to the repository after the deadline will be considered as a late submission and we do not accept late submissions. Please do not submit a regrade request for this.
Regrade Requests
Please use this link to raise a regrade request if you think you didn't receive a correct grade.Rubric
| Criteria | Description | Points |
|---|---|---|
| Part A | There is no pre-lab quiz for this lab | 0 |
| Part B: Git merge conflict and resolution | A text file with your git merge conflict and resolution | 15 |
| Part B: Git pull request | A screenshot of GitHub showing the log of the git pull request. | 15 |
| Part B: Bash and git practice commands | A text file containing the practice shell commands and their outputs. | 10 |
| Part B: Regex | A text file containing your regex expression | 10 |
| Part C: Take home section | Canvas Quiz | 30 |
| In-class check-in | You showed your work to the TA or CM. | 20 |
| 100 |