Learn Linux Shell Scripting – Fundamentals of Bash 4.4: A comprehensive guide to automating administrative tasks with the Bash shell

Learn Linux Shell Scripting –  Fundamentals of Bash 4.4

A comprehensive guide to automating administrative tasks with the Bash shell

Sebastiaan Tammer

BIRMINGHAM - MUMBAI

Learn Linux Shell Scripting – Fundamentals of Bash 4.4

Copyright © 2018 Packt Publishing

All rights reserved. No part of this book may be reproduced, stored in a retrieval system, or transmitted in any form or by any means, without the prior written permission of the publisher, except in the case of brief quotations embedded in critical articles or reviews.

Every effort has been made in the preparation of this book to ensure the accuracy of the information presented. However, the information contained in this book is sold without warranty, either express or implied. Neither the author, nor Packt Publishing or its dealers and distributors, will be held liable for any damages caused or alleged to have been caused directly or indirectly by this book.

Packt Publishing has endeavored to provide trademark information about all of the companies and products mentioned in this book by the appropriate use of capitals. However, Packt Publishing cannot guarantee the accuracy of this information.

Commissioning Editor: Vijin Boricha

Acquisition Editor: Rohit Rajkumar

Content Development Editor: Abhishek Jadhav

Technical Editor: Prachi Sawant

Copy Editor: Safis Editing

Project Coordinator: Jagdish Prabhu

Proofreader: Safis Editing

Indexer: Priyanka Dhadke

Graphics: Tom Scaria

Production Coordinator: Tom Scaria

First published: December 2018

Production reference: 1281218

Published by Packt Publishing Ltd.

Livery Place

35 Livery Street

Birmingham

B3 2PB, UK.

ISBN 978-1-78899-559-7

www.packtpub.com

mapt.io

Mapt is an online digital library that gives you full access to over 5,000 books and videos, as well as industry leading tools to help you plan your personal development and advance your career. For more information, please visit our website.

Why subscribe?

Spend less time learning and more time coding with practical eBooks and Videos from over 4,000 industry professionals

Improve your learning with Skill Plans built especially for you

Get a free eBook or video every month

Mapt is fully searchable

Copy and paste, print, and bookmark content

PacktPub.com

Did you know that Packt offers eBook versions of every book published, with PDF and ePub files available? You can upgrade to the eBook version at www.PacktPub.com and as a print book customer, you are entitled to a discount on the eBook copy. Get in touch with us at [email protected] for more details.

At www.PacktPub.com, you can also read a collection of free technical articles, sign up for a range of free newsletters, and receive exclusive discounts and offers on Packt books and eBooks. 

Contributors

About the author

Sebastiaan Tammer is a Linux enthusiast from the Netherlands. After attaining his BSc in information sciences, he graduated with an MSc in business informatics, both from Utrecht University. His professional career started in Java development, before he pivoted into a Linux opportunity. Because of this dual background, he feels most at home in a DevOps environment.

Besides working extensively with technologies such as Puppet, Chef, Docker, and Kubernetes, he has also attained the RHCE and OSCP certificates. He spends a lot of time in and around Bash. Whether it is creating complex scripting solutions or just automating simple tasks, there is hardly anything he hasn't done with Bash!

I would like to thank my girlfriend, Sanne, for all the help and support she has given me throughout the years. She has had to endure the late nights studying, me fixing stuff (which I had inevitably broken only hours earlier), and my endless storytelling about all those exciting new technologies. Thanks for the enormous amount of patience and love. I could not have done it without you!

About the reviewer

Heathe Kyle Yeakley holds degrees in technical communications and network management. He began his IT career in 1999 doing entry-level help desk support. In 2003, he began his first enterprise data center job performing tape backup and recovery for the United States Federal Aviation Administration. He worked in the aerospace sector for several years, during which time he worked on a wide range of products, including HP Tru64 Unix, Red Hat Enterprise Linux, Solaris 10, Legato Networker, Symantec NetBackup, HP and NetApp storage arrays, Spectra Logic and ADIC tape libraries, VMware, and HP Blade servers.

He currently works for Agio, where he and his coworkers deliver managed IT services to some of the world's most prestigious companies.

Packt is searching for authors like you

If you're interested in becoming an author for Packt, please visit authors.packtpub.com and apply today. We have worked with thousands of developers and tech professionals, just like you, to help them share their insight with the global tech community. You can make a general application, apply for a specific hot topic that we are recruiting an author for, or submit your own idea.

Table of Contents

Title Page

Copyright and Credits

Learn Linux Shell Scripting – Fundamentals of Bash 4.4

About Packt

Why subscribe?

PacktPub.com

Contributors

About the author

About the reviewer

Packt is searching for authors like you

Preface

Who this book is for

What this book covers

To get the most out of this book

Download the example code files

Download the color images

Conventions used

Get in touch

Reviews

Disclaimer

Introduction

What is Linux?

What is Bash?

Summary

Setting Up Your Local Environment

Technical requirements

Choosing between a virtual machine and a physical installation

Setting up VirtualBox

Creating an Ubuntu virtual machine

Creating the virtual machine in VirtualBox

Installing Ubuntu on the virtual machine

Accessing the virtual machine via SSH

Summary

Questions

Further reading

Choosing the Right Tools

Technical requirements

Using graphical editors for shell scripting

Atom

Atom installation and configuration

Notepad++

Using command-line editors

Vim

Vim summary

.vimrc

Vim cheat sheet

nano

Combining graphical editors with command-line editors when writing shell scripts

Summary

Questions

Further reading

The Linux Filesystem

Technical requirements

The Linux filesystem explained

What is a filesystem?

What makes the Linux filesystem unique?

Structure of the Linux filesystem

Tree structure

Overview of top-level directories

What about multiple partitions?

/bin/, /sbin/, and /usr/

/etc/

/opt/, /tmp/, and /var/

Everything is a file

Different types of files

Summary

Questions

Further reading

Understanding the Linux Permissions Scheme

Technical requirements

Read, write, and execute

RWX

Users, groups, and others

Manipulating file permissions and ownership

chmod, umask

sudo, chown, and chgrp

sudo

chown, chgrp

Working with multiple users

Advanced permissions

File attributes

Special file permissions

Access Control Lists (ACLs)

Summary

Questions

Further reading

File Manipulation

Technical requirements

Common file operations

Copying

Removing

Renaming, moving, and linking

Archiving

Finding files

locate

find

Summary

Questions

Further reading

Hello World!

Technical requirements

First steps

The shebang

Running scripts

Readability

Comments

Script header

Verbosity

Verbosity in comments

Verbosity of commands

Verbosity of command output

Keep It Simple, Stupid (KISS)

Summary

Questions

Further reading

Variables and User Input

Technical requirements

What is a variable?

Why do we need variables?

Variables or constants?

Variable naming

Dealing with user input

Basic input

Parameters and arguments

Interactive versus non-interactive scripts

Combining positional arguments and read

Summary

Questions

Further reading

Error Checking and Handling

Technical requirements

Error checking

Exit status

Functional checks

Test shorthand

Variable refresher

Bash debugging

Error handling

if-then-exit

if-then-else

Shorthand syntax

Error prevention

Checking arguments

Managing absolute and relative paths

Dealing with y/n

Summary

Questions

Further reading

Regular Expressions

Technical requirements

Introducing regular expressions

What is a regular expression?

grep

Greediness

Character matching

Line anchors

Character classes

Globbing

What is globbing?

Similarities with regular expressions

More globbing

Advanced globbing

Disabling globbing, and other options

Using regular expressions with egrep and sed

Advanced grep

Introducing egrep

sed, the stream editor

Stream editing

In-place editing

Line manipulation

Final remarks

Summary

Questions

Further reading

Conditional Testing and Scripting Loops

Technical requirements

Advanced if-then-else

A recap on if-then-else 

Using regular expressions in tests

The elif condition

Nesting

Getting help

The while loop

The until loop

Creating an interactive while loop

The for loop

Globbing and the for loop

Loop control

Breaking the loop

The continue keyword

Loop control and nesting

Summary

Questions

Further reading

Using Pipes and Redirection in Scripts

Technical requirements

Input/output redirection

File descriptors

Redirecting output

stdout

stderr

Redirect all output

Special output redirection

/dev/null

/dev/zero

Input redirection

Generating a password

Advanced redirecting

Redirecting redirections

Command substitution

Process substitution

Pipes

Binding stdout to stdin

Practical examples

Yet another password generator

Setting passwords in a script

tee

Here documents

Heredocs and variables

Using heredocs for script input

Here strings

Summary

Questions

Further reading

Functions

Technical requirements

Functions explained

Hello world!

More complexity

Variable scopes

Practical examples

Error handling

Augmenting functions with parameters

Colorful

Returning values

Function libraries

Source

More practical examples

Current working directory

Type checking

Yes-no check

Summary

Questions

Further reading

Scheduling and Logging

Technical requirements

Scheduling with at and cron

at

Time syntax

The at queue

at output

cron

crontab

Syntax for the crontab

Logging script results

Crontab environment variables

PATH

SHELL

MAILTO

Logging with redirection

Final logging considerations

A note on verbosity

Summary

Questions

Further reading

Parsing Bash Script Arguments with getopts

Technical requirements

Positional parameters versus flags

Using flags on the command line

The getopts shell builtin

The getopts syntax

Multiple flags

Flags with arguments

Combining flags with positional arguments

Summary

Questions

Further reading

Bash Parameter Substitution and Expansion

Technical requirements

Parameter expansion

Parameter substitutions – recap

Default values

Input checking

Parameter length

Variable manipulation

Pattern substitution

Pattern removal

Case modification

Substring expansion

Summary

Questions

Further reading

Tips and Tricks with Cheat Sheet

Technical requirements

General tips and tricks

Arrays

The history command

Creating your own aliases

Command-line shortcuts

Fun with exclamation marks

Running commands from the history

Keyboard shortcuts

Copying and pasting from the terminal

Reverse search

Cheat sheet for interactive commands

Navigation

cd

ls

pwd

File manipulation

cat

less

touch

mkdir

cp

rm

mv

ln

head

tail

Permissions and ownership

chmod

umask

chown

chgrp

sudo

su

useradd

groupadd

usermod

Summary

Final words

Assessments

Chapter 2

Chapter 3

Chapter 4

Chapter 5

Chapter 6

Chapter 7

Chapter 8

Chapter 9

Chapter 10

Chapter 11

Chapter 12

Chapter 13

Chapter 14

Chapter 15

Chapter 16

Other Books You May Enjoy

Leave a review - let other readers know what you think

Preface

Shell scripts allow us to program commands in chains and have the system execute them as a scripted event, just like batch files. This book will start with an overview of Linux and Bash shell scripting, and then quickly deep dive into helping you set up your local environment, before introducing you to tools that are used to write shell scripts. The next set of chapters will focus on helping you understand Linux under the hood and what Bash provides the user. Soon, you will have embarked on your journey along the command line. You will now begin writing actual scripts instead of commands, and will be introduced to practical applications for scripts. The final set of chapters will deep dive into the more advanced topics in shell scripting. These advanced topics will take you from simple scripts to reusable, valuable programs that exist in the real world. The final chapter will leave you with some handy tips and tricks and, as regards the most frequently used commands, a cheat sheet containing the most interesting flags and options will also be provided.

After completing this book, you should feel confident about starting your own shell scripting projects, no matter how simple or complex the task previously seemed. We aim to teach you how to script and what to consider, to complement the clear-cut patterns that you can use in your daily scripting challenges.

Who this book is for

This book targets new and existing Linux system administrators, as well as Windows system administrators or developers who are interested in automating administrative tasks. No prior shell scripting experience is required, but if you do possess some experience, this book will quickly turn you into a pro. Readers should have a (very) basic understanding of the command line.

What this book covers

Chapter 1, Introduction, primes you for the remainder of the book. Aided by some background in Linux and Bash, you should be better able to understand how and why shell scripting can provide clear benefits to you.

Chapter 2, Setting Up Your Local Environment, helps you to prepare your local machine for the examples and exercises throughout the rest of the book. You will be shown how to set up an Ubuntu 18.04 Linux virtual machine on your local machine, using VirtualBox. This virtual machine will be used to write, run, and debug commands and scripts in this book.

Chapter 3, Choosing the Right Tools, introduces you to the tools that will be used to write shell scripts. Two different kinds of tools will be described: IDE editors (Atom, Notepad++), and terminal-based editors (vim and nano). You will be encouraged to initially write scripts in an IDE, and troubleshoot scripts in a terminal-based editor, to most resemble real-world use.

Chapter 4, The Linux Filesystem, coves how the Linux filesystem is organized by exploring the virtual machine created in the previous chapters. You will achieve this by performing your first command-line actions, such as cd, pwd, and ls. Context regarding the different structures will be provided so that you can use this information when writing scripts. And, most importantly, the concept of everything is a file will be explained.

Chapter 5, Understanding the Linux Permissions Scheme, gets you acquainted with permissions under Linux, once again by exploring the virtual machine. Commands such as sudo, chmod, and chown will be used to interactively learn about file and directory privileges. The skills acquired in this chapter will be heavily used in shell scripting, so it is imperative that you gain exposure to both the successful execution of commands as well as failure messages.

Chapter 6, File Manipulation, introduces you to the most relevant file manipulation commands, including the most commonly used flags and modifiers for those commands. This will be achieved by means of commands inside the virtual machine.

Chapter 7, Hello World!, educates you in terms of thinking ahead and developing good habits when it comes to writing scripts. You will write your first actual shell script during this chapter.

Chapter 8, Variables and User Input, introduces you to variables and user input. You will see how parameters are used by Bash, and where the differences lie between parameters and arguments. User input will be handled and used to produce new functions in your scripts. Finally, the difference between interactive and non-interactive scripts will be clarified and discussed.

Chapter 9, Error Checking and Handling, gets you familiar with (user) input, and error checking and handling. Introducing user input into a script is bound to result in more errors, unless the script specifically deals with the possibility of users submitting incorrect or unexpected input. You will learn how to best deal with this.

Chapter 10, Regular Expressions, gets you familiar with regular expressions, which are often used in shell scripting. The most common patterns and uses for these regular expressions will be presented. Basic usage of sed will be covered in this chapter, complementing regular expression explanations.

Chapter 11, Conditional Testing and Scripting Loops, discusses the different kind of loops and the relevant control structures that are used in shell scripting with Bash.

Chapter 12, Using Pipes and Redirection in Scripts, introduces you to redirection on Linux. This chapter will start with the basic input/output redirection, before continuing to stream redirection and pipes.

Chapter 13, Functions, introduces you to functions. Functions will be presented as blocks of code that are grouped together in such a way that they can be reused, often with different arguments, to produce a slightly different end result. You will learn to understand the benefit to reusing code, and planning scripts accordingly.

Chapter 14, Scheduling and Logging, teaches you how to schedule scripts and how to make sure these scheduled scripts perform the task they were intended for, by using the crontab and the at command, coupled with proper logging.

Chapter 15, Parsing Bash Script Arguments with getopts, helps you to improve your scripts by adding flags instead of positional parameters, thereby making the scripts much easier to use.

Chapter 16, Bash Parameter Substitution and Expansion, shows how previous patterns used in earlier scripts can be optimized by means of parameter expansion, substitution, and variable manipulation.

Chapter 17, Tips and Tricks with Cheat Sheet, provides you with some handy tips and tricks that are not necessarily used in Bash scripts, but that are very convenient when working on the terminal. For the most frequently used commands, a cheat sheet containing the most interesting flags and options will be provided so that you can use this chapter as reference while scripting.

To get the most out of this book

You will require an Ubuntu 18.04 Linux virtual machine to follow along with the book. We will guide you through setting this up during in the second chapter. You will only truly learn shell scripting if you follow along with all code examples. The entire book has been written with this in mind, so be sure to follow this advice!

Download the example code files

You can download the example code files for this book from your account at www.packtpub.com. If you purchased this book elsewhere, you can visit www.packtpub.com/support and register to have the files emailed directly to you.

You can download the code files by following these steps:

Log in or register at www.packtpub.com.

Select the SUPPORT tab.

Click on Code Downloads & Errata.

Enter the name of the book in the Search box and follow the onscreen instructions.

Once the file is downloaded, please make sure that you unzip or extract the folder using the latest version of:

WinRAR/7-Zip for Windows

Zipeg/iZip/UnRarX for Mac

7-Zip/PeaZip for Linux

The code bundle for the book is also hosted on GitHub at https://github.com/PacktPublishing/Learn-Linux-Shell-Scripting-Fundamentals-of-Bash-4.4. In case there's an update to the code, it will be updated on the existing GitHub repository.

We also have other code bundles from our rich catalog of books and videos available at https://github.com/PacktPublishing/. Check them out!

Download the color images

We also provide a PDF file that has color images of the screenshots/diagrams used in this book. You can download it here: https://www.packtpub.com/sites/default/files/downloads/9781788995597_ColorImages.pdf.

Conventions used

There are a number of text conventions used throughout this book.

CodeInText: Indicates code words in text, database table names, folder names, filenames, file extensions, pathnames, dummy URLs, user input, and Twitter handles. Here is an example: Let's try to copy the /tmp/ directory into our home directory.

A block of code is set as follows:

#!/bin/bash

echo Hello World!

When we wish to draw your attention to a particular part of a code block, the relevant lines or items are set in bold:

reader@ubuntu:~/scripts/chapter_10$ grep 'use' grep-file.txt

We can use this regular file for testing grep.

but in the USA they use color (and realize)!

Any command-line input or output is written as follows:

reader@ubuntu:~/scripts/chapter_10$ grep 'e.e' character-class.txt

eee

e2e

e e

Bold: Indicates a new term, an important word, or words that you see on screen. For example, words in menus or dialog boxes appear in the text like this. Here is an example: Click the Install button and watch the installation.

Warnings or important notes appear like this.

Tips and tricks appear like this.

Get in touch

Feedback from our readers is always welcome.

General feedback: Email [email protected] and mention the book title in the subject of your message. If you have questions about any aspect of this book, please email us at [email protected].

Errata: Although we have taken every care to ensure the accuracy of our content, mistakes do happen. If you have found a mistake in this book, we would be grateful if you would report this to us. Please visit www.packtpub.com/submit-errata, selecting your book, clicking on the Errata Submission Form link, and entering the details.

Piracy: If you come across any illegal copies of our works in any form on the internet, we would be grateful if you would provide us with the location address or website name. Please contact us at [email protected] with a link to the material.

If you are interested in becoming an author: If there is a topic that you have expertise in, and you are interested in either writing or contributing to a book, please visit authors.packtpub.com.

Reviews

Please leave a review. Once you have read and used this book, why not leave a review on the site that you purchased it from? Potential readers can then see and use your unbiased opinion to make purchase decisions, we at Packt can understand what you think about our products, and our authors can see your feedback on their book. Thank you!

For more information about Packt, please visit packtpub.com.

Disclaimer

The information within this book is intended to be used only in an ethical manner. Do not use any information from the book if you do not have written permission from the owner of the equipment. If you perform illegal actions, you are likely to be arrested and prosecuted to the full extent of the law. Packt Publishing does not take any responsibility if you misuse any of the information contained within the book. The information herein must only be used while testing environments with proper written authorizations from appropriate persons responsible.

Introduction

Before we start writing shell scripts, we need to have some context about our two most relevant components: Linux and Bash. We'll give an explanation of Linux and Bash, look into the history behind the two technologies, and discuss the current state of both.

The following topics will be covered in this chapter:

What is Linux?

What is Bash?

What is Linux?

Linux is a generic term that refers to different open source operating systems that are based on the Linux kernel. The Linux kernel was originally created by Linus Torvalds in 1991, and open sourced in 1996. A kernel is a piece of software that is designed to act as an intermediate layer between low-level hardware (such as the processor, memory, and input/output devices) and high-level software, such as an operating system. Apart from the Linux kernel, most Linux operating systems rely heavily on GNU project utilities; for example, the Bash shell is a GNU program. Because of this, Linux operating systems are referred to by some as GNU/Linux. The GNU project, where GNU stands for GNU's Not Unix! (a recursive acronym), is a collection of free software, a lot of which is found in most Linux distributions. This collection includes many tools, but also an alternative kernel called GNU HURD (which is not as widespread as the Linux kernel).

Why do we need a kernel? Since a kernel sits between hardware and the operating system, it provides an abstraction for interacting with hardware. This is why the Linux ecosystem has grown so large: the kernel can be used freely, and it handles a lot of low-level operations on many types of hardware. Creators of an operating system can therefore spend their time making an easy-to-use, beautiful experience for their users, instead of having to worry about how the users' pictures are going to be written to the physical disk(s) attached to the system.

The Linux kernel is so-called Unix-like software. As you might suspect, this implies that it is similar to the original Unix kernel, which was created between 1971 and 1973 at Bell Labs, by Ken Thompson and Dennis Ritchie. However, the Linux kernel is only based on Unix principles and does not share code with Unix systems. Famous Unix systems include the BSDs (FreeBSD, OpenBSD, and so on) and macOS.

Linux operating systems are broadly used for one of two purposes: as a desktop or as a server. As a desktop, Linux can serve as a replacement for the more commonly used Microsoft Windows or macOS. However, most Linux usage is accounted for the server landscape. At the time of writing, it is estimated that around 70% of all servers on the internet use a Unix or Unix-like operating system. The next time you're browsing the news, reading your mail, or are scrolling through your favorite social media website, remember that there's a big chance the pages you are being shown have been processed by one or more Linux servers.

There are many distributions, or flavors, of Linux. Most Linux operating systems fall within distribution families. A distribution family is based on a common ancestor, and often use the same package management tools. One of the more well-known Linux distributions, Ubuntu, is based on the Debian distribution family. Another prominent Linux distribution, Fedora, is based on the Red Hat family. Other notable distribution families include SUSE, Gentoo, and Arch.

Not many people realize how many devices run the Linux kernel. For example, the most common smartphone operating system in use today, Android (with a market share of around 85%), uses a modified version of the Linux kernel. The same goes for many smart TVs, routers, modems, and various other (embedded) devices. If we were to include Unix and other Unix-like software, we can safely say that most of the devices in the world run on these kernels!

What is Bash?

The most commonly used shell in Linux systems is the Bourne-again shell, or Bash. The Bash shell is based on the Bourne shell, known as sh. But what is a shell?

A shell is, in essence, a user interface. Most often, it is used to refer to a text-based interface, also called a command-line interface (CLI). However, it is called a shell because it can be seen as a shell around the kernel; this means that it applies not just to CLIs, but just as well to graphical user interfaces (GUIs). When we refer to a shell in this book, we are talking about a CLI, and unless stating differently, we're talking about the Bash shell.

The purpose of a shell, both CLI and GUI, is to allow the user to interact with the system. After all, a system that does not offer interaction would be hard to justify, not too mention hard to use! Interaction in this context means many things: typing on your keyboard will make letters appear on your screen, moving your mouse will change the location of the cursor on screen, giving the command to delete a file (either with a CLI or GUI) will remove the bytes from the disk, and so on.

In the earliest days of Unix and computers, no GUIs were available, so all work was performed via a CLI. To connect to the shell on a running machine, a video terminal was used: often this would be a very simple monitor combined with a keyboard, which was connected with, for example, a RS-232 serial cable. Commands entered on this video terminal were processed by the shell running on the Unix machine.

Luckily for us, things have changed quite a bit since the first computers. Today, we no longer use dedicated hardware to connect to shells. A piece of software running in a GUI, a terminal emulator, is used for interaction with the shell. Lets take a quick look at how connecting to a Bash shell with a terminal emulator can look:

In the preceding screenshot, we're connected to a Linux virtual machine (we'll be setting this up in the next chapter), using a terminal emulator (GNOME Terminal) via the Secure Shell (SSH) protocol. A few interesting things to note:

We're on a CLI interface; we do not have access to, nor do we need, a mouse

We're connected to an Ubuntu machine, but we're running this within another operating system (Arch Linux, in this case)

We're presented with a welcome message by Ubuntu 18.04, showing some general information about the system

Besides using the Bash shell for direct system interaction, it provides another important functionality: the ability to execute multiple commands sequentially, with or without user interaction, tailored to a specific goal. This might sound complicated, but it's actually pretty simple: we're talking about Bash scripts, the subject of this book!

Summary

In this chapter, you've read about the GNU/Linux operating systems and the Linux kernel, what a kernel really is, and how big an impact Linux distributions have on daily life. You've also learned what a shell is, and that the most common Linux shell, Bash, can be both used to interact with a Linux system, and is also utilized to write shell scripts.

In the next chapter, we'll set up a local environment which we will use throughout the rest of the book, in both the examples and exercises.

Setting Up Your Local Environment

In the previous chapter, we ventured into some context for the wonderful world of Linux and Bash. Since this is a practical, exercise-driven book, we're going to use this chapter to set up a machine where you can follow along with the examples and perform the exercises at the end of each chapter. This can either be a virtual machine or a physical installation; that is up to you. We will discuss this in the first part of this chapter, before continuing with the installation of VirtualBox and, finally, creating an Ubuntu virtual machine.

The following command will be introduced in this chapter: ssh and exit.

The following topics will be covered in this chapter:

Choosing between a virtual machine and a physical installation

Setting up VirtualBox

Creating an Ubuntu virtual machine

Technical requirements

To complete the exercises in this chapter (and the following chapters), you will need either a PC or laptop with at least 2 GHz of CPU power, 10 GB of hard disk space, and about 1 GB of RAM to spare. Pretty much all hardware created in the last 5 years should suffice.

Choosing between a virtual machine and a physical installation

A virtual machine is an emulation of a physical machine. This means it runs inside a physical machine, as opposed to directly on the hardware. A physical machine has direct access to all hardware, such as the CPU, the RAM, and other devices such as the mouse, the keyboard, and the monitor. It is, however, impossible to share the CPU or the RAM between multiple physical machines. Virtual machines do not directly get access to hardware, but through an emulation layer, which means resources can be shared between multiple virtual machines.

Because we're discussing Bash shell scripting in general, in theory it does not matter what kind of an installation is performed. As long as that installation runs a compatible Linux operating system with Bash 4.4 or later, all exercises should work. There are, however, many advantages to using a virtual machine over a physical installation:

There is no need to remove your current preferred operating system, or set up a complicated dual-boot configuration

Virtual machines can be snapshotted, which allows recovery from critical failures

You are able to run (many) different operating systems on a single machine

Unfortunately, there are also drawbacks associated with virtual machine use:

Because you're running a virtual operating system within an already running operating system, there is some overhead from the virtualization (in comparison to running a bare-metal installation)

Since you're running multiple operating systems at the same time, you will need more resources than with a bare-metal installation

In our opinion, modern computers are fast enough to make the drawbacks almost trivial, while the advantages provided by running Linux in a virtual machine are very helpful. Because of this, we will only be explaining a virtual machine setup in the rest of this chapter. If you feel confident enough to run Linux as a physical installation (or perhaps you already have Linux running somewhere!), feel free to explore the rest of the book with that machine.

You might have a Raspberry Pi or another single-board computer running Linux in your house from a previous project. While these machines are indeed running a Linux distribution (Raspbian), they are probably running it on a different architecture: ARM instead of x86. Because this can cause unexpected results, we recommend only using x86 devices for this book.

If you want to be sure all examples and exercises work as seen in this book, run an Ubuntu 18.04 LTS virtual machine in VirtualBox with the recommended specifications of 1 CPU, 1 GB RAM, and a 10 GB hard disk: this setup is described in the rest of this chapter. Even if many other types of deployment should work, you would not want to be banging your head against the wall for hours when an exercise is not working, before discovering it was caused by your setup.

Setting up VirtualBox

To use virtual machines, we need software called a hypervisor. A hypervisor manages resources between the host machine and the virtual machines, provides access to disks, and has an interface to manage it all. There are two different types of hypervisors: type-1 and type-2. Type-1 hypervisors are the so-called bare-metal hypervisors. These are installed instead of a regular operating system such as Linux, macOS, or Windows, directly on the hardware. These types of hypervisors are used for corporate servers, cloud services, and so on. For this book, we will use a type-2 hypervisor (also called hosted hypervisors): these are installed within another operating system, as a piece of software not much different than, for example, a browser.

There are many type-2 hypervisors. The most popular choices at the time of writing are VirtualBox, VMware workstation player, or OS-specific variants such as QEMU/KVM on Linux, Parallels Desktop on macOS, and Hyper-V on Windows. Because we are going to use a virtual machine throughout this book, we do not assume anything about the host machine: you should work comfortably with whatever operating system you prefer. Because of this, we've chosen to use VirtualBox as our hypervisor, since it runs on Linux, macOS, and Windows (and even others!). Furthermore, VirtualBox is free and open source software, which means you can just download and use it.

Presently, VirtualBox is owned by Oracle. You can download the installer for VirtualBox from https://www.virtualbox.org/. Installation should not be hard; follow the instructions by the installer.

After installing a type-2 hypervisor such as VirtualBox, be sure to restart your computer. Hypervisors often need some kernel modules loaded, which is easiest to achieve by rebooting.

Creating an Ubuntu virtual machine

In this book, we're scripting with Bash, which means we do not need a GUI for our Linux installation. We have chosen to use Ubuntu Server 18.04 LTS as the virtual machine operating system, for a number of reasons:

Ubuntu is considered a beginner-friendly Linux distribution

The 18.04 is a Long-Term Support (LTS) release, which means it will receive updates until April 2023

Because an Ubuntu server offers a CLI-only installation, it is easy on system resources and representative of real-life servers

At the time of writing, Ubuntu is maintained by Canonical. You can download the ISO image from https://www.ubuntu.com/download/server. Download the file now, and remember where you save this file, since you'll need it soon.

Should the preceding download link no longer work, you can go to your favorite search engine and search for Ubuntu Server 18.04 ISO download. You should find a reference to the Ubuntu archives, which will have the required ISO.

Creating the virtual machine in VirtualBox

First, we will start with creating the virtual machine to host our Ubuntu installation:

Open VirtualBox and choose Machine | New in the menu toolbar.

For reference, we have circled the Machine entry in the menu toolbar given in the following screenshot. Choose a name for the virtual machine (this can be a different name than the server name, but we like to keep it the same for simplicity), set the Type to Linux, and Version to Ubuntu (64-bit). Click Next:

On this screen, we determine memory settings. For most servers, 1024 MB of RAM is a great start (and is recommended by VirtualBox for virtual machines as well). If you have beefy hardware, this can be set to 2048 MB, but 1024 MB should be fine. Make your selection and press Next:

Once again, the recommended values by VirtualBox are perfect for our needs. Press Create to start the creation of the virtual hard disk:

Virtual hard disks can be many different types. VirtualBox defaults to its own format, VDI, as opposed to VMDK, which is the format used by VMware (another