Linux Roadmap¶

Step by step guide to learning Linux in 2024

Navigation Basics¶

Basic Commands¶

# Change directory
cd /path/to/directory  

# List contents of a directory
ls  

# View current working directory
pwd  

Moving Files /Directories¶

mv [options] source destination

Creating & Deleting Files /Dirs¶

touch newfile.txt

cat > newfile.txt

Directory Hierarchy Overview¶

Overview of File System Hierarchy Standard

/usr
  |- bin/
  |- etc/
  |- games/
  |- include/
  |- kerberos/
  |- lib/
  |- libexec/       
  |- local/
  |- sbin/
  |- share/
  |- src/
  |- tmp -> ../var/tmp/
  |- X11R6/

Editing Files¶

There’s a variety of text editors available in Linux by default, these include: nano, vi/vim, emacs, and gedit. Each of these has its own learning curve and set of commands

nano filename

vim example.txt

Shell & Others Basics¶

Linux Shell Basics¶

pwd

Command Path in Shell Basics¶

echo $PATH

Environment Variables Under Shell Basics¶

# List all environment variables
$ env

# Print a particular variable like PATH
$ echo $PATH

Command Help¶

man [command]

help [command]

Redirects In Shell Basics¶

ls -al > file_list.txt

Super User¶

# This would prompt for root password and switch you to root usermode
$ su -

# To perform a command as superuser (if allowed in sudoers list)
$ sudo <command>

Working with Files¶

Working with files is an essential part of Linux and it’s a skill every Linux user must have

touch example.txt

ls

Linux File Permissions¶

-rwxr--r-- 1 root root 4096 Jan 1 12:00 filename

Linux File Permissions and Ownership Explained with Examples

Linux File Permissions in 5 Minutes

Archiving¶

Linux offers powerful utilities for archiving, where multiple files and directories are combined into a single file, primarily for backup and simplification of distribution. The main tools used for this purpose are tar, gzip, and bzip2

# To create a tar archive:
tar cvf archive_name.tar directory_to_archive/

# To extract a tar archive:
tar xvf archive_name.tar

# To create a gzip compressed tar archive:
tar cvzf archive_name.tar.gz directory_to_archive/

#To create a bzip2 compressed tar archive:
tar cvjf archive_name.tar.bz2 directory_to_archive/

Copying and Renaming Files¶

cp /path/to/original/file /path/to/copied/file

mv /path/to/original/file /path/to/new/file

Soft and Hard Links¶

In Unix-like operating systems like Linux, soft (symbolic) and hard links are simply references to existing files that allow users to create shortcuts and duplication effects within their file system.

# Create a hard link
ln source_file.txt hard_link.txt

# Create a soft link
ln -s source_file.txt soft_link.txt

Text Processing¶

Text processing is an essential task for system administrators and developers. Linux, being a robust operating system, provides powerful tools for text searching, manipulation, and processing.

Users can utilize commands like awk, sed, grep, and cut for text filtering, substitution, and handling regular expressions

Although being primarily a command-line operating system, Linux also offers numerous GUI-based text editors including gedit, nano, and vim, which make text editing convenient for both beginners and advanced users.

Below is a simple example using grep command to search for the term “Linux” in a file named “sample.txt”

grep 'Linux' sample.txt

stdout / stdin / stderr , cut , paste , sort , tr , head , tail , join , 
split , pipe , tee , nl ,wc,expand, unexpand , uniq.grep ,awk

Stdout and stderr¶

The concepts of stdout and stderr in Linux belong to the fundamentals of Linux text processing. In Linux, when a program is executed, three communication channels are typically opened, namely, STDIN (Standard Input), STDOUT (Standard Output), and STDERR (Standard Error)

$ command > stdout.txt 2>stderr.txt

Cut Command¶

cut OPTION... [FILE]...

echo "one,two,three,four" | cut -d "," -f 2

Paste¶

paste file1.txt file2.txt > combined.txt

Sort¶

sort filename.txt

sort filename.txt > sorted_filename.txt

sTr-Command¶

echo 'hello' | tr 'a-z' 'A-Z'

Head Command¶

head file.txt

head -n 5 file.txt

Tail Command¶

tail /var/log/syslog

join Command¶

# Syntax
join file1.txt file2.txt

Split Command¶

split [options] [input [prefix]]

split -l 500 bigfile.txt 

Pipe Commands¶

ls | grep .txt

Tee Command¶

command | tee file

Number Lines¶

nl [options] [file_name]

WC¶

wc myfile.txt

Expand¶

expand filename

expand -t 4 filename

Unexpand¶

unexpand -t 4 file.txt

Uniq¶

sort names.txt | uniq

GREP¶

grep "pattern" fileName

awk¶

awk '{print $1,$2}' filename

Server Review¶

The process of reviewing a server in Linux involves assessing the server’s performance, security, and configuration to identify areas of improvement or any potential issues.

# A command often used for showing memory information
free -m

# A command for showing disk usage
df -h

# A command for showing CPU load
uptime

Uptime Load¶

$ uptime
 10:58:35 up 2 days, 20 min,  1 user,  load average: 0.00, 0.01, 0.05

Auth Logs¶

These logs, usually located in /var/log/auth.log (for Debian-based distributions) or /var/log/secure (for Red Hat and CentOS).

tail /var/log/auth.log

Services Running¶

Linux has a variety of tools to achieve this, such as: systemctl, service, netstat, ss and lsof.

For example, the command systemctl is widely used on Linux systems to list all running services:

systemctl --type=service 

Evaluating Available Memory¶

Using various command-line tools provided by Linux, such as free, vmstat, and top

$ free -h
              total        used        free      shared  buff/cache   available
Mem:           15Gi       10Gi       256Mi       690Mi       5.3Gi       4.2Gi
Swap:         8.0Gi       1.3Gi       6.7Gi

Process Management¶

Process management is integral part of any operating system and Linux is no different.

ps aux

top

kill -SIGTERM pid
kill -SIGKILL pid

Background and Forground Process¶

In Linux environment, a process can be run in either the foreground (fg) or the background (bg).

command &

CTRL + Z       # This will pause the process
bg             # This resumes the paused process in the background

fg

Listing and Finding Processes (proc)¶

# list all running processes
ps -ef 

# display ongoing list of running processes 
top

# alternatively, for a more user-friendly interface
htop

# view specifics of a particular PID
cat /proc/{PID}/status

Proc Signals under Process Management¶

kill -SIGSTOP 12345

Kill Processes¶

kill [signal or option] PID(s)

Proc Priorities Under Process Management¶

ps -eo pid,pri,user

renice +5 -p [PID]   # Increase priority by 5 units for process ID [PID]

Process Forking in Process Management¶

#include<sys/types.h>
#include<unistd.h>
#include<stdio.h>

int main()
{
    pid_t child_pid;

    // Try creating a child process
    child_pid = fork();

    // If a child is successfully created
    if(child_pid >= 0)
    printf("Child created with PID: %d\n", child_pid);
    else
    printf("Fork failed\n");
    return 0;

User Management¶

Linux operating system offers a structured user management system, allowing multiple users to interact with the same system in an isolated manner. This includes defining user roles, assigning permissions, groups, ownership and other related aspects, which are crucial tasks for Linux administrators.

sudo adduser newuser

sudo deluser newuser

Create, Update, and Delete Users¶

Administrators can create new users using commands like useradd or adduser, update user details such as home directory or login shell with usermod, and delete users using userdel

User Management: Users & Groups¶

Each user belongs to one or more groups, allowing administrators to grant specific privileges without full superuser access. Commands like groupadd, groupdel, groupmod, usermod, and gpasswd are used to manage groups

User Management: Managing Permissions¶

Permissions are categorized into read, write, and execute types and can be set for the file owner (user), the owning group, and others. Commands like chmod, chown, and chgrp are used to view and manipulate these permissions.

Service Management (systemd)¶

Service Management in Linux refers to the system of controlling the services (or “daemons”) that Linux starts and stops during the process of booting up and shutting down your computer. These services perform various functions and provide processes that aren’t attached to the user interface.

Various commands involved in service management in Linux include systemctl start, systemctl stop, systemctl restart, systemctl reload, systemctl status, and systemctl enable/disable, among others.

# Start sshd service
sudo systemctl start sshd

# Check status of sshd service
sudo systemctl status sshd

Service Status¶

systemctl status apache2.service

Start Stop Service¶

# To start a service
sudo systemctl start service_name   

# To stop a service
sudo systemctl stop service_name   

# To restart a service
sudo systemctl restart service_name   

Checking Logs¶

Several essential logs generated by system processes, users and administrator actions can be found in /var/log directory. Logs can be accessed and viewed using several commands. For example, the dmesg command can be used to display the kernel ring buffer. Most system logs are managed by systemd and can be checked using the command journalctl.

journalctl

journalctl -u service_name

Creating Services¶

For instance, we could create a simple my_service.service file:

[Unit]
Description=My Custom Service
After=network.target

[Service]
ExecStart=/path/to/your/executable

[Install]
WantedBy=multi-user.target

Package Management¶

inux distributions use various package managers. Some of the commonly used are apt (Advanced Packaging Tool) for Debian-based distributions, yum (Yellowdog Updater, Modified) and dnf (Dandified YUM) for Red-Hat-based distributions, and pacman for Arch Linux.

For instance, to install a package in a Debian-based distribution, you would use the following command in apt:

sudo apt-get install <package-name>

Linux Package Management: Repositories¶

sudo apt update      # command to update the repository in Ubuntu
sudo yum update      # command to update the repository in CentOS or Fedora
raco pkg update      # command in Racket to update all installed packages

Snap¶

sudo snap install [package-name]

Finding and Installing Packages¶

The ability to efficiently find and install software packages is a fundamental skill when working with Linux based systems. Linux package management tools such as apt, yum, or dnf are used to automate the process of installing, upgrading, configuring, and removing software packages in a consistent manner.

For example, on a Debian-based system like Ubuntu you would use apt or apt-get to install a new package like so:

sudo apt-get update
sudo apt-get install package-name

sudo dnf update
sudo dnf install package-name

Listing Installed Packages¶

Each Linux distribution may come with its own package management system. Examples include apt in Debian-based systems, dnf in Fedora, zypper in OpenSUSE, and pacman in Arch Linux.

Below is the command for listing installed packages in an apt package manager:

sudo apt list --installed

For dnf package manager, you would use:

dnf list installed

Installation, Removal, and Upgrade of Packages¶

Linux distributions use different package managers such as apt for Debian/Ubuntu based distributions, yum and dnf for Fedora/RHEL/CentOS, and zypper for SUSE.

sudo apt-get install packagename

Disk & FileSystems¶

Linux uses a variety of filesystems to allow us to store and retrieve data from the hardware of a computer system such as disks. The filesystem defines how data is organized, stored, and retrieved on these storage devices. Examples of popular Linux filesystems include EXT4, FAT32, NTFS, and Btrfs.

df -T

Inodes¶

In a Linux filesystem, an inode (index node) is a core concept that represents a filesystem object such as a file or a directory

# Retrieve the inode of a file
ls -i filename

Filesystems¶

Linux operating system provides multiple ways to handle the data storage through the concept of filesystems under disks.

df -T

Mounts¶

The mount command in Linux is used for mounting filesystems.

mount /dev/sdb1 /mnt

Adding Disks¶

The following are common commands to manage disks:

Use lsblk to list all block devices (disk and partitions). Use fdisk /dev/sdX to create a new partition on a disk. Use mkfs.ext4 /dev/sdX1 to create a new filesystem on a partition. Use mount /dev/sdX1 /mount/point to mount a filesystem to a directory.

# example commands to add new disk
lsblk                     # list all disks and partitions
sudo fdisk /dev/sdb       # let's suppose new disk is /dev/sdb
sudo mkfs.ext4 /dev/sdb1  # make filesystem(e.g., ext4) on partition 1
sudo mount /dev/sdb1 /mnt # mount new filesystem to /mnt directory

Linux Swap under Disks Filesystems¶

Swap space in Linux is used when the amount of physical memory (RAM) is full. If the system needs more memory resources and the physical memory is full, inactive pages in memory are moved to the swap space. Swap space is a portion of a hard disk drive (HDD) that is used for virtual memory.

fallocate -l 1G /swapfile # creates a swap file
chmod 600 /swapfile # secures the swap file by preventing regular users from reading it
mkswap /swapfile # sets up the Linux swap area
swapon /swapfile # enables the file for swapping

Linux Logical Volume Manager (LVM)¶

LVM works on 3 levels: Physical Volumes (PVs), Volume Groups (VGs), and Logical Volumes (LVs).

PVs are the actual disks or partitions. VGs combine PVs into a single storage pool. LVs carve out portions from the VG to be used by the system. To create an LVM, you need to follow these steps in Linux:

pvcreate /dev/sdb1
vgcreate my-vg /dev/sdb1
lvcreate -L 10G my-vg -n my-lv

Booting Linux¶

Booting Linux refers to the process that the Linux operating system undergoes when a computer system is powered on.

GRUB_DEFAULT=0
GRUB_TIMEOUT=5
GRUB_DISTRIBUTOR=`lsb_release -i -s 2> /dev/null || echo Debian`
GRUB_CMDLINE_LINUX_DEFAULT="quiet splash"
GRUB_CMDLINE_LINUX=""

Introduction to Logs¶

Linux utilizes various log message levels from emerg (the system is unusable) to debug (debug-level messages). During the boot process, messages from various components of the system like kernel, init, services, etc., are stored. Many Linux distributions use systemd logging system, journalctl, which holds the logs of the boot process.

Viewing boot messages can occur in real-time with the dmesg command. It’s used to read and print the kernel ring buffer. Or they can be accessed via the logging setup of your system, which often includes text files in /var/log.

dmesg | lesss

Boot Loaders¶

# This command updates the GRUB bootloader 
sudo update-grub

Networking¶

Networking is a crucial aspect in the Linux environment. It enables Linux systems to connect, interact, and share resources with other systems, be it Linux, Windows, macOS or any other operating system.

Linux adopts a file-based approach for network configuration, storing network-related settings and configurations in standard files, such as /etc/network/interfaces or /etc/sysconfig/network-scripts/, depending on the Linux distribution.

Perhaps one of the most popular commands related to networking on a Linux system is the ifconfig command:

ifconfig

TCP/IP¶

# To view all active TCP/IP network connections
netstat -at

Subnetting¶

# Display current routing table
$ route -n 

# Add a new subnet
$ route add -net xxx.xxx.xxx.x/xx gw yyy.yyy.yyy.y

Ethernet, ARP and RARP¶

DHCP¶

sudo apt-get install isc-dhcp-serve

IP Routing¶

$ ip route show

DNS Resolution in Networking on Linux¶

nslookup www.example.com

dig www.example.com

Netfilter¶

iptables -A INPUT -i eth0 -s 192.168.0.0/24 -m netfilter --netfilter-name example --action drop 

SSH (Secure Shell)¶

ssh username@server_ip_address

Linux File Transfer under Networking¶

Linux provides several command-line tools and applications for network-based file transfers. These tools support various standard protocols such as FTP, HTTP, SCP, SFTP, and NFS. Some of the most well-known commands for file transfer include scp, rsync, and wget.

For instance, when transferring a file from a local machine to a remote server, the scp command can be utilized as follows:

scp /path/to/local/file username@remote:/path/to/destination

Backup Tools¶

Some of the popular and powerful backup tools in Linux include rsync, tar, dump, restore, and various GUI based tools such as Deja Dup and Back In Time. These tools provide various features such as incremental backups, automation, scheduling, and encryption support.

For instance, a basic usage of rsync can be shown below

rsync -avz /source/directory/ /destination/directory

Shell Programming¶

Shell programming, also known as shell scripting, is an integral part of the Linux operating system. A shell script is essentially a program that the system’s shell executes

#!/bin/bash
# My first script
echo "Hello, World!"

Debugging in Shell Programming Under Linux¶

bash -x script.sh

Conditionals in Shell Programming¶

#!/bin/sh
a=10
b=20

if [ $a -lt 20 ]
then
   echo "a is less than b"
elif [ $a -gt 20 ]
then
   echo "a is greater than b"
else
   echo "a is equal to b"
fi

Loops¶

for i in 1 2 3
do
   echo "$i"
done

Literals in Shell Programming on Linux¶

#!/bin/bash
# Example of literals in shell script

StringLiteral="This is a string literal"
NumericLiteral=125
echo $StringLiteral
echo $NumericLiteral

Variables in Shell Programming on Linux¶

Variables fall into two broad categories: System Variables and User-Defined Variables

# Create a User-Defined Variable
MY_VARIABLE="Hello World"

# Print the value of the Variable
echo $MY_VARIABLE  # Output: Hello World

Troubleshooting¶

# example of using a command-line tool for troubleshooting
top

ICMP¶

In Linux systems, common command-line tools related to ICMP include ping and traceroute, both used to diagnose the state of the network and often part of troubleshooting efforts.

# Use of ICMP via the ping command to send an echo request to a specific host
ping www.google.com

Ping¶

ping <target IP or hostname>

Traceroute¶

acing route in Linux can be achieved by executing the traceroute command which allows you to discover the routes that internet protocol packets follow when traveling to their destination.

$ traceroute www.example.com

Netstat¶

# List all connections with numerical values.
netstat -n

Packet Analysis¶

Tools like tcpdump and Wireshark are common utilities for this very purpose

sudo tcpdump -i eth0

Containrization¶

docker run -it ubuntu bash

In Linux, this technology can be utilized using various open-source platforms like Docker and Kubernetes

https://roadmap.sh/linux