Amazing Linux Command Line Tips and Tricks

Linux Command Line Tips & Tricks

In an operating system, Command line is a tool which provides the typing interface to perform commands on computer. This article will walk you trough some time saving tips to working with Linux command line.

Switch working directory to home directory

cd ~

In Linux ~ refers to home directory. You can use cd ~ to switch current working directory to home directory.

Execute previous command

!!

Print Or get process ID of current Shell

echo $$

Inside script “$$” return the process ID of the process in which it is executing

List last n commands used

Command history returns history of commands performed typically 1000 or configured count

history 

Pass number to get last n commands

history 5

Repeat last nth command

!-n executes the nth last command from the history. Where following command executes last 5th command

!-5

Repeat nth command from history

!n executes the command by it’s number from history . Where following command executes the command at 535th entry in command history.

!535

Repeat last command starting with given characters

!string executes the very last command which starts with given string of characters. For example following command executes last command which starts with lsb .

!lsb
lsblk
NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT
loop0 7:0 0 140.7M 1 loop /snap/gnome-3-26-1604/74
loop1 7:1 0 143.5M 1 loop /snap/gnome-3-28-1804/23

Executes last command which contains given characters

!?chars executes the last command which contains given characters. For Example following example executes command which contains getKT.

echo "hello! welcome to getKT"
hello! welcome to getKT
!?getKT
echo "hello! welcome to getKT"
hello! welcome to getKT

Reuse the last word from previous command

!$ refers to the last word in the last command executed in the terminal. In the following example we used created file name using !$ .

echo "hello world" > testfile
# cat !$
cat testfile
hello world

Get process number of last background command

echo $!

Populate command from history while typing

CTRL + R populates last command that matches as you type

~#
(reverse-i-search)`if': ifconfig

Execute command in background

To execute given command in background append & to the command

# command &

For example, the command “sleep 50 &” would execute the command in background because of suffix &. So, it doesn’t block command line.

sleep 50 &
[2] 9416

Suspend the current command

CTRL + z suspends long running command and puts it in background to gives you access to the terminal .

Send command to background

Command which is sent to background can be called as background job (task) . The command bg is used to put given command by it’s JOB_SPEC to background.

# bg --help
bg: bg [job_spec ...]
    Move jobs to the background.
    
    Place the jobs identified by each JOB_SPEC in the background, as if they
    had been started with `&'.  If JOB_SPEC is not present, the shell's notion
    of the current job is used.
    
    Exit Status:
    Returns success unless job control is not enabled or an error occurs.

Where job_spec can be,

  • %n : Job number n
  • %str : Refers to job command which start with str
  • %?str : Refers to job command which contains str
  • %% or %+ : Refers current job
  • %- : Refers the previous job

Bring last background job back to foreground

command fg brings the last background job to foreground. The command which executes in foreground can be called as foreground job(task)

fg

Bring last nth background job to foregrond

fg n brings last nth background job to foreground where n is background job ID not PID

fg 3

jobs command lists all background jobs by it’s job ID and command

Easy cursor navigation by words and lines

Following short cuts will help you to navigate cursor quickly in side command line

Move cursor left by one wordESC + B
Move cursor right by one wordESC + F
Jump to the beginning of the line CTRL + A
Jump to the end of the lineCTRL + E
Delete the previous word CTRL + W
Clear the lineCTRL + U

Unfreeze Linux terminal from accidental CTRL + S

If you freeze your terminal because of habitual CTRL+S. Don’t worry frozen terminal can be unfrozen by CTRL + Q .

Linux terminal copy and paste

Most of the terminals support copy and paste with following short cuts

CopyCTRL + SHIFT+ C
PasteCTRL + SHIFT + V

Also you can use middle mouse button to paste copied or selected text.

Some terminal may support CTRL +C and CTRL +V. On windows on putty, select text for copy and then right click to paste

Linux Shell Command Tips For Files and Directories

Linux Shell Command Tips For Files and Directories

This article would give you quick Linux command tips to perform operation and work with files and directories effectively

Create Multiple Directories at once

mkdir  path/to/{test,work,pictures,newDir}

No spaces around comma ( , ) in side curly braces {}

Above command creates all given directories inside curly braces {} at specified location. Here in this case “test, work, pictures and newDir”

Create Multiple Files Using touch At once

Multiple files can be created at once by passing them enclosed by curly braces {} and separated by comma (,)

touch {pictures,work,fun} 
touch /path/to/{pictures,work,fun} 

Remove All Files Except Specified

By using following syntax you can remove all files except the one or more specified using command rm .

rm !(<file_to_keep>|<another_file_to_keep>)

Let’s say we have following files

ls
cg  manage  python  test  world

We want to keep files test and cg and delete rest, here is the way

rm !(test|cg)

Same thing can also be done by using the command find .

find . -type f ! -name 'filename'  -delete

Create Directories Recursively

Use command line option -p on command “mkdir” to create directories recursively without getting error

mkdir -p /home/demo/server/www

Since we passed option -p mkdir command will create all directories recursively if any directory in the specified path doesn’t exist

Rename Files By Using Regular Expression

There is a handy command called “rename” to rename multiple files according to given pattern. This rename command uses the sed command syntax.

Let’s learn this command by example. For suppose we have following files

ls
testfile  work  worker  workspace

I want to rename work in every file to test. We can achieve this by following command

rename 's/work/test/' * 

As said syntax is similar to command sed. Describing ‘s/work/test’. Where,

  • s is the command which stands for substitute
  • work in ‘s/work/test’ is the text to replace
  • test in ‘s/work/test’ is the text to get replaced

If you want to see what files are going to get renamed instead of taking action. You can do it so by passing command line argument -n .

rename -n 's/work/test/' *
rename(work, test)
rename(worker, tester)
rename(workspace, testspace)

Display File Type Based On Content

We can recognize the type of file by it’s extension but it is not reliable. Actual file type by it’s content can be displayed using command “file” .

$ file <path/to/file>

Command file is more capable than querying file type

Pass desired file path (absolute or relative) to the command file to get it’s type

file welcome.mp3
welcome.mp3: MPEG ADTS, layer III, v2,  24 kbps, 16 kHz, Monaural

Query mime type of file instead of verbose description with command line option -i .

file -i  welcome.mp3
welcome.mp3: audio/mpeg; charset=binary

Print only File type without echoing file name

file -b cg
ASCII text

Create Symbolic Links

Basically there are two types of symbolic links. Hard Link and Soft (Symbolic) Link . Think of these are short cuts in windows

Hard links associate file names of created links with same inode as a result they share the same data block on the disk. It is perceived as independent copy of files.

A file is said to be completely delete only if the file and all of it’s hard links are deleted.

Symbolic Link or Symlink or Soft Link is a file which points to another file. Unlike hard Links, soft links point to path on the file system instead of physical location(inode).

Hard link not allowed for directories, hard links can only be created for files. Where as, soft links can be created for both files and directories

Command ln is used to create Symbolic Links

ln <source> <link>

Create Hard Link

ln  /path/to/source  /path/to/link

Create Soft Link

The command ln created soft link command line argument -s is passed

ln -s  /path/to/source  /path/to/link

Display File Meta Data

The command stat can be used to display file or file system meta data.

stat testfile
File: testfile
Size: 0 Blocks: 0 IO Block: 4096 regular empty file
Device: fd00h/64768d Inode: 1612373 Links: 1
Access: (0644/-rw-r--r--) Uid: ( 0/ root) Gid: ( 0/ root)
Access: 2019-03-25 21:30:20.240532016 +0530
Modify: 2019-03-25 21:30:20.240532016 +0530
Change: 2019-03-25 21:30:20.240532016 +0530
Birth: -

Move All Files and Folders Except Specified

Following command would help you to move all files and folder in current working directory except specified once.

mv !(<file_to_keep>|<another_file_to_keep>) destination

ls

 abc  fileA  fileB  mdir  test  xyz

Following command will move all but specified to destination directory

mv !(fileA|fileB)   /opt/dst

Above command will move all files but specified fileA and fileB to destination /opt/dst

Move all files but specified using the command find

We can also move all but specified using the command find. Using the command find will give us advantage of more fine control over filters.

Move file & folders inside current working directory to specified destination

find -maxdepth 1 -not -name "abc" -exec mv {} /opt/dst \;

Describing above command. Where,

  • Option -maxdepth 1 will force find to look for files & folders in current working or given directory only
  • Option -not negates the match. Spits out all but matched
  • Option -name “abc” . -name takes string too look up files which contain given name. Here, it means to look for file named or contains “abc”.
  • -exec takes external command to execute on matched files.
  • {} will be replaced by matched file/folder path in every match.

You can spice it up, if you are comfortable with regular expressions to match multiple files

find -maxdepth 1 -type f  -regextype egrep -not  -regex './(fileA|fileB)' -exec mv {} /opt/dst \; 

Above command match only files (-type f) other than fileA and fileB and move
(-exec mv {} /opt/dst \;) operation on matched

Switch to Previous Working Directory

As we switch directories or change current working directories using command cd . Linux updates the environment variable both OLDPWD and PWD. we can use environment variable OLDPWD to switch back to previous working directory.

Switch to old working directory

cd $OLDPWD

We can also switch to previous working directory using “cd -“

cd -

List Hidden Files

List all files including hidden files

ls -a
ls -la

Delete Files Based On Their Age

The command find can be used to filter files or folder by their age and perform desired action

find /path/to/files -type f -mtime <time> -exec rm {} \;

find /path/to/files -type f  -mtime +7 -exec rm {} \;

Describing Above Command

  • /path/to/files is the path to look for files
  • -type f is to limit find lookup by files
  • -mtime +5 => -mtime is used to specify time like how old the file can be. +7 means, find files older than 7 days.
  • -exec allows you to execute external command on filters such as rm.
  • {} will be replaced by found file/folder.
  • ;\ is required at the end for this command to work.

Display Files Inside tar file or archive

tar -tf   filename.tar

Read PDF File In Linux Terminal

Well, there is nothing magical we need to read PDF files inside Linux terminal.All you need is the command less .

less /path/to/file.pdf

Empty The File Without Deleting

If you want remove the content of the file instead of deleting it, you don’t have to open a file. Instead you can use the following syntax to empty file.

>  filename

Print First n Lines Of A File

The command head helps you print first n lines of file. It takes command line option -n by default it is 10 if not passed

head filename

Following command prints first 5 lines since command line arg -n 5 is passed

head -n 5 filename

Print Last n Lines Of A File

The command tail helps you to print last n lines of the file. Number of lines it prints is 10 by default if n is not passed.

tail filename

Following command prints only last 5 lines since command line arg -n 5 is passed

tail -n 5 filename

If file tend to change often (ex: log file) we can force tail command to listen for added lines at the end and print on console

tail -f filename.log

Install Gnome Desktop Environment (GUI) on CentOS/RHEL

Install GNOME GUI On CentOS

The linux distribution CentOS is widely used on server side without GUI. Which might not come with desktop environment. If you want to install GUI for any kind of purpose like connecting to server over VNC or using it on desktop. This article will help you to install GNOME Desktop environment

GNOME is part of GNU Project. It’s official GNU desktop platform. GNOME stands form GNU Network Object Model Environment .

You can more about GNOME graphical shell at it’s official websites

List Installed Desktop Environments

Sometimes it is possible that desktop environment you are looking for to install is installed but not enable. List installed desktop environments using following command

ls -l /usr/share/xsessions/

If you can see your desktop environment skip to Start System with GUI

Install GNOME dekstop environment

On centos all packages related to GNOME dekstop environment are available as group “GNOME Desktop”. Install it using “yum groups install”

yum -y groups install "GNOME Desktop" 

It might take a while to install all packages of group “GNOME Desktop”, and also depends on internet speed.

Start System With GUI

After installation you can start GUI from command line manually as follows

startx

(OR) Start our new GNOME GUI systemctl

systemctl isolate graphical.target

Installing GNOME GUI won’t start system with GUI on reboot. You have to enable it, to do so change run level , which is 3 (multi-user networking mode) by default. Change the target to run level 5 which means “run level 3 + display manager” to start system normally with GUI

Check current target

systemctl get-default
multi-user.target

Change default target to start system with GUI

systemctl set-default graphical.target 
rm '/etc/systemd/system/default.target'
ln -s '/usr/lib/systemd/system/graphical.target' '/etc/systemd/system/default.target'

Verify Updated target

systemctl get-default
graphical.target

If you see “graphical.target” , you are good to go. System will start with GUI.

On first run of GNOME GUI, you will be prompted with gnome initial setup. Follow steps

  • Select System Language
  • Select Keyboard layout
  • Following along till the end, you are done.

Uninstall GNOME Desktop Environment

For some reason if you want to uninstall GNOME GUI, you can do it using “yum groups remove ”

yum groups remove "GNOME Desktop"

(or)

yum groupremove "GNOME Desktop"

Optionally you can remove any related entries from file ~/.xinitrc .

Quick Summary (Turbo mode)

sudo yum -y groups install "GNOME Desktop"   # Install Gnome Dekstop
sudo systemctl set-default graphical.target # Enable GUI on startup
sudo reboot

Troubleshooting GNOME GUI installation

  • Make sure system is connected to network and able to download packages
  • If you don’t see package group install/enable corresponding YUM Repository
  • yum plugin protectbase might block install/update of packages. Manage your repositories.
  • To solve Issue “conflicts with file from package grub2-common” . Upgrade grub2-common “yum update grub2-common”

Endianness. Little Endian vs Big Endian

Endianness. Little Endian vs Big Endian

Endianness describes the byte order in which bytes of large values are stored, transferred and accessed. Based on order in which bytes are stored, transmitted and accessed there are two main types of endianness called little-endian and big-endian .

Before we understand Little Endian and Big Endian we need to understand how memory and it’s addressing work.

Computer Memory Address

Computer memory is generally abstracted as an ordered series of bytes. Memory address is a number which is reference to a specific memory location in which data(by bytes) is stored . Where computer divides memory into bytes, each byte is assigned a unique number which is a memory address so called byte addressable. Whole memory (like RAM) is addressed by bytes regardless of whether it is being used or not for storage. So that CPU can track where data and instructions are stored in RAM by using memory address.

Memory Addressing. Flat Memory Model
Memory Addressing. Flat Memory Model

First memory location(byte) is assigned lowest possible memory address and further locations are addressed with increasing memory address. Large values are divided into bytes and placed in each memory location.

Question: How large values which require more than one byte to store or transmit are arranged ?

Order in which bytes of large values are stored or transmitted over network is described by Endianness or Byte Order .

To better understand byte order. Have a look at terminology Least Significant Byte, Most Significant Byte, Left Most Byte and Right Most Byte.

In any positional number system including binary we count position from right to left. As we write numbers(from left to write) in binary representation of decimal number we call,

  • Byte which is having least positional value is called Least Significant Byte
  • Least Significant Byte can also be called as Right Most Byte since it comes right most as we write from left to right
  • Byte which is having greatest positional value is called Most Significant Byte
  • Most Significant Byte can also be called as Left Most Byte since it comes left most as write from left to right

Following Image illustrates terms Least Significant Byte and Most Significant Byte. By taking number 0x1A2B3C4D which is of size 32 bits.

Bit Numbering. Terms - Least Significant Byte, Most Significant Byte
Bit Numbering. Terms – Least Significant Byte and Most Significant Byte

Least significant byte can also be referred as LSB or LSByte
Most significant byte can also be referred as MSB or MSByte

Endianness

The Origin Of Words Of Endianness

Words Little Endian and Big Endian are originated from the 1726 novel Gulliver’s Travels by Jonathan Swift. In which civil war breaks out over whether the big end or the little end of the egg is proper to crack open. Words Little Endian and Big Endian are used to name two groups who fight over little end and big end of the egg is proper way to break egg receptively .

The origin of words of Endianness. Little Endian and Big Endian
The origin of words of Endianness. Little Endian and Big Endian


The novel further describes an intra-Lilliputian quarrel over the practice of breaking eggs. Traditionally, Lilliputians broke boiled eggs on the larger end; a few generations ago, an Emperor of Lilliput, the Present Emperor’s great-grandfather, had decreed that all eggs be broken on the smaller end after his son cut himself breaking the egg on the larger end. The differences between Big-Endians (those who broke their eggs at the larger end) and Little-Endians had given rise to “six rebellions… wherein one Emperor lost his life, and another his crown”. The Lilliputian religion says an egg should be broken on the convenient end, which is now interpreted by the Lilliputians as the smaller end. The Big-Endians gained favour in Blefuscu.

Excerpt of story from wiki Lilliput and Blefuscu

Endianness. The Byte Order

Endianness is also called as Byte Order which describes order in which bytes of large values are stored in memory or transmitted over network in a transmission protocol or a stream (ex. an audio, video streams).

There are two main endianness formats based on whether Most Significant Byte (MSB) or Least Significant Byte (LSB) comes first, when storing data or transmitting streams. Which are Little-Endian and Big-Endian

  • If Least Significant Byte (LSB) is stored or transmitted first, it is said to be Little-Endian
  • If Most Significant Byte (MSB) is stored or transmitted first, it is said to be Big-Endian

Using words Little-Endian and Big-Endian for byte order is analogous to cracking egg on little end and big end respectively

Endianness - The Byte Order - Little Endian and Big Endian
Endianness – The Byte Order – Little Endian and Big Endian. Cracking Egg Analogy

It is interesting topic in computer science, since these two formats are conflicting. Using wrong byte order may potentially read wrong data or corrupt when writing.

Most Computers prefer to handle all of it’s data in single byte order. Systems native byte order is default

You may not need to worry about endianness and endianness conversion unless you work with assembly, drivers, middle level languages like c, embedded systems and low level networking transports. High level programming languages will take care of it for you.

Endianness - Little Endian vs Big Endian
Endianness – Little Endian vs Big Endian

Little Endian

Like Little-Endians crack egg on little end, least significant byte is stored or transmitted over network first in Little-Endian format. Little endian format is also called as host byte order.

Little-Endian is also known as the “Intel convention”. As Intel used little-endian format from it’s earlier processors.

  • Least significant byte is stored or transmitted first
  • When bytes of large values are stored in memory “Least Significant Byte” is placed first in memory at lowest memory address
  • Numeric significance or positional value is increased with increasing memory address
  • Easier mathematical computation. Mathematical operations mostly work from LSB

Little Endian
Little-Endian

Big Endian

Like Big-Endians crack egg on big end, most significant byte is stored or transmitted over network first in Big-Endian format. Big endian format is also called as network byte order.

Over decades most of processors took transition to little endian at present many processors(laptops, desktops and servers) come in little endian. Yet, big endian is widely used for network transmission. So it is called “network byte order” .

Big-Endian is also known as the “Motorola convention”.Back then as it was widely used by Motorola processors.

  • Most significant byte is stored or transmitted first
  • When bytes of large values are stored in memory “Most Significant Byte” is placed first in memory at lowest memory address
  • Numeric significance or positional value is decreased with increasing memory address
  • Quick sign checking. Since MSB stores first.
  • Better human readability as it is one to one mapping. Bytes are in the same order as we write.
Big Endian
Big-Endian

Check Native Endianness (Native Byte Order)

Check native “byte order” of system using linux command line (terminal)

You can use the command “lscpu” to get CPU information including it’s byte order

# lscpu
Architecture:        x86_64
CPU op-mode(s):      32-bit, 64-bit
Byte Order:          Little Endian
CPU(s):              2
On-line CPU(s) list: 0,1
Thread(s) per core:  1
Core(s) per socket:  2
Socket(s):           1
NUMA node(s):        1
Vendor ID:           GenuineIntel
CPU family:          6
Model:               158
Model name:          Intel(R) Core(TM) i7-7700HQ CPU @ 2.80GHz
Stepping:            9
CPU MHz:             2808.002
BogoMIPS:            5616.00
Hypervisor vendor:   KVM
Virtualization type: full
L1d cache:           32K
L1i cache:           32K
L2 cache:            256K
L3 cache:            6144K
NUMA node0 CPU(s):   0,1
Flags:               fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx rdtscp lm constant_tsc rep_good nopl xtopology nonstop_tsc cpuid pni pclmulqdq ssse3 cx16 pcid sse4_1 sse4_2 x2apic movbe popcnt aes xsave avx rdrand hypervisor lahf_lm abm 3dnowprefetch invpcid_single pti fsgsbase avx2 invpcid rdseed clflushopt

As you can see endianness(byte order) in the above output “Byte Order: Little Endian“.

Check native byte order using python programming language

Python sys module has attribute called “byteorder” where it’s value describes native byte order of the system .

  • sys.byteorder will be “big” if it’s big-endian (most significant byte first) platform
  • sys.byteorder will be “little” if it is little-endian (least significant byte first) platform
>>> import sys
>>> sys.byteorder
little

Check native byte order of system using java

In java “java.nio.ByteOrder” provides flags “BIG_ENDIAN” and “LITTLE_ENDIAN” to denote Big-Endian and Little-Endian respectively. It has the static method “nativeOrder()” to get system’s native byte order.

java.nio.ByteOrder.nativeOrder()

Check endianness of system using C program

#include <stdio.h>
int main()
{
unsigned int i = 1;
char *c = (char*)&i;
if (*c)
printf("Little Endian System");
else
printf("Big Endian System");
getchar();
return 0;
}

Above c program prints Endianess of the system. In the above program we have used character pointer “c” to dereference integer “i“. Since size of character is 1 byte it will contain only first byte of integer.

  • *c will return 1 which is Least Significant Byte on Little-Endian machine.
  • *c will return 0 which is Most Significant Byte on Big-Endian machine

Following “c” program prints memory representation of given 32-bit integer. You can see byte order in action, order in which given 32-bit integer is stored in memory either by Little-Endian or Big-Endian.

Above program will print ” 4d 3c 2b 1a” if machine is Little-Endian (Least Significant Byte Fist) or “1a 2b 3c 4d” if machien is Big-Endian (Most Significant Byte Fist)

Convert Endianness

It is required sometimes to convert endianness. You don’t have to worry about converting endiannes if you are working application level. For data exchange between machines common format is used. Yet if machine is little endian often data sent over network is big endian. Big-Endian is standard byte order for network communication. In such a case we have to convert data that we receive in Big-Endian to host fomat little endian and when we send data back to network need to convert Little-Endian to Big-Endian. There are many utility function to perform this conversion.

Following function are available in c to help endianness conversion

htons() – Host to Network Short
htonl() – Host to Network Long
ntohs() – Network to Host Short
ntohl() – Network to Host Long

Python library NumPy provides way to convert endianness. NumPy library uses c types and low level data structures behind.

How Systems Of Different Endianness Interact

Files that are created on little-endian machine won’t be read as they were created on big-endian machines without conversion. This is the reason, some programs binary file formats which are endianness independent. Where files are endianness dependent byte order marks are used in headers of file to hint a program endianness of the file.

Example: TIFF files use MM and II in header (begning of file) for big-endian and little-endian respectively. Where MM and II are palindromes these are endianness independent, interpreted same on both little-endian and big-endian machines.

Unicode uses byte order mark in the beginning of the stream to indicated endianness of following unicode bytes.

Endianness problems. Who needs to worry about byte order

You don’t need to worry much if you work with application layer protocols or high level programming languages, but it is good to know.

Think what’s the out put of following program

#include <stdio.h>

void foo(void *d){
printf("Data = %d\n", *(char *)d);
}

int main(void){
unsigned int i = 1;
foo(&i);
}

Summary

  • Words Little-Endian and Big-Endian came from 1726 novel Gulliver’s Travels by Jonathan Swift
  • System is said to be Little-Endian if it stores or transmits Least Significant Byte First
  • System is said to be Big-Endian if it stores or transmits Most Significant Byte First
  • Command “lscpu” on linux will print CPU information including Byte Order (Endianness)
  • Currently m
  • ost of processors are little endian
  • Big Endian is mostly used in network protocols so called network byte order .
  • You don’t have to worry about byte order (endianness) compatibility, system will take care of it behind the senses. Unless you work with assembly, c/c++ language, drivers, embedded system and low level network transports.
  • Choose One endianness and stick to it to avoid obscured bugs.