Setting up Minikube and kubectl for Kubernetes Development

K8s Development Pack

Minikube is a tool that makes it easy to run and setup kubernetes cluster for development locally on laptop or desktop. Minikube runs a single node cluster inside given Virtual Machine. It is also possible to run minikube on host without VM on Linux. It’s goal is to enable fast local development and support all Kubernetes features that fit. This article walks you through installation of minikube and kubectl for kubernetes development for different platforms.

Minikube installation requires Hypervisor and corresponding driver for minikube. Minikube supports following drivers

  • virtualbox
  • vmwarefusion
  • kvm2
  • kvm
  • hyperkit
  • xhyve
  • hyperv
  • none

The derive “none” is the special one. Which works on only Linux. It is helpful if you want to run Minikube on host without VM. Especially it is helpful if minikube is configured to use inside virtual machine.

Install kubectl

The command line tool kubectl is used to manage application on Kubernetes cluster.

It is important to install kubectl which is compatible with kubernetes cluster version. Choose kubectl version that is within one minior version difference of targeted kuberntes cluster. i.e, v1.2 client should work with v1.1, v1.2 and v1.3 .
You can check kubectl version using command “kubectl version

Install kubectl on Windows

Install kubectl using Chocolaty

choco install kubernetes-cli

Install kubectl using Scoop

scoop install kubectl 

Install kubectl on macOS

Install kubectl using HomeBrew

brew install kubernetes-cli

Install kubectl using MacPorts

sudo port selfupdate
sudo port install kubectl

Install kubectl on Linux

Install kubectl using package manager

For Debian base distributions

sudo apt-get update && sudo apt-get install -y apt-transport-https
curl -s https://packages.cloud.google.com/apt/doc/apt-key.gpg | sudo apt-key add -
echo "deb https://apt.kubernetes.io/ kubernetes-xenial main" | sudo tee -a /etc/apt/sources.list.d/kubernetes.list
sudo apt-get update
sudo apt-get install -y kubectl

For Red Hat base distributions

cat < /etc/yum.repos.d/kubernetes.repo
[kubernetes]
name=Kubernetes
baseurl=https://packages.cloud.google.com/yum/repos/kubernetes-el7-x86_64
enabled=1
gpgcheck=1
repo_gpgcheck=1
gpgkey=https://packages.cloud.google.com/yum/doc/yum-key.gpg https://packages.cloud.google.com/yum/doc/rpm-package-key.gpg
EOF
yum install -y kubectl

Install kubectl from binaries

Current Stable version number can be found at,

https://storage.googleapis.com/kubernetes-release/release/stable.txt

Download Binaries of stable kubectl version

curl -LO https://storage.googleapis.com/kubernetes-release/release/$(curl -s https://storage.googleapis.com/kubernetes-release/release/stable.txt)/bin/linux/amd64/kubectl

If you are looking for a specific version, replace $(curl -s https://storage.googleapis.com/kubernetes-release/release/stable.txt) with version number that you want to install.

Once the file kubectl is downloaded make it executable

chmod +x ./kubectl

Move kubectl to one of executables PATH, i.e /usr/local/bin

sudo mv ./kubectl /usr/local/bin/kubectl 

Configure kubectl

Starting Minikube for the first time (command minikube start) will automatically create a context called “minikube” which is configured to point kubectl to the minikube cluster.

Default configuration files that kubectl reads can be found at,

  • ~/.kube/config on Linux
  • %USERPROFILE%/.kube/config on Windows

Install Minikube

Prerequisites

  • VT-x or AMD-v Virtualization must be enabled in BIOS
  • Hypervisor
  • kubectl on host

On Linux, the driver “none” (–vm-driver=none) is supported. We don’ need any kind of hypervisor but the docker (container tooling) if driver “none” is used. It is not recommended for production.

Install Minikube On Windows

Prerequisites

  • Hypervisor such as VirtualBox or HyperV
  • VT-x/AMD-v virtualization must be enabled in BIOS

VirtualBox is recommended. Minikube uses virtualbox driver by default on windows

Install Minikube Using Chocolatey

This is the easiest way to install Minikube on windows using Chocolatey.

choco install minikube kubernetes-cli

It is required to open power shell as administrator (run as administrator )

Install Minikube Using Windows Installer

Download the installer at minikube-installer-latest.exe and execute to install.

Install Minikube Manually

To manually install Minikube on Windows,

Install Specific Version

Specific version can be downloaded and installed from minikube-releases

Install Minikube On Linux

Install Minikube Using Binaries

Download Binaries and change permissions to make it executable

curl -Lo minikube https://storage.googleapis.com/minikube/releases/latest/minikube-linux-amd64 
chmod +x minikube  

Add Minikube executable to path

sudo cp minikube /usr/local/bin && rm minikube

Install Minikube On macOS

We can install Minikube on macOS using Hombrew

brew cask install minikube

It is also possible to install Minikube from binaries just in case specific version is required

(OR)

Download Minikube Binary, make it executable and add it to path

curl -Lo minikube https://storage.googleapis.com/minikube/releases/latest/minikube-darwin-amd64 
chmod +x minikube
 sudo mv minikube /usr/local/bin

kubectl Cluster Configuration for minikube

The command minikube start creates a context called “minikube” for kubectl. Minikube sets this context default on installation. This context contains the configuration to communicate with Minikube cluster.

Display kubectl configuration

kubectl config view

The command kubectl can be configured with multiple contexts to communicate with different kubernetes clusters. If you need to switch back to minikube context

kubectl config use-context minikube

Start Minikube

Starting Minikube for the first time will configure cluster and kubectl.

Start Minikube

minikube start

Above command uses the default driver of corresponding platform. We can instruct minikube to use specific driver when we start for the first time . For example, we want to use HyperV as hypervisor. Thus driver will be hyperv

minikube start --vm-driver=hyperv

Start Minikube with specific version

It is also possible to specify version of Kubernetes using command line argument
–kubernetes-version. For example to specify version 1.11.3 command would be

minikube start --kubernetes-version v1.11.3

Test Installed Minikube

To test if Minikube is installed successfully and the command line tool kubectl is pointing to installed Minikube

Check Minikube status

minikube status
minikube: Running
cluster: Running
kubectl: Correctly Configured: pointing to minikube-vm at 10.0.2.15

Check kubectl version

kubectl version
Client Version: version.Info{Major:"1", Minor:"10", GitVersion:"v1.10.2", GitCommit:"81753b10df112992bf51bbc2c2f85208aad78335", GitTreeState:"clean", BuildDate:"2018-04-27T09:22:21Z", GoVersion:"go1.9.3", Compiler:"gc", Platform:"linux/amd64"}
Server Version: version.Info{Major:"1", Minor:"10", GitVersion:"v1.10.0", GitCommit:"fc32d2f3698e36b93322a3465f63a14e9f0eaead", GitTreeState:"clean", BuildDate:"2018-03-26T16:44:10Z", GoVersion:"go1.9.3", Compiler:"gc", Platform:"linux/amd64"}

If you see both “Client Version” and “Server Version”. It means both kubectl (client) and Kubernetes (master) version are aligned and kubectl is able to communicate with k8s API Server.

Stop Cluster

minikube stop

This command shuts down the Minikube (Virtual Machine) but preserves cluster state and data. So, starting cluster again will restore the previous state and data.

Restarting cluster doesn’t require a drive parameter since minikube is already configured and previous state is stored.

Delete Cluster

minikube delete

This command is used to delete whole cluster. This command shuts down and deletes Minikube (Virtual Machine), deletes cluster state and data.

You might want to re-setup your Minikube if cluster is malfunctioning by deleting it using minikube delete .

Cleanup For Fresh Start

To cleanup installed old minikube

Delete Minikube Cluster

minikube delete

Remove Configurations

rm -rf ~/.minikube

Troubleshooting Minikube

  • Check compatibility of kubernetes server and client(kubectl)
  • If possible cleanup old minikube and it’s configuration if it is causing conflicts
  • Make sure system is connected to internet and domain names are getting resolved
  • Make sure ports that are required by minikube/kubernetes are free if driver “none” is used on Linux
  • Make sure container tooling like docker is compatible and up to date

Error : crictl not found in system path

crictl is a Container Runtime Interface (CRI) CLI. Which is required by minikube. Install crictl using following commands

VERSION="v1.13.0" 
wget https://github.com/kubernetes-sigs/cri-tools/releases/download/$VERSION/crictl-$VERSION-linux-amd64.tar.gz
sudo tar zxvf crictl-$VERSION-linux-amd64.tar.gz -C /usr/local/bin
rm -f crictl-$VERSION-linux-amd64.tar.gz

On Ubuntu you can get more details of a failure of ” minikube start” by using command journalctl

journalctl -r

Delete old cluster before “minikube start”. Following command could help

minikube delete && minikube start --kubernetes-version v1.11.3 --vm-driver=none --bootstrapper kubeadm

You can replace –vm-driver=none with supported driver for the hypervisor you are using. –kubernetes-version v1.11.3 is optional unless you are looking for specific Kubernetes version

Top # Search Engines List For Webmasters

Top Search Engines List

Web search engine is a software which crawls whole internet and indexes (stores) to make World wide web available to you at your finger tips.

Regardless of who(audience) you are targeting either domestic(local) or global . To better reach you audience you need to do SEO (Search Engine Optimization). Search engines crawl internet by following links they encounter but we can speed up things by feeding these search engines manually using web master tools they offer.

If you really care about global exposure and getting traffic from across the glob don’t just limit your self by one search engine. Optimize your site for multiple search engines & multiple languages.

Google

Well, well. I think google needs no introduction. Google is the most popular search engine in the world with highest search engine mark share.

www.google.com

Google search console(and google analytics) is the go to place for webmaster to manage their properties, submit sitemaps and monitor traffic.

Google webmaster tools

Google Search Console

Google search console tools & reports will help you to submit/update sitemap, analyse and measure search traffic, spot search trends, measure performance, identify issues, monitor search impressions/click trough rate and fix issues. More than anything google search console helps you to make sure your site is visible in google search results.

Google Analytics

Google analytics is a service that provides statics and analytical tools for Search Engine Optimization (SEO). It is lot powerful than it seems. Google analytics can be used to monitor site traffic and optimize websites for both users and search engines.

Bing

www.bing.com

Bing is the search engine which is owned by Microsoft. Bing is decedent of it’s previous search engines “MSN Search”, “windows live search” and “Live Search”, these search engines were replaced by (re-branded as ) Bing. Microsoft officially replaced Live Search by Bing on 2009. Now, bing is one of biggest search engine competing with google and others. As of 2019 it is in second place with market share 2.41%.

Big Webmaster Tools

Bing webmaster tools are like google search console and google analytics. Use Bing webmaster tools to submit your website and it’s sitemap to get indexed by Bing.

Add your site and verify against Bing to start monitoring your search appearances.

Yahoo

www.yahoo.com Yahoo Search

Yahoo! search engine is owned by american company yahoo which was launched in 1995. As of 2019 Yahoo reserves 3rd place in search engine market share after Bing. Yahoo and Microsoft had a deal in 2009 where yahoo search would be powered by Bing. Yahoo launched it’s localized versions in many countries.

As of writing this article Yahoo! doesn’t offer any webmaster tools or portal to manage websites since it uses Bing behind. Adding your site on Bing would help.

Baidu

www.baidu.com

Baidu is the biggest popular search engine in China. It is china’s local search engine. Because of the fact that china having so many internet consumers. Search engine baidu stands fourth in a row after yahoo according to market share.

Yandex

yandex.com

Yandex is the web search engine in Russia which is owned by Russian corporation Yandex. Yandex search engine is a strong competitor for google in Russia. As of 2019 Yandex is in 4th place after Baidu according to it’s market share.

Yandex Webmaster

Yandex Webmaster is the portal where you can submit sitemap, monitor search engine appearance, stats and performance.

DuckDuckGo

duckduckgo.com duck.com ddg.gg

The web search engine DuckDuckGo known as DDG . Which is known as DuckDuckGo the search engine that doesn’t track you. DuckDuckGo is getting popular, more and more people are using it who are concerned about privacy and online tracking. It is minimalistic, doesn’t store personal information, doesn’t follow with ads and tracking. Since it doesn’t track, it won’t provide an personalized search results.

As of 2019 DuckDuckGo has market share of 0.41%. DDG gets revenue from Yahoo-Bing Search alliance network and affiliate links.

The search engine DuckDuckGo doen’t offer any webmaster tools to submit your website. Submitting your site and sitemap at Google and Bing would help. Duckduckgo uses multiple search sources to index, once your site is picked by other search engines, it will show up on DuckDuckGo as well.

www.naver.com

The Naver is a regional web search engine which is quite popular and ranked first in South Korea. Naver has more than 72% of market share in South Korea. Naver offers other suite of tools including web search like translation service etc.

Naver Featured Services Naver Webmaster Tools

Seznam

www.seznam.cz Seznam.cz Search

Seznam.cz is the regional search engine which serves “Czech Republic” and “Slovakia”. It is founded in 1996 which offers 15+ other services along with internet search. Seznam is the strongest competitor in Czech Republic for google. Seznam ranks second after google in Czech Republic.

Other Regional Search Engines

Japan: Goo Israel: Walla

Install Gnome Desktop Environment (GUI) on CentOS/RHEL

Install GNOME GUI On CentOS

The linux distribution CentOS is widely used on server side without GUI. Which might not come with desktop environment. If you want to install GUI for any kind of purpose like connecting to server over VNC or using it on desktop. This article will help you to install GNOME Desktop environment

GNOME is part of GNU Project. It’s official GNU desktop platform. GNOME stands form GNU Network Object Model Environment .

You can more about GNOME graphical shell at it’s official websites

List Installed Desktop Environments

Sometimes it is possible that desktop environment you are looking for to install is installed but not enable. List installed desktop environments using following command

ls -l /usr/share/xsessions/

If you can see your desktop environment skip to Start System with GUI

Install GNOME dekstop environment

On centos all packages related to GNOME dekstop environment are available as group “GNOME Desktop”. Install it using “yum groups install”

yum -y groups install "GNOME Desktop" 

It might take a while to install all packages of group “GNOME Desktop”, and also depends on internet speed.

Start System With GUI

After installation you can start GUI from command line manually as follows

startx

(OR) Start our new GNOME GUI systemctl

systemctl isolate graphical.target

Installing GNOME GUI won’t start system with GUI on reboot. You have to enable it, to do so change run level , which is 3 (multi-user networking mode) by default. Change the target to run level 5 which means “run level 3 + display manager” to start system normally with GUI

Check current target

systemctl get-default
multi-user.target

Change default target to start system with GUI

systemctl set-default graphical.target 
rm '/etc/systemd/system/default.target'
ln -s '/usr/lib/systemd/system/graphical.target' '/etc/systemd/system/default.target'

Verify Updated target

systemctl get-default
graphical.target

If you see “graphical.target” , you are good to go. System will start with GUI.

On first run of GNOME GUI, you will be prompted with gnome initial setup. Follow steps

  • Select System Language
  • Select Keyboard layout
  • Following along till the end, you are done.

Uninstall GNOME Desktop Environment

For some reason if you want to uninstall GNOME GUI, you can do it using “yum groups remove ”

yum groups remove "GNOME Desktop"

(or)

yum groupremove "GNOME Desktop"

Optionally you can remove any related entries from file ~/.xinitrc .

Quick Summary (Turbo mode)

sudo yum -y groups install "GNOME Desktop"   # Install Gnome Dekstop
sudo systemctl set-default graphical.target # Enable GUI on startup
sudo reboot

Troubleshooting GNOME GUI installation

  • Make sure system is connected to network and able to download packages
  • If you don’t see package group install/enable corresponding YUM Repository
  • yum plugin protectbase might block install/update of packages. Manage your repositories.
  • To solve Issue “conflicts with file from package grub2-common” . Upgrade grub2-common “yum update grub2-common”

Endianness. Little Endian vs Big Endian

Endianness. Little Endian vs Big Endian

Endianness describes the byte order in which bytes of large values are stored, transferred and accessed. Based on order in which bytes are stored, transmitted and accessed there are two main types of endianness called little-endian and big-endian .

Before we understand Little Endian and Big Endian we need to understand how memory and it’s addressing work.

Computer Memory Address

Computer memory is generally abstracted as an ordered series of bytes. Memory address is a number which is reference to a specific memory location in which data(by bytes) is stored . Where computer divides memory into bytes, each byte is assigned a unique number which is a memory address so called byte addressable. Whole memory (like RAM) is addressed by bytes regardless of whether it is being used or not for storage. So that CPU can track where data and instructions are stored in RAM by using memory address.

Memory Addressing. Flat Memory Model
Memory Addressing. Flat Memory Model

First memory location(byte) is assigned lowest possible memory address and further locations are addressed with increasing memory address. Large values are divided into bytes and placed in each memory location.

Question: How large values which require more than one byte to store or transmit are arranged ?

Order in which bytes of large values are stored or transmitted over network is described by Endianness or Byte Order .

To better understand byte order. Have a look at terminology Least Significant Byte, Most Significant Byte, Left Most Byte and Right Most Byte.

In any positional number system including binary we count position from right to left. As we write numbers(from left to write) in binary representation of decimal number we call,

  • Byte which is having least positional value is called Least Significant Byte
  • Least Significant Byte can also be called as Right Most Byte since it comes right most as we write from left to right
  • Byte which is having greatest positional value is called Most Significant Byte
  • Most Significant Byte can also be called as Left Most Byte since it comes left most as write from left to right

Following Image illustrates terms Least Significant Byte and Most Significant Byte. By taking number 0x1A2B3C4D which is of size 32 bits.

Bit Numbering. Terms - Least Significant Byte, Most Significant Byte
Bit Numbering. Terms – Least Significant Byte and Most Significant Byte

Least significant byte can also be referred as LSB or LSByte
Most significant byte can also be referred as MSB or MSByte

Endianness

The Origin Of Words Of Endianness

Words Little Endian and Big Endian are originated from the 1726 novel Gulliver’s Travels by Jonathan Swift. In which civil war breaks out over whether the big end or the little end of the egg is proper to crack open. Words Little Endian and Big Endian are used to name two groups who fight over little end and big end of the egg is proper way to break egg receptively .

The origin of words of Endianness. Little Endian and Big Endian
The origin of words of Endianness. Little Endian and Big Endian


The novel further describes an intra-Lilliputian quarrel over the practice of breaking eggs. Traditionally, Lilliputians broke boiled eggs on the larger end; a few generations ago, an Emperor of Lilliput, the Present Emperor’s great-grandfather, had decreed that all eggs be broken on the smaller end after his son cut himself breaking the egg on the larger end. The differences between Big-Endians (those who broke their eggs at the larger end) and Little-Endians had given rise to “six rebellions… wherein one Emperor lost his life, and another his crown”. The Lilliputian religion says an egg should be broken on the convenient end, which is now interpreted by the Lilliputians as the smaller end. The Big-Endians gained favour in Blefuscu.

Excerpt of story from wiki Lilliput and Blefuscu

Endianness. The Byte Order

Endianness is also called as Byte Order which describes order in which bytes of large values are stored in memory or transmitted over network in a transmission protocol or a stream (ex. an audio, video streams).

There are two main endianness formats based on whether Most Significant Byte (MSB) or Least Significant Byte (LSB) comes first, when storing data or transmitting streams. Which are Little-Endian and Big-Endian

  • If Least Significant Byte (LSB) is stored or transmitted first, it is said to be Little-Endian
  • If Most Significant Byte (MSB) is stored or transmitted first, it is said to be Big-Endian

Using words Little-Endian and Big-Endian for byte order is analogous to cracking egg on little end and big end respectively

Endianness - The Byte Order - Little Endian and Big Endian
Endianness – The Byte Order – Little Endian and Big Endian. Cracking Egg Analogy

It is interesting topic in computer science, since these two formats are conflicting. Using wrong byte order may potentially read wrong data or corrupt when writing.

Most Computers prefer to handle all of it’s data in single byte order. Systems native byte order is default

You may not need to worry about endianness and endianness conversion unless you work with assembly, drivers, middle level languages like c, embedded systems and low level networking transports. High level programming languages will take care of it for you.

Endianness - Little Endian vs Big Endian
Endianness – Little Endian vs Big Endian

Little Endian

Like Little-Endians crack egg on little end, least significant byte is stored or transmitted over network first in Little-Endian format. Little endian format is also called as host byte order.

Little-Endian is also known as the “Intel convention”. As Intel used little-endian format from it’s earlier processors.

  • Least significant byte is stored or transmitted first
  • When bytes of large values are stored in memory “Least Significant Byte” is placed first in memory at lowest memory address
  • Numeric significance or positional value is increased with increasing memory address
  • Easier mathematical computation. Mathematical operations mostly work from LSB

Little Endian
Little-Endian

Big Endian

Like Big-Endians crack egg on big end, most significant byte is stored or transmitted over network first in Big-Endian format. Big endian format is also called as network byte order.

Over decades most of processors took transition to little endian at present many processors(laptops, desktops and servers) come in little endian. Yet, big endian is widely used for network transmission. So it is called “network byte order” .

Big-Endian is also known as the “Motorola convention”.Back then as it was widely used by Motorola processors.

  • Most significant byte is stored or transmitted first
  • When bytes of large values are stored in memory “Most Significant Byte” is placed first in memory at lowest memory address
  • Numeric significance or positional value is decreased with increasing memory address
  • Quick sign checking. Since MSB stores first.
  • Better human readability as it is one to one mapping. Bytes are in the same order as we write.
Big Endian
Big-Endian

Check Native Endianness (Native Byte Order)

Check native “byte order” of system using linux command line (terminal)

You can use the command “lscpu” to get CPU information including it’s byte order

# lscpu
Architecture:        x86_64
CPU op-mode(s):      32-bit, 64-bit
Byte Order:          Little Endian
CPU(s):              2
On-line CPU(s) list: 0,1
Thread(s) per core:  1
Core(s) per socket:  2
Socket(s):           1
NUMA node(s):        1
Vendor ID:           GenuineIntel
CPU family:          6
Model:               158
Model name:          Intel(R) Core(TM) i7-7700HQ CPU @ 2.80GHz
Stepping:            9
CPU MHz:             2808.002
BogoMIPS:            5616.00
Hypervisor vendor:   KVM
Virtualization type: full
L1d cache:           32K
L1i cache:           32K
L2 cache:            256K
L3 cache:            6144K
NUMA node0 CPU(s):   0,1
Flags:               fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx rdtscp lm constant_tsc rep_good nopl xtopology nonstop_tsc cpuid pni pclmulqdq ssse3 cx16 pcid sse4_1 sse4_2 x2apic movbe popcnt aes xsave avx rdrand hypervisor lahf_lm abm 3dnowprefetch invpcid_single pti fsgsbase avx2 invpcid rdseed clflushopt

As you can see endianness(byte order) in the above output “Byte Order: Little Endian“.

Check native byte order using python programming language

Python sys module has attribute called “byteorder” where it’s value describes native byte order of the system .

  • sys.byteorder will be “big” if it’s big-endian (most significant byte first) platform
  • sys.byteorder will be “little” if it is little-endian (least significant byte first) platform
>>> import sys
>>> sys.byteorder
little

Check native byte order of system using java

In java “java.nio.ByteOrder” provides flags “BIG_ENDIAN” and “LITTLE_ENDIAN” to denote Big-Endian and Little-Endian respectively. It has the static method “nativeOrder()” to get system’s native byte order.

java.nio.ByteOrder.nativeOrder()

Check endianness of system using C program

#include <stdio.h>
int main()
{
unsigned int i = 1;
char *c = (char*)&i;
if (*c)
printf("Little Endian System");
else
printf("Big Endian System");
getchar();
return 0;
}

Above c program prints Endianess of the system. In the above program we have used character pointer “c” to dereference integer “i“. Since size of character is 1 byte it will contain only first byte of integer.

  • *c will return 1 which is Least Significant Byte on Little-Endian machine.
  • *c will return 0 which is Most Significant Byte on Big-Endian machine

Following “c” program prints memory representation of given 32-bit integer. You can see byte order in action, order in which given 32-bit integer is stored in memory either by Little-Endian or Big-Endian.

Above program will print ” 4d 3c 2b 1a” if machine is Little-Endian (Least Significant Byte Fist) or “1a 2b 3c 4d” if machien is Big-Endian (Most Significant Byte Fist)

Convert Endianness

It is required sometimes to convert endianness. You don’t have to worry about converting endiannes if you are working application level. For data exchange between machines common format is used. Yet if machine is little endian often data sent over network is big endian. Big-Endian is standard byte order for network communication. In such a case we have to convert data that we receive in Big-Endian to host fomat little endian and when we send data back to network need to convert Little-Endian to Big-Endian. There are many utility function to perform this conversion.

Following function are available in c to help endianness conversion

htons() – Host to Network Short
htonl() – Host to Network Long
ntohs() – Network to Host Short
ntohl() – Network to Host Long

Python library NumPy provides way to convert endianness. NumPy library uses c types and low level data structures behind.

How Systems Of Different Endianness Interact

Files that are created on little-endian machine won’t be read as they were created on big-endian machines without conversion. This is the reason, some programs binary file formats which are endianness independent. Where files are endianness dependent byte order marks are used in headers of file to hint a program endianness of the file.

Example: TIFF files use MM and II in header (begning of file) for big-endian and little-endian respectively. Where MM and II are palindromes these are endianness independent, interpreted same on both little-endian and big-endian machines.

Unicode uses byte order mark in the beginning of the stream to indicated endianness of following unicode bytes.

Endianness problems. Who needs to worry about byte order

You don’t need to worry much if you work with application layer protocols or high level programming languages, but it is good to know.

Think what’s the out put of following program

#include <stdio.h>

void foo(void *d){
printf("Data = %d\n", *(char *)d);
}

int main(void){
unsigned int i = 1;
foo(&i);
}

Summary

  • Words Little-Endian and Big-Endian came from 1726 novel Gulliver’s Travels by Jonathan Swift
  • System is said to be Little-Endian if it stores or transmits Least Significant Byte First
  • System is said to be Big-Endian if it stores or transmits Most Significant Byte First
  • Command “lscpu” on linux will print CPU information including Byte Order (Endianness)
  • Currently m
  • ost of processors are little endian
  • Big Endian is mostly used in network protocols so called network byte order .
  • You don’t have to worry about byte order (endianness) compatibility, system will take care of it behind the senses. Unless you work with assembly, c/c++ language, drivers, embedded system and low level network transports.
  • Choose One endianness and stick to it to avoid obscured bugs.