Linux Fundamental 1

 

Log into local & remote graphical and text mode consoles

• Local GUI - local GUI system login based on user
• Remote GUI - VNC software
• Remote text-mode login - over ssh putty, telnet

Read, and use System Documentation

• ls --help
• man journalctl
• man man

For Searching the command ( in Following example i am searching for directory ) - If you partially remember the command :

$apropos director

##It updates the manual page index cache on Linux system
$sudo mandb

$apropos director
$apropos -s 1,8 director
$apropos "NFS mounts"

Command Type

• Internal or built-in command: such as echo, cd, pwd, set, etc
• External command:  such as cp, mv, date, etc

To know the command type use type command

$ type cp
$ type mv
$ type echo

Basics of Bash Shell

.bash_profile is mainly set for configuring and customizing the user settings.

We can load the scripts, env variables, etc when user is logged into the system.

Basically Bash provide two modes to interact with shell

1. login
2. non-login

login mode

• with ssh when you log in to a system you get an interactive login shell.
• startup file such as .bash_profile is executed by interactive login shell.
• Login shells typically read a file that does things like setting environment variables: /etc/profile and ~/.profile for the traditional Bourne shell, ~/.bash_profile additionally for bash, /etc/zprofile and ~/.zprofile for zsh, /etc/csh.login and ~/.login for csh, etc.

non-login mode

• with the already logged-in shell we get interactive non-login shell(while opening new terminal)
• .bashrc file are getting executed by interactive non-login shell.
• .bashrc is best for aliases and bash related function
• When you start a shell in a terminal in an existing session (screen, X terminal, Emacs terminal buffer, a shell inside another, etc.), you get an interactive, non-login shell. That shell might read a shell configuration file (~/.bashrc for bash invoked as bash, /etc/zshrc and ~/.zshrc for zsh, /etc/csh.cshrc and ~/.cshrc for csh

When we log in with interactive shell login bash looks for .bash_profile in home directory if not found it looks for .bash_login if it is also not found then it looks for .profile and execute it.

In /etc/profile, we can set global environment variables.

Content of .bash_profile can be following:

if [ -f ~/.bashrc ];
  then
  .  ~/.bashrc; 
fi
JAVA_HOME=/usr/share/
PATH=$PATH:$HOME/bin:$JAVA_HOME/bin
export PATH

Content of .bashrc can be following:

echo "I am in .bashrc file"
alias ll='ls -lrt'

help, man command - To know the command in details:

$ls --help
$man journalctl

There are two types of Path Linux :

Absolute directory - /home/satish/file.pdf

Relative directory - with respect to current directory

Hard Link :

$ echo "Pictures of the mountain" > Pictures/mountain.jpg

$ stat Pictures/mountain.jpg

Why do we need hard link?

• suppose Ramesh and Suresh are working on a system in their respective home directory they need to have a common set of data on the system, of course they can upload the files in their current respective working directory however it will duplicate the data on system and it can be a example of waste of storage, instead one can upload the data in their home directory and we can create hard link for another.
• One more example, if we have a file a.txt. If we create a hard link to the file and then delete the file, we can still access the file using hard link. But if we create a soft link of the file and then delete the file, we can’t access the file through soft link and soft link becomes dangling. Basically hard link increases reference count of a location while soft links work as a shortcut (like in Windows) 

See the Hard Link info:

$ echo "Pdf Files of the Satish" > Files/filename.pdf

$ stat Files/filename.pdf

$ cp -r /home/ramesh/Files /home/suresh/Files  | not a good way

# ln path_to_target_file path_to_link_file

$ ln /home/ramesh/Files/filename.pdf  /home/suresh/Files/filename.pdf 
$ stat Files/filename.pdf

##even ramesh delete the following files it won't impact of suresh file since the file has 2 hard links

$rm /home/ramesh/Files/filename.pdf
$rm /home/suresh/Files/filename.pdf
now data is completely removed

Limitations:

• You can only hard link to files not to directories
• Only hardlink to files on the same filesystem
• Users should belong to same group

$ useradd -a -G family ramesh
$ useradd -a -G family suresh
$ chmod 660 /home/ramesh/Files/filename.pdf

Soft link | Symbolic link: This is very similar to shortcut in Windows

# ln -s path_to_target_file path_to_link_file

path_to_target_file is the file or directory that our soft link is going to point to 
path_to_link_file is where our soft link will actually be created

$ chmod 660 /home/ramesh/Files/filename.pdf

Soft link of symlink in linux:

In Linux Symlink or soft link is just like shortcuts in windows:

$ ln -s /path/to/file /path/to/symlink
$ ln -s TARGET link_name

To update

$ ln -sf /path/to/file /path/to/symlink

$ cd /usr/bin
$ ln -s python3 python

Change group of the file/directory:

rwx owner group others
421
- no permission
$ chgrp group_name file/directory

## see the list of groups user belogns to 

$ groups
$ sudo chown satish 1.pdf
$ sudo chown ramesh:family 1.pdf
$ sudo chmod u=w 1.pdf
$ sudo chmod g=rw 1.pdf
$ chmod u+rw,g=r,o= 1.pdf
$ stat 1.pdf

Change group of the file/directory:

$ useradd -a -G family ramesh
$ useradd -a -G family suresh
$ chmod 660 /home/ramesh/Files/filename.pdf

SUID, SGID, and sticky bit:

Start at 0

SUID = 4

SGID = 2

Sticky = 1

you could remember this as a USER , GROUP ,STICKY
                                                       4           2             1

SUID(Set user identification bit) | works on file:

Whenever any user runs any executable file, it runs under the ownership of user who has executed it(check pid), however when SUID bit is set on that executable file it will only run under the ownership of file irrespective of who has executed it.

for example, passwd command have SUID bit enabled. 

This is required because passwd command write password in /etc/shadow file and this file does not have write permission to any other user except root.

$ sudo chmod 4755 /usr/local/bin/test
$ sudo chmod u+s /usr/local/bin/test

S - means no executable permission

s - means executable permission i.e chmod 4755 filename, will have small s

SUID enabled means if anybody else were to be able to execute that file, it would be executed as the owner, instead of as that user who executed it

SGID(Set Group identification bit) | works on file and directory with different meaning: 

On file:
On file SGID is almost same as SUID on file, just user group as a analogy instead of user.

$ sudo chmod 2755 /user/local/bin/test
$ sudo chmod g+s /user/local/bin/test

On Directory:

After setting SGID on particular directory, all the files and directory that will be getting created under it will have ownership of the group that is associated with that particular directory.

Since this operation is recursive hence if you create any directory under that directory it will have SGID bit set as well.

$ sudo chgrp -R devops /users/pankaj/test/
$ sudo chmod 2755 /users/pankaj/test/
$ sudo chmod g+s /users/pankaj/test/

## find file who has SUID bit enabled
$ find . -perm /4000

## fine file who has GUID enabled
$ find . -perm /2000

## following file will have SUID and GUID enabled
$ chmod 67555 filename

Sticky Bits:

Suppose a directory is accessible to all the users for creating files.

However you don't want anyone(except root) to delete other's ownership file regardless of the directory permission, in such case you need to set sticky bit on directory.

It will make sure the person who has created this file they can only delete it.

$ chmod +t /tmp/users-file
$ chmod o+t /tmp/users-file
OR
$ chmod 1755 /tmp/users-file

### To remove sticky bits
$ chmod -t /tmp/users-file
OR
$ chmod 0755 /tmp/users-file

t at the end - lower case because executable permission set + sticky bit on

T - because execute bit off

Find command in Linx:

## find the file that has SUID, GUID, both

$ find . -perm /4000
$ find . -perm /2000
$ find . -perm /6000


## find the file with jpg extension
$ find /usr/share/ -name '*.jpg'


## find the file which size is greater than 10MB
$ find /lib64/ -size +10M


##files that were modified on the last minutes
$ find /dev/ -mmin -1


## find [/path/to/directory] [search_parameter]
$ find /bin/ -name file1.txt
$find -name file1.txt


## go-there find it,
$ find -iname felix


## all file that start with lower case
$ find -name "f*"


## file modified 5 mins exactly ago
$ find -mmin 5


## find all files that were modifed 5 mins ago
$ find -mmin -5


## any files that were five min to infinity
$ find -mmin +5


## 24 hour periods
find -mtime 2
## you are getting an error in application and you suspect that someone changed some file permissions in the wroing way
you could find file with permission changed in the last
five minutes

$ find -cmin -5
$ find -size 512k
c bytes
k kilobytes
M megabytes
G gigabytes
greater that 512k


## file with f size 512k
$ find -name "f*" -size 512k #AND operatot
$ find -name "f*" -o -size 512k #or operatot


## exclude param
$ find -not -name "f*" #not operator
$ find \! -name "f*" #alternate not operator


## based on permission
$ find -perm 664 #exact permission
$ find -perm -664 #atlease 664 permission
$ find -perm /664 #any of these permission


## owner can read and write
$ find -perm 600 


## find the files which other user can't read
$ find \! -perm -o=r


## find files that can be read by either the user or the group or others
but at lease one of them should be able to read
$ find -perm /u=r,g=r,o=r

Sed & Grep Command in Linux:

Sed

Sed is very much helpful when you want to do search, replace, insert or delete, these kinds of operations on files and folders:

1st use case:

• when you want to search for a word 'gnu' and replace it with 'public' in a file test-file.txt

$sed 's/gnu/public/g' test-file.txt

#### By default sed is case sensitive
## s - substitute
## g - global

$ sed -i 's/String/Sonar/g' /root/file.xml
Here -i means --in-place means replace in file

#### TO make it case-insensitive
$ sed -i 's/String/Sonar/gI' /root/file.xml 

2nd use case:

• perform above operation and print the replaced line as well

$sed -n 's/gnu/public/g' test-file.txt

3rd use case:

• delete a line that match the searching pattern

$ sed '/gnu/d' test-file.txt
$ sed '/\<code\>/d' /home/BSD.txt > /home/BSD_DELETE.txt

4th use case:

• when you want to do multiple operations at a time with sed like following example shows searching for a  word "gnu" replacing it with "public" and deleting the line that contains "links"

$sed -e 's/gnu/public/g' -e "/links/d" test-file.txt

5th use case:

• sometimes you want to remove comment or # from a file, following example will replace # that starts with # and replace it with blank

$ sed "s/^#//" file-test.txt

6th use case:

• sometime you just want to replace the word with boundary, for example suppose you want to replace and with is, so the word land should not be replaced. 

$ sed -e 's/\band\b/is/g' file-test.txt   ##It will just print, not replace in file
$ sed -i 's/\band\b/is/g' file-test.txt   ## it will replace in file

Grep:

grep is very much helpful when you are searching for a pattern with multiple filter and you want that line to display on screen.

1st use case:

• when you want to search for a word 'GNU'.

$grep 'GNU' test-file.txt

2nd use case:

• perform above operation but it should be case insensitive.

$grep -i 'Gnu' test-file.txt

3rd use case:

• perform above operation but it should be case insensitive and display the line number as well.

$ grep -in 'Gnu' test-file.txt

4th use case:

• perform above operation but it should be Recursive.

$ grep -inr 'Gnu' test-file.txt

OR

$ grep -inR 'Gnu' test-file.txt
Note: -R will grep all symbolic links as well.

5th use case:

• perform above operation but it should only display the file name.

$ grep -inlR 'Gnu' test-file.txt
$ grep -inlR 'Gnu' *
$ grep -inlR 'Gnu' .

*  means in files and folders
. means in current directory

6th use case:

• sometime you want to print the line that do no match the patterns..

$grep -v 'GNU' test-file.txt

7th use case:

• sometime you want to print the line that start with matching patterns..

$grep '^GNU' test-file.txt

8th use case:

• sometime you want to print the line that end with matching patterns.

$grep 'GNU$' test-file.txt

9th use case:

• sometime you want to search for multiples matching patterns at a time..

$grep -e 'GNU|gnu|public' test-file.txt

sed and grep together with xargs

$grep -ilrn "100.72.60.10" /root/main.yaml | xargs sed -i -e 's/192.68.60.10/10.2.0.0/g'

$grep -ilrn "Pass@123!" /root/main.yaml | xargs sed -i -e 's/Pass@123\!/Password/g'

To avoid this error "-bash: !/g: event not found" escape the !

xargs command in linux:

• xargs command is very helpful when you are combining with another command.
• Basically xargs command expect input from standard i/o and then it just echo the command and perform the operation.
• It can be also very helpful while filtering out some data from file.

Some Examples:
In the following example i am searching from *.java file and trying to grep println

[pankaj@host]$ find . -name *.java | xargs grep -i 'println'
System.out.println("Age = " + age);
System.out.println("Date = " + date);

In the following example i am searching from *.java file and removing all of them.

[pankaj@host]$ find . -name "*.java" | xargs rm -rf
Hello World.java

Note : The above example can't delete the file which has whitespace in the filename. so you need to use the following command

[pankaj@host]$ find . -name "*.java" -print0 | xargs -0 rm -rf

Sometime you want to export list of user account or group of your Linux system:

[pankaj@host]$ cut -d: -f1 < /etc/passwd | xargs

find the file build.xml and trying to grep the version of the code:

[pankaj@host]$ find . -name "build.xml" | xargs grep -n "1.02.01" | wc -l

Sometime you want to replace the old version with new version:

[pankaj@host]$ find . -name "build.xml" | xargs sed -i "s#1.02.01#1.02.02#g"

Sometime you want to copy a users file in some other dir with preserving folder structure:

[pankaj@host]$ find . -user anita | cpio -pdm /official
###### (-updm for overwrite destination content.)

[pankaj@host]$ find . -type f -user anita -exec install -v {} /dest/path/{} \;

### if you don't want to preserver folder structure #####
$ find . -user  | xargs cp -t /target/path/

echo command in Linux :

echo '{
   "debug": true,
   "tls": true,
   "tlscert": "/var/docker/cert.pem",
   "tlskey": "/var/docker/servercert.pem",
   "hosts": ["tcp://192.168.0.0:2343"]
}' > /etc/docker/daemon.json

echo '100.68.0.100 server1 jumphost' >> /etc/hosts

ps command in Linux :

[pankaj@host]$ ps
This command will list all the running processes on current shell session.

[pankaj@host]$ ps -u root
This command will list all of the processes and commands that is getting run by root user

[pankaj@host]$ ps aux
This command will list all the processes by all users of the current system

[pankaj@host]$ ps -ef | less
This command will list all the running process of a system.

[pankaj@host]$ ps -aux | less
This command will list all the running processes of a system with some additional details like memory utilization, cpu utilization etc.

[pankaj@host]$ ps -aux --sort=-pmem | less
This command will list all the running processes of a system based on memory utilization
 
[pankaj@host]$ ps -aux --sort=-pcpu | less
This command will list all the running processes of a system based on CPU utilization

[pankaj@host]$ ps -aux | grep -v grep | grep -e tail-e bash

see the operating system release details:

[pankaj@host]$ cat /etc/os-release
[pankaj@host]$ cat /etc/redhat-release
[pankaj@host]$ cat /etc/*release

tar command in Linux :

Making archive file:
[pankaj@host]$ tar -cvf ./my-file.tar ./folder-name/
[pankaj@host]$ tar -cvzf ./my-file.tar.gz ./folder-name/

Even you can exclude some sub directory:
[pankaj@host]$ tar -cvzf ./my-file.tar.gz ./folder-name/ --exclude="folder-name/subf"

Make archive file with multiple files or directory:
[pankaj@host]$ tar -cvzf ./file.tar.gz file1.txt file2.txt


add file to archive.tar
$ tar --append --file archive.tar file2 = $tar rf archive.tar file2

Extracting tar or compressed tar file:
[pankaj@host]$ tar -xvf ./my-file.tar.gz

Extracting tar or compressed tar file to specific location:
[pankaj@host]$ tar -xvf file1.tar -C ./data/node

without extracting list the contents of archive file. 
[pankaj@host]$ tar -tf file1.tar

-c   create a new archive
-z   compress and archive through gzip
-f   allows you the specify the filename of the archive
-v   verbose
-x   extract archive file

$tar -czvf name-of-archive.tar.gz /path/to/directory-or-file

tr (translate) command in Linux :

$cat file.txt
Hello

$cat file.txt | tr -d "[a-z]"
H

$cat file.txt | tr -d [:lower:]
H

$cat file.txt 
{Hello}

$cat file.txt | tr '{}' '[]'
[Hello]

## Write file with cat command
$cat > outputfile
Hello
world 
ctrl + d
$cat outputfile
Hello
world

readlink command in Linux :

$which java
/usr/bin/java

$ll /usr/bin/java
/usr/bin/java -> /etc/alternatives/java

$ll /etc/alternatives/java
/etc/alternatives/java -> /usr/lib/jvm/java-11-openjdk-amd64/bin/java

$readlink /usr/bin/java
/etc/alternatives/java

$readlink -f /usr/bin/java
/usr/lib/jvm/java-11-openjdk-amd64/bin/java

dos2unix/unix2dos command in Linux :
Sometime when you are copying text from windows to linux or linux to windows then we might face some issues

in windows line end with CR \r followed by \n however in linux \n

so it is always recommended to convert the Dos to unix AND Unix to Dos whenever required.

$dos2unix file.sh
$unix2dos file.sh

Remove folders except one folder "haproxy":

shopt -s extglob
rm -rf !(haproxy)

Insert a line at line number 3 in file:

$ awk 'NR==3{print "I am at line number3"}1' filename

compare directory in linux:

diff -qr directory1/ directory2/

How to make files IMMUTABLE (Unchangeable) in Linux :

## To set
chattr +i /tmp/test

## To unset
chattr +i /tmp/test

Linux kernel manages following things mainly:

• Memory management
• process management
• device drivers
• system calls & security

User or program calls system calls to interact with kernel space.

Total cores is the number of cores per socket x number of sockets

To see the kernal Version:

$uname -r

CPU architecture info:

$lscpu

See the Memory info:

$lsmem --summary

Thanks for reading!!