Unix – Command grep, sed and awk – Best Practice

Below Commands are used in real time practice in work order. Many times we had a problem to deal with issues and incident. below following command is used to fix the incident. these Linux command is tested and usually used in operation.

Unix – Command Examples:

1. Unix – Command grep -To print capital letter words

grep -oP ‘\b[A-Z]{3,}\b’

example :

vicky@pwr-VirtualBox:~$ cat fruit.txt

apple berry banana

VICKY

vicky@pwr-VirtualBox:~$ grep -oP '\b[A-Z]{3,}\b' fruit.txt

VICKY

example:

sachin@pwr-VirtualBox:~$ cat num.txt

100,200,500,100,

200,300,400,000

sachin@pwr-VirtualBox:~$ grep '\([0-9]\) *\1' num.txt

100,200,500,100,

200,300,400,000

2. Unix – Command sed – To Replace first 12 numbers.

Replace first 12 numbers of credit card to ****
sed ‘s/[0-9]\+ /**** /g’

Substitute “multiple digits followed by space” by “**** “.
Last 4 digits aren’t followed by space, hence, won’t be replaced by asterisk

To rearrange the columns 4,3,2,1 seperated by space
sed -r ‘s/([0-9]* )([0-9]* )([0-9]* )([0-9]*)/\4 \3\2\1/’

*************INSERT************************

1. To insert a new column (say serial number) before the 1st column
$ awk -F, ‘{$1=++i FS $1;}1’ OFS=, file

2. To insert a new column after the last column
$ awk -F, ‘{$(NF+1)=++i;}1’ OFS=, file

3. Add 2 columns after the last column:
$ awk -F, ‘{$(NF+1)=++i FS “X”;}1’ OFS=, file

$ awk -F, ‘{$1=toupper($1)}1’ OFS=, file

5. Update 2nd column by adding 10 to the variable:
$ awk -F, ‘{$2+=10;}1’ OFS=, file

Duplicate
awk ‘{ col = $2 ” ” $2; $2 = col; print }’ file

$ awk -F, ‘{$1=substr($1,0,3)}1’ OFS=, file

##Rearrange the order

awk ‘BEGIN { FS=”,”; OFS=”,”; } {print $3,$1,$2}’ file

***REMOVE COLUMN in a CSV******

$ cut -d, -f2-4,7-9 –complement temp.csv
1,5,6,10
1,5,6,10
1,5,6,10

$ cut -d, -f2-4,7-9 temp.csv
2,3,4,7,8,9
2,3,4,7,8,9

************************************************

$ awk ‘NR==1 && /Medicine/’ file
Medicine,200

to exactly match only for Rent or Medicine,

$ awk -F, ‘$1 ~ /^Rent$|^Medicine$/’ file
Medicine,200
Rent,900
Medicine,600

$ awk -F, ‘$1 !~ /Medicine/’ file
Grocery,500
Rent,900
Grocery,800

$ awk ‘NR==1 && /Medicine/’ file
Medicine,200

*************SPLIT**************

awk -F, ‘{print > $1}’ file1

$ awk ‘/START/{x=”F”++i;}{print > x;}’ file2

Split and add header and trailer in every splitted file
awk ‘BEGIN{getline f;}NR%3==2{x=”FAN”++i;a[i]=x;print f>x;}{print > x}END{for(j=1;j<i;j++)print> a[j];}’ splitfiles.txt

split(“auto-da-fe”, a, “-“)
a[1] = “auto”
a[2] = “da”
a[3] = “fe”

************CHARACTER POSITION **********************

awk ‘BEGIN { print index(“peanut”, “an”) }’

echo “RAMSITALSKHMAN|1223333” | grep -aob ‘|’

echo “RAMSITALSKHMAN|1223333″|awk ‘BEGIN{print index($0,”|”)}’

awk ‘{match($0, regex)

awk ‘BEGIN { print substr(“peanut”,3,3) }’

#!/bin/bash
s1=”Trying to see if 172.22.23.12 is reachable; ping successful”
s2=”ping successful”
awk -v a=”$s1″ -v b=”$s2″ ‘BEGIN{print index(a,b)}’ >>index.log1

********SUBSTITUTE********************

awk ‘BEGIN {
str = “peanueat”
sub(/ea/, “MONEY”, str)
print str
}’

##GSUB — for all occurances