Linux Journey - 02 Text-Fu PART II

Linux Journey

02. Text-Fu PART II

Important:

My goal here is to fully understand what I am doing with Linux. Over the years I noticed that I learn best, if I document it and try to explain it to myself in different words. I know that the webpage https://linuxjourney.com/ is already written for beginners never the less I want to rephrase, document and repeat the commands here.

My summary may not be complete so I would recomment you that you actually learn Linux from https://linuxjourney.com/ yourself :-)

2.7 paste

paste is very similar to cat. It merges lines together in a file. Example:

root@PREDATOR-01:/home/linux_journey# cat test.txt
apples
pears
pineapples
bananas
strawberries
cherries
root@PREDATOR-01:/home/linux_journey# paste -s test.txt
apples  pears   pineapples      bananas strawberries    cherries
root@PREDATOR-01:/home/linux_journey#

As you can see, after using paste -s merged the lines which were seperated.

With -d we can change the delimiter to something else. For example Spaces

paste -d " " -s test.txt

2.8 head

Very important to get a short preview of a large file. head shows just the first 10 lines. It is very useful if you combine head with pipes or tee. If you want to have for example 20 lines you can use the flag -n Number of lines.

Example: head /var/log/syslog head -n 20 /var/log/syslog

2.9 tail

Very similar to head. You get the last 10 lines by default. You can also change via the flag -n the lines you want to see. Very nice as an Linux Administrator is the flag -f. It shows you everything that is getting added to that file.

root@PREDATOR-01:/home/linux_journey# tail -f /var/log/syslog

2.10 join and split

With join we can merge multiple files together by a common field.

Example:

root@PREDATOR-01:/home/linux_journey# cat test.txt test1.txt
1 apples
2 pears
3 pineapples
4 bananas
5 strawberries

1 monday
2 tuesday
3 wednesday
4 thursday
5 friday
root@PREDATOR-01:/home/linux_journey# join test.txt test1.txt
1 apples monday
2 pears tuesday
3 pineapples wednesday
4 bananas thursday
5 strawberries friday

They are joined by the common field 1,2,3,4,5.

We can split now via split into lines or bytes. You get files names xaa, xab and so on.

 root@PREDATOR-01:/home/linux_journey# cat xaa
1 apples monday
2 pears tuesday

2.11 sort

This command is very useful for sorting and changing big text files. I make an example:

root@PREDATOR-01:/home/linux_journey# cat sort-test.txt
Thomas
Albert
Chris
Franz
Jeff
Richard
Elon
Tony
root@PREDATOR-01:/home/linux_journey# sort sort-test.txt
Albert
Chris
Elon
Franz
Jeff
Richard
Thomas
Tony
root@PREDATOR-01:/home/linux_journey#

By default the sort command sorts the content alphabetically. It is also possible to sort in other ways.

Common Flags:

• -n sorts numbers
• -r sorts descending alphabetically
• -k sorts lines based on a specified field (column)
• -u removes duplicates lines from the output

2.12 tr (Translate)

This command allows you to translate characters into other characters.

Example:

TR(1)                                               User Commands                                               TR(1)

NAME
       tr - translate or delete characters

SYNOPSIS
       tr [OPTION]... SET1 [SET2]

root@PREDATOR-01:/home/linux_journey# cat test.txt
test hello test
root@PREDATOR-01:/home/linux_journey# cat test.txt | tr "a-z" "A-Z"
TEST HELLO TEST

2.13 uniq (Unique)

uniq is similar to sort and very useful when used together. uniqonly identifies duplicates if the lines are adjacent.

Good ways to use both commands:

root@PREDATOR-01:/home/linux_journey# cat sort-test.txt
Chris
Chris
Chris
Thomas
Chris
Franz
root@PREDATOR-01:/home/linux_journey# sort sort-test.txt | uniq
Chris
Franz
Thomas
root@PREDATOR-01:/home/linux_journey#

After sorting via sort we could delete all duplicates from the file.

with -c we can count, how often the duplicates appear:

root@PREDATOR-01:/home/linux_journey# sort sort-test.txt | uniq -c
      4 Chris
      1 Franz
      1 Thomas

other good flags:

• -d show only lines that appear more than once
• -i ignores upper and lowercatter and treats them as the same
• -wcheck X characters for uniqueness

2.14 wc and nl (word count and number lines)

Wordcount wc shows the total count of words in a file:

/home/linux_journey# wc sort-test.txt
 6  6 37 sort-test.txt

It display the number of lines, number of words and number of bytes, respectively.

To just see just the count of a certain field, use the -l, -w, or -c respectively.

With nl you can count lines.

root@PREDATOR-01:/home/linux_journey# nl sort-test.txt
     1  Chris
     2  Chris
     3  Chris
     4  Thomas
     5  Chris
     6  Franz

2.15 grep

grep is probably my most used text processing command. It allows to search files for character that matches a pattern.

Example:

root@PREDATOR-01:/home/linux_journey# grep Th sort-test.txt
Thomas

I searched here for “Th` and found Thomas. Very very useful.

It works also very good for completly different tasks, for example finding a service.

root@PREDATOR-01:/home/linux_journey# systemctl |grep -i netw
  networkd-dispatcher.service
                                                    loaded active running   Dispatcher daemon for systemd-networkd

I searched in my services for “netw” and found the networkd service. With -i I ignore upper/lower letters. (Case sensitive)