Some Really Useful Linux Command Line Tricks

Posted by on Feb 21, 2009 in Featured, Linux7 comments

cliMany people, especially users new to the world of Linux, are afraid of the command line. However, the command line is incredibly powerful, and in many cases can save time, and execute tasks much faster than GUIs. Personally, I love the command line, and what follows is a list of tips and tricks I find invaluable in my daily activities. The list is by no means exhaustive (I’ll certainly be adding to it as and when I find time), and isn’t limited to built in command line tools, but includes bash utilities, application commands, and combinations of these commands. Some of them I have covered before in previous articles, but are worth listing here – where this is the case, a link to the relevant article is given for reference and further reading. Finally, I don’t purport to be an expert – these are just commands I find particularly useful on a (near) daily basis. If you have any comments, suggestions, additions, or spot any mistakes, please comment below. So, anyway, enough small talk, and on with the show…

1. Recursively Search and Replace Text in Multiple Files

I’ve already covered this one extensively in a previous article, Recursively Search and Replace Terms in Multiple Files with grep, xargs and sed, so I’ll not go into too much detail here about how the command works, and how arcuments are passed from one command to another. The command basically combines the utilities grep, xargs and sed to recursively find files within a directory (and its sub-directories) containing the desired search term (using grep), then pass them through (via xargs) to sed to replace each instance of the search term with the target term. The command structure is as follows:

grep -lr -e '<searchterm>' * | xargs sed -i 's/<searchterm>/<targetterm>/g'

Where <searchterm> is the string we are looking for, and <targetterm> is the string we wish to replace it with. For example:

grep -lr -e 'apples' * | xargs sed -i 's/apples/oranges/g'

will search all the files within the current directory, and its sub-directories for the string “apples”, and replace all instances of “apples” within those files with “oranges”. This already powerful command combination becomes even more powerful with the use of regular expressions in place of <searchterm> and <targetterm>.

2. Create a Multiple Directory Hierarchy with a Single Command

A majority of users are familiar with the mkdir command to create a single directory from the command. Howerver, it the -p option, the mkdir command can be told to create multiple directories with a single command. In its most basic form, it can be used to create a single directory, with a single child, grand child and so on:

mkdir -p top/lev1/lev2/lev3

Will create the directory structure as follows:

|-top
|---lev1
|-----lev2
|-------lev3

The command’s -p option becomes really powerful, however, when combined with nested directories structures contained in {curly braces}. The {curly braces} can be used to group directories and sub directories, in order to create a complex directory hierarchy with a single command:

mkdir -p coolapp/{docs/{en_uk/{html,text,pdf},en_us/{html,text,pdf}},includes/{functions,language/{en_uk,en_us},classes},templates,tmp}

Will produce the following complex hierarchy:

|-coolapp
|---docs
|-----en_uk
|-------html
|-------pdf
|-------text
|-----en_us
|-------html
|-------pdf
|-------text
|---includes
|-----classes
|-----functions
|-----language
|-------en_uk
|-------en_us
|---templates
|---tmp

Now there’s no excuse for creating each directory with separate commands – with careful planning, a whole application structure can be created quickly.

3. Output a Directory Structure in a Visual Tree Format

This one, I picked up from http://www.centerkey.com/tree/ and will output a directory structure as shown in the examples above, when run from the current directory! Very cool!

ls -R | grep ":$" | sed -e 's/:$//' -e 's/[^-][^/]*//--/g' -e 's/^/ /' -e 's/-/|/'

4. Access and Search Command History, and Execute Commands Quickly

There are a couple of ways to access your command/bash history in order to re-execute previous commands. If the command was recently executed, using the Up and Down Arrow keys it’s possible to quickly navigate through your command history. Entering double exclamation marks – !! – will run the last command. For commands executed further in the past, you can view them by running:

more ~/.bash_history

Or more conveniently with the history command:

history

The history command will output your command line history in two columns, the first being a number, and the second the command itself:

1 ls
2 history
3 history | more
4 history
5 pwd
6 cd ~/Documents

The reason for the number, is that it allows for quick execution of past commands, when the number is preceded with an ! exclamation mark. So, for example, if I wanted to execute the fifth command in the list above (pwd), I could simply run:

!5

Running double !! exclamation marks will simple run the previous command:

!!

Would run cd ~/Documents from the example history list above.

As can be seen in the history list above, history can also be piped | through to more, which is particularly useful if you have executed hundreds of commands, and run history only to see them all stream through the terminal in one go. Piping through to more will allow the history to be viewed page by page:

history | more

So far so good. But what if you can’t remember a particular command, or only partially remember a long command? Manually looking through hundreds of lines can be a strain, even without an eye burning theme. The grep command can be used to search for a term, before being piped through to more:

history | grep '<searchterm>' | more

This will restrict the results to those found with the <searchterm>, so:

history | grep 'ls' | more

will only return the list of commands found in history containing “ls”. As before, the command number is also returned, and thus can easily be executed by preceding the number with an ! exclamation mark.One additional point about running !nn or !! – it can also be run with sudo, to execute previous commands as root:

sudo !!
sudo !5

If there are hundreds of commands in history containing “ls”, or lots of repeated commands in history containing “ls”, the search can be further refined to include only unique results as follows:

history | grep 'ls' | sed 's/^[ 0-9]* //' | sort | uniq | more

This takes the output from history, pipes it to grep to filter only the output containing “ls”, then passes the output from grep to sed, to strip out the command numbers, then through to sort, which, well, sorts the output, then from sort to uniq, which returns only unique lines, and finaly to our friend more (which can be substituted for ess if you prefer). We have to strip out the command numbers for uniq to work, since it will see “3 ls -la” and “67 ls -ls” as unique values! This does mean copying and pasting the command instead of calling the number with an ! exclamation mark, but that’s fine for me.

5. Backup and Restore MySQL Databases with Compression

This one obviously won’t work if you don’t have MySQL installed! While GUI tools such as PHPMyAdmin, MySQL Workbench (currently in Alpha for Linux), and [insert MySQL GUI of Choice here] are great for visuay representing databases, once again, nothing beats the command line for speed when it comes to backing up and restoring databases. Using a combination of MySQL’s built in dump feature, and the gzip tool, we can quickly backup any database on the system and compress it on the fly. The syntax is as follows:

mysqldump -u <uname> -p<pass> <dbname> | gzip -9 > backupfile.sql.gz

Note that there is no space between the -p option and the password itself – this isn’t a typo! For example:

mysqldump -u mydb_username -pmydb_password mydb_dbname | gzip -9 > mydb_dbname-backup.sql.gz

It’s also possible to time/date stamp the output file, by incorporating the date command into the filename:

mysqldump -u mydb_username -pmydb_password mydb_dbname | gzip -9 > mydb_dbname-backup-`date +%d-%m-%Y`.sql.gz

These commands will create the backup file in the current directory.

Restoring the Database is just as simple:

gunzip < backupfile.sql.gz | mysql -u <uname> -p<pass> <dbname>

For example:

gunzip mydb_dbname-backup-21-02-2009.sql.gz | mysql -u mydb_username -pmydb_password mydb_dbname

6. Command Line Dictionary

This is a nice, quick Dictionary reference for writers, and much faster than any online GUI dictionary. It uses cURL to retrieve definitions from dict.org, and is executed as follows:

curl dict://dict.org/d:<theword>

where <theword> is whatever you want to look up:

curl dict://dict.org/d:incontrovertible

Will output:

220 aspen.miranda.org dictd 1.9.15/rf on Linux 2.6.18-6-k7 <39151475.31648.1235199452@aspen.miranda.org>
250 ok
150 1 definitions retrieved
151 "Incontrovertible" gcide "The Collaborative International Dictionary of English v.0.48"
Incontrovertible In*con`tro*ver"ti*ble, a.
Not controvertible; too clear or certain to admit of dispute;
indisputable. --Sir T. Browne. --
{In*con`tro*ver"ti*ble*ness}, n. -- {In*con`tro*ver"ti*bly},
adv.
[1913 Webster]
.
250 ok [d/m/c = 1/0/16; 0.000r 0.000u 0.000s]
221 bye [d/m/c = 0/0/0; 0.000r 0.000u 0.000s]

If that’s too much to remember, it’s really easy to create a bash function in your .bashrc file, which takes a single argument and outputs the results as above. Just run the following (only once) from a Terminal to create the function:

echo 'function mydict() { curl dict://dict.org/d:"$@" ;}' >> ~/.bashrc

Then close the Terminal by either pressing Ctrl+D or typing:

exit

The next time you open a terminal session, you can simply run:

mydict incontrovertible

which will call the function, query the dictionary and return the results as above.

Update: Thanks to loomsen, in comment #2, who pointed out that it isn’t necessary to restart the Terminal. Simply running:

source .bashrc

from your home directory will reload the edited .bashrc containing the new function.

7. When it all gets too cluttered…

This is so simple and obvious, it strains credulity. When the terminal just gets way too cluttered – and it probably will be after testing the above – simply run:

clear

for a nice clean terminal window.

That’s all for now – hope you find something useful in there! Finally, if you have any really useful CLI tips and tricks, please leave a comment below.

Tags: , ,