1. Recursively Search and Replace Text in Multiple Files

I’ve already covered this one extensively in a previous article, Recursively Search and Replace Terms in Multiple Files with grep, xargs and sed, so I’ll not go into too much detail here about how the command works, and how arcuments are passed from one command to another. The command basically combines the utilities grep, xargs and sed to recursively find files within a directory (and its sub-directories) containing the desired search term (using grep), then pass them through (via xargs) to sed to replace each instance of the search term with the target term. The command structure is as follows:

grep -lr -e '<searchterm>' * | xargs sed -i 's/<searchterm>/<targetterm>/g'

Where <searchterm> is the string we are looking for, and <targetterm> is the string we wish to replace it with. For example:

grep -lr -e 'apples' * | xargs sed -i 's/apples/oranges/g'

will search all the files within the current directory, and its sub-directories for the string “apples”, and replace all instances of “apples” within those files with “oranges”. This already powerful command combination becomes even more powerful with the use of regular expressions in place of <searchterm> and <targetterm>.

2. Create a Multiple Directory Hierarchy with a Single Command

A majority of users are familiar with the mkdir command to create a single directory from the command. Howerver, it the -p option, the mkdir command can be told to create multiple directories with a single command. In its most basic form, it can be used to create a single directory, with a single child, grand child and so on:

mkdir -p top/lev1/lev2/lev3

Will create the directory structure as follows:

|-top
|---lev1
|-----lev2
|-------lev3

The command’s -p option becomes really powerful, however, when combined with nested directories structures contained in {curly braces}. The {curly braces} can be used to group directories and sub directories, in order to create a complex directory hierarchy with a single command:

mkdir -p coolapp/{docs/{en_uk/{html,text,pdf},en_us/{html,text,pdf}},includes/{functions,language/{en_uk,en_us},classes},templates,tmp}

Will produce the following complex hierarchy:

|-coolapp
|---docs
|-----en_uk
|-------html
|-------pdf
|-------text
|-----en_us
|-------html
|-------pdf
|-------text
|---includes
|-----classes
|-----functions
|-----language
|-------en_uk
|-------en_us
|---templates
|---tmp

Now there’s no excuse for creating each directory with separate commands – with careful planning, a whole application structure can be created quickly.

3. Output a Directory Structure in a Visual Tree Format

This one, I picked up from http://www.centerkey.com/tree/ and will output a directory structure as shown in the examples above, when run from the current directory! Very cool!

ls -R | grep ":$" | sed -e 's/:$//' -e 's/[^-][^/]*//--/g' -e 's/^/ /' -e 's/-/|/'

4. Access and Search Command History, and Execute Commands Quickly

There are a couple of ways to access your command/bash history in order to re-execute previous commands. If the command was recently executed, using the Up and Down Arrow keys it’s possible to quickly navigate through your command history. Entering double exclamation marks – !! – will run the last command. For commands executed further in the past, you can view them by running:

more ~/.bash_history

Or more conveniently with the history command:

history

The history command will output your command line history in two columns, the first being a number, and the second the command itself:

1 ls
2 history
3 history | more
4 history
5 pwd
6 cd ~/Documents

The reason for the number, is that it allows for quick execution of past commands, when the number is preceded with an ! exclamation mark. So, for example, if I wanted to execute the fifth command in the list above (pwd), I could simply run:

!5

Running double !! exclamation marks will simple run the previous command:

!!

Would run cd ~/Documents from the example history list above.

As can be seen in the history list above, history can also be piped | through to more, which is particularly useful if you have executed hundreds of commands, and run history only to see them all stream through the terminal in one go. Piping through to more will allow the history to be viewed page by page:

history | more

So far so good. But what if you can’t remember a particular command, or only partially remember a long command? Manually looking through hundreds of lines can be a strain, even without an eye burning theme. The grep command can be used to search for a term, before being piped through to more:

history | grep '<searchterm>' | more

This will restrict the results to those found with the <searchterm>, so:

history | grep 'ls' | more

will only return the list of commands found in history containing “ls”. As before, the command number is also returned, and thus can easily be executed by preceding the number with an ! exclamation mark.One additional point about running !nn or !! – it can also be run with sudo, to execute previous commands as root:

sudo !!
sudo !5

If there are hundreds of commands in history containing “ls”, or lots of repeated commands in history containing “ls”, the search can be further refined to include only unique results as follows:

history | grep 'ls' | sed 's/^[ 0-9]* //' | sort | uniq | more

This takes the output from history, pipes it to grep to filter only the output containing “ls”, then passes the output from grep to sed, to strip out the command numbers, then through to sort, which, well, sorts the output, then from sort to uniq, which returns only unique lines, and finaly to our friend more (which can be substituted for ess if you prefer). We have to strip out the command numbers for uniq to work, since it will see “3 ls -la” and “67 ls -ls” as unique values! This does mean copying and pasting the command instead of calling the number with an ! exclamation mark, but that’s fine for me.

5. Backup and Restore MySQL Databases with Compression

This one obviously won’t work if you don’t have MySQL installed! While GUI tools such as PHPMyAdmin, MySQL Workbench (currently in Alpha for Linux), and [insert MySQL GUI of Choice here] are great for visuay representing databases, once again, nothing beats the command line for speed when it comes to backing up and restoring databases. Using a combination of MySQL’s built in dump feature, and the gzip tool, we can quickly backup any database on the system and compress it on the fly. The syntax is as follows:

mysqldump -u <uname> -p<pass> <dbname> | gzip -9 > backupfile.sql.gz

Note that there is no space between the -p option and the password itself – this isn’t a typo! For example:

mysqldump -u mydb_username -pmydb_password mydb_dbname | gzip -9 > mydb_dbname-backup.sql.gz

It’s also possible to time/date stamp the output file, by incorporating the date command into the filename:

mysqldump -u mydb_username -pmydb_password mydb_dbname | gzip -9 > mydb_dbname-backup-`date +%d-%m-%Y`.sql.gz

These commands will create the backup file in the current directory.

Restoring the Database is just as simple:

gunzip < backupfile.sql.gz | mysql -u <uname> -p<pass> <dbname>

For example:

gunzip mydb_dbname-backup-21-02-2009.sql.gz | mysql -u mydb_username -pmydb_password mydb_dbname

6. Command Line Dictionary

This is a nice, quick Dictionary reference for writers, and much faster than any online GUI dictionary. It uses cURL to retrieve definitions from dict.org, and is executed as follows:

curl dict://dict.org/d:<theword>

where <theword> is whatever you want to look up:

curl dict://dict.org/d:incontrovertible

Will output:

220 aspen.miranda.org dictd 1.9.15/rf on Linux 2.6.18-6-k7 <39151475.31648.1235199452@aspen.miranda.org>
250 ok
150 1 definitions retrieved
151 "Incontrovertible" gcide "The Collaborative International Dictionary of English v.0.48"
Incontrovertible In*con`tro*ver"ti*ble, a.
Not controvertible; too clear or certain to admit of dispute;
indisputable. --Sir T. Browne. --
{In*con`tro*ver"ti*ble*ness}, n. -- {In*con`tro*ver"ti*bly},
adv.
[1913 Webster]
.
250 ok [d/m/c = 1/0/16; 0.000r 0.000u 0.000s]
221 bye [d/m/c = 0/0/0; 0.000r 0.000u 0.000s]

If that’s too much to remember, it’s really easy to create a bash function in your .bashrc file, which takes a single argument and outputs the results as above. Just run the following (only once) from a Terminal to create the function:

echo 'function mydict() { curl dict://dict.org/d:"$@" ;}' >> ~/.bashrc

Then close the Terminal by either pressing Ctrl+D or typing:

exit

The next time you open a terminal session, you can simply run:

mydict incontrovertible

which will call the function, query the dictionary and return the results as above.

Update: Thanks to loomsen, in comment #2, who pointed out that it isn’t necessary to restart the Terminal. Simply running:

source .bashrc

from your home directory will reload the edited .bashrc containing the new function.

7. When it all gets too cluttered…

This is so simple and obvious, it strains credulity. When the terminal just gets way too cluttered – and it probably will be after testing the above – simply run:

clear

for a nice clean terminal window.

That’s all for now – hope you find something useful in there! Finally, if you have any really useful CLI tips and tricks, please leave a comment below.

Related posts:

1. Quickly archive multiple directories into separate archive files I recently needed to quickly archive several hundred directories, and...

Related posts brought to you by Yet Another Related Posts Plugin.

I was thinking a while back “wouldn’t it be cool if there was a desktop reporting tool for Google Analytics – I could just run it, and it’d sit there quietly in the background waiting from me to press Alt+Tab”, when a friend pointed me to an interesting Bundle on YouBundle – “Best Adobe AIR Desktop Applications“. Sat amongst the application list, was the “Google Analytics Reporting Suite” – Just what I need! The only thing is, I didn’t have Adobe AIR installed… So, here’s how I went about getting it installed…

Installing Adobe AIR 1.5 Linux

64 Bit Users Note: Adobe AIR is not yet available as a 64 Bit application, so you’ll need to install 32 bit libraries first. If you haven’t got them already, in a Terminal, run:

sudo apt-get install ia32-libs

Adobe AIR 1.5 is available for Linux. The binary can be downloaded from http://get.adobe.com/air (it auto detects your system). Once the file had downloaded to my desktop, I opened a Terminal (Applications -> Accessories -> Terminal), made the AdobeAIRInstaller.bin executable, and ran the file:

cd ~/Desktop
chmod +x AdobeAIRInstaller.bin
sudo ./AdobeAIRInstaller.bin

This fires up the Adobe AIR installer GUI, and it’s installed in a couple of clicks. It needs to be run with sudo, since it installs into /opt. The uninstaller, by the way, is located in the Applications -> Accessories menu.

Additional 64 Bit step

I didn’t actually run this, as everything seems to run fine, but in a Terminal, run:

sudo cp /usr/lib/libadobecertstore.so /usr/lib32

Installing Google Analytics Reporting Suite

This part should be easy – it should just be a case of visiting http://www.adobe.com/cfusion/exchange/index.cfm?event=extensionDetail&loc=en_us&extid=1282521 and clicking on “Install”. However, for some strange reason, the page wouldn’t detect my flash plugin properly – even though I have the latest 64 Bit version (albeit Alpha) installed from Adobe – so instead of the install box, I just had a box telling me to install Flash. No worries – with some sifting through the page’s HTML source code, I found the flash installation application just points to http://www.aboutnico.be/updates/gas32.air, so I just entered that into the browser, and when Firefox prompted, selected “Open with Adobe AIR Application Installer” – this downloads the file, and begins the installation.

I went through the motions – enter Admin password, choose location to install etc. and that was it. Once installed, it’s launched from Applications -> Accessories -> Analytics Reporting Suite. No more pesky logging in (except, of course, for the initial account setup).

A Note about TweetDeck

Many people seem to be having problems getting TweetDeck to work, so after some digging around (see comments below) I found a solution on http://beegod.org/tweetdeck-on-hardy-heron-64bit. It basically involves using getlibs to resolve dependency issues with the libgnome.keyring.so 32 bit library. In a Terminal, run:

wget http://www.boundlesssupremacy.com/Cappy/getlibs/getlibs-all.deb
sudo dpkg -i getlibs-all.deb
sudo getlibs -l libgnome-keyring.so.0.1.1

Postscript: Some people may be wondering why the image at the head of the post says “Adobe AIR for Linux beta”. Well, I originally wrote this post some time ago in November 2008, using Adobe AIR v1.1 Beta, but the whole setup was a little buggy – i.e. I couldn’t configure any account information in the Google Analytics application. It seemed a little pointless publishing a post when the software didn’t work, so I waited until everything did work – i.e. Adobe AIR v1.5. I just can’t be bothered to find another image, upload it etc…

No related posts.

Related posts brought to you by Yet Another Related Posts Plugin.

Getting JRE set up and Ready (Optional)

If you’d like to set up 64 Bit (or 32 Bit) JRE to run Eclipse, follow these instructions.

Installing the Base: Eclipse 3.4 (Ganymede) JEE

First, I downloaded the 64 Bit version of Eclipse IDE for Java EE Developers to my desktop (32 Bit version here). Once downloaded, I extracted the archive, then moved the files to /opt:

cd ~/Desktop
tar -zxvf eclipse-jee-ganymede-SR1-linux-gtk-x86_64.tar.gz
sudo mv eclipse /opt

For 32 Bit:

cd ~/Desktop
tar -zxvf eclipse-jee-ganymede-SR1-linux-gtk.tar.gz
sudo mv eclipse /opt

Installing PDT 2

Simple so far. Once moved (installed), I ran Eclipse for the first time from the command line:

cd /opt/eclipse
./eclipse

and selected a workspace within my home directory once prompted. From here, installing PDT 2 is an easy task – adding two sites to the Update Manager (far easier in Ganymede than Europa), and letting Eclipse to the rest. Once Eclipse had started, I went to Help -> Software Updates. The Software Updates and Add-ons window appeared, and I navigated to the Available Software tab. From there, I clicked the “Manage Sites” button, followed by “Add” once the Site Manager appeared. The only additional prerequisite to install for PDT is the Dynamic Languages Toolkit, or DLTK. So, I added:

http://download.eclipse.org/technology/dltk/updates-dev/1.0/

in the Dialog, and clicked OK. I then needed to add the PDT update site, so clicked add, and pasted:

http://download.eclipse.org/tools/pdt/updates/2.0/interim/

Click OK to close the Available Sites Manager. The rest is just as easy – two new sites should appear in the Available Software sites list. So, I expanded the DLTK site (the small triangle to the left of the site name), then the Dynamic Languages Toolkit option, and checked “Dynamic Languages Toolkit – Core Frameworks (Incubation)”, from the top of the list. Finally, I repeated the process with the PDT Update site – expanded PDT Update Site -> PDT SDK 2.0.0 and checked “PDT Runtime Feature” from the list. That’s it. Click Install and follow the prompts.

Once installed, Eclipse recommends restarting the application. Accept, restart, and enjoy a shiny new PHP IDE, with PDT 2.

Setting Up the Environment

Once Eclipse Ganymede has reloaded, click on the Workspace icon to open the IDE workspace for the first time. The default Perspective is JEE. To change this, go to Window -> Open Perspective -> Other, then scroll down and select PHP. Other Views, relating to other plugins can be inserted into this perspective, and the perspective saved for future use. Personally, I like to add the SQL Development views, and MyLin, for live access to Bugzilla (Window -> Sow View -> Other).

A new PHP Project can be started by right clicking in Project Explorer (left window pane), and selecting New PHP Project (“Other” may need selecting on the first run, followed by navigating to and selecting PHP Project from the list).

That’s it. Far easier than previous versions, and much less effort required in the process! We like that.

No related posts.

Related posts brought to you by Yet Another Related Posts Plugin.

I wanted the latest version of Sun’s JRE installed on my system, and to have it set up as the default Java Runtime Environment whenever I ran a Java executable, or Jar file. It’s actually a pretty easy process, so this is a relatively short tutorial, but I’ll cover both the 32 bit and 64 bit installations, since aside from differences in file names, the set up is identical.

Changelog

Update 12/10/09: JRE 1.6.0 u16 now available!

Update 19/08/09: JRE 1.6.0 u15 now available!

Update 29/06/09: JRE 1.6.0 u14 now available!

Update 29/03/09: JRE 1.6.0 u13 now available!

Update 05/02/09: Updated for u12, which now includes the 64 bit Browser Plugin!

Installing JRE

First, decide on which architecture will be installed, and download the relevant file from the main Java website – if you’re running a 64 Bit Linux OS, download the 64 Bit JRE.

At the time of writing, 1.6.0 u16 can be downloaded from the following: 32 bit JRE, and 64 bit JRE. I wanted to install the x64 version, and so downloaded jre-6u16-linux-x64.bin to my Desktop. I also wanted to install it in its own directory in /opt, and so created a directory for it to sit in:

cd /opt
sudo mkdir java

I was also experimenting with the 32 and 64 bit versions, so made two extra directories within java:

cd java
sudo mkdir 32 64

I then moved the relevant file(s) to the respective directories, and made them executable:

32 bit:

sudo mv ~/Desktop/jre-6u16-linux-i586.bin /opt/java/32
sudo chmod 755 /opt/java/32/jre-6u16-linux-i586.bin

64 bit:

sudo mv ~/Desktop/jre-6u16-linux-x64.bin /opt/java/64
sudo chmod 755 /opt/java/64/jre-6u16-linux-x64.bin

The final part of the installation simply involves executing the binary file:

32 bit:

cd /opt/java/32
sudo ./jre-6u16-linux-i586.bin

64 bit:

cd /opt/java/64
sudo ./jre-6u16-linux-x64.bin

Regardless of architecture, this should create a sub directory called jre1.6.0_16.

Setting JRE 1.6.0 u16 as Default

The process simply involves telling the system that there is an alternative Java binary available, and to use this binary to execute an “java” commands:

32 bit:

sudo update-alternatives --install "/usr/bin/java" "java" "/opt/java/32/jre1.6.0_16/bin/java" 1
sudo update-alternatives --set java /opt/java/32/jre1.6.0_16/bin/java

64 bit:

sudo update-alternatives --install "/usr/bin/java" "java" "/opt/java/64/jre1.6.0_16/bin/java" 1
sudo update-alternatives --set java /opt/java/64/jre1.6.0_16/bin/java

Follwing the second command, there should be output to the terminal something along the lines of:

Using '/opt/java/32/jre1.6.0_16/bin/java' to provide 'java'.

or,

Using '/opt/java/64/jre1.6.0_16/bin/java' to provide 'java'.

depending on the architecture installed.

To double check everything is as it should be, in a Terminal run:

java -version

This should output something along the lines of:

java version "1.6.0_xx"
Java(TM) SE Runtime Environment (build 1.6.0_xx)
Java HotSpot(TM) 64-Bit Server VM (build xxxx, mixed mode)

That’s it. Now every time “java” is run, either explicitly from the Terminal, or via a Java executable (such as the Eclipse IDE), the newly installed /opt/java/64/jre1.6.0_16/bin/java binary will be used as opposed to the binary installed by default. Simple, eh?

Installing the Browser Plugin

Note: For the 64 bit plugin, it is strongly recommended that you use the version of Firefox available for installation via your distribution’s repositories.

As with most Firefox plugins, they can be installed in ~/.mozilla/plugins – if it doesn’t exist, it can be created:

mkdir ~/.mozilla/plugins

It may also be necessary to uninstall any previous Java installations, such as IcedTea etc.:

sudo apt-get remove icedtea-gcjwebplugin

Note: if you’re upgrading from a previous JRE install, remove the old plugin first:

32 Bit:

rm ~/.mozilla/plugins/libjavaplugin_oji.so

64 Bit:

rm ~/.mozilla/plugins/libnpjp2.so

The Plugin can now be installed by either copying the file:

32 Bit:

cp /opt/java/32/jre1.6.0_16/plugin/i386/ns7/libjavaplugin_oji.so ~/.mozilla/plugins/

64 Bit:

cp /opt/java/64/jre1.6.0_16/lib/amd64/libnpjp2.so ~/.mozilla/plugins/

or by creating a symbolic link to the file (this is the best option)

32 Bit:

ln -s /opt/java/32/jre1.6.0_16/plugin/i386/ns7/libjavaplugin_oji.so ~/.mozilla/plugins/

64 Bit:

ln -s /opt/java/64/jre1.6.0_16/lib/amd64/libnpjp2.so ~/.mozilla/plugins/

Once installed, restart Firefox. There are two methods to see if the Plugin is installed – navigate to about:plugins

or to the Java Tester Page

That’s it. Enjoy

No related posts.

Related posts brought to you by Yet Another Related Posts Plugin.