You’ll need to register to post, but anyone can read them. I’ll eventually be on the look out for moderators, and the forums may possibly even lead to enlisting future guest writers for 64 Bit Jungle too.

They’re obviously looking a little sparse at the moment, since they’ve only just opened, but please don’t let that put you off registering and posting to kick things off.

Enjoy!

No related posts.

Related posts brought to you by Yet Another Related Posts Plugin.

32 Bit users

32 Bit users need only download the precompiled deb from one of the MySQL Mirrors, and once downloaded, the deb can be installed by opening a Terminal and running the following:

sudo dpkg -i mysql-workbench-oss-5.1.9-1_i386.deb

64 Bit users

To compile MySQL Workbench from source, a bunch of dependencies need to be installed, which can be done easily by running the following in a Terminal:

sudo apt-get install build-essential autoconf automake libtool libzip-dev libxml2-dev libsigc++-2.0-dev libglade2-dev libgtkmm-2.4-dev libglu1-mesa-dev libgl1-mesa-glx mesa-common-dev libmysqlclient15-dev uuid-dev liblua5.1-dev libpixman-1-dev libpcre3-dev libgnome2-dev libgtk2.0-dev libpango1.0-dev libcairo2-dev python-dev

The source package can be downloaded once again from one of the MySQL Mirrors. As usual, I downloaded to my Desktop, and ran the following to compile and install:

cd
mkdir mysqlwb-src mysql-workbench
mv Desktop/mysql-workbench-oss-5.1.9.tar.gz mysqlwb-src
cd mysqlwb-src
tar -zxvf mysql-workbench-oss-5.1.9.tar.gz
cd mysql-workbench-oss-5.1.9
./autogen.sh
make install DESTDIR=/home/<username>/mysql-workbench

I’m using the DESTDIR option, since with Workbench being in Beta, I don’t really want it installing system wide.

Unless you’re lucky enough to have a machine with ridiculous specs, this will take a while – so go make a brew, coffee, smoke a cigarette, or [instert short term recreational activity here], and come back in a few.

No, really – it takes a long time…

Once installed (took a long time, eh?), running it is easy – simply navigate to the installation directory and run the executable from the Terminal:

cd ~/mysql-workbench/usr/local/bin
./mysql-workbench

No related posts.

Related posts brought to you by Yet Another Related Posts Plugin.

1. Recursively Search and Replace Text in Multiple Files

I’ve already covered this one extensively in a previous article, Recursively Search and Replace Terms in Multiple Files with grep, xargs and sed, so I’ll not go into too much detail here about how the command works, and how arcuments are passed from one command to another. The command basically combines the utilities grep, xargs and sed to recursively find files within a directory (and its sub-directories) containing the desired search term (using grep), then pass them through (via xargs) to sed to replace each instance of the search term with the target term. The command structure is as follows:

grep -lr -e '<searchterm>' * | xargs sed -i 's/<searchterm>/<targetterm>/g'

Where <searchterm> is the string we are looking for, and <targetterm> is the string we wish to replace it with. For example:

grep -lr -e 'apples' * | xargs sed -i 's/apples/oranges/g'

will search all the files within the current directory, and its sub-directories for the string “apples”, and replace all instances of “apples” within those files with “oranges”. This already powerful command combination becomes even more powerful with the use of regular expressions in place of <searchterm> and <targetterm>.

2. Create a Multiple Directory Hierarchy with a Single Command

A majority of users are familiar with the mkdir command to create a single directory from the command. Howerver, it the -p option, the mkdir command can be told to create multiple directories with a single command. In its most basic form, it can be used to create a single directory, with a single child, grand child and so on:

mkdir -p top/lev1/lev2/lev3

Will create the directory structure as follows:

|-top
|---lev1
|-----lev2
|-------lev3

The command’s -p option becomes really powerful, however, when combined with nested directories structures contained in {curly braces}. The {curly braces} can be used to group directories and sub directories, in order to create a complex directory hierarchy with a single command:

mkdir -p coolapp/{docs/{en_uk/{html,text,pdf},en_us/{html,text,pdf}},includes/{functions,language/{en_uk,en_us},classes},templates,tmp}

Will produce the following complex hierarchy:

|-coolapp
|---docs
|-----en_uk
|-------html
|-------pdf
|-------text
|-----en_us
|-------html
|-------pdf
|-------text
|---includes
|-----classes
|-----functions
|-----language
|-------en_uk
|-------en_us
|---templates
|---tmp

Now there’s no excuse for creating each directory with separate commands – with careful planning, a whole application structure can be created quickly.

3. Output a Directory Structure in a Visual Tree Format

This one, I picked up from http://www.centerkey.com/tree/ and will output a directory structure as shown in the examples above, when run from the current directory! Very cool!

ls -R | grep ":$" | sed -e 's/:$//' -e 's/[^-][^/]*//--/g' -e 's/^/ /' -e 's/-/|/'

4. Access and Search Command History, and Execute Commands Quickly

There are a couple of ways to access your command/bash history in order to re-execute previous commands. If the command was recently executed, using the Up and Down Arrow keys it’s possible to quickly navigate through your command history. Entering double exclamation marks – !! – will run the last command. For commands executed further in the past, you can view them by running:

more ~/.bash_history

Or more conveniently with the history command:

history

The history command will output your command line history in two columns, the first being a number, and the second the command itself:

1 ls
2 history
3 history | more
4 history
5 pwd
6 cd ~/Documents

The reason for the number, is that it allows for quick execution of past commands, when the number is preceded with an ! exclamation mark. So, for example, if I wanted to execute the fifth command in the list above (pwd), I could simply run:

!5

Running double !! exclamation marks will simple run the previous command:

!!

Would run cd ~/Documents from the example history list above.

As can be seen in the history list above, history can also be piped | through to more, which is particularly useful if you have executed hundreds of commands, and run history only to see them all stream through the terminal in one go. Piping through to more will allow the history to be viewed page by page:

history | more

So far so good. But what if you can’t remember a particular command, or only partially remember a long command? Manually looking through hundreds of lines can be a strain, even without an eye burning theme. The grep command can be used to search for a term, before being piped through to more:

history | grep '<searchterm>' | more

This will restrict the results to those found with the <searchterm>, so:

history | grep 'ls' | more

will only return the list of commands found in history containing “ls”. As before, the command number is also returned, and thus can easily be executed by preceding the number with an ! exclamation mark.One additional point about running !nn or !! – it can also be run with sudo, to execute previous commands as root:

sudo !!
sudo !5

If there are hundreds of commands in history containing “ls”, or lots of repeated commands in history containing “ls”, the search can be further refined to include only unique results as follows:

history | grep 'ls' | sed 's/^[ 0-9]* //' | sort | uniq | more

This takes the output from history, pipes it to grep to filter only the output containing “ls”, then passes the output from grep to sed, to strip out the command numbers, then through to sort, which, well, sorts the output, then from sort to uniq, which returns only unique lines, and finaly to our friend more (which can be substituted for ess if you prefer). We have to strip out the command numbers for uniq to work, since it will see “3 ls -la” and “67 ls -ls” as unique values! This does mean copying and pasting the command instead of calling the number with an ! exclamation mark, but that’s fine for me.

5. Backup and Restore MySQL Databases with Compression

This one obviously won’t work if you don’t have MySQL installed! While GUI tools such as PHPMyAdmin, MySQL Workbench (currently in Alpha for Linux), and [insert MySQL GUI of Choice here] are great for visuay representing databases, once again, nothing beats the command line for speed when it comes to backing up and restoring databases. Using a combination of MySQL’s built in dump feature, and the gzip tool, we can quickly backup any database on the system and compress it on the fly. The syntax is as follows:

mysqldump -u <uname> -p<pass> <dbname> | gzip -9 > backupfile.sql.gz

Note that there is no space between the -p option and the password itself – this isn’t a typo! For example:

mysqldump -u mydb_username -pmydb_password mydb_dbname | gzip -9 > mydb_dbname-backup.sql.gz

It’s also possible to time/date stamp the output file, by incorporating the date command into the filename:

mysqldump -u mydb_username -pmydb_password mydb_dbname | gzip -9 > mydb_dbname-backup-`date +%d-%m-%Y`.sql.gz

These commands will create the backup file in the current directory.

Restoring the Database is just as simple:

gunzip < backupfile.sql.gz | mysql -u <uname> -p<pass> <dbname>

For example:

gunzip mydb_dbname-backup-21-02-2009.sql.gz | mysql -u mydb_username -pmydb_password mydb_dbname

6. Command Line Dictionary

This is a nice, quick Dictionary reference for writers, and much faster than any online GUI dictionary. It uses cURL to retrieve definitions from dict.org, and is executed as follows:

curl dict://dict.org/d:<theword>

where <theword> is whatever you want to look up:

curl dict://dict.org/d:incontrovertible

Will output:

220 aspen.miranda.org dictd 1.9.15/rf on Linux 2.6.18-6-k7 <39151475.31648.1235199452@aspen.miranda.org>
250 ok
150 1 definitions retrieved
151 "Incontrovertible" gcide "The Collaborative International Dictionary of English v.0.48"
Incontrovertible In*con`tro*ver"ti*ble, a.
Not controvertible; too clear or certain to admit of dispute;
indisputable. --Sir T. Browne. --
{In*con`tro*ver"ti*ble*ness}, n. -- {In*con`tro*ver"ti*bly},
adv.
[1913 Webster]
.
250 ok [d/m/c = 1/0/16; 0.000r 0.000u 0.000s]
221 bye [d/m/c = 0/0/0; 0.000r 0.000u 0.000s]

If that’s too much to remember, it’s really easy to create a bash function in your .bashrc file, which takes a single argument and outputs the results as above. Just run the following (only once) from a Terminal to create the function:

echo 'function mydict() { curl dict://dict.org/d:"$@" ;}' >> ~/.bashrc

Then close the Terminal by either pressing Ctrl+D or typing:

exit

The next time you open a terminal session, you can simply run:

mydict incontrovertible

which will call the function, query the dictionary and return the results as above.

Update: Thanks to loomsen, in comment #2, who pointed out that it isn’t necessary to restart the Terminal. Simply running:

source .bashrc

from your home directory will reload the edited .bashrc containing the new function.

7. When it all gets too cluttered…

This is so simple and obvious, it strains credulity. When the terminal just gets way too cluttered – and it probably will be after testing the above – simply run:

clear

for a nice clean terminal window.

That’s all for now – hope you find something useful in there! Finally, if you have any really useful CLI tips and tricks, please leave a comment below.

Related posts:

1. Quickly archive multiple directories into separate archive files I recently needed to quickly archive several hundred directories, and...

Related posts brought to you by Yet Another Related Posts Plugin.

The script itself, can be downloaded here: export_amarok_covers. Simply right click and save it to the Desktop.

System Prerequisites

export_amarok_covers requires Python, python-dev and if your Amarok database is in MySQL, the Python’s MySQL libraries and connectors, as well as python-setuptools which contains scripts such as easy_install. These are easy to set up in Debian/Ubuntu systems. From a Terminal, run:

sudo apt-get install python python-dev python-mysqldb python-setuptools

Python Libraries

export_amarok_covers also requires several Python libraries and dependencies – argparse, ConfigObj, PIL, path.py and SQLAlchemy. Unfortunately, the script creator’s suggested method for installing the dependencies as follows:

sudo easy_install PIL ConfigObj path.py SQLAlchemy argparse

didn’t work for me, so I manually downloaded each library, then installed with easy_install:

wget http://cheeseshop.python.org/packages/source/a/argparse/argparse-0.8.0.zip http://jaist.dl.sourceforge.net/sourceforge/configobj/configobj-4.5.3.zip http://effbot.org/downloads/Imaging-1.1.6.tar.gz http://pypi.python.org/packages/source/p/path.py/path-2.2.zip#md5=941660081788282887f652510d80e64e http://pypi.python.org/packages/source/S/SQLAlchemy/SQLAlchemy-0.5.2.tar.gz#md5=616893cbfb7b04a4f0ca9587bded3a1c
sudo easy_install argparse-0.8.0.zip configobj-4.5.3.zip Imaging-1.1.6.tar.gz path-2.2.zip SQLAlchemy-0.5.2.tar.gz

Final Tweaks

The first time I ran the script, it threw some errors at me, regarding the script’s MySQL connection variables mapping method, which actually turned out to be a problem with my amarokrc file, in which for some reason, some MySQL connection parameters were missing. To remedy this, I edited the file:

gedit ~/.kde/share/config/amarokrc

and added to the [MySQL] section the following lines:

MySqlHost=localhost
MySqlPort=3306
MySqlDbName=AMAROK_DB_NAME

Running export_amarok_covers

Once everything is set up, it’s just a case of making the script executable and running it, so from a Terminal, run:

cd ~/Desktop
chmod +x export_amarok_covers
./export_amarok_covers

There are also a number of options, listed on the script creator’s site, which allow you to specify the name of the output file, create Desktop Entries, and so on.

When the script has finished running (it’s very fast), you should find each album art image within the location of the respective album.

Now I need to work on batch importing the covers to the MP3’s ID3 tags!

No related posts.

Related posts brought to you by Yet Another Related Posts Plugin.

OK, it’s a little cheeky, but “Top 10″ lists seem to be the in thing at the moment. So, here’s my Top 10 11 list of Top 10 nn Lists for Linux – although they’re in no particular order, so it’s more of a simple list of 10 11 Top 10 nn lists:

1. 10 Of The Best Linux Desktop Customization Screenshots To Inspire Your Creativity – This is a nice list of 10 rather nice Linux desktop customisations. It even lists the Themes, etc. used where the information is available.
   
2. Top 10 Linux console applications – This is a great list from linux.com if you’re into CLI.
3. Top 25 Linux Games – OK, I’m cheating with this on (and so is the original Author if you compare the title with the URL), but I really like games
4. 10 things you didn’t know you could do in Ubuntu – mostly Ubuntu specific, but some things carry over to other Distros. #5 is particularly useful for those who Dual boot.
5. 10 Best Security Live CD Distros (Pen-Test, Forensics & Recovery) – Useful if you’re in the industry, or just curious.
6. Ten Cool Coreutils Commands – Another list for CLI fans, or even those just learning CLI. This is a list of more obscure, and forgotten commands.
7. 10 things I’ve overheard about my Linux laptop while on public transportation – speaks for itself. Pretty amusing.
8. Lazy Linux: 10 essential tricks for admins – “How to be a more productive Linux systems administrator”. A nice list for beginner sysadmins.
9. Top 10 Linux Based Gadgets Just in Time for Christmas – cool stuff.
10. The 10 Best Linus Torvalds Quotes –
11. 8 Unix Tricks You Probably Didn’t Know – Some more great command line tricks

No related posts.

Related posts brought to you by Yet Another Related Posts Plugin.

I was thinking a while back “wouldn’t it be cool if there was a desktop reporting tool for Google Analytics – I could just run it, and it’d sit there quietly in the background waiting from me to press Alt+Tab”, when a friend pointed me to an interesting Bundle on YouBundle – “Best Adobe AIR Desktop Applications“. Sat amongst the application list, was the “Google Analytics Reporting Suite” – Just what I need! The only thing is, I didn’t have Adobe AIR installed… So, here’s how I went about getting it installed…

Installing Adobe AIR 1.5 Linux

64 Bit Users Note: Adobe AIR is not yet available as a 64 Bit application, so you’ll need to install 32 bit libraries first. If you haven’t got them already, in a Terminal, run:

sudo apt-get install ia32-libs

Adobe AIR 1.5 is available for Linux. The binary can be downloaded from http://get.adobe.com/air (it auto detects your system). Once the file had downloaded to my desktop, I opened a Terminal (Applications -> Accessories -> Terminal), made the AdobeAIRInstaller.bin executable, and ran the file:

cd ~/Desktop
chmod +x AdobeAIRInstaller.bin
sudo ./AdobeAIRInstaller.bin

This fires up the Adobe AIR installer GUI, and it’s installed in a couple of clicks. It needs to be run with sudo, since it installs into /opt. The uninstaller, by the way, is located in the Applications -> Accessories menu.

Additional 64 Bit step

I didn’t actually run this, as everything seems to run fine, but in a Terminal, run:

sudo cp /usr/lib/libadobecertstore.so /usr/lib32

Installing Google Analytics Reporting Suite

This part should be easy – it should just be a case of visiting http://www.adobe.com/cfusion/exchange/index.cfm?event=extensionDetail&loc=en_us&extid=1282521 and clicking on “Install”. However, for some strange reason, the page wouldn’t detect my flash plugin properly – even though I have the latest 64 Bit version (albeit Alpha) installed from Adobe – so instead of the install box, I just had a box telling me to install Flash. No worries – with some sifting through the page’s HTML source code, I found the flash installation application just points to http://www.aboutnico.be/updates/gas32.air, so I just entered that into the browser, and when Firefox prompted, selected “Open with Adobe AIR Application Installer” – this downloads the file, and begins the installation.

I went through the motions – enter Admin password, choose location to install etc. and that was it. Once installed, it’s launched from Applications -> Accessories -> Analytics Reporting Suite. No more pesky logging in (except, of course, for the initial account setup).

A Note about TweetDeck

Many people seem to be having problems getting TweetDeck to work, so after some digging around (see comments below) I found a solution on http://beegod.org/tweetdeck-on-hardy-heron-64bit. It basically involves using getlibs to resolve dependency issues with the libgnome.keyring.so 32 bit library. In a Terminal, run:

wget http://www.boundlesssupremacy.com/Cappy/getlibs/getlibs-all.deb
sudo dpkg -i getlibs-all.deb
sudo getlibs -l libgnome-keyring.so.0.1.1

Postscript: Some people may be wondering why the image at the head of the post says “Adobe AIR for Linux beta”. Well, I originally wrote this post some time ago in November 2008, using Adobe AIR v1.1 Beta, but the whole setup was a little buggy – i.e. I couldn’t configure any account information in the Google Analytics application. It seemed a little pointless publishing a post when the software didn’t work, so I waited until everything did work – i.e. Adobe AIR v1.5. I just can’t be bothered to find another image, upload it etc…

No related posts.

Related posts brought to you by Yet Another Related Posts Plugin.

Update 05/02/09: Sun have actually released the update – no more beta -  so I’ve modified “Install Java JRE 1.6.0 (Update x) in Linux as the Default Java Runtime, including Firefox Browser Plugin” to reflect this (even the title has changed – although the URL remains the same!). The following tutorial remains for reference etc.

Back in January 2003, a bug was submitted to Sun pointing out that there is no 64 Bit Browser plugin support for JRE. Almost 6 years later, the issue seems to have been addressed: with the latest Beta release of JRE6 Update 12, Sun have finally developed and included the required Plugin to run 64 Bit Java, in a 64 Bit Web Browser on a 64 Bit OS. So, I decided to give it a try, and see what’s what. Since this is a Beta release (currently b3), and also deals specifically with 64 Bit, I’m keeping this post separate from my main HowTo on installing JRE: Install Java JRE 1.6.0 (Update x) on Hardy as the Default Java Runtime, which will be updated to include the new information when JRE6 u12 Final is released – hopefully sometime early this year (2009).

The first step is to download the installation file, jre-6u12-ea-bin-b03-linux-amd64-22_dec_2008.bin, which as usual was saved to my cluttered desktop. I have previous versions of JRE installed in /opt/java, but the directoty can be created as follows, if it doesn’t already exist:

sudo mkdir /opt/java

Once the installation file had downloaded, I moved it to the /opt/java directory, made it executable, and ran the setup:

cd /opt/java
sudo mv ~/Desktop/jre-6u12-ea-bin-b03-linux-amd64-22_dec_2008.bin /opt/java
sudo chmod 755 jre-6u12-ea-bin-b03-linux-amd64-22_dec_2008.bin
sudo ./jre-6u12-ea-bin-b03-linux-amd64-22_dec_2008.bin

After reading and agreeing to the Licensing terms, the executable extracts all the relevant files into a subdirectory: jre1.6.0_12

Setting JRE6 u12 as the Default Java Runtime

To install JRE6 u12 as the default Java for the system, simply run the following two commands:

sudo update-alternatives --install "/usr/bin/java" "java" "/opt/java/jre1.6.0_12/bin/java" 1
sudo update-alternatives --set java /opt/java/jre1.6.0_12/bin/java

Following the execution of both commands, there should be output along the lines of:

Using '/opt/java/jre1.6.0_12/bin/java' to provide 'java'.

Installing the Browser Plugin

The part I’m really interested in for the moment, is getting the 64 Bit JRE Browser Plugin installed, which thankfully is very easy. As with most Firefox plugins, they can be installed in ~/.mozilla/plugins – if it doesn’t exist, it can be created:

mkdir ~/.mozilla/plugins

It may also be necessary to uninstall any previous Java installations, such as IcedTea etc.:

sudo apt-get remove icedtea-gcjwebplugin

The Plugin can be installed by either copying the file:

cp /opt/java/jre1.6.0_12/lib/amd64/libnpjp2.so ~/.mozilla/plugins/

or by creating a symbolic link to the file (the best option, since it allows updating to the final release without recopying the file etc.)

ln -s /opt/java/jre1.6.0_12/lib/amd64/libnpjp2.so ~/.mozilla/plugins/

Once installed, restart Firefox. There are two methods to see if the Plugin is installed – navigate to about:plugins

or to the Java Tester Page

Looks good so far – I can finally upload photos to Facebook using the Java Applet!

No related posts.

Related posts brought to you by Yet Another Related Posts Plugin.

Getting JRE set up and Ready (Optional)

If you’d like to set up 64 Bit (or 32 Bit) JRE to run Eclipse, follow these instructions.

Installing the Base: Eclipse 3.4 (Ganymede) JEE

First, I downloaded the 64 Bit version of Eclipse IDE for Java EE Developers to my desktop (32 Bit version here). Once downloaded, I extracted the archive, then moved the files to /opt:

cd ~/Desktop
tar -zxvf eclipse-jee-ganymede-SR1-linux-gtk-x86_64.tar.gz
sudo mv eclipse /opt

For 32 Bit:

cd ~/Desktop
tar -zxvf eclipse-jee-ganymede-SR1-linux-gtk.tar.gz
sudo mv eclipse /opt

Installing PDT 2

Simple so far. Once moved (installed), I ran Eclipse for the first time from the command line:

cd /opt/eclipse
./eclipse

and selected a workspace within my home directory once prompted. From here, installing PDT 2 is an easy task – adding two sites to the Update Manager (far easier in Ganymede than Europa), and letting Eclipse to the rest. Once Eclipse had started, I went to Help -> Software Updates. The Software Updates and Add-ons window appeared, and I navigated to the Available Software tab. From there, I clicked the “Manage Sites” button, followed by “Add” once the Site Manager appeared. The only additional prerequisite to install for PDT is the Dynamic Languages Toolkit, or DLTK. So, I added:

http://download.eclipse.org/technology/dltk/updates-dev/1.0/

in the Dialog, and clicked OK. I then needed to add the PDT update site, so clicked add, and pasted:

http://download.eclipse.org/tools/pdt/updates/2.0/interim/

Click OK to close the Available Sites Manager. The rest is just as easy – two new sites should appear in the Available Software sites list. So, I expanded the DLTK site (the small triangle to the left of the site name), then the Dynamic Languages Toolkit option, and checked “Dynamic Languages Toolkit – Core Frameworks (Incubation)”, from the top of the list. Finally, I repeated the process with the PDT Update site – expanded PDT Update Site -> PDT SDK 2.0.0 and checked “PDT Runtime Feature” from the list. That’s it. Click Install and follow the prompts.

Once installed, Eclipse recommends restarting the application. Accept, restart, and enjoy a shiny new PHP IDE, with PDT 2.

Setting Up the Environment

Once Eclipse Ganymede has reloaded, click on the Workspace icon to open the IDE workspace for the first time. The default Perspective is JEE. To change this, go to Window -> Open Perspective -> Other, then scroll down and select PHP. Other Views, relating to other plugins can be inserted into this perspective, and the perspective saved for future use. Personally, I like to add the SQL Development views, and MyLin, for live access to Bugzilla (Window -> Sow View -> Other).

A new PHP Project can be started by right clicking in Project Explorer (left window pane), and selecting New PHP Project (“Other” may need selecting on the first run, followed by navigating to and selecting PHP Project from the list).

That’s it. Far easier than previous versions, and much less effort required in the process! We like that.

No related posts.

Related posts brought to you by Yet Another Related Posts Plugin.

Thankfully, Linux has dozens of command line utilities which can aid in just such a horribly monotonous task. In this case, I used grep, xargs and sed to undertake the task. I’ve written briefly before about grep, and how it can be used to recursively search files – extending the same concept, and throwing xargs and sed into the equation, it’s possible to recursively search files, and replace terms within those files. The command structure is as follows:

grep -lr -e '<searchterm>' * | xargs sed -i 's/<searchterm>/<targetterm>/g'

What is basically happening, is the command is telling Linux to find all the files containing <searchterm> and replace all occurrences with <targetterm>. grep is called first to find any files containing <searchterm>. When <searchterm> is matched, the information output by grep (i.e. the filename) is passed to xargs, which then executes sed. sed looks for the <searchterm> within the file passed via grep and xargs and replaces all instances of <searchterm> with <targetterm>.

A practical use can be changing all instances of “apples” to “oranges” in every html file contained in public_html and its sub-directories:

cd /home/username/public_html
grep -lr -e 'apples' *.html | xargs sed -i 's/apples/oranges/g'

To break it down further:

1. grep is asked to list anything containing “apples”. The -r option tells grep to do this recursively, through all sub-directories, and the -l option tells grep to only list the file names containing “apples” (the default behaviour of grep is to ouput the filename, and every line within the file containing “apples”). The -e option tells grep that the search term (“apples”) may contain a regular expression, and to disregard the leading hyphen in the search term, so as not to interpret the search term as an option (all options passed to Linux commands begin with a hyphen). Finally, *.html tells grep to search all HTML files. This of course could be changed to *.php, or *.txt for example.
2. Once grep finds an instance of “apples” within a file, the filename is piped | through to xargs, which allows additional commands to be executed without interrupting the original command, and, once the additional command has finished, to continue running the original command. In this case, sed can be executed without interrupting grep. xargs basically allows extending commands, feeding the output of one to another. For example, grep finds one or more instances of “apples” in fruit.html, and simply outputs the file name fruit.html (not the contents of the file itself, or even the lines containing “apples”). “fruit.html” is then piped through to xargs.
3. xargs takes the output fed to it and effectively uses it as input to sed – in this case, the filename, “fruit.html”. sed is a command for streaming, filtering and altering text. sed is asked to look at fruit.html (sent to it from grep, through the pipe and via xargs), find any occurrences of “apples” and replace them with “oranges”. The -i option tells sed to do this in-place – i.e. to work with fruit.html to it, not to create a new file etc. Everything within the single quotes is the expression sed works with. First, s tells sed to substitute “apples” for “oranges”. The / is the delimiter, which effectively separates the search term and target term, and also separates those in turn from the options passed to sed (s and g). Finally, the g switch tells sed to make these changes global within the file – change all instances of <searchterm> as opposed to only the first instance (which is the default behaviour of sed).
4. Once sed has finished editing fruit.html, the command loops back to grep, and the process continues until all instances in all files are changed.

What makes this combination of commands so powerful, is it’s not only possible to change words, or phrases, but complex regular expressions can be employed to search and replace intricate patterns within multiple files.

With this incredibly versatile set of commands in my toolbox, it took a matter of seconds to complete my task, instead of potentially dozens of minutes.

Finally, this isn’t the only and exclusive method for achieving the same result – there are numerous commands which may be strung together to the same effect, but as a frequent user of grep, xargs and sed, this seemed the sensible method

Linux: turning monotonous tasks into tasks. (OK, not a great slogan, but I was never really interested in marketing…)

Related posts:

1. Quickly archive multiple directories into separate archive files I recently needed to quickly archive several hundred directories, and...

Related posts brought to you by Yet Another Related Posts Plugin.

wget http://www.poweriso.com/poweriso-1.3.tar.gz
tar -zxvf poweriso-1.3.tar.gz
./poweriso convert /path/to/source.daa -o /path/to/target.iso -ot iso

However, there is also an Open Source version available, developed by Luigi Auriemma. The Zip file contains a precompiled Windozzze binary, along with the source code – which is what we’re interested in for Linux. Once it’s compiled:

wget http://aluigi.altervista.org/mytoolz/daa2iso.zip
unzip daa2iso.zip
cd src
make

it’s really easy to use:

./daa2iso /path/to/source.daa /path/to/target.iso

The new ISO image can be now burned with whatever application you choose.

No related posts.

Related posts brought to you by Yet Another Related Posts Plugin.