Bash Script 9: Altering Files with sed

When writing bash scripts, sometimes you'll want to do a search and replace on a file. If you already know regular expressions, this is super easy! If not, read on, because that's what's being covered in this post.

Regular Expressions

A search and replace expressions can be simple. The simplest is just changing out text directly. Here's an example:

s/One/Two/g

The forward slash '/' divides the expression into 4 parts. The first part is just the 's', which means we are going to search and replace. The second part is the match expression. In this case, we want to match the text 'One'. The third part is the replace expression. Here, 'Two' is the replacement string. The fourth part gives some extra flags. Here, it's just 'g', which means perform this globally and replace all instances of 'One'. Other flags include 'i' for case-insensitive searches and 'm' for a multi-line match.

Special Match Characters

In the match expression, there are some special characters that will match certain things.

Character	Meaning
Period '.'	Matches any character
Carot '^'	Matches the start of a line
Dollar Sign '$'	Matches the end of a line
Question Mark '?'	Means the preceding character may be present or absent
Star '*'	Means the preceding character may be absent, present, or repeat over and over
Plus Sign '+'	Means the preceding character is present and may repeat over and over

Here is an example:

s/^MyKey=.*$/MyKey=NewValue/g

In this example, we are looking for 'MyKey=' at the beginning of a line in the file. The '.*$' at the end of the match will match the rest of the line, so this will replace 'MyKey=ASDF' and 'MyKey=Value' with 'MyKey=NewValue'. If you want to search for that actual character, simply escape it by adding a backslash before it.

S/This is a sentence\./This sentence has no period now /g

Special Matches

There are some special escaped characters that we can use for matches:

Match String	What is matches
\s	Spaces and Tabs
\d	Numbers (does not include period, dollar sign, percent, negative sign, etc.)
\w	All alphabet characters, numbers and the underscore
\S	Everything except spaces and tabs
\D	Everything except numbers
\W	Everything except alphabet characters, numbers, and the underscore

These can be used in combination with each other and the special characters above. Here are some examples:

s/^\w+/Replace the first word/g

s/^\d+/Numbers!/g

s/^\s\s\s\s/  /g

Using Regular Expressions with sed

Now that we've covered the basics, here is how to use them with sed. [bash] sed -i -e 's/^MyLine.*$/YourLine/g' file_to_update.txt [/bash] This time, we're replacing all lines that start with 'MyLine' to 'YourLine'. The -i parameter tells sed to do an inline replace. Removing this causes the output to go to your terminal. The -e parameter tells sed that a regular expression is next. It's really that easy to use sed. The hard part is crafting the the replace expression.