What is a Regular Expression?

It’s a string pattern written in a compact syntax, that allows us to quickly check

whether a given string matches or contains a given pattern.

The power of regular expressions is that they can specify patterns, not just

fixed characters. Many examples in this articles can be found on:

Googles Python Course

Basic patterns

a, X, 9

ordinary characters just match themselves exactly.

. ^ $ * + ? { [ ] | ( )

meta-characters with special meanings (see below)

. (a period)

matches any single character except newline ‘n’

w

matches a “word” character: a letter or digit or underbar [a-zA-Z0-9_].

It only matches a single character not a whole word.

W

matches any non-word character.

w+

matches one or more words / characters

b

boundary between word and non-word

s

matches a single whitespace character, space, newline, return, tab, form

S

matches any non-whitespace character.

t, n, r

tab, newline, return

D

matches anything but a digit

d

matches a decimal digit [0-9]

d{1,5}

matches a digit between 1 and 5 in lengths.

Recommended Python Training For Python training, our top recommendation is DataCamp. Free Trial

{n} d{5}

matches for 5 digits in a row

^

match the start of the string

$

match the of the string end

*

matches 0 or more repetitions

?

matches 0 or 1 characters of whatever precedes it

use . to match a period or to match a slash.

If you are unsure if a character has special meaning, such as ‘@’, you can

put a slash in front of it, @, to make sure it is treated just as a character.

re.findall

The findall() is probably the single most powerful function in the re module

and we will use that function in this script.

In the example below we create a string that have a text with many email

addresses.

We then create a variable (emails) that will contain a list of all the found

email strings.

Lastly, we use a for loop that we can do something with for each email string

that is found.

We can also apply this for files. If you have a file and want to iterate over

the lines of the file, just feed it into findall() and let it return a list of

all the matches in a single step

read() returns the whole text of a file in a single string.

(If you want to read more about file handling in Python, we have written a

‘Cheat Sheet’ that you can find here)

# Open file f = open('test.txt', 'r') # Feed the file text into findall(); it returns a list of all the found strings strings = re.findall(r'some pattern', f.read())

re.search

The re.search() method takes a regular expression pattern and a string and

searches for that pattern within the string.

The syntax is re.search(pattern, string).

where:

pattern

regular expression to be matched.

string

the string which would be searched to match the pattern anywhere in the string.

It searches for first occurrence of RE pattern within string with optional flags.

If the search is successful, search() returns a match object or None otherwise.

Therefore, the search is usually immediately followed by an if-statement to test

if the search succeeded.

It is common to use the ‘r’ at the start of the pattern string, that designates

a python “raw” string which passes through backslashes without change which is

very handy for regular expressions.

This example searches for the pattern ‘word:’ followed by a 3 letter word.

The code match = re.search(pat, str) stores the search result in a variable

named “match”.

Then the if-statement tests the match, if true the search succeeded and

match.group() is the matching text (e.g. ‘word:cat’).

If the match is false, the search did not succeed, and there is no matching text.

str = 'an example word:cat!!' match = re.search(r'word:www', str) # If-statement after search() tests if it succeeded if match: print 'found', match.group() ## 'found word:cat' else: print 'did not find'

As you can see in the example below, I have used the | operator, which search for either pattern I specify.

import re programming = ["Python", "Perl", "PHP", "C++"] pat = "^B|^P|i$|H$" for lang in programming: if re.search(pat,lang,re.IGNORECASE): print lang , "FOUND" else: print lang, "NOT FOUND"

The output of above script will be:

Python FOUND

Perl FOUND

PHP FOUND

C++ NOT FOUND

re.sub

The re.sub() function in the re module can be used to replace substrings.

The syntax for re.sub() is re.sub(pattern,repl,string).

That will replace the matches in string with repl.

In this example, I will replace all occurrences of the re pattern (“cool”)

in string (text) with repl (“good”).

import re text = "Python for beginner is a very cool website" pattern = re.sub("cool", "good", text) print text2

Here is another example (taken from Googles Python class ) which searches for all

the email addresses, and changes them to keep the user (1) but have

yo-yo-dyne.com as the host.

re.compile

With the re.compile() function we can compile pattern into pattern objects,

which have methods for various operations such as searching for pattern matches

or performing string substitutions.

Let’s see two examples, using the re.compile() function.

The first example checks if the input from the user contains only letters,

spaces or . (no digits)

Any other character is not allowed.

import re name_check = re.compile(r"[^A-Za-zs.]") name = raw_input ("Please, enter your name: ") while name_check.search(name): print "Please enter your name correctly!" name = raw_input ("Please, enter your name: ")

The second example checks if the input from the user contains only numbers,

parentheses, spaces or hyphen (no letters)

Any other character is not allowed

import re phone_check = re.compile(r"[^0-9s-()]") phone = raw_input ("Please, enter your phone: ") while phone_check.search(phone): print "Please enter your phone correctly!" phone = raw_input ("Please, enter your phone: ")

The output of above script will be:

Please, enter your phone: s

Please enter your phone correctly!

It will continue to ask until you put in numbers only.

Find Email Domain in Address

Let’s end this article about regular expressions in Python with a neat script I

found on stackoverflow.

@

scan till you see this character

[w.]

a set of characters to potentially match, so w is all alphanumeric characters,

and the trailing period . adds to that set of characters.

+

one or more of the previous set.

Because this regex is matching the period character and every alphanumeric

after an @, it’ll match email domains even in the middle of sentences.

import re s = 'My name is Conrad, and [email protected] is my email.' domain = re.search("@[w.]+", s) print domain.group()

outputs:

@gmail.com

More Reading

https://developers.google.com/edu/python/regular-expressions

http://www.doughellmann.com/PyMOTW/re/

http://www.daniweb.com/