worldofitech
Python RegEx: In this tutorial, you will find out about regular expressions (RegEx), and use Python’s re module to work with RegEx (with the help of examples).
A Regular Expression (RegEx) is an arrangement of characters that characterizes an inquiry design. For instance,
^a...s$
The above code defines a RegEx pattern. The pattern is: any five-letter string starting with a and ending with s.
A pattern defined using RegEx can be used to match against a string.
| Expression | String | Matched? |
|---|---|---|
^a...s$ | abs | No match |
alias | Match | |
abyss | Match | |
Alias | No match | |
An abacus | No match |
Python has a module named re to work with RegEx. Here’s an example:
import re
pattern = '^a...s$'
test_string = 'abyss'
result = re.match(pattern, test_string)
if result:
print("Search successful.")
else:
print("Search unsuccessful.")
Here, we used re.match() function to search pattern within the test_string. The method returns a match object if the search is successful. If not, it returns None.
There are other a few functions characterized in the re module to work with RegEx. Before we investigate that, we should find out about standard articulations themselves.
In this article, you will learn-
- 1 Specify Pattern Using RegEx
- 2 MetaCharacters
- 3 [] – Square brackets
- 4 . – Period
- 5 ^ – Caret
- 6 $ – Dollar
- 7 * – Star
- 8 + – Plus
- 9 ? – Question Mark
- 10 {} – Braces
- 11 | – Alternation
- 12 () – Group
- 13 \ – Backslash
- 14 Special Sequences
- 15 Python RegEx
- 16 re.findall()
- 17 Example 1: re.findall()
- 18 re.split()
- 19 Example 2: re.split()
- 20 re.sub()
- 21 Example 3: re.sub()
- 22 re.subn()
- 23 Example 4: re.subn()
- 24 re.search()
- 25 Example 5: re.search()
- 26 Match object
- 27 match.group()
- 28 Example 6: Match object
- 29 match.start(), match.end() and match.span()
- 30 match.re and match.string
- 31 Using r prefix before RegEx
- 32 Example 7: Raw string using r prefix
Specify Pattern Using RegEx
To specify regular expressions, metacharacters are used. In the above example, ^ and $ are metacharacters.
MetaCharacters
Metacharacters are characters that are deciphered in an extraordinary manner by a RegEx motor. Here’s a list of metacharacters:
[] – Square brackets
Square brackets specifies a set of characters you wish to match.
| Expression | String | Matched? |
| [abc] | a | 1 match |
| ac | 1 match | |
| hey test | No match | |
| abc de ca | 5 matches |
Here, [abc] will match if the string you are trying to match contains any of the a, b or c.
You can also specify a range of characters using – inside square brackets.
[a-e] is the same as [abcde].[1-4] is the same as [1234].
[0-39] is the same as [01239].
You can complement (invert) the character set by using caret ^ symbol at the start of a square-bracket. [^abc] means any character except a or b or c.
[^0-9] means any non-digit character.
. – Period
A period matches any single character (except newline ‘\n’).
| Expression | String | Matched? |
| .. | a | No match |
| ac | 1 match | |
| acd | 1 match | |
| acde | 2 matches (contains 4 characters) |
^ – Caret
The caret symbol ^ is used to check if a string starts with a certain character.
| Expression | String | Matched? |
| ^a | a | 1 match |
| ^ab | abc | 1 match |
| bac | No match | |
| abc | 1 match | |
| acb | No match (starts with a but not followed by b) |
$ – Dollar
The dollar symbol $ is used to check if a string closes with a specific character.
| Expression | String | Matched? |
| a$ | a | 1 match |
| ^ab | formula | 1 match |
| cab | No match |
* – Star
The star symbol * matches zero or more occurrences of the pattern left to it.
| Expression | String | Matched? |
| ma*n | mn | 1 match |
| man | 1 match | |
| maaan | 1 match | |
| main | No match (a is not followed by n) | |
| woman | 1 match |
+ – Plus
The plus symbol + matches one or more occurrences of the pattern left to it.
| Expression | String | Matched? |
| ma+n | mn | No match (no a character) |
| man | 1 match | |
| maaan | 1 match | |
| main | No match (a is not followed by n) | |
| woman | 1 match |
? – Question Mark
The question mark symbol? matches zero or one occurrence of the pattern left to it.
| Expression | String | Matched? |
| ma?n | mn | 1 match |
| man | 1 match | |
| maaan | No match (more than one a character) | |
| main | No match (a is not followed by n) | |
| woman | 1 match |
{} – Braces
Consider this code: {n,m}. This means at least n, and at most m repetitions of the pattern left to it.
| Expression | String | Matched? |
| a{2,3} | abc dat | No match |
| abc daat | 1 match (at daat) | |
| aabc daaat | 2 matches (at aabc and daaat) | |
| aabc daaaat | 2 matches (at aabc and daaaat) |
Let’s try one more example. This RegEx [0-9]{2, 4} matches at least 2 digits but not more than 4 digits
| Expression | String | Matched? |
| [0-9]{2,4} | ab123csde | 1 match (match at ab123csde) |
| 12 and 345673 | 3 matches (12, 3456, 73) | |
| 1 and 2 | No match |
| – Alternation
Vertical bar | is used for alternation (or operator).
| Expression | String | Matched? |
| a|b | cde | No match |
| ade | 1 match (match at ade) | |
| acdbea | 3 matches (at acdbea) |
Here, a|b match any string that contains either a or b
() – Group
Parentheses () are used to group sub-patterns. For example, (a|b|c)xz match any string that matches either a or b or c followed by xz
| Expression | String | Matched? |
| (a|b|c)xz | ab xz | No match |
| abxz | 1 match (match at abxz) | |
| axz cabxz | 2 matches (at axzbc cabxz) |
\ – Backslash
Kickback \ is used to escape various characters including all metacharacters. For instance,
\$a coordinate if a string contains $ followed by a. Here, $ isn’t deciphered by a RegEx motor in an exceptional manner.
If you are uncertain if a character has uncommon importance or not, you can put \ before it. This ensures the character isn’t treated in a special way.
Special Sequences
Special sequences make usually used examples simpler to compose. Here’s a rundown of special sequences:
\A – Matches if the specified characters are at the start of a string.
| Expression | String | Matched? |
| \Athe | the sun | Match |
| In the sun | No match |
\b – Matches if the specified characters are at the beginning or end of a word.
| Expression | String | Matched? |
| \bfoo | football | Match |
| foo\b | a football | Match |
| afootball | No match | |
| the foo | Match | |
| the afoo test | Match | |
| the afootest | No match |
\B – Opposite of \b. Matches if the specified characters are not at the beginning or end of a word.
| Expression | String | Matched? |
| \Bfoo | football | No match |
| foo\B | a football | No match |
| afootball | Match | |
| the foo | No match | |
| the afoo test | No match | |
| the afootest | Match |
\d – Matches any decimal digit. Equivalent to [0-9]
Expression String Matched? \d 12abc3 3 matches (at 12abc3) Python No match
\D – Matches any non-decimal digit. Equivalent to [^0-9]
Expression String Matched? \d 1ab34″50 3 matches (at 1ab34″50) 1345 No match
\s – Matches where a string contains any whitespace character. Equivalent to [ \t\n\r\f\v].
| Expression | String | Matched? |
| \s | Python RegEx | 1 match |
| PythonRegEx | No match |
\S – Matches where a string contains any non-whitespace character. Equivalent to [^ \t\n\r\f\v].
| Expression | String | Matched? |
| \S | a b | 2 matches (at a b) |
| No match |
\w – Matches any alphanumeric character (digits and letters in order). Equal to [a-zA-Z0-9_]. Coincidentally, underscore _ is additionally viewed as an alphanumeric character.
| Expression | String | Matched? |
| \w | 12&”: ;c | 3 matches (at 12&”: ;c) |
| %”> ! | No match |
\W – Matches any non-alphanumeric character. Equivalent to [^a-zA-Z0-9_]
Expression String Matched? \w 1a2%c 1 match (at 1a2%c) Python No match
\Z – Matches if the specified characters are at the end of a string.
| Expression | String | Matched? |
| Python\Z | I like Python | 1 match |
| I like Python Programming | No match | |
| Python is fun. | No match |
Tip: To build and test regular expressions, you can use RegEx tester tools such as regex101. This tool not only helps you in creating regular expressions, but it also helps you learn it.
Now you understand the basics of RegEx, let’s discuss how to use RegEx in your Python code.
Python RegEx
Python has a module named re to work with regular expressions. To use it, we need to import the module.
import re
The module defines several functions and constants to work with RegEx.
re.findall()
The re.findall() method returns a list of strings containing all matches.
Example 1: re.findall()
# Program to extract numbers from a string import re string = 'hello 12 hi 89. Howdy 34' pattern = '\d+' result = re.findall(pattern, string) print(result) # Output: ['12', '89', '34']
If the pattern is not found, re.findall() returns an empty list.
re.split()
The re.split method splits the string where there is a match and returns a list of strings where the splits have occurred.
Example 2: re.split()
import re string = 'Twelve:12 Eighty nine:89.' pattern = '\d+' result = re.split(pattern, string) print(result) # Output: ['Twelve:', ' Eighty nine:', '.']
If the pattern is not found, re.split() returns a list containing the original string.
You can pass the max split argument to the re.split() method. It’s the maximum number of splits that will occur.
import re string = 'Twelve:12 Eighty nine:89 Nine:9.' pattern = '\d+' # maxsplit = 1 # split only at the first occurrence result = re.split(pattern, string, 1) print(result) # Output: ['Twelve:', ' Eighty nine:89 Nine:9.']
By the way, the default value of the max split is 0; meaning all possible splits.
re.sub()
The syntax of re.sub() is:
re.sub(pattern, replace, string)
The method returns a string where matched occurrences are replaced with the content of the replace variable.
Example 3: re.sub()
# Program to remove all whitespaces import re # multiline string string = 'abc 12\ de 23 \n f45 6' # matches all whitespace characters pattern = '\s+' # empty string replace = '' new_string = re.sub(pattern, replace, string) print(new_string) # Output: abc12de23f456
If the pattern is not found, re.sub() returns the original string.
You can pass count as a fourth parameter to the re.sub() method. If omitted, it results in 0. This will replace all occurrences.
import re # multiline string string = 'abc 12\ de 23 \n f45 6' # matches all whitespace characters pattern = '\s+' replace = '' new_string = re.sub(r'\s+', replace, string, 1) print(new_string) # Output: # abc12de 23 # f45 6
re.subn()
The re.subn() is similar to re.sub() expect it returns a tuple of 2 items containing the new string and the number of substitutions made.
Example 4: re.subn()
# Program to remove all whitespaces
import re
# multiline string
string = 'abc 12\
de 23 \n f45 6'
# matches all whitespace characters
pattern = '\s+'
# empty string
replace = ''
new_string = re.subn(pattern, replace, string)
print(new_string)
# Output: ('abc12de23f456', 4)
re.search()
The re.search() strategy takes two contentions: an example and a string. The technique searches for the principal area where the RegEx design delivers a match with the string.
If the search is successful, re.search() restores a match object; if not, it returns none.
match = re.search(pattern, str)
Example 5: re.search()
import re
string = "Python is fun"
# check if 'Python' is at the beginning
match = re.search('\APython', string)
if match:
print("pattern found inside the string")
else:
print("pattern not found")
# Output: pattern found inside the string
Here, the match contains a match object.
Match object
You can get the methods and attributes of a match object using dir() function.
Some of the commonly used methods and attributes of match objects are:
match.group()
The group() method returns the part of the string where there is a match.
Example 6: Match object
import re
string = '39801 356, 2102 1111'
# Three digit number followed by space followed by two digit number
pattern = '(\d{3}) (\d{2})'
# match variable contains a Match object.
match = re.search(pattern, string)
if match:
print(match.group())
else:
print("pattern not found")
# Output: 801 35
Here, the match variable contains a match object.
Our pattern (\d{3}) (\d{2}) has two subgroups (\d{3}) and (\d{2}). You can get the part of the string of these parenthesized subgroups. Here’s how:
>>> match.group(1)
'801'
>>> match.group(2)
'35'
>>> match.group(1, 2)
('801', '35')
>>> match.groups()
('801', '35')
match.start(), match.end() and match.span()
The start() function returns the index of the start of the matched substring. Similarly, end() returns the end index of the matched substring.
>>> match.start() 2 >>> match.end() 8
The span() function returns a tuple containing the start and end index of the matched part.
>>> match.span() (2, 8)
match.re and match.string
The re-attribute of a matched object returns a regular articulation object. Additionally, string property restores the passed string.
>>> match.re
re.compile('(\\d{3}) (\\d{2})')
>>> match.string
'39801 356, 2102 1111'
If you want to learn more, visit the Python 3 re module.
Using r prefix before RegEx
When r or R prefix is used before a regular expression, it means raw string. For example, ‘\n’ is a new line whereas r’\n’ means two characters: a backslash \ followed by n.
Backlash \ is used to escape various characters including all metacharacters. However, using r prefix makes \ treat as a normal character.
Example 7: Raw string using r prefix
import re string = '\n and \r are escape sequences.' result = re.findall(r'[\n\r]', string) print(result) # Output: ['\n', '\r']
Please feel free to give your comment if you face any difficulty here.
