Pattern-matching text with re

Text patterns with re

Raw text arrives messy: log lines like "order 12, item 345", user-typed phone numbers, IDs buried in free-form comments. A regular expression (regex) is a compact pattern language for "text that looks like this," and Python's re module runs those patterns. When a data engineer needs to pull every order ID out of a million log lines, or reject rows whose zip_code field is not five digits, regex does it in one pass instead of a hand-written character loop.

Write patterns as raw strings

Always write patterns as raw strings (r"..."). Regex leans on the backslash (\d, \w, \s), and in a normal Python string the backslash is an escape character. r"\d" is the two characters backslash-d, exactly what the regex engine wants; a plain "\d" is fragile because \d is not a valid string escape, and in Python 3.12 and later it triggers a SyntaxWarning.

The pieces you will use most

Table

Two rows quietly cause most regex surprises. * can match zero characters, so a pattern built only from * always succeeds without consuming anything, and + is greedy by default, so it runs to the LAST possible match rather than the first.

Pattern	Matches	Note
\d	one digit, 0 to 9	\D is the negation: one non-digit
\w	one letter, digit, or underscore	\W negates it
\s	one whitespace character	space, tab, or newline
.	any single character	except a newline, unless re.DOTALL
+	one or more of what precedes it	greedy: use +? to take as few as possible
*	zero or more of what precedes it	can match nothing at all
[abc]	any single one of a, b, or c	[^abc] means any character EXCEPT those
( )	a capture group	its contents come back from .group(1)

The three workhorse functions

import re

re.findall(r"\d+", "a1 b22")   # ['1', '22']  every match, as a list
re.search(r"\d+", "abc7")       # a Match at '7', or None if nothing matches
re.sub(r"\d", "#", "a1b2")      # 'a#b#'  replace every match

re.findall returns a list of every non-overlapping match, left to right. That is exactly the Apply task: re.findall(r"\d+", "a1b22") gives ["1", "22"], and [] when there are no digits. re.sub(pattern, replacement, text) swaps every match for the replacement string, which is the Practice task: re.sub(r"\d", "#", "a1b2") gives "a#b#".

Pitfalls

\d matches one digit; \d+ matches a whole run. re.findall(r"\d", "a1b22") returns ['1', '2', '2'] (three separate digits), while re.findall(r"\d+", "a1b22") returns ['1', '22']. Reach for + when you want whole numbers, and drop it when you want single characters (as in redaction, where replacing each digit one at a time is fine).

A capture group changes what findall returns. With no group it returns the full match; with one group it returns only that group; with several it returns tuples:

re.findall(r"\w+@\w+", "a@x b@y")     # ['a@x', 'b@y']
re.findall(r"(\w+)@(\w+)", "a@x b@y") # [('a', 'x'), ('b', 'y')]

So do not wrap your whole pattern in () out of habit; it silently reshapes the result.

Interview nuance: quantifiers are greedy by default. .* matches as much as it can, then backtracks. re.findall(r"<.*>", "<a><b>") returns ['<a><b>'], not the two tags you probably wanted. Add ? to make it lazy: re.findall(r"<.*?>", "<a><b>") returns ['<a>', '<b>']. Interviewers use this to check whether you understand that the engine explores text by backtracking, which is also why a careless pattern over adversarial input can blow up to quadratic time.