Top 5 Hidden sed Tricks You Didn’t Know Existed

CLI text-processing one-liners

Everyone knows sed for quick find-and-replace, but it can also splice files together in memory, run shell commands, sample lines, and tee matches out to another file. Here are five surprisingly useful tricks—with copy-ready snippets and portability notes for GNU vs. BSD/macOS.

Table of Contents

1. Run shell commands inside replacements (e flag)
2. Select every nth line with step addressing
3. De-duplicate consecutive lines (no uniq needed)
4. Write matches to a separate file with w
5. Multi-line editing by “slurping” lines
6. Cheat sheet

---

1) Run shell commands inside replacements (e flag)

Scope: GNU sed (Linux, Homebrew gsed on macOS). Executes the right-hand side as a shell command and substitutes its output. Powerful—handle with care.

Use-case: Fill placeholders in a template with dynamic values (date, git hash, hostname, etc.).

# template.txt:
# Build {{DATE}} • Commit {{GIT}} • Host {{HOST}}

sed -E \
  -e 's/\{\{DATE\}\}/date +%F/e' \
  -e 's/\{\{GIT\}\}/git rev-parse --short HEAD/e' \
  -e 's/\{\{HOST\}\}/hostname/e' \
  template.txt

macOS/BSD alternative (no e flag)

# Use the shell to expand variables, then plain sed:
DATE=$(date +%F) GIT=$(git rev-parse --short HEAD) HOST=$(hostname) \
  sed -E "s/\{\{DATE\}\}/$DATE/; s/\{\{GIT\}\}/$GIT/; s/\{\{HOST\}\}/$HOST/" template.txt

---

2) Select every nth line with step addressing

Use-case: Sample a gigantic log: print every 3rd line (or 10th, etc.).

# Every 3rd line (GNU sed):
sed -n '1~3p' big.log

# Every 10th line:
sed -n '1~10p' big.log

POSIX/BSD sed workaround

# Print 1st, 4th, 7th, ... line (no ~ support):
sed -n '1p; n; n; :a; n; n; n; p; ba' big.log

---

3) De-duplicate consecutive lines (no uniq needed)

Use-case: Collapse runs of identical adjacent lines to a single line (e.g., noisy logs).

# Input:
# A
# A
# B
# B
# B
# C

sed '$!N; /^\(.*\)\n\1$/!P; D' file.txt
# Output:
# A
# B
# C

---

4) Write matches to a separate file with w

Use-case: Extract error messages to errors.txt while leaving stdout clean or doing other processing.

# Capture everything after "ERROR: " into errors.txt
sed -nE 's/^ERROR:\s+(.*)$/\1/w errors.txt' server.log

---

5) Multi-line editing by “slurping” lines

Use-case: Join backslash-continued lines or do a substitution that crosses line breaks.

# Join lines ending with backslash "\" into a single logical line:
sed -e ':a' -e '/\\$/N; s/\\\n//; ta' source.txt

# Replace patterns that span a newline (e.g., "foo\nbar" → "foobar"):
sed ':a;N;$!ba; s/foo\nbar/foobar/g' input.txt

---

Quick Copy-Paste Cheat Sheet

# 1) Dynamic replacements (GNU sed):
sed -E 's/\{\{DATE\}\}/date +%F/e; s/\{\{GIT\}\}/git rev-parse --short HEAD/e' template.txt

# 2) Every n-th line (GNU sed):
sed -n '1~5p' big.log

# 3) Collapse adjacent duplicates:
sed '$!N; /^\(.*\)\n\1$/!P; D' file.txt

# 4) Write matches to a file:
sed -nE 's/^ERROR:\s+(.*)$/\1/w errors.txt' server.log

# 5) Multi-line “slurp” tricks:
sed -e ':a' -e '/\\$/N; s/\\\n//; ta' source.txt
sed ':a;N;$!ba; s/foo\nbar/foobar/g' input.txt

Pro tip: When a one-liner gets gnarly, save it as a script file and run sed -Ef script.sed input.txt for readability and version control.