Frequently-Asked Questions about sed, the stream editor

This is Section 2 of the sed FAQ.
Go back to Section 1, or forward to Section 3.
Go up to the index.

2. BASIC SED

2.1. What is sed?

"sed" stands for Stream EDitor. Sed is a non-interactive editor, written by the late Lee E. McMahon in 1973 or 1974. A brief history of sed's origins may be found in an early history of the Unix tools, at <http://www.columbia.edu/~rh120/ch106.x09>.

Instead of altering a file interactively by moving the cursor on the screen (as with a word processor), the user sends a script of editing instructions to sed, plus the name of the file to edit (or the text to be edited may come as output from a pipe). In this sense, sed works like a filter -- deleting, inserting and changing characters, words, and lines of text. Its range of activity goes from small, simple changes to very complex ones.

Sed reads its input from stdin (Unix shorthand for "standard input," i.e., the console) or from files (or both), and sends the results to stdout ("standard output," normally the console or screen). Most people use sed first for its substitution features. Sed is often used as a find-and-replace tool.

     sed 's/Glenn/Harold/g' oldfile >newfile

will replace every occurrence of "Glenn" with the word "Harold", wherever it occurs in the file. The "find" portion is a regular expression ("RE"), which can be a simple word or may contain special characters to allow greater flexibility (for example, to prevent "Glenn" from also matching "Glennon").

My very first use of sed was to add 8 spaces to the left side of a file, so when I printed it, the printing wouldn't begin at the absolute left edge of a piece of paper.

     sed 's/^/        /' myfile >newfile   # my first sed script
     sed 's/^/        /' myfile | lp       # my next sed script

Then I learned that sed could display only one paragraph of a file, beginning at the phrase "and where it came" and ending at the phrase "for all people". My script looked like this:

     sed -n '/and where it came/,/for all people/p' myfile

Sed's normal behavior is to print (i.e., display or show on screen) the entire file, including the parts that haven't been altered, unless you use the -n switch. The "-n" stands for "no output". This switch is almost always used in conjunction with a 'p' command somewhere, which says to print only the sections of the file that have been specified. The -n switch with the 'p' command allow for parts of a file to be printed (i.e., sent to the console).

Next, I found that sed could show me only (say) lines 12-18 of a file and not show me the rest. This was very handy when I needed to review only part of a long file and I didn't want to alter it.

# the 'p' stands for print

sed -n 12,18p myfile

Likewise, sed could show me everything else BUT those particular lines, without physically changing the file on the disk:

# the 'd' stands for delete

sed 12,18d myfile

Sed could also double-space my single-spaced file when it came time to print it:

     sed G myfile >newfile

If you have many editing commands (for deleting, adding, substituting, etc.) which might take up several lines, those commands can be put into a separate file and all of the commands in the file applied to file being edited:

#  'script.sed' is the file of commands
     # 'myfile' is the file being changed

sed -f script.sed myfile  # 'script.sed' is the file of commands

It is not our intention to convert this FAQ file into a full-blown sed tutorial (for good tutorials, see section 2.3). Rather, we hope this gives the complete novice a few ideas of how sed can be used.

2.2. What versions of sed are there, and where can I get them?

2.2.1. Free versions

Note: "Free" does not mean "public domain" nor does it necessarily mean you will never be charged for it. All versions of sed in this section except the CP/M versions are based on the GNU general public license and are "free software" by that standard (for details, see http://www.gnu.org/philosophy/free-sw.html). This means you can get the source code and develop it further.

At the URLs listed in this category, sed binaries or source code can be downloaded and used without fees or license payments.

2.2.1.1. Unix platforms

ssed v3.60

ssed is the version recommended by the FAQ maintainers, since it shares the same codebase with GNU sed, has the most options, and is free software (you can get the source). Though there were earlier version of ssed distributed, sites for these are not being listed.

       http://sed.sourceforge.net/grabbag/ssed
       http://freshmeat.net/project/sed/

GNU sed v4.0.5

This is the latest official version of GNU sed. It offers in-place text replacement as an option switch.

       ftp://ftp.gnu.org/pub/gnu/sed/sed-4.0.5.tar.gz
       http://freshmeat.net/project/sed

BSD multi-byte sed (Japanese)

Based on the latest version of GNU sed, which supports multi-byte characters.

       ftp://ftp1.freebsd.org/pub/FreeBSD/FreeBSD-stable/packages/Latest/ja-sed.tgz

GNU sed v3.02.80

An alpha test release which was the base for the development of ssed and GNU sed v4.0.

       ftp://alpha.gnu.org/pub/gnu/sed/sed-3.02.80.tar.gz

GNU sed v3.02a

Interim version with most features of GNU sed v3.02.80.

GNU sed v3.02

       ftp://ftp.gnu.org/pub/gnu/sed/sed-3.02.tar.gz

Precompiled versions:

GNU sed v3.02-8

source code and binaries for Debian GNU/Linux

       http://www.debian.org/Packages/stable/base/sed.html

For some time, the GNU project <http://www.gnu.org> used Eric S. Raymond's version of sed (ESR sed v1.1), but eventually dropped it because it had too many built-in limits. In 1991 Howard Helman modified the GNU/ESR sed and produced a flexible version of sed v1.5 available at several sites (Helman's version permitted things like \<...\> to delimit word boundaries, \xHH to enter hex code and \n to indicate newlines in the replace string). This version did not catch on with the GNU project and their version of sed has moved in a similar but different direction.

sed v1.3, by Eric Steven Raymond (released 4 June 1998)

       http://catb.org/~esr/sed-1.3.tar.gz

Eric Raymond <esr@snark.thyrsus.com> wrote one of the earliest versions of sed. On his website <http://www.catb.org/~esr/> which also distributes many freeware utilities he has written or worked on, he describes sed v1.1 this way:

"This is the fast, small sed originally distributed in the GNU toolkit and still distributed with Minix. The GNU people ditched it when they built their own sed around an enhanced regex package -- but it's still better for some uses (in particular, faster and less memory-intensive)." (Version 1.3 fixes an unidentified bug and adds the L command to hexdump the current pattern space.)

2.2.1.2. OS/2

GNU sed v3.02.80

       http://www2s.biglobe.ne.jp/~vtgf3mpr/gnu/sed.htm

GNU sed v3.02

       http://hobbes.nmsu.edu/pub/os2/util/file/sed-3_02-r2-bin.zip # binaries
       http://hobbes.nmsu.edu/pub/os2/util/file/sed-3_02-r2.zip     # source

2.2.1.3. Microsoft Windows (Win3x, Win9x, WinNT, Win2K)

GNU sed v4.0.5

32-bit binaries and docs. Precompiled versions not available (yet).

GNU sed v3.02.80

32-bit binaries and docs, using DJGPP compiler. For details on new features, see Unix section, above.

       http://www.student.northpark.edu/pemente/sed/sed3028a.zip # DOS binaries
       ftp://alpha.gnu.org/pub/gnu/sed/sed-3.02.80.tar.gz        # source
       ftp://ftp.simtel.net/pub/simtelnet/gnu/djgpp/v2gnu/sed3028b.zip # binaries
       ftp://ftp.simtel.net/pub/simtelnet/gnu/djgpp/v2gnu/sed3028d.zip # docs
       ftp://ftp.simtel.net/pub/simtelnet/gnu/djgpp/v2gnu/sed3028s.zip # source

GNU sed v2.05

32-bit binaries, no docs. Requires 80386 DX (SX will not run) and must be run in a DOS window or in a full screen DOS session under Microsoft Windows. Will not run in MS-DOS mode (outside Win/Win95). We recommend using the latest version of GNU sed.

       http://www.simtel.net/pub/win95/prog/gsed205b.zip
       ftp://ftp.cdrom.com/pub/simtelnet/win95/prog/gsed205b.zip

GNU sed v1.03

modified by Frank Whaley.

This version was part of the "Virtually UN*X" toolset, hosted by itribe.net; that website is now closed. Gsed v1.03 supported Win9x long filenames, as well as hex, decimal, binary, and octal character representations.

The Cygwin toolkit:

       http://www.cygwin.com

Formerly know as "GNU-Win32 tools." According to their home page, "The Cygwin tools are Win32 ports of the popular GNU development tools for Windows NT, 95 and 98. They function through the use of the Cygwin library which provides a UNIX-like API on top of the Win32 API." The version of sed used is GNU sed v3.02.

Minimalist GNU for Windows (MinGW):

       http://www.mingw.org
       http://mingw.sourceforge.net

According to their home page, "MinGW ('Minimalist GNU for Windows') refers to a set of runtime headers, used in building a compiler system based on the GNU GCC and binutils projects. It compiles and links code to be run on Win32 platforms ... MinGW uses Microsoft runtime libraries, distributed with the Windows operating system." The version of sed used is GNU sed v3.02.

sed v1.5 (a/k/a HHsed), by Howard Helman

Compiled with Mingw32 for 32-bit environments described above. This version should support Win95 long filenames.

       http://www.dbnet.ece.ntua.gr/~george/sed/OLD/sed15.exe
       http://www.student.northpark.edu/pemente/sed/sed15exe.zip

2.2.1.4. MS-DOS

sed v1.6 (from HHsed), by Walter Briscoe

This is a forthcoming version, now in beta testing, but with many new features. It corrects all the bugs in sed v1.5, and adds the best features of sedmod v1.0 (below). It is available in 16-bit and 32-bit compiled versions for MS-DOS. Sorry, no URLs available yet.

sed v1.5 (a/k/a HHsed), by Howard Helman

uncompiled source code (Turbo C)

       ftp://ftp.simtel.net/pub/simtelnet/msdos/txtutl/sed15.zip
       ftp://ftp.cdrom.com/pub/simtelnet/msdos/txtutl/sed15.zip

DOS executable and documentation

       ftp://ftp.simtel.net/pub/simtelnet/msdos/txtutl/sed15x.zip
       ftp://ftp.cdrom.com/pub/simtelnet/msdos/txtutl/sed15x.zip

sedmod v1.0, by Hern Chen

       http://www.ptug.org/sed/SEDMOD10.ZIP
       http://www.student.northpark.edu/pemente/sed/sedmod10.zip
       ftp://garbo.uwasa.fi/pc/unix/sedmod10.zip

GNU sed v3.02.80

See section 2.2.1.3 ("Microsoft Windows"), above.

GNU sed v2.05

Does not run under MS-DOS.

GNU sed v1.18

32-bit binaries and source, using DJGPP compiler. Requires 80386 SX or better. Also requires 3 CWS*.EXE extenders on the path. See section 5.5 ("What is CSDPMI*B.ZIP and why do I need it?"), below. We recommend using a newer version of GNU sed.

       http://www.simtel.net/pub/simtelnet/gnu/djgpp/v2gnu/sed118b.zip
       ftp://ftp.cdrom.com/pub/simtelnet/gnu/djgpp/v2gnu/sed118b.zip
       http://www.simtel.net/pub/simtelnet/gnu/djgpp/v2gnu/sed118s.zip
       ftp://ftp.cdrom.com/pub/simtelnet/gnu/djgpp/v2gnu/sed118s.zip

GNU sed v1.06

16-bit binaries and source. Should run under any MS-DOS system.

       http://www.simtel.net/pub/gnu/gnuish/sed106.zip
       ftp://ftp.cdrom.com/pub/simtelnet/gnu/gnuish/sed106.zip

2.2.1.5. CP/M

ssed v2.2, by Chuck A. Forsberg

Written for CP/M, ssed (for "small/stupid stream editor) supports only the a(ppend), c(hange), d(elete) and i(nsert) options, and apparently doesn't support regular expressions. A -u switch will "unsqueeze" compressed files and was used mainly in conjunction with DIF.COM for source code maintenance. (file: ssed22.lbr)

change, by Michael M. Rubenstein

Rubenstein released a version of sed called CHANGE.COM (the TTOOLS.LBR archive member CHANGE.CZM is a "crunched" file). CHANGE.COM supports full RE's except grouping and backreferences, and its only function is global substitution. (file: ttools.lbr)

2.2.1.6. Macintosh v8 or v9

Since sed is a command-line utility, it is not customary to think of sed being used on a Mac. Nonetheless, the following instructions from Aurelio Jargas describe the process for running sed on MacOS version version 8 or 9.

(1) Download and install the Apple DiskCopy application

       ftp://ftp.apple.com/developer/Development_Kits

(2) Download and install Apple MPW

       ftp://ftp.apple.com/developer/Tool_Chest/Core_Mac_OS_Tools/MPW_etc./

(3) Download and expand Matthias Neeracher's GNU sed for MPW. (They seem to have misnumbered the sed filename.)

       ftp://sunsite.cnlab-switch.ch/software/platform/macos/src/mpw_c/sed-2.03.sit.bin

(4) Enter the sed-3.02 directory and doubleclick the 'sed' file

(5) MPW Shell will open up. It will be a command window instead of a command line, but sed should work as expected. For example:

       echo aa | sed 's/a/Z/g'<ENTER>

Note that ENTER is different from RETURN on an iMac. Apple also has its own version of sed on MPW, called "StreamEdit", with a syntax fairly similar to that of normal sed.

2.2.2. Shareware and Commercial versions

2.2.2.1. Unix platforms

       [ Additional information needed. ]

2.2.2.2. OS/2

Hamilton Labs:

       http://www.hamiltonlabs.com/cshell.htm

A sizable set of Unix/C shell utilities designed for OS/2. Price is $350 in the US, $395 elsewhere, with FedEx shipping, unconditional guarantee, unlimited support and free updates. A demo version of the suite can be downloaded from this site, but a stand-alone copy of sed is not available.

2.2.2.3. Windows 95/98, Windows NT, Windows 2000

Hamilton Labs:

       http://www.hamiltonlabs.com/cshell.htm

A sizable set of Unix/C shell utilities designed for Win9x, WinNT, and Win2K. Price is $350 in the US, $395 elsewhere, with FedEx shipping, unconditional guarantee, unlimited support and free updates. A demo version of the suite can be downloaded from this site, but a stand-alone copy of sed is not available.

Interix:

       http://www.interix.com

Interix (formerly known as OpenNT) is advertised as "a complete UNIX system environment running natively on Microsoft Windows NT", and is licensed and supported by Softway Systems. It offers over 200 Unix utilities, and supports Unix shells, sockets, networking, and more. A single-user edition runs about $200. A free demo or evaluation copy will run for 31 days and then quit; to continue using it, you must purchase the commercial version.

MKS NuTCRACKER Professional

       http://www.datafocus.com/products/nutc/

A different, yet related product line offered by MKS (Mortice Kern Systems, below); the awkward spelling "NuTCRACKER" is intentional. Various packages offer hundreds of Unix utilities for Win32 environments. Sed is not available as a separate product.

UnixDos:

       http://www.unixdos.com

UnixDos is a suite of 82 Unix utilities ported over to the Windows environments. There are 16-bit versions for Win3.x and 32-bit versions for WinNT/Win95. It is distributed as uncrippled shareware for the first 30 days. After the test period, the utilities will not run and you must pay the registration fee of $50.

Their version of sed supports "\n" in the RHS of expressions, and increases the length of input lines to 10,000 characters. By special arrangement with the owners, persons who want a licensed version of sed only (without the other utilities) may pay a license fee of $10.

U/WIN:

       http://www.research.att.com/sw/tools/uwin/

U/WIN is a suite of Unix utilities created for WinNT and Win95 systems. It is owned by AT&T, created by David Korn (author of the Unix korn shell), and is freely distributed only to educational institutions, AT&T employees, or certain researchers; all others must pay a fee after a 90-day evaluation period expires. U/WIN operates best with the NTFS (WinNT file system) but will run in degraded mode with the FAT file system and in further degraded mode under Win95. A minimal installation takes about 25 to 30 megs of disk space. Sed is not available as a separate file for download, but comes with the suite.

2.2.2.4. MS-DOS

Mix C/Utilities Toolchest

       http://www.mixsoftware.com/product/utility.htm

According to their web page, "The C/Utilities Toolchest adds over 40 powerful UNIX utilities to your MS-DOS operating system. The result is an environment very similar to UNIX operating systems, yet 100% compatible with MS-DOS programs and commands." The toolchest costs $19.95, with source code available for an additional fee. Mix C's version of sed is not available separately.

MKS (Mortice Kern Systems) Toolkit

       http://www.mks.com

Sed comes bundled with the MKS Toolkit, which is distributed only as commercial software; it is not available separately.

Thompson Automation Software

       http://www.tasoft.com

The Thompson Toolkit contains over 100 familiar Unix utilities, including a version of the Unix Korn shell. It runs under MS-DOS, OS/2, Win3.x, Win9x, and WinNT. Sed is one of the utilities, though Thompson is better known for its version of awk for DOS, TAWK. The toolkit runs about $150; sed is not available separately.

2.3. Where can I learn to use sed?

2.3.1. Books

       Sed & Awk, 2d edition, by Dale Dougherty & Arnold Robbins
       (Sebastopol, Calif: O'Reilly and Associates, 1997)
       ISBN 1-56592-225-5
       http://www.oreilly.com/catalog/sed2/noframes.html

About 40 percent of this book is devoted to sed, and maybe 50 percent is devoted to awk. The other 10 percent covers regexes and concepts common to both tools. If you prefer hard copy, this is definitely the best single place to learn to use sed, including its advanced features.

The first edition is also very useful. Several typos crept into the first printing of the first edition (though if you follow the tutorials closely, you'll recognize them right away). A list of errors from the first printing of sed & awk is available at <http://www.cs.colostate.edu/~dzubera/sedawk.txt>, and errors in the 2nd are at <http://www.cs.colostate.edu/~dzubera/sedawk2.txt>, though most of these were corrected in later printings. The second edition tells how POSIX standards have affected these tools and covers the popular GNU versions of sed and awk. Price is about (US) $30.00

-----

       Mastering Regular Expressions, 2d ed., by Jeffrey E. F. Friedl
       (Sebastopol, Calif: O'Reilly and Associates, 2002)
       ISBN 0-596-00289-0
       http://regex.info
       http://www.oreilly.com/catalog/regex2/
       http://public.yahoo.com/~jfriedl/regex/ (for the first edition)

Knowing how to use "regular expressions" is essential to effective use of most Unix tools. This book focuses on how regular expressions can be best implemented in utilities such as perl, vi, emacs, and awk, but also touches on sed as well. Friedl's home page (above) gives links to other sites which help students learn to master regular expressions. His site also gives a Perl script for determining a syntactically valid e-mail address, using regexes:

       http://public.yahoo.com/~jfriedl/regex/code.html

-----

       Awk und Sed, by Helmut Herold.
       (Bonn: Addison-Wesley, 1994; 288 pages)
       2nd edition to be released in March 2003
       ISBN 3-8273-2094-1
       http://www.addison-wesley.de/main/main.asp?page=home/bookdetails&ProductID=37214

2.3.2. Mailing list

If you are interested in learning more about sed (its syntax, using regular expressions, etc.) you are welcome to subscribe to a sed-oriented mailing list. In fact, there are two mailing lists about sed: one in English named "sed-users", moderated by Sven Guckes; and one in Portuguese named "sed-BR" (for sed-Brazil), moderated by Aurelio Marinho Jargas. The average volume of mail for "sed-users" is about 35 messages a week; the average volume of mail for "sed-BR" is about 15 messages a week.

       sed-BR mailing list:    http://br.groups.yahoo.com/group/sed-br/
       sed-users mailing list: http://groups.yahoo.com/group/sed-users/

To subscribe to sed-users, send a blank message to:

       sed-users-subscribe@yahoogroups.com

To unsubscribe from sed-users, send a blank message to:

       sed-users-unsubscribe@yahoogroups.com

2.3.3. Tutorials, electronic text

The original users manual for sed, by Lee E. McMahon, from the 7th edition UNIX Manual (1978), with the classic "Kubla Khan" example and tutorial, in formatted text format:

       http://sed.sourceforge.net/grabbag/tutorials/sed_mcmahon.txt

The source code to the preceding manual. Use "troff -ms sed" to print this file properly:

       http://plan9.bell-labs.com/7thEdMan/vol2/sed
       http://cm.bell-labs.com/7thEdMan/vol2/sed

"Do It With Sed", by Carlos Duarte

       http://www.dbnet.ece.ntua.gr/~george/sed/OLD/sedtut_1.html

"Sed: How to use sed, a special editor for modifying files automatically", by Bruce Barnett and General Electric Company

       http://www.grymoire.com/Unix/Sed.html

U-SEDIT2.ZIP, by Mike Arst (16 June 1990)

       ftp://ftp.cs.umu.se/pub/pc/u-sedit2.zip
       ftp://ftp.uni-stuttgart.de/pub/systems/msdos/util/unixlike/u-sedit2.zip
       ftp://sunsite.icm.edu.pl/vol/wojsyl/garbo/pc/editor/u-sedit2.zip
       ftp://ftp.sogang.ac.kr/pub/msdos/garbo_pc/editor/u-sedit2.zip

U-SEDIT3.ZIP, by Mike Arst (24 Jan. 1992)

       http://www.student.northpark.edu/pemente/sed/u-sedit3.zip
       CompuServe DTPFORUM, "PC DTP Utilities" library, file SEDDOC.ZIP

Another sed FAQ

       http://www.dreamwvr.com/sed-info/sed-faq.html

sed-tutorial, by Felix von Leitner

       http://www.math.fu-berlin.de/~leitner/sed/tutorial.html

"Manipulating text with sed," chapter 14 of the SCO OpenServer "Operating System Users Guide"

       http://ou800doc.caldera.com/SHL_automate/CTOC-Manipulating_text_with_sed.html

"Combining the Bourne-shell, sed and awk in the UNIX environment for language analysis," by Lothar Schmitt and Kiel Christianson. This basic tutorial on the Bourne shell, sed and awk downloads as a 71-page PostScript file (compressed to 290K with gzip). You may need to navigate down from the root to get the file.

       ftp://ftp.u-aizu.ac.jp/u-aizu/doc/Tech-Report/1997/97-2-007.tar.gz
       available upon request from Lothar Schmitt <lothar@u-aizu.ac.jp>

2.3.4. General web and ftp sites

       http://sed.sourceforge.net/grabbag             # Collected scripts
       http://main.rtfiber.com.tw/~changyj/sed/       # Yao-Jen Chang
       http://www.math.fu-berlin.de/~guckes/sed/      # Sven Guckes
       http://www.math.fu-berlin.de/~leitner/sed/     # Felix von Leitner
       http://www.dbnet.ece.ntua.gr/~george/sed/      # Yiorgos Adamopoulos
       http://www.student.northpark.edu/pemente/sed/  # Eric Pement

       http://spacsun.rice.edu/FAQ/sed.html
       ftp://algos.inesc.pt/pub/users/cdua/scripts.tar.gz (sed and shell scripts)

"Handy One-Liners For Sed", compiled by Eric Pement. A large list of 1-line sed commands which can be executed from the command line.

       http://sed.sourceforge.net/sed1line.txt
       http://www.student.northpark.edu/pemente/sed/sed1line.txt

"Handy One-Liners For Sed", translated to Portuguese

       http://wmaker.lrv.ufsc.br/sed_ptBR.html

The Single UNIX Specification, Version 3 (technical man page)

       http://www.opengroup.org/onlinepubs/007904975/utilities/sed.html

Getting started with sed

       http://www.cs.hmc.edu/tech_docs/qref/sed.html

masm to gas converter

       http://www.delorie.com/djgpp/faq/converting/asm2s-sed.html

mail2html.zip

       http://www.crispen.org/src/#mail2html

sample uses of sed in batch files and scripts (Benny Pederson)

       http://users.cybercity.dk/~bse26236/batutil/help/SED.HTM

dc.sed - the most complex and impressive sed script ever written. This sed script by Greg Ubben emulates the Unix dc (desk calculator), including base conversion, exponentiation, square roots, and much more.

       http://sed.sourceforge.net/grabbag/scripts/dc_overview.htm

If you should find other tutorials or scripts that should be added to this document, please forward the URLs to the FAQ maintainer.

This is the end of Section 2 of the sed FAQ.
Go back to Section 1, or forward to Section 3.
Go up to the index.

Site Links
The Books I Own
Main Page
  Vi in Emacs
  Linux on Vaio
  Study NZ
  Utilities
  Programming Fun?
  SED FAQ
  C Language
  Source Code Browsers
  C Struct Packing
  Walt Disney World
  PPP RFCs
  FSM/HSM
  Tcl/Tk
  Photographs of Flowers
  Random Photogaphs
  Put this on your site!
  SQLite
The Sundial Bridge
Repetitive Strain Injury (RSI)
Selling Software Online (MicroISV)
Tcl Tk Life-Savers
The Experience Shows!
Green Tips
.htaccess tricks
Web-Site Development Online Tools
Blog