URL:         http://www.math.fu-berlin.de/~guckes/sed/
Created:     Sun Aug 18 18:18:18 CET 1996
Last update: Tue Jun 06 12:00:00 MET DST 2000

SED - the stream editor

Scripts can really help a lot. And SED definitely is one of the best scripting programs around!


SED Pages - Overview

Scripts | mailing list | other solutions | more... | sed links

Scripts

Here are some scripts that are really helpful. Enjoy!
insert after column [000606]
Task: Insert a character/string after the n-th column in every line.
Example:
	IN:
	1234567890123456
	abcdefghijklmnop
	The quick brown

	OUT:
	123|456789|01|23456
	abc|defghi|jk|lmnop
	The| quick| b|rown
Idea: Use '^' to anchor the line at the beginning and use '.' to match each character within the line. Group the characters with "\(\)" pairs and use back references in the replacement ("\1", "\2", etc); insert the additional characters/strings between the back references.
Solution: sed -e 's/^\(...\)\(......\)\(..\)/\1\|\2\|\3\|/' If you don't want to count the dots the you can use "\{n\}" for an exact repetition of a single dot: sed -e 's/^\(.\{3\}\)\(.\{6\}\)\(.\{2\}\)/\1\|\2\|\3\|/'
Problem posed by Fred Distenfeld Fred_Distenfeld@lnotes5.bankofny.com [000605]

8bit to HTML
Task: Convert all (well, most) "8bit characters" aka "ISO characters" (ASCII 128-255) to their equivalent in HTML.
8bit2html.sed
Author: Xose Ramos in98xora@mikkeliamk.fi [980421]

Extracting IP numbers
IP numbers are "number addresses" of hosts. They typically appear in some log files of server/client programs. Extracting these from the logs is a frequent task of many programmers. IP numbers consists of four numbers with 8bit values (0-255), separated by (three) dots.
Examples: 127.0.0.1 - the default address of the "localhost".
sed -n 's/[^0-9]*\([0-9]\{1,3\}\.[0-9]\{1,3\}\.[0-9]\{1,3\}\.[0-9]\{1,3\}\).*/\1/p' infile > outfile
	string		read as		explanation
	s		"substitute"
	/		delimiter	start of pattern
	[^0-9]*		non-digits	need not be anchored to '^';
					use of ".*" would take away
					from the following digits!
	\(		"start group"	- start of IP number
	[0-9]\{1,3\}	a number	at least one, at most three digits
	\.		a literal dot	seperating the numbers
	[0-9]\{1,3\}	a number	at least one, at most three digits
	\.		a literal dot	seperating the numbers
	[0-9]\{1,3\}	a number	at least one, at most three digits
	\.		a literal dot	seperating the numbers
	[0-9]\{1,3\}	a number	at least one, at most three digits
	\)		"end group"	end of IP number
	.*		"everything"	this should match the rest of the line
	/		delimiter	separates input pattern from
						substitution pattern
	\1		first group	references the matched first group
					this should contain the IP number
	/		delimiter	separates input pattern from
	p		"print"		prints the output pattern (default)
Result of email with Adam Brothers abrothers@comfax.com [980121]

Mail Folder Weedout
Do you keep a mail log? Do you read all those "Received:" lines when looking at mails? No? Then you might as well do away with them! Here is the script:

# 950614, 950623, 951018, 951020 # Purpose: Weeds (deletes) unwanted header lines from "folders" # (text files containing emails) # Installation: Save this script as "weedout.sed". # Usage: sed -f weedout.sed folder > folder.weeded # :again /^Received:/{ N s/^.*\n// :blah /^[ ]/{ N s/^.*\n// b blah } b again } # # Comment the lines which you want to keep with a "#" # NOTE: Case of characters matters! # /^Approved:/d /^Content-.*:/d /^Distribution:/d /^Errors-to:/d /^Errors-To:/d /^Full-Name:/d /^In-Reply-To:/d /^Lines:/d /^Message-ID:/d /^MIME-Version:/d /^Message-Id:/d /^Mime-Version:/d /^NNTP-Posting-Host:/d /^Organisation:/d /^Organization:/d /^Path:/d /^Phone:/d /^Post:/d /^Precedence:/d /^Received:/d /^References:/d /^Return-Receipt-to:/d /^Return-Receipt-To:/d /^Reply-To:/d /^Resent:/d /^Return-Path:/d /^Sender:/d /^Sent:/d /^Status:/d /^Supercedes:/d /^Supersedes:/d /^Telephone:/d /^X-[a-zA-Z-]*:/d # /^X-Face:/d # /^X-Location:/d # /^X-Mailer:/d # /^X-Status:/d # /^X-Sun-Charset:/d # /^X-URL:/d # /^X400/d # End of script

ELM filter log weedout
This script changes the text that ELM's "filter" program writes to its logfile so that you will save some bytes. I got annoyed by the long strings "Mailing message to" and "Message saved in folder" and by the slashes in the date format, so I did away with them. Also, my home directory was always written in full length so I cut it down to the name of the directory that contains the inboxes (folders where incoming mail is filtered to).

Here it is:

s/^filter (\(.*\) guckes): Mailing message to/\1/ s/^filter (\(.*\) guckes): Message saved in folder/\1/ s/\/home\/emailer\/guckes\/NEW\/// s/.new$// s/0-// s/guckes/IN/ s/^\(..\)\/\(..\)\/\(..\)/\3\1\2/

LaTeX Umlauts to HTML umlauts
Ever wanted to convert a LaTeX text to HTML? Some of the available scripts do not convert the umlauts - so I had to find a quick solution to this problem.

Again, this is fairly simple:

s/\\"a/\ä/g s/\\"A/\Ä/g s/\\"o/\ö/g s/\\"O/\Ö/g s/\\"u/\ü/g s/\\"U/\Ü/g s/\\\{ss\}/\ß/g


SED - Other solutions

Here are some examples to problems where SED is not the right tool. There are other tools about which are designed for a purpose and you should use them, too!

Deleting newlines from a file

Use tr:
  tr -d \\012 < bar

Weeding mails with mawk

See Fefe's page about mawk:
http://www.math.fu-berlin.de/~leitner/mawk/

Links to other SED stuff

SED Documentation

The SEDer's grab bag [980611,980720,990831]
http://seders.icheme.org/ [defunct?!]
Pages with and without frames. Scripts, tutorials; guest book. and an archive of the seders mailing list.
Maintainer: Casper Boden-Cummins mister_pink@bigfoot.com
Well, Mister Pink is a bit busy right now, but you can send him email about his page, anyway. :-)

Custom SED [980527]
http://www.ptug.org/sed/custom_sed.html
"Custom SED" is a new kind of SED that shall "always capable of running existing sed scripts portably and efficiently. The changes should be portable to all contemporary Unixes and if the underlying GNU code permits, to the DOS/Windows environments."

Fefe's tutorial on SED
http://www.math.fu-berlin.de/~leitner/sed/tutorial.html
If you have never used sed and if you don't want to buy a book about it then you should read this short tutorial.
Author: Felix von Leitner leitner@math.fu-berlin.de
See also his page on sed:
http://www.math.fu-berlin.de/~leitner/sed/

sed(0)
http://nano.am.cs.vu.nl/man/Minix/1.7.4/man1/sed.1.html
Minix 1.7.4 sed manual (html) - ultra-short

Misc

Dictionaries
Dictionaries are useful for spell checking and other things. Here are some sites:
ftp://ftp.funet.fi/pub/doc/dictionaries/DEC-collection
ftp://ftp.funet.fi/pub/doc/dictionaries
ftp://ftp.fu-berlin.de//pub/unix/security/dictionaries/

sed-2.05
http://hpux.csc.liv.ac.uk/hppd/hpux/Gnu/sed-2.05/
Info about GNU sed for HP Unix.

SED Exercise
http://research.umbc.edu/~tarr/cs491c/fall94/sed/sed_ex.html
An sheet with exercises for SED (written for the course CMSC491 "Scripting Languages").

sed
http://www.che.uc.edu/~nauss/ch981/unix_for_beginners.html#sed
"UNIX for beginners" - it also mentions SED:
VII-7 *sed generating files with instructions for all files of a certain kind: for example, get all files named m24_cav*.o, put the string 'draw ' in front of each of them and put them in a file: \ls -c m24_cav*.o | sed -e s/"m"/"draw m"/g > odraw.mac. The resulting file can be executed by O immediately.
VII-8 *sed sed stream editor; for example: sed -e s/random/rannew/g conezd.com > new.com or (using a script file) sed -f sed.script test.f > new.f."

text2html.sed
http://info.cern.ch/hypertext/WWW/Tools/bin/txt2html.sed
TODO

sed
http://mathssun5.lancs.ac.uk:2080/~maa036/GNU/GNUWeb/sed.html
TODO

UNIX man pages : sed (1B)
http://melmac.corp.harris.com/cgi-bin/man-pages?sed+1B
TODO

sed106.zip
http://oak.oakland.edu/pub/simtelnet/gnu/gnuish/sed106.zip
TODO

sed
http://plan9.att.com/magic/man2html/1/sed
TODO

FrameFree in Sed!
http://reality.sgi.com/grafica/framefree/sed.html
TODO

sed --> Perl
http://rphc1.physik.uni-regensburg.de/psi/perl-4.036/man/manl/s2p.l.html
TODO

sed106.zip
http://simtel.coast.net/SimTel/gnu/SimTel/vendors/gnu/gnuish/sed106.zip
TODO

sed Version 2.05
http://src.doc.ic.ac.uk/gnu/sed-2.05.tar.gz
TODO

Man page of sed(1)
http://web.cnam.fr/Docs/man/OSF1-2.0/sed.1.html
TODO

u-sedit2.zip
http://wuarchive.wustl.edu/systems/ibmpc/garbo.uwasa.fi/editor/u-sedit2.zip
TODO

http://www-usa.partner.digital.com/www-swdev/pages/Home/TEC
TODO

sed.zip
http://www-usa.partner.digital.com/www-swdev/pages/Home/TECH/software/csw/sed/sed.zip
TODO

The Islamic Audio Studio at the Islamic Center of
TODO

???
http://www.ccs.ornl.gov/news/guide/xps_man/sed.1.html
TODO

sed
http://www.che.uc.edu/~nauss/ch981/unix_for_beginners.html#sed2
TODO

sunos sed.1v
TODO

Vector Design THE COMMON for Software : dos/gnu/sed
TODO

SED documentation
TODO

http://www.cs.hut.fi/~sti/manuals/sed.html
TODO

UNIX Programmer's Manual[1]
TODO

sed and awk
TODO

sunos sed.1v
TODO

Introduction to sed
TODO

By Dale Dougherty
TODO

sed & awk
TODO

sed - stream edito
TODO

How to make a global change to a UNIX file with sed editor
TODO

gnused.zip/
TODO

sed(1)
TODO


FTP Links - getting SED

TODO! check these links.

ftp://ftp.ibp.fr/pub/simtelnet/msdos/txtutl/sed15x.zip
sed15x.zip - contains the bare minimum from sed15.zip (see below)
executable, manual, and a readme.

ftp://ftp.ibp.fr/pub/simtelnet/msdos/txtutl/sed15.zip
sed15.zip - source for a DOS sed (from October 1991) (60K).
Requires installing by compiling.

SED not in GNU dist
ftp://prep.ai.mit.edu/pub/gnu/sed-3.0.README
A readme that explains why sed 3.0 has been withdrawn from the GNU distribution:
sed 3.0 has been withdrawn from distribution.  It has major revisions,
which mostly seem to be improvements; but it turns out to have bugs too
which cause trouble in some common cases.

Tom Lord won't be able to work fixing the bugs until May [1996?].
So in the mean time, we've decided to withdraw sed 3.0 from distribution
and make version 2.05 once again the recommended version.

GNU sed 2.05
ftp://prep.ai.mit.edu/pub/gnu/sed-2.05.tar.gz
GNU sed 2.05 - the last "working" official release.

...
ftp://garbo.uwasa.fi/pc/editor/u-sedit2.zip"
TODO

...
ftp://uiarchive.cso.uiuc.edu/pub/systems/pc/winsite/winnt/miscutil/psed109.zip
psed109.zip  B    62423  960101  sed binary for NT/PowerPC
TODO


SED - Mailing List aka "seders"

There is a mailing list for people interested in SED - it's name is "seders". The maintainer is Al Aab af137@freenet.toronto.on.ca. You send you mails to him and he will decide whether he will distribute it.


Links to other pages about SED

SED FAQ [980508,990826]
http://www.cornerstonemag.com/sed/sedfaq.html
http://www.dbnet.ece.ntua.gr/~george/sed/sedfaq.html
Looks pretty cool. (Still have to fully read it.)

SED One-Liners [990826]
http://www.cornerstonemag.com/sed/sed1line.txt

SEDers UK [961005,980720]
http://seders.icheme.org/


Send feedback on this page to
Sven Guckes guckes-sed@math.fu-berlin.de