TCP for the Uninitiated - Part I (Introduction and Background)
By: Erik Iverson
The number of users on the Internet has grown exponentially since its
inception. The Internet as we know it today had a rather humble beginning. It
was an experimental network invented by the Department Of Defense, and more
specifically, the Advanced Research Projects Agency (ARPA). At the time, there
were only four nodes (communication endpoints) connected to the Internet
(ARPANET). Although that sounds miniscule by today's standards, it was an
incredible accomplishment for the time. This all happened in the late sixties
and early seventies. Unfortunately, the way that the ARPANET communicated was
prone to crashing and therefore not very reliable. In the mid seventies, Vinton
Cerf and Robert Kahn suggested a new set of protocols. This was the birth of
TCP/IP and by the mid eighties it was fairly standard. It is still the protocol
of choice for the Internet and networks worldwide.
Not amazingly, few understand even the most elementary of concepts of how the
Internet functions. Most believe it is just a combination of Internet Explorer,
Netscape Navigator, and Microsoft Outlook. By reading this, you should gain an
understanding of some common terms you hear when discussing the Internet, the
big picture of how the Internet works, and even some details about how specific
applications function over the Internet. Lastly, and most importantly, I hope to
show you how you can start to learn things about computer networks and the
Internet on your own, and to provide you with links and programs that will help
you in that cause.
First, we should define the Internet. If you're reading this you probably
know what a computer network is. The Internet is the largest computer network,
composed of all the clients, servers, and routers that communicate with each
other across the globe.
You'll find that a lot of people confuse the terms Internet and World Wide Web
(WWW). The WWW, along with e-mail, are the two most popular services the
Internet offers to the common person. So when you hear "I am on the
Internet", the large majority of people are referring to the WWW. We shall
see that there are many more parts to the Internet than the WWW and e-mail; they
are just the ones that get all the glory in pop culture. The reason for this is
ease of use. These services offer a familiar and easy to learn point and click
interface which the common person is able to grasp.
So, since we are all familiar with the WWW, let us examine it to see how a
typical Internet service works. We will now define a new term, protocol. A
protocol is a set of rules that allow communication between hosts. By host I
mean either a server or a client. Think of it this way: there are many different
operating systems. Windows is the most popular, but there is also Linux,
Solaris, OpenBSD, and Macintosh to name a few. All these machines must be able
to "talk" to each other on the Internet. A Windows machine must be
able to connect to an OpenBSD machine and fetch a web page for instance. Or
vice-versa. This is where protocols come to the rescue. They standardize, or
attempt to, how machines communicate to each other on the Internet.
To put it another way, if I know only English, I can't communicate to someone
who is speaking Chinese. We need to speak the same language, and in the case of
the Internet, the "language" is called TCP/IP.
The Internet uses many layers of protocols to get the job done. The main
layers are the physical layer, link layer, network layer, transport layer, and
the application layer.
Let's start with the physical layer, or hardware layer. This is the
"lowest level" that deals with electronics and transfer of data across
lines. It transmits the "1's" and "0's" that you are
probably familiar with. For a decent look at this aspect, check out the "Voice
and Data Communications Handbook."
The link layer won't be touched on much here either. It puts data into units
called frames. The frames contain a header with certain information. On a LAN,
the frames header would contain the source and destination MAC addresses on the
network cards. Link layer protocols can carry any type of data across the link;
it doesn't matter what protocols the higher layers use.
The network layer has to do with routing information across the Internet. You
have probably heard the term "packet" thrown around fairly liberally.
When talking about the Internet Protocol (IP), the correct term to use would be
datagram. Routers are responsible to get datagrams where they should be. The
decision is, for the most part, based on the destination IP address of the
datagram. There are many different routing protocols one can choose from, each
with their own strengths and weaknesses. This is another topic, however. What
you should know is that the network layer uses IP, and is responsible for
routing packets from one place to another.
The next level is the transport layer, the meat of the discussion. Two main
protocols are available for this layer, the User Datagram Protocol (UDP) and the
Transmission Control Protocol (TCP). UDP is an unreliable protocol; it does not
guarantee message delivery. UDP gives its best effort of getting data somewhere.
The good part about UDP is the overhead on the network is lower. When most
people hear "unreliable", they think of negative things. UDP is not a
bad thing per se; it just doesn't do any checking to see if a particular piece
of data ever gets to a host. As it stands, applications should do their own
checking to see if a piece of data arrives using UDP.
TCP connections are reliable; if data gets lost or garbled, TCP will
retransmit it until it is received properly. We will see how this works in more
detail shortly. TCP makes sure data gets to the host in the right order, with
the correct message, and without duplicating the message.
The "highest" layer in computer networks is the application layer.
Different Internet services, or programs, have different protocols. Some
examples of application layer protocols are Telnet, File Transfer Protocol,
Internet Relay Chat, and E-mail. The WWW has its own protocol too; it is the
Hyper Text Transfer Protocol, better known as http. This is why you type http
before most web addresses. It lets the browser know what protocol to use.
As you are seeing, protocols play a vital role in the Internet functioning.
Let's take a look at some highlights of the Transmission Control Protocol, or
TCP makes sure that two computers have a reliable connection. With the
millions of packets moving across the Internet, things get lost sometimes. TCP
makes sure that when this happens, data gets to the intended host in an orderly,
reliable manner. How does this work? Well, we are not writing a TCP stack today
so we'll go over the basics and you can learn more from there. I want to hit the
main points without overwhelming you with details. When you understand the
concepts, the details are learned rather easily.
First, we'll tackle the issue of ports. You might have heard or ports before.
"Ftp to my port 21, telnet to my port 23, use port 6667 to get on
IRC," your Internet savvy friend says. Then again, perhaps you've never
heard of ports before. Ports are analogous to doors on a house. Machines can
have open ports and closed ports. Machines on the Internet can open up ports
numbered 0 to 65535, which for you binary number folks means a port is a 2-byte
integer. By and large, open ports will offer services to someone. For example, a
machine running a WWW server has port 80 open. Port 80 was chosen for the
standard WWW port. Unless you direct it otherwise, your browser will connect to
port 80 on www.altavista.com or port 80 on www.zdnet.com when you try and visit
these sites. Having a standard port helps so that everyone is on the same page.
But don't think that the WWW wouldn't work if they had chosen port 45, or 199.
It's just a number used so we, and programs, know how to find it. Below are some
common services and their port numbers:
21-File Transfer Protocol (FTP)
22-Secure Shell (SSH)
25-Simple Mail Transfer Protocol (SMTP)
80-World Wide Web (WWW)
110-Post Office Protocol (POP)
These are not set in stone. It would be one and the same to have Telnet
running on port 21, and FTP on port 23. The port numbers are there for
standardization; there is nothing magic about them. I often find people who
think there is something special about the numbers; there isn't! If you are
running a server of some type, more likely than not, you will be able to tell it
what port you want it running on through an options menu or file. More often
than not, though, you will want to leave it on the default port, so it is easily
So somebody thinks they're clever and wants to run a "hidden" web
server, one that only certain people can find. He gets a web server and
configures it to listen on port 75 instead of the common port 80. Has he hidden
the web server? Maybe from his little brother, but probably not even him! There
are tools widely available for every operating system called port scanners.
These programs will, among other things, search a target machine for every open
port and report back to the user what ports are open on that machine. If it is a
more advanced port scanner, it might also tell them what service is running on
that port, and even the operating system you are running. So if you want to set
up a "private" web server, use some sort of password authentication
method. Simply hiding the server on a different port is not going to help much.
Security through obscurity is a bad idea when dealing with computers; get it out
of your head as soon as possible!
So now you hopefully understand that servers will have ports open with
services running on them. One machine can have as many ports open as it feels
like, up to the maximum limit of 65535 of course. The point is that you don't
need separate machines for an FTP and a WWW server for instance; they can be
running on the same computer. You can't however, have two things running on the
same port on the same machine. This isn't entirely true, I suppose, if the
machine had two IP addresses, but by and large it is the case.
So let's see ports in action and learn a new concept, how connections are
established using TCP. There is no better tool to see TCP in action than a
network sniffer. There are a great many sniffers available for any operating
system. Since I am on a Windows machine right now, and it is a popular OS, we'll
use a sniffer made for Windows. The one we will be using for our demonstrations
is called ButtSniff, and was written by L0pht Heavy Industries (www.l0pht.com).
They released ButtSniff as a plugin to the popular Back Orifice remote
administration tool (www.cultdeadcow.com), but it is a great standalone
application too! Without further ado...
We are going to view my computer connecting to www.l0pht.com, port 80. We
will see what is called the 3-way handshake in action. We'll show you the output
and then describe it. Pay attention! This is where a lot of people get confused,
because I'll be talking about things like SYN and ACK flags. Don't get scared,
it's really an intuitive concept. The scientists who thought of TCP were no
Since TCP is supposed to be reliable, it needs a way of making sure data
arrives in the right order. Naturally, numbers are used to keep track of where
the data belongs. Think of each TCP segment (packet) as being labeled with a
number, called a "sequence number". Each segment also has an ACK
number, which is the segment number it is expecting from the other host. TCP is
a two-way protocol. There is an "out-going" data stream and an
"incoming" data stream. Confused? Let's look at this in action and I
guarantee it will become clear in time.
Buttsniff Dump of directing browser to www.l0pht.com
Source IP: 220.127.116.11 Target IP: 18.104.22.168
TCP Length: 0 Source Port: 3038 Target Port: 80 Seq: 08A236AC
Flags: S Window: 8192 TCP ChkSum: 43386 UrgPtr: 0
Source IP: 22.214.171.124 Target IP: 126.96.36.199
TCP Length: 0 Source Port: 80 Target Port: 3038 Seq: BE51108E
Flags: SA Window: 16384 TCP ChkSum: 54084 UrgPtr: 0
Source IP: 188.8.131.52 Target IP: 184.108.40.206
TCP Length: 0 Source Port: 3038 Target Port: 80 Seq: 08A236AD
Flags: A Window: 8192 TCP ChkSum: 1870 UrgPtr: 0
Step 1 of 3-way handshake
To the uninitiated, this mess might just confuse you more. Here comes the
explanation. Note the items in bold, as I will concentrate on them. The IP is
the 4-byte address that each computer on the Internet has. Each computer must
have a unique IP address if they wish to receive traffic on the Internet. So
you'll see here in the 3-way handshake demo, that there are two IP addresses.
Mine, which is currently a computer at the University of Minnesota - Twin
Cities, and www.l0pht.com, which I assume is in Boston. You'll see my machine
sends the first packet, which only makes sense. I am the one who wants to start
a connection. L0pht has no idea that I want to connect to their machine, until
their machine receives my first packet. It knows I want to initiate a connection
because the TCP packet has a special flag set, which shows up as "S"
on the sniffer output. "S" stands for the SYN, or synchronization
flag. It is the first step of the 3-way handshake.
Other things to note that are interesting about this packet are the source
port and the destination port, particularly the destination port, 80. It is 80
because we are trying to access the WWW service on www.l0pht.com. The source
port is picked by the operating system we are using, in this case Windows. It
will find a port that is unused on your machine and use that. So when you access
a web site, you're port 80 is not open. And if it is open, it probably
does not have anything with you having your web browser open. It would most
likely mean you are running a web server yourself. So could the source port by
80 by random chance? Most likely not! In almost all cases, the OS won't pick a
port below 1024; those are supposed to be reserved for things like services.
They are called the "well known ports."
So in this case our machine at Minnesota has IP address 220.127.116.11 and is
using port number 3038 for this connection. The L0pht's machine has IP address
18.104.22.168 and is using port number 80.
Step 2 of the 3-way handshake
Now L0pht knows we want to establish a connection with them. They have
received our packet with the SYN flag set, they respond by sending us a packet
with two flags set. The SYN (synchronization) and ACK (acknowledge) flags are
both set on the reply packet. This is the second part of the 3-way handshake.
More on the rest of the info in that packet in a moment.
Step 3 of the 3-way handshake
The third and final piece to the puzzle is our packet that we send to L0pht
with only the ACK flag set. So go over it in your head until you remember: SYN,
SYN/ACK, ACK - SYN, SYN/ACK, ACK. You've now got the order of the 3-way
handshake down, but what does it mean?
Remember how I said before that for TCP to be reliable it had to number the
data it sent? We're going to learn about that now. The two numbers in bold above
are the Sequence and Acknowledgement numbers, also known as SEQ and ACK. Notice
when we send the first packet of the 3-way handshake that there is a SEQ number
but not an ACK number. This sequence number, the one on the initial SYN packet
sent to establish the connection, is called the initial sequence number.
The TCP stack decides what the initial sequence number should be. Why not just
start with the number 1 you may ask? We will touch this in a future tutorial.
Now notice the SEQ and ACK numbers and how they correspond with each other. What
is an ACK number then, and why is it 0 in the initial sequence packet? A machine
sends an ACK to acknowledge that it has received a packet. The ACK number is one
greater than the last SEQ number received from the other host. The ACK number
can be thought of as the number that that particular end of the connection wants
to receive next. TCP is full duplex; data can travel both ways. There is an
outgoing data stream and an incoming one, too. So each end of the connection has
it's own Sequence number and keeps track of the sequence number it wants to
receive from the other end in the form of an ACK number.
So using our example, our initial sequence number is: 08A236AC. This
is a hex number, which uses the standard 0-9, as well as A-F to extend the
number system to base 16 (for example, 08A236AC + 1 = 08A236AD). Our initial
packet has no ACK number, since we don't know what the L0pht's initial sequence
number is (the ACK number shows up as 0, but this just means it does not really
have one). The L0pht's machine receives our SYN packet, with the initial
sequence number of 08A236AC. They reply with their own initial sequence
number (decided by the operating system, in this case BE51108E), and the
ACK number is our initial sequence number + 1 (08A236AD)! Notice the
addition of one since the ACK number is the number they want to receive next,
not the number they have already received. That is a vital concept. Notice the
L0pht's packet has both the SYN and ACK flags set, since it contains both the
initial sequence number and an ACK number, letting our machine know what to send
Our machine receives that packet and peeks at the ACK number, and sees that
it is 08A236AD. So our machine sends the L0pht a packet with sequence
number 08A236AD, and also has an ACK number. The ACK number is the
L0pht's initial sequence number + 1, or BE51108F. Notice that only the
ACK flag is seta the SYN is only used when giving an initial sequence number
(that is, when first connecting).
Hopefully you can see how this works now. It should be clear how TCP retains
a reliable connection. Let's take one end of the connection, ours. If the
L0pht's machine sent us a packet with a sequence number lower than what we are
expecting, we can discard it because it is old data. If it is higher than what
we were expecting, then we must have missed some data. When this happens, we do
not send an ACK for the new packet until we have received all the packets before
it. When the L0pht's machine realizes we have not acknowledged receiving some of
the data it sent (that is, when it times out), it will send it again. We receive
it this time, and continue data transmission. All this occurs at immense speed
and is transparent to the end user.
This wraps up section one. I don't want to lay too much information on you at
once. There is plenty to know about TCP and the Internet Protocols, and I myself
am learning more about it each day. Let me list the things that I hope you can
take away from this tutorial. Lastly, there are some references which have been
invaluable to me and that I highly recommend to further your knowledge.
- IP addresses
- 3-way handshake
- Layered protocols
- Function of Internet Protocol (IP)
- Simple differences between TCP and UDP
- Basic truths about TCP
- Basic concept of roles of Sequence (SEQ) and Acknowledgment (ACK) numbers
Attrition. Presentation on TCP/IP. http://www.attrition.org/news/content/tcpip_defcon.zip
Bates, Regis, et al. "Voice
and Data Communications Handbook." McGraw-Hill, 1998.
L0pht. Buttsniff. www.l0pht.com.
File in packetstorm.securify.com
Feit, Sidnie. "TCP/IP:
Architecture, Protocols, and Implementation with Ipv6 and IP Security."
Stevens, Richard. "TCP/IP
Illustrated Volume 1." Addison-Wesley, 1994.