Blog posts for 2005

Added a page to sign up for an account at brianbondy.com. I plan to offer several services which will be announced over time, but they will need to store information, and therefore you wll need to create an account. You can create an account here

Added a page to the Other section that allows you to get your external IP address. You can access that page here.

I archived the old front page. You can access that page here

The official FTP standard is defined in RFC 959 [Postel and Reynolds 1985]. This article is in no way complete nor should it be used as an implementation guide alone.

FTP uses two TCP connections to transfer files:*

1. The control connection

The control connection is made by the server listening to the default port for FTP which is 21. This port can differ, but the default is 21. The FTP server waits for a connection from the FTP client. The client does an active open from any port to port 21. The control connection once established, remains open for the duration of the communication between client and server. The control connection is used to specify what to do.

1. The data connection

Once the control connection is used to specify that it would like to transfer a file, a data connection is established to actually transfer the file. The data connection is used for sending files, receiving files, and getting a list of files in the current directory.

FTP requests:

All FTP requests are 3 or 4 digit ASCII commands. Each command can have 0, 1, or more optional arguments. The most popular FTP commands are as follows:

• ABOR - abort a file transfer
• ACCT - send account information
• APPE - append to a remote file
• CDUP - CWD to the parent of the current directory
• CWD - change working directory
• HELP - return help on using the server
• DELE - delete a remote file
• LIST - list remote files
• MDTM - return the modification time of a file
• MKD - make a remote directory
• MODE - set transfer mode
• NLST - name list of remote directory
• NOOP - do nothing
• PASS - send password
• PASV - enter passive mode
• PORT - open a data port
• PWD - print working directory
• QUIT - terminate the connection
• RETR - retrieve a remote file
• REIN - reinitialize the connection
• RMD - remove a remote directory
• RNFR - rename from
• RNTO - rename to
• STAT - return server status
• SITE - site-specific commands -SIZE - return the size of a file
• STOR - store a file on the remote host
• STOU - store a file uniquely
• STRU - set file transfer structure
• SYST - return system type
• TYPE - set transfer type -USER - send username

FTP replies:

All FTP replies start with 3 digit status code in ASCII. An optional message can follow this 3 digit status code.

The first (left most) digit in the error code has special meaning.

• 1xx: Used for positive preliminary replies.
• 2xx: Used for positive completion replies.
• 3xx: Used for positive intermediate replies. Which means everything was OK, but another command is expected.
• 4xx: Transient negative completion reply.
• 5xx: Permanent negative completion reply.

The second (middle) digit in the error codes also have special meaning.

• x0x: Syntax error
• x1x: Information
• x2x: Connection related
• x3x: Authentication related
• x4x: Unspecified
• x5x: Filesystem status

The third (right most) digit in the error code, is just used to be more specific.

Most FTP replies are 1 line replies, and can be read up to the first CR LF. However, some may be multi-line replies and you should therefore check to see if the 4th digit is a hyphen. If the 4th digit is a hypen, then you should keep reading lines until the 4th digit is not a hypen. All lines in a multiline reply will start with the same 3 digit status code. The final line in a multi line reply will not have a hyphen.

Using telnet to communicate with FTP:

As with many other ASCII protocols, you can communicate with an FTP server via telnet alone. To do this in windows, simply go to start menu | Run | command.com In the prompt type:

> telnet myservername.com 21
220 Microsoft FTP service

> USER brian
331 Password required for user brian

> PASS mypassword
230 User brian logged in.

> SYST
215 Windows_NT

You won't be able to easily test your data connection this way, but you can test out many of the FTP commands and see the replies that they give.

Debugging FTP:

As with every other protocol, when implementing or learning FTP, it is a good idea to use ethereal. You can download Ethereal/Wireshark. If you are using ethereal 0.10.9 or above, you will not need to download winPCAP.

I just purchased a new 64-Bit AMD computer and installed Windows 64-Bit AMD for the first time.

Here’s a short rundown on what I found out during my first night of using it.

There are 2 main types of 64-bit processors, IA64 and x64, and they run different versions of windows 64-bit.
That means that every driver built for Windows 64-Bit needs to be compiled twice.

The IA64 processor was called the Itanium. It was a huge flop because of problems with the way they did the caching. It was replaced with the Itanium 2, which has a better design for cache. Itanium 2 does not mean dual processors, but instead it means the 2nd design of Itanium (version 2). It is referred to the Itanium 2 CPU. It is now referred to as IA64 only.

With x64 you can install both Windows 32 bit or Windows 64 bit. With IA64 you can only install the IA64 Windows. You can run 32-bit apps on all variants. For the 64-bit Windows versions you can run the applications through an emulator that works transparently called Windows 32 on Windows 64 (WOW64)

The same does not hold for 32-bit drivers though. WOW64 is only for user mode applications.

On 64-bit machines there are actually 2 different registries. A 32-bit registry and a 64-bit registry. They are treated as different registries by 32-bit and 64-bit applications but the 32-bit registry is just a subkey for the 64-bit registry. Windows will automatically route an application to the proper registry depending on if it’s 32-bit or 64-bit.

Because of WOW64, any application that was compiled before on a 32-bit machine, will still work in 64-bit Windows. This isn't because it supports it direclty, but because windows is using WOW64 to translate the 32-bit calls to 64-bit ones. If however a program depends on a driver, or if it integrates with the windows shell, then that program will no longer work. I believe IPC is also translated on the fly by WOW64.

Some other useful information.

These 2 directories are both for 64-bit applications/drivers only. Pretty great naming! :), but I'm sure it will save many headaches for inf file authors.

• C:\Windows\System32\
• C:\Program Files\

These 2 directories are for 32-bit apps and drivers only, they use 32-bit emulation

• C:\Windows\SysWOW64\
• C:\Program Files (x86)\

So developing for 64-bit machines can get a little tricky if you actually want it to run as a 64-bit program. If it's running as a 32-bit program, then your ints will still be 4 bytes, and everything else you hold true will also still hold.

Right now, when I want to make a 64-bit program, I use a makefile instead of using Visual Studio directly. You can use different NSIS .exes to generate the different installers. I suggest to build each project with 3 different versions of NSIS. Then if you want to, you can build a 32-bit installer wrapper that combines each of your installers.

Vectors, strings, and maps are all part of the C++ standard template library (STL). You can use them in any C++ program. It doesn't matter what operating system you're programming on, if it has C++ support, then it has STL support.

Before using this tutorial you should have a good grasp at C++ templates, as well as everyting that you should know before templates. A great tutorial for beginners is at cplusplus.com

To use STL you need the following includes:

//If you want to use maps
#include <map>

//If you want to use vectors
#include <vector>

//If you want to use strings
#include <string>

And you will need to always prefix string, vector and map with std:: unless you put this:

using namespace std;

Here is an example on how to use STL strings:

string s = “hi”;//s holds "hi"
s += “ Brian”;//s now holds “hi Brian”
int iSize = s.size();//iSize holds 8
char *p = s.c_str();//p holds a pointer to an array of characters with a 0 at the end.

All character arrays, when holding strings, have a 0 at the end, this is very important. They don’t have the character “0” at the end, but they have an ascii value of 0 at the end. This zero is used to indicate the end of the char array. When you call the string's c_str() method, it will append a 0 for you at the end of the char array that it returns. You could have also used s.data(), which means the exact same thing as s.c_str().

Many people don't know this, but strings can also store binary data. By binary data I just mean any data that is non text. I.e. there may be zero's intermixed througout the string.

So for example:

string s;
//The size of pBinaryData is uiLen1; copy a new string object into s that holds the binary data from pBinaryData
s = string(pBinaryData, uiLen1);
s.append(pMoreBinaryData, uiLen2);
assert(s.size() == uiLen1 + uiLen2);

In the example above s.c_str() will now hold the binary data from both calls. You no longer have to worry about forgetting to delete binary data that you allocate. If you created pBinaryData and pMoreBinaryData on the heap, you can delete[] them after the call to append. Anything that you put in a string will have it's own memory, you don't need to worry about freeing it.

Here is an example on how to use STL vectors:

You use a vector to store a list of values. A value can be a number, string, or any other class object that you create. This list can grow to any number of elements, unlike an array.

vector<string> v;
v.push_back(“hi”);
v.push_back(“hi2”);
v.push_back(“hi3”);
assert(v.size() == 3);

for(int a = 0; a < v.size(); ++a)
{
  string s = v[a];
  s += “Brian”;
}

//The above "for loop" is pointless obviously :).  It will go through each element in the vector, and copy the vector element's data to a variable "s"
// When I go s+= "Brian" it won’t modify the vector contents in any way.  Because when you access each elemetn v[a], it will create a copy of the string.
// After the for loop, the vector is left unchanged.

Notice how I wrote vector, that means a vector that holds strings. You could have also wrote vector, or vector.

Vector of your own class objects:

class Cat
{
public:
string str1;
  string str2;
  string str3;
  int x;
};

vector<Cat> vMyCats;

Cat minou;

minou.str1 = “hi”;
minou.str2 = “hi2”;
minou.str3 = “hi3”;
minou.x = 5;
vMyCats.push_back(minou);
assert(vMyCats.size() == 1);
//vMyCats holds a vector of Cat objects.  Each cat has 3 strings and 1 integer.

Here is an example on how to use STL maps:

You use a map to store a lookup table. i.e. to associate something with something else.

map<string, string> myMap;
myMap.insert(pair<string,string>(“firstName”, “brian”);
myMap.insert(pair<string,string>(“lastName”, “bondy”);
myMap.insert(pair<string,string>(“age”, “24”);
string strFirstName = myMap[“firstName”];
assert(strFirstName == "brian");

You can use a map with other things then a string: map myMap;

FROM can be any type, for example int, Cat, string,
TO can be any type, for example int, Cat, string

The type you use for the FROM you use in the brackets to do lookups
The type you use in the TO is the value that’s associated with it.

Here's a quick tutorial on how email works, SMTP, POP3, IMAP, Webmail, ...

What is a Standard?

A standard is a set of rules that are followed by all developers around the world. Some standards include HTTP, SMTP, POP3, … There is official documentation that describes each individual standard and most standards have been around for 0 to 30 years.
Each standard document is a very detailed explanation of what the standard is and how it works. Typically a standard has an RFC number associated with it, but there are many different types of standards.

Protocols

SMTP and POP3 are ‘standards’. Each standard describes a different protocol. A protocol is any kind of communication between 2 or more computers.

What is SMTP?

SMTP is the ‘standards’ protocol that is used to send email. Your computer uses SMTP to send email. See RFC 821, August 1982

What is POP3?

POP3 is the ‘standards’ protocol that is used to receive email. Your computer uses POP3 to receive email. POP3 is also referred to as simply POP. See RFC 1939, May 1996.

POP3 typically will connect to the mail server and download messages to your computer. It can then optionally delete the message from the server (which it is usually setup to do).

How Email works

1. User A wants to send an email to user B.
   
2. User A writes up an email and presses send.
   
3. User A’s computer, uses SMTP communication to send the email to User A’s (Yes A, not B) SMTP server.
   
4. User A’s SMTP server, sends the email to user B’s SMTP server using SMTP communication.
   
5. User B when he feels like it, contacts his SMTP server and uses POP3 to download the messages.

Some important notes:

The only way to send email is to use SMTP. (Actually you can also use MAPI and some other things but let's not get into that)

The only way to receive email is to use POP3. (Actually there is also IMAPv4, but we'll pretend that POP3 is the only way)

How Email Applications work:

SMTP communication is present on your computer, no matter what email client you use. Any time an email is sent out, your computer uses SMTP to send the email. It doesn’t matter if you're using Eudora, Outlook, Outlook Express, Mozilla Thunderbird, or a custom made program. All programs use SMTP to send emails.

By using standards you are guaranteed that, even know user A uses Outlook, and user B uses Eudora, and they both have different SMTP servers both of the users will be able to communicate.

What is HTTP?

Before I can get to what web mail is, you first need to know what HTTP is. HTTP is just another standard protocol. But HTTP is meant to download files and web pages, unlike SMTP which is meant to send emails. See HTTP 1.1 RFC 2616, June 1999.

What is web mail?

Web mail is an online web page that allows you to send and receive emails using HTTP.

But wait a minute, didn’t I just say that the ONLY way to send email was using SMTP?

Yes! What the web page does, is provide you with a form that you fill out. Your computer doesn’t know that it is any different from a form that you fill out to enter your credit card information, or a form that you fill out to enter your home address, or a form that you fill out to sign into another web site. All your computer knows is that you are filling out a form.

When you press the send button, your web browser sends the form to the server. The server knows that this form is for email though. So the server interprets the form and extracts the needed information. The HTTP server then uses SMTP to send the message. Because the only way that a message is going to get from User A to User B is using SMTP.

What the web browser has done is fooled you into thinking that you are sending an email. But what’s really happening, is that your web browser is filling out a form, and then the web server is using SMTP to send your email.

Can you give me a web mail walk through ?

1. User A wants to send an email to User B, User A is going to use web mail.
2. User A uses his browser to type in an internet address (for example: www.hotmail.com).
3. User A’s computer uses HTTP to contact the server and ask for the web page that is used for web mail in this case.
4. The server responds (using HTTP) to User A’s computer with a web page that gives him options to compose mail, check mail, …
5. User A clicks on the compose a message link. Again User A’s computer uses HTTP to contact the server.
6. The server responds (using HTTP) to User A’s computer with the web page (which contains a form) that allows User A to compose a message.
7. User A fills in the web page and presses send. The page is sent back to the server using HTTP.
8. In the background, unknown to User A, the web server uses SMTP to send the email to User B. Why? Because the only way to send an email is to use SMTP
9. The server responds (using HTTP) to User A’s computer with a web page that says the email was sent.

How does the web server use SMTP?

Since SMTP is a standard protocol it uses SMTP in the same way any program would use SMTP. See the section ‘How email works’.

What is IMAPv4?

I mentioned IMAPv4 earlier. IMAPv4 is a second method used by email clients to retrieve your emails. IMAPv4 is also referred to as more simply IMAP. IMAPv4 is more complex than POP3, but gives you the ability to work on your email from multiple computers. If you use more than one computer, and you'd like to access your email from both computers, IMAP is the way to go.

IMAP stores all of its data on the mail server. In that way each mail client from each different computer can be in sync. When you read an email from one computer, your work computer will also see that the message is read. Since data is stored on the server, IMAP email accounts are typically more expensive.