A firewall controls access in and out of your computer so that only the programs that you identify will be able to access the internet (this blocks trojans from contacting their writer) and blocks all incoming accesses except those in response to outgoing requests.
A good firewall hides the existence of your computer when you are connected to the internet by refusing to respond to any incoming requests that are not responses to outgoing requests from authorised programs on your computer and which are not on its list of programs where the communication is allowed to be initiated from the other end.
Firewall software can vary in what it grants access to automatically and what it asks you about. Generally a firewall that grants access automatically is more likely to be effective for the average computer user because it is working that access off of a list of programs supplied to it by the people who wrote the firewall and they specialise in determining what software should legitimately be able to access the internet. With that sort of firewall you should only get asked about whether new software you just installed should have access if it is something not typically installed on most computers.
Less useful are firewalls that don't try to auto configure access and which ask every time. The person will see so many popups about whether to grant access to a program that they may just start answering 'yes' all the time which defeats the purpose of having a firewall.
Another aspect to firewalls is how they store the information as to what does and doesn't have access. This data needs to be encrypted so as to prevent viruses and trojans from being able to directly update the firewall controls to give themselves access.
There are two issues with the firewall built into Windows XP that make it rather useless. First it is only half a firewall in that it only blocks incoming requests that are not responses to requests sent and does not prevent anything on the computer from sending to the internet (and so once a trojan gets in it doesn't prevent it from communicating). Second, it stores its information in plain text in the registry. So once a trojan gets in it can update the registry to add itself to the list of programs stored there so that even if you find and delete it that it can get in again without the firewall blocking it. More recent versions of Windows implement a more effective firewall.
This article written by Stephen Chapman, Felgall Pty Ltd.