First of all, do not statically set or get rid of your swap. Doing so is more or less saying that you know more about what is going on in the internals of your operating system than Microsoft does.
One thing to keep in mind is that all of your graphics cards, all pci devices, etc that have addressable memory on them are taking away from the maximum amount of memory in your system. After you subtract that memory from your physically addressable memory, you will have the portion that your OS and Applications share.
If you need to extend the amount of RAM your apps have access to, there is a switch that can be set in boot.ini, but this can constrain the amount of memory windows has to work with.
Tim Slattery did a good article on windows ram usage.
There is a great explanation of the page file's importance and usage in response to this serverfault question.
To gain more usable memory your best bet is to upgrade to a 64-bit OS which will be able to make use of a full 3.5+ gigs of RAM.
Even though no single process has high memory usage, there are still lots of processes running -- that all adds up.
Using ps and other command line tools, you can drill deeper into this.
First, use options to ps to limit what is shown to just resident memory and the name of the command. -m sorts output by memory usage, -a shows all users' processes, -x shows processes not associated with a terminal (i.e. most mac apps).
$ ps -axm -o "rss,comm"
Here's the first few lines I see:
RSS COMM
210256 /Applications/Firefox.app/Contents/MacOS/firefox-bin
158276 /Applications/Google Chrome.app/Contents/MacOS/Google Chrome
155360 /Applications/iTunes.app/Contents/MacOS/iTunes
It's no shock to me that these are at the top (and the Chrome process shown is just one of a bunch). The RSS column is expressed as 1024 byte blocks. So iTunes is using about 151.7 MB.
That output is formatted well enough that you could throw it into a spreadsheet to analysis if you like analyzing things that way. I'll stick to the command line a little longer
String together a bunch more to see the sum of all processes:
$ ps -axm -o "rss,comm" | awk 'BEGIN { s=0;}; {s=s+$1;}; END { printf("%.2f GB\n", (s/1024.0/1024));}'
The output I see for that is 2.44 GB, not exactly what I see for Active memory in Activity Monitor, but close enough that I can use ps to get to the bottom of this.
You can add together just those processes using more than 100MB:
$ ps -axm -o "rss,comm" | awk 'BEGIN { c=0;s=0;}; ($1 > 100000) {c=c+1;s=s+$1;}; END { printf("%.2f GB from %d processes\n", (s/1024.0/1024),c);}'
0.98 GB from 8 processes
I haven't told you exactly what is using your mac's memory, but these tools will help you discover.
Best Answer
It is a good thing that you never get 4GB RAM. It would be very bad for your system if all your applications used all your real memory. There must be some free memory at all times for the system to run smoothly and virtual memory paging to work properly. I would be worried if your free memory is under 50 MB.
Apple has a support document explaining memory usage shown in Activity Monitor.
I like iStat Menus for giving me a quick snapshot of my memory/CPU/Network usage.
You can see what processes are using a lot of memory with Activity Monitor and quit processes there.
Unless your Page outs value under the System Memory tab is very high compared to the Page ins value, or you get excessive disk thrashing, I wouldn't worry about it. I think the best way to limit memory usage, given the set of applications you are using, is simply to keep the number of tabs/windows you have open in Firefox to a minimum. 4 GB should be more than enough to run that set of applications in Mac OS X.
Frankly, in my opinion there's not much you can do besides limiting the number of applications/startup items you launch.
The most important thing is ensuring you have enough disk space for Virtual Memory paging. The X Lab has an excellent article explaining memory usage and how to determine if you have sufficient memory.