Just to give my 2 pennies worth
If you add pae to your 32 bit installation:http://www.webupd8.org/2009/10/use-more ... buntu.html
You should have the facility to use more ram in your VMs
I installed 32 bit first on my test machine but could not be bothered re-doing it for 64. I had more memory than the system would use, so having had a similar problem when I was going through a Fedora phase I found the PAE method.
Seems to work fine and reduces the the issues I have with windows applications in VBox.
Hope that helps (also think its about 150/200 MB download/install)