I was on the right tracks when I was thinking network card but instead of the server it turns out its the WiFi card in my laptop. I was running an Intel 4965AGN Wifi card. I replaced it back with the origional Intel BG WiFi card and Wireless has now been stable at about 2MB/s up/down for a couple of days.
It reminded me that Ubuntu had the same issue when I first started using that, the problems were very simular including stalling in transmission, intimitent breaks in transmission and low transmission speeds. I'm assuming Ubuntu has a different kernel which has ironed out these problems, Debian is yet to fully support the intel 4965 WiFi card.
As a test I swapped them back over, making sure all 3 aerials were connected correctly and the problem reared its ugly head again before going back to BG.