If your server for your xbox is linux only, I really do suggest going to ext3 over NTFS, being that ntfs is not as stable in linux, and ever so often disk-check is tripped off.
If you dual Boot and need access to media through windows, then I suggest to create your xbox media server in a virtual machine, so that File access would be done by windows itself. Virtual Box lets you share folders between environments, VMware-workstation allows fast share but costs mucho. You'll need to install their associated tools, and in the end gets compex.
If you want a convertable server that you boot off a live cd, or dual boot to a small partition, then I suggest you try to make your smb.conf configured using appropaate buffers and packet sizes, and timings, since they are set really conservative. (reasing being microsoft salted their code, and it still hasen't been reversed engineered as well as microsoft can do them self).