Instead of compiling...
If using a Debian stable or LMDE (testing) you can activate the SID (unstable) repo and grab the Xfce 4.10, then deactivate (or deleate) those repos so as not get other updates things you don't want.
P.S. I did with a straight LMDE awile back. I since uninstalled LMDE so don't know if caused promblem down road!