I did some more snooping around and got the buttons to work again!
I had to enter this line in my config
Option "Device" "/dev/input/event3"
I got "mouse03" from the event handler via "cat /proc/bus/input/devices" in the terminal
I read somewhere that without specifying the device path the evdev driver can crap out.
Now to just remap a few buttons with xmodmap and btnx and I'm back and running again.