There's only one place to change the size in Totem, but I think this may be a bug. At least for earlier version that is the case.
You may actually have to tinker directly with the config file.
~/home/.gnome2/totem_config
You have to change this appropriately
# subtitle size
# { tiny small normal large very large huge }, default: 1
#subtitles.separate.subtitle_size:small
Uncomment to make it effective