Kazyn, Fedora's list of common bugs in their current beta includes this as a bug for the GTX 580, but it's the same kernel bug that produces the crash when it tries to load the open source (nouveau) Nvidia driver. Here's what they say:
It has been reported that a bug in the nouveau video driver causes systems with an NVIDIA GeForce GTX 580 video adapter to hang as soon as the X graphical environment starts up. This issue also affects installation, so as soon as X is started, installation hangs.
This issue can be worked around by disabling hardware acceleration. To do this, pass the kernel parameter nouveau.noaccel=1. You can append kernel parameters at boot time - of a live image, DVD or network install image, or an installed system - by hitting the Tab key when the system reaches the bootloader menu, and typing in the desired additional parameter.
You can make the workaround 'permanent' on an installed system either by editing it into the /boot/grub2/grub.cfg file, or by creating a file named /etc/modprobe.d/noaccel.conf containing a single line reading:
options nouveau noaccel=1
Of course, turning off acceleration permanently is not what we want to do. Fedora has patched their kernels and I hope I pick it up when I update the beta. I don't know if Ubuntu/Mint have updated their kernels, but it's worth asking.
I'm expecting a 550TI to arrive tomorrow and plan to use the Fedora beta on it, so I've been concerned about this, too.