ExFAT support is not natively installed in Ubuntu. I found the link below with instructions that worked:
http://apcmag.com/how-to-enable-exfat-in-ubuntu.htm
I used it to install exfat support on Ubuntu 12.04 LTS (32-bit).
I used the following commands through the Terminal:
sudo -s
apt-add-repository ppa:relan/exfat
apt-get install fuse-exfat
Before doing apt-get I had to do:
apt-get update
After that I was able to plug in my USB drive and Ubuntu recognized it.