Comment 7 for bug 1250277

I had the same problem which I think was caused by xorg defaulting to Software Rendering(due to faulty graphics driver) , hence high CPU Usage.

I solved my problem by installing intel graphics driver first and then reinstalling xserver-xorg

My Solution:

Step 1:
 Install latest intel grphics driver
 goto https://01.org/linuxgraphics/downloads/2013/intelr-graphics-installer-1.0.3-linux

Step 2: Reinstall xserver-xorg
sudo apt-get remove --purge xserver-xorg
sudo apt-get install xserver-xorg
sudo dpkg-reconfigure xserver-xorg