Some things to try, in order of likelihood:

1) Reinstall Custom Edition
2) Update your video drivers (ATI here) (nVidia here) (If you're not sure, try downloading Everest. It should be able to tell you the name of your adapter and give you a link for a driver download)
3) Update DirectX9 (Latest version here)
4) Check your graphics card temperature, it should be less than 80 Celsius (SIW can tell you temperatures: download here)