If your cooling is good, then you should find the max voltage your memory is rated for and put it up to that, then go a bit more gradual on your timings, first try 5-5-4-15, if thats ok under 8 hours of Memtest86+, then try 5-4-4-15
overclocking is a time consuming and gradual process, normally...