If I were you, I wouldn't use a height map.

If you insist, then to make them look better you just need to put in a little effort to move the vertices out of that grid pattern or better yet simplify the cliffs along their length so they're comprised of less vertical sections, then add detail.