I usually go for one up from easy, depending on the options. Also depending on the game, if it is an RPG where you are supposedly a great warrior or whatever, I find it just silly to play it on hard where you struggle to kill a dog.
A game like Fallout 3 though it kind of fits in with the whole story etc to have it as a challenge. Skyrim- You are Dragonborn, you should be able to deal with crabs easily enough!
I play games for fun though, I know some people like to grind games or whatever, but I don't see the appeal myself.
Something like Fifa I always play on a pretty easy difficulty, there is no way that jody craddock can outrun Theo Walcott, it makes no sense