Man of Honour
This isn't a dilemma for me. I always choose to be happy.
If you don't think that the current job is going to get any better nor more flexible, then my suggestion would be to seriously think about the new job. Budget with it in mind, see where you can cut back on your outgoings, and maybe give that budget a trial run for a few months. Worst case you'll have a chunk of money that can go into savings, and it'll give you an idea of the things you can and can't live without.An opportunity may arise where I can take a job that is term time only, obviously the wage is severely reduced and we would struggle financially until both kids are at school and my wife upped her hours at work.
If you feel that money's going to be tight is there anything you can do to supplement your income? Do you have any skills that you can employ on a freelance basis from home?
This. Having lived in this area for 16 years I have always had an hours travel to work. Bizarrely though it has been from where I live to the the Northallerton area !
One thing I will say is that your career will likely span a long, long period of time, and various opportunities will arise during it. Therefore, taking what might seem a step backwards in the short term may open up new doors later down the line. Whereas, your kids will only be young once and you may feel more fulfilled spending quality time with them at this stage.
I think we're pretty close to each other, or have been... know that feeling.
This isn't a dilemma for me. I always choose to be happy.