I'm very careful in what I'd say to someone who isn't sure whether to have kids - it's a totally personal choice etc, but considering you're already on this journey I'll happily say the below.
Yes, there are plenty of downsides. And when you look at them in isolation, it sounds like total...