Marriage has a respect from the government, not usually given to other relationships. Until two people file for divorce, they can do whatever they want between themselves and their children. But the minute someone files for divorce, it’s on. Yes, you can leave the state with your children even if your spouse disapproves. Keep in mind though that if you leave, you should certainly tell him where you are going, but also that if a divorce is filed within 6 months of you leaving Florida, you run the risk of having to relocate back to Florida. File the divorce first and in the Petition for Dissolution of marriage, request to relocate.