Are you officer or enlisted? I know a few people who have and a few people who haven't been able to get reimbursement in graduate school while in the military. I can't speak to every contract, but you should see your retention NCO to see if you would be capable of balancing your personal finances and school costs if you stepped down from a full time job. I know when I reread my contract from when I was enlisted they covered masters courses but not doctorate programs, but then other options exist that involve commissioning in for a period of time after graduation (mostly in medical, but there may be other research opportunities).

Further, check what expectations your potential university programs have with regards to tuition reimbursement and waivers based on teaching and research. If you can afford it and it will make you more over the course of your career, do it. Otherwise you could see about doing a master's degree through night classes, and whether or not that is an option from your school.