The average combat is meant to go about three rounds. At level 5 monk and level 3 ranger, casting hunter's mark with the bonus action gets us 2d6 on the first round and 4d6 for the next two each, assuming you have the ki for flurry and you're attacking the same target over those three rounds. That's 10d6 over 3 rounds, which equals 11.67 avg dpr + ranger subclass fun.

With 8 levels of monk and instead flurrying the first turn, you're getting 2 x (1d6 + mod). Assuming two asi has got us to max modifier, then that's 2d6 + 10 or 17 avg dpr. The monk die would go up further with monk levels also.

17 > 11.67. Therefore 8 levels of monk is the better bet.