Markov state models (MSMs) provide some of the simplest mathematical and physical descriptions of dynamical and thermodynamical properties of complex systems. However, typically, the large dimensionality of biological systems studied makes them prohibitively expensive to work in fully Markovian regimes. In this case, coarse graining can be introduced to capture the key dynamical processes—slow degrees of the system—and reduce the dimension of the problem. Here, we introduce several possible options for such Markovian coarse graining, including previously commonly used choices: the local equilibrium and the Hummer Szabo approaches. We prove that the coarse grained lower dimensional MSM satisfies a variational principle with respect to its slowest relaxation time scale. This provides an excellent framework for optimal coarse graining, as previously demonstrated. Here, we show that such optimal coarse graining to two or three states has a simple physical interpretation in terms of mean first passage times and fluxes between the coarse grained states. The results are verified numerically using both analytic test potentials and data from explicit solvent molecular dynamics simulations of pentalanine. This approach of optimizing and interpreting clustering protocols has broad applicability and can be used in time series analysis of large data.