An extended transfer operator approach to identify separatrices in open flows.

Vortices of coherent fluid volume are considered to have a substantial impact on transport processes in turbulent media. Yet, due to their Lagrangian nature, detecting these structures is highly nontrivial. In this respect, transfer operator approaches have been proven to provide useful tools: Approximating a possibly time-dependent flow as a discrete Markov process in space and time, information about coherent structures is contained in the operator's eigenvectors, which is usually extracted by employing clustering methods. Here, we propose an extended approach that couples surrounding filaments using "mixing boundary conditions" and focuses on the separation of the inner coherent set and embedding outer flow. The approach refrains from using unsupervised machine learning techniques such as clustering and uses physical arguments by maximizing a coherence ratio instead. We show that this technique improves the reconstruction of separatrices in stationary open flows and succeeds in finding almost-invariant sets in periodically perturbed flows.