Strong Coulomb repulsion and spin-orbit coupling are known to give rise to exotic physical phenomena in transition metal oxides. Here, we report magnetic and transport characteristics of (001) oriented epitaxial SrMnxIr1-xO3 thin films, having both 3d and 5d elements on the perovskite B sites. With the increase of Mn concentration, perpendicular magnetic anisotropy decreases gradually in accompany with the magnetic easy axis tilting away from the out-of-plane [001] direction. X-ray absorption spectroscopy reveals that Mn eg electrons preferentially occupy the "d" _(〖"3z" 〗^"2" "-" "r" ^"2" ) orbital, which produces the observed perpendicular magnetic anisotropy in the framework of spin-orbital coupling. A planar topological Hall effect appears in SrMnxIr1-xO3 films with x about 0.30 when the magnetic field is applied along the current, which is a result of the noncoplanar spin structure due to the competition among the perpendicular magnetic anisotropy, the magnetic exchange interaction and the Zeeman energy. These results provide an example to show the subtle balance among complex competitions in materials with both strong correlation and spin-orbit coupling.