With exceptional quantum confinement, 2D monolayer semiconductors support a strong excitonic effect, making them an ideal platform for exploring light-matter interactions and as building blocks for novel optoelectronic devices. Different from the well-known in-plane excitons in transition metal dichalcogenides (TMD), the out-of-plane excitons in indium selenide (InSe) usually show weak emission, which limits their applications as light sources. Here, by embedding InSe in an anisotropic gap plasmon nanocavity, we have realized plasmon-enhanced linearly polarized photoluminescence with an anisotropic ratio up to ∼140, corresponding to degree of polarization (DoP) of ∼98.6%. Such polarization selectivity, originating from the polarization-dependent plasmonic enhancement supported by the "nanowire-on-mirror" nanocavity, can be well tuned by the InSe thickness. Moreover, we have also realized an InSe-based light-emitting diode with polarized electroluminescence. Our research highlights the role of excitonic dipole orientation in designing nanophotonic devices and paves the way for developing InSe-based optoelectronic devices with polarization control.