The cell membranes of different cells deviate significantly in lipid compositions and thus provide varying biological environments to modulate the diffusion, organization and the resultant function of biomacromolecules. However, the detailed modulation mechanism remains elusive especially in consideration of the current overuse of the simplified membrane models such as the pure phosphatidylcholine (PC) membrane. In this work, with the typical membrane-active peptide melittin, we demonstrated that a more complicated membrane environment, such as the bacterial (IME) or plasma membrane (PM), would significantly change the organization and dynamics of melittin, by using molecular dynamics simulations as a "computational microscope". It was found that in these membrane systems, adding melittin would cause a varying degree of reduction in the lateral diffusion of lipids due to the different assembly states of peptides. Melittin tended to aggregate to oligomers in the pure PC membrane, mostly as a tetramer or trimer, while in IME or PM, its degree of oligomerization was significantly reduced. More surprisingly, melittin displayed a strong affinity with ganglioside GM3 in PM, leading to the formation of melittin-GM3 nanoclusters, which hindered its diffusion and further oligomerization. Additionally, small changes in the residue sequence of melittin could modulate the degree or structure of the peptide oligomer. Our work provides a typical example of a study on the organization and dynamics of pore-forming peptides in specific membrane environments and has great significance on the optimization of peptide sequences and the design of helix bundles in the membrane for target biological function.