The main objective of the current study was to investigate the effect of time of day on visual and auditory, short-term memory (STM), and long-term memory (LTM) distortions using a hybrid Deese-Roediger-McDermott procedure. In Experiment 1, we used semantically related words, whereas in Experiment 2 - words were characterized by phonological similarity. The results showed a relationship between modality and types of stimuli. In STM, more semantic errors were found in the evening for items presented visually and more errors following auditory presentation for phonologically similar words. In LTM, the analysis revealed a higher rate of semantic distortions in the evening hours for auditorily presented words. For words with phonological similarity, we observed more errors in the evening without the effect of modality. The results support the hypothesis that more reliance is placed on elaborative processing in the evening and more on maintenance processing in the morning; however, this is not modality invariant.