I argue that there is tension in Wittgenstein's position on trivialities (i.e. tautologies and contradictions) in the Tractatus, as it contains the following claims: (A) sentences are pictures; (B) trivialties are not pictures; (C) trivialities are sentences. A and B follow from the "picture theory" of language which Wittgenstein proposes, while C contradicts it. I discuss a way to resolve this tension in light of Logicality, a hypothesis recently developed in linguistic research. Logicality states that trivialities are excluded by the grammar, and that under the right analysis sentences which look trivial are in fact contingent. The tools necessary to support Logicality, I submit, were not available to Wittgenstein at the time, which explains his commitment to C. I end the paper by commenting on some points of contact between analytic philosophy and the generative enterprise in linguistics which are brought into relief by the discussion.