In the twilight zone of protein sequence homology: do protein language models learn protein structure?

We believe this opens the way for further research both on remote homology prediction and on the broader goal of learning sequence- and structure-rich representations of protein molecules. All code, data, and models are made publicly available.