Randomized controlled trials (RCTs) have traditionally been considered the gold standard for medical evidence. However, in light of emerging methodologies in data science, many experts question the role of RCTs. Within this context, experts in the USA and Canada came together to debate whether the primacy of RCTs as the gold standard for medical evidence, still holds in light of recent methodological advances in data science and in the era of big data. The purpose of this manuscript, aims to raise awareness of the pros and cons of RCTs and observational studies in order to help guide clinicians, researchers, students, and decision-makers in making informed decisions on the quality of medical evidence to support their work. In particular, new and underappreciated advantages and disadvantages of both designs are contrasted. Innovations taking place in both of these research methodologies, which can blur the lines between the two, are also discussed. Finally, practical guidance for clinicians and future directions in assessing the quality of evidence is offered.