Bisphenol A is a chemical used to make certain types of plastics and is found in numerous consumer products. Because scientific studies have raised concerns about Bisphenol A's potential impact on human health, it has been removed from some (but not all) products. What many consumers do not know, however, is that Bisphenol A is often replaced with other, less-studied chemicals whose health implications are virtually unknown. This type of situation is known as a potential 'regrettable substitution', because the substitute material might actually be worse than the material that it replaces. Regrettable substitutions are a common concern among policymakers, and they are a real-world manifestation of the tension that can exist between the desire to avoid risk (known possible consequences that might or might not occur) and ambiguity (second-order uncertainty), which is itself aversive. In this article we examine how people make such trade-offs using the example of Bisphenol A. Using data from Study 1, we show that people have inconsistent preferences toward these alternatives and that choice is largely determined by irrelevant contextual factors such as the order in which the alternatives are evaluated. Using data from Study 2 we further demonstrate that when people are informed of the presence of substitute chemicals, labeling the alternative product as 'free' of Bisphenol A causes them to be significantly more likely to choose the alternative despite its ambiguity. We discuss the relevance of these findings for extant psychological theories as well as their implications for risk, policy and health communication.