Women at high risk of breast cancer face a difficult decision whether to take medications like tamoxifen to prevent a first breast cancer diagnosis. Decision aids (DAs) offer a promising method of helping them make this decision. But concern lingers that DAs might introduce cognitive biases.We recruited 663 women at high risk of breast cancer and presented them with a DA designed to experimentally test potential methods of identifying and reducing cognitive biases that could influence this decision, by varying specific aspects of the DA across participants in a factorial design.Participants were susceptible to a cognitive bias - an order effect - such that those who learned first about the risks of tamoxifen thought more favorably of the drug than women who learned first about the benefits. This order effect was eliminated among women who received additional information about competing health risks.We discovered that the order of risk/benefit information influenced women's perceptions of tamoxifen. This bias was eliminated by providing contextual information about competing health risks.We have demonstrated the feasibility of using factorial experimental designs to test whether DAs introduce cognitive biases, and whether specific elements of DAs can reduce such biases.