The recent expansion of digital financial products leads to severe consumer protection issues such as fraud and scams. As these potentially decrease trust in digital services, especially in developing countries, avoiding victimization has become an important policy objective. In an online experiment, we first investigate how well individuals in Kenya identify phone scams using a novel measure of scam identification ability. We then test the effectiveness of scam education, a commonly used approach by banks and institutions for fraud and scam prevention. We find that common tips on how to spot scams do not significantly improve individuals' scam identification ability, i.e., the distinction of scams from genuine messages. This null effect is driven by an increase in correctly identified scams and a decrease in correctly identified genuine messages. We interpret this as an increase in caution. In addition, we find suggestive evidence that genuine messages which contain scam-like features are more likely to be misclassified, highlighting the importance of a careful design of official communication.