With a large number of possible attacks on commercial electronic devices reported, the security of hardware devices and systems has become an urgent problem over the past few decades. Accordingly, various solutions and countermeasures have also been explored to mitigate these attacks. Machine learning, as one of the fastest-growing research areas, also makes a unique impact on the landscape of vulnerabilities and countermeasures of hardware security. In this paper, we provide a survey of the double-edged sword impact of machine learning techniques on the security of hardware. We enumerate both the effective countermeasures and destructive attacks based on pure machine learning methods, as well as the integration of machine learning and other methods, such as side-channel analysis. Without loss of generality, we also discuss the security concerns associated with hardware devices that are used as carriers and accelerators for machine learning algorithm implementations. Specifically, we present the security issues of FPGAs in two different application scenarios: 1) as a standalone computing resource and 2) as a public-leased computing resource shared by multiple users.