Functional MRI is a powerful tool, but like most powerful tools, it works best when operated with care and consideration. In this talk, I selectively review a number of methodological and statistical issues that are routinely overlooked in neuroimaging studies, yet threaten the validity of many common inferences. These include concerns about measurement error, construct validity, statistical confounding, causal attribution, and generalizability of results. Drawing on both contemporary examples from neuroimaging and decades of domain-general psychometric research, I demonstrate how researchers who ignore such concerns run a substantial risk of getting major conclusions wrong--or, worse, not even wrong. For principled reasons, I do not, however, discuss any solutions to these problems.