The objectives of this study were to estimate the prevalence of, and factors associated with, anxiety in epilepsy. We conducted a cross-sectional analysis using data from the Neurological Disease and Depression Study. The prevalence of anxiety and associated factors were assessed using descriptive statistics and logistic regression. Of the total sample (n = 250 patients), nearly 40.0% of participants had anxiety according to the Hospital Anxiety and Depression Scale. The most prevalent symptom of anxiety was “worrying thoughts” (35.6%). After adjustment for age and sex, depression (odds ratio [OR] = 8.97, 95% confidence interval [CI] = 4.38–18.40), medication side effects (OR = 1.79, 95% CI = 1.04–3.05), smoking (OR = 4.35, 95% CI = 2.27–8.31), and illicit substance use (OR = 2.42, 95% CI = 1.18–4.96) were significantly associated with higher odds of anxiety, whereas higher education (OR = 0.47, 95% CI = 0.28–0.80) was associated with lower odds of anxiety. Furthermore, participants with anxiety reported more severe epilepsy, debilitating seizures, and overall lower quality of life. Evidence from our study reveals a high prevalence of anxiety in persons with epilepsy and that anxiety is associated with a variety of negative outcomes. These findings further emphasize the need for more studies to understand the impact of anxiety and its relationship with various sociodemographic and clinical factors.