Background: Within China's hierarchical medical system, many patients seek medical care in different hospitals independently without integrated management. As a result, multi-hospital visiting is associated with fragmented service utilization and increased incidence of polypharmacy behaviors, especially for patients with chronic disease. It has been confirmed that factors from the perspective of patients may cause polypharmacy behaviors in Chinese community patients; whether having a usual primary care provider for chronic disease patients could reduce the polypharmacy behaviors and the effect size remains unanswered, and that is what our study aimed to answer. Methods: Our study adopted a cluster sampling method to select 1,196 patients with hypertension or diabetes and measured some information about them. The propensity score weighting method was adopted to eliminate the influence of confounding bias, and then a multivariate logistic regression model was conducted to test the relationship between having a usual primary care provider and polypharmacy behaviors. Results: Patients without usual primary care providers were significantly correlated with polypharmacy behaviors (OR = 2.40, 95%CI: 1.74-3.32, p < 0.001), and the corresponding marginal effect is 0.09 (95%CI: 0.06-0.12). Patients who suffer from two kinds of diseases (OR = 3.05, 95%CI: 1.87-5.10, p < 0.001), with more than three kinds of diseases (OR = 21.03, 95%CI: 12.83-35.65, p < 0.001), with disease history of 20 years and above (OR = 1.66, 95%CI: 1.14-2.42, p = 0.008), who communicate frequently with doctors (OR = 3.14, 95%CI: 1.62-6.19, p < 0.001), alcoholic patients (OR = 2.14, 95%CI: 1.08-4.19, p = 0.027), who used to have meat-based food (OR = 1.42, 95%CI: 1.00-2.00, p = 0.049), and have vegetarian-based diet (OR = 1.42, 95%CI: 1.00-2.00, p = 0.049) are more likely to have polypharmacy behaviors, while patients aged between 65 and 75 years (OR = 0.50, 95%CI: 0.33-0.77, p = 0.020), used to be brain workers (OR = 0.67, 95%CI: 0.45-0.99, p = 0.048), with disease history between 10 and 20 years (OR = 0.56, 95%CI: 0.37-0.83, p = 0.005), have had adverse drug reactions (OR = 0.64, 95%CI: 0.45-0.93, p = 0.019), and participated in medical insurance for urban and rural residents (OR = 0.35, 95%CI: 0.21-0.58, p < 0.001) were less likely to have polypharmacy behaviors. Conclusion: The results suggest that having a usual primary care provider may reduce the incidence of having polypharmacy behaviors; we can take intervention measures to promote establishing a long-term relationship between patients and primary care providers.