We first investigate the nondipole effect of asymmetric quantum dots (QDs) in plasmonic antennas of shell, rod, triangle, and disk. The rod and triangle are found to have greater nondipole effect compared to shell and disk due to the larger electric field gradients resulting from high surface curvature. For a 3nm-radius QD adjacent to the rod end or the corner of triangle, the coupling strength of plasmon-QD interaction has a 10% enhancement for dipolar plasmonic mode. The multipole expansion shows that the enhancement is mainly contributed by the dipole-quadrupole transition interference, which will be suppressed in a symmetrical dimer structure. We further explore the nondipole eﬀect in the plasmonic nanocavity formed by a gold nanotip and substrate. The nondipole eﬀect is found to increase the quantum nonlinearities and lead to better single-photon purity. Our work demonstrates the impact of nondipole eﬀect on plasmon-QD strong coupling and potential applications in quantum optics.