It is two decades since the first reports that the insulator-to-metal transition (IMT) in vanadium dioxide (VO2) occurred on an ultrafast time scale, followed by growing interest in the potential use of this strongly correlated oxide in a variety of switching schemes. At first glance, VO2 would seem to be ideally suited to a variety of applications in electro-optics and all optical switching: The IMT occurs on a sub-picosecond time scale; it is fully reversible and has a large dielectric contrast at wavelengths in the near- to mid-infrared; and the material itself is fully compatible with many optical and electronic materials of interest. However, there are also well-known difficulties, chief among them the fact that the IMT, if fully completed, is accompanied by a structural phase transition (SPT) that requires nanoseconds to return from the rutile, metallic state to the monoclinic insulating ground state – thus essentially limiting switching speeds to time scales similar to those in amorphous-to-crystalline transitions in chalcogenide glasses. Here we discuss the ways in which the very considerable advantages of VO2 as a modulating or threshold switch can be amplified by deploying it appropriately in silicon photonic modulators, switchable metasurfaces, plasmonic heterostructures, and two-dimensional materials that can support phonon polariton optics. We focus particularly on ways of tailoring the physical properties of the VO2 component of a system to meet the requirements of operating in particular wavelength regions, meeting specific threshold requirements and choosing electrical or optical initiation of the IMT.