Infrared small target detection (IRSTD) plays an essential role in many fields such as air guidance, tracking, and surveillance. However, due to the tiny sizes of infrared small targets, which are easily confused with background noises and lack clear contours and texture information, how to learn more discriminative small target features while suppressing background noises is still a challenging task. In this paper, a context-aware cross-level attention fusion network for IRSTD is proposed. Specifically, a self-attention-induced global context-aware module obtains multilevel attention feature maps with robust positional relationship modeling. The high-level feature maps with abundant semantic information are then passed through a multiscale feature refinement module to restore the target details and highlight salient features. Feature maps at all levels are fed into a channel and spatial filtering module to compress redundant information and remove background noises, which are then used for cross-level feature fusion. Furthermore, to overcome the lack of publicly available datasets, a large-scale multiscene infrared small target dataset with high-quality annotations is constructed. Finally, extensive experiments on both public and our self-developed datasets demonstrate the effectiveness of the proposed method and the superiority compared with other state-of-the-art approaches.