A hybrid shaping scheme, carrier-less amplitude and phase modulation geometric-probabilistic hybrid shaping 16QAM (CAP-GPS-16QAM), is proposed for data center networks. The probabilistic shaping is achieved by applying the proposed probabilistic subset shaping to the constellation after geometric shaping (GS). An intensity modulation/direct detection experimental system of 5 GBaud is built to verify the proposal. The results show that the gains of the proposed scheme are 2.4, 2.6, and 3 dB compared with CAP-GS-16QAM, CAP-PAS-16QAM, and CAP-square-16QAM, respectively, at BER = 10 − 3. Meanwhile, in terms of generalized mutual information, the proposal improves the performance by 0.078, 0.125, and 0.153 bits/symbol compared with CAP-GS-16QAM, CAP-PAS-16QAM, and CAP-square-16QAM, respectively.
