17 April 2023 Another scale-guided parallel transformer for image aesthetic assessment
Lili Shen, Shaohu Xu, Jing Zhang, Bo Peng
Author Affiliations +
Abstract

Image aesthetic assessment (IAA) is a challenging task in computer vision fields, which aims to automatically evaluate image beauty by simulating human perception on image aesthetic. With the development of deep learning, although convolutional neural network (CNN)-based IAA approaches have achieved extraordinary progress, CNN experiences difficulty to capture long-distance relationships among visual elements. There is a strong correlation between image layout and image semantic information for image aesthetic. In order to solve this problem, an another scale-guided parallel transformer is proposed, including a multiscale local feature extractor (ME), a feature projection (FP), and an another scale-guided parallel feature fusion transformer (AST). The ME captures primary local features with classic ResNet at multiple scales. The FP performs dimension transformation on feature maps for each scale, which can obtain feature token and aesthetic token. The AST with two parallel transformer encoders is exploited to highlight the significant regions in the holistic image, in which the feature tokens and the aesthetic token from another scale are grouped together to obtain interscale guidance. The final score distribution is achieved by weighting multiple aesthetic tokens with learnable parameters for unified aesthetics assessment. Extensive experiments on two public datasets, including aesthetic visual analysis and aesthetics and attributes database, demonstrate that the proposed method outperforms the state-of-the-art methods across three different tasks.

© 2023 SPIE and IS&T
Lili Shen, Shaohu Xu, Jing Zhang, and Bo Peng "Another scale-guided parallel transformer for image aesthetic assessment," Journal of Electronic Imaging 32(2), 023035 (17 April 2023). https://doi.org/10.1117/1.JEI.32.2.023035
Received: 12 November 2022; Accepted: 30 March 2023; Published: 17 April 2023
Advertisement
Advertisement
RIGHTS & PERMISSIONS
Get copyright permission  Get copyright permission on Copyright Marketplace
KEYWORDS
Transformers

Visualization

Feature extraction

Semantics

Image quality

Education and training

Image fusion

Back to Top