Decoder-Only Image Registration

Jia, Xi ORCID: https://orcid.org/0000-0002-7034-5050, Lu, Wenqi, Cheng, Xinxing ORCID: https://orcid.org/0000-0001-5084-682X and Duan, Jinming ORCID: https://orcid.org/0000-0002-5108-2128 (2025) Decoder-Only Image Registration. IEEE Transactions on Medical Imaging. ISSN 0278-0062

Preview

Accepted Version Available under License Creative Commons Attribution. Download (2MB) | Preview

Abstract

In unsupervised medical image registration, encoder-decoder architectures are widely used to predict dense, full-resolution displacement fields from paired images. Despite their popularity, we question the necessity of making both the encoder and decoder learnable. To address this, we propose LessNet, a simplified network architecture with only a learnable decoder, while completely omitting a learnable encoder. Instead, LessNet replaces the encoder with simple, handcrafted features, eliminating the need to optimize encoder parameters. This results in a compact, efficient, and decoder-only architecture for 3D medical image registration. We evaluate our decoder-only LessNet on five registration tasks: 1) inter-subject brain registration using the OASIS-1 dataset, 2) atlas-based brain registration using the IXI dataset, 3) cardiac ES-ED registration using the ACDC dataset, 4) inter-subject abdominal MR registration using the CHAOS dataset, and 5) multi-study, multi-site brain registration using images from 13 public datasets. Our results demonstrate that LessNet can effectively and efficiently learn both dense displacement and diffeomorphic deformation fields. Furthermore, our decoder-only LessNet can achieve comparable registration performance to benchmarking methods such as Voxel-Morph and TransMorph, while requiring significantly fewer computational resources. Our code and pre-trained models are available at https://github.com/xi-jia/LessNet.