Sensitivity, parameter selection, and generalization of augmentation-based auxiliary-domain adversarial domain adaptation for object detection in foggy and rainy driving weather

The framework of Li et al. (IEEE Transactions on Intelligent Vehicles, 2025; preliminary version WACV 2023) adapts a clear-weather object detector to foggy and rainy driving scenes by combining image-level and object-level adversarial alignment, an Adversarial Gradient Reversal Layer (AdvGRL) that up-weights hard examples during alignment, and an augmentation-synthesized auxiliary domain that acts as the negative of a domain-level metric (triplet) regularizer, with the source features as the anchor and the target features as the positive. The method is demonstrated only on Faster R-CNN with synthetic-weather benchmarks (Cityscapes to Foggy/Rainy Cityscapes) and a clear-weather cross-camera transfer (Cityscapes to KITTI), with ad hoc choices of auxiliary-domain synthesis and of the AdvGRL hardness and overflow thresholds. The source leaves open the following questions: how sensitive the gains are to the auxiliary-domain construction and whether a principled construction criterion exists; whether a general rule for selecting the AdvGRL hardness and overflow thresholds can be established beyond the few tested settings; whether the framework behaves as reported on other detector families, more diverse driving datasets, and real severe weather; and how complete the learned weather invariance actually is, beyond qualitative t-SNE visualization.

Background

Object detection networks such as Faster R-CNN localize and classify traffic participants in images, and they are typically trained under the assumption that training and test images follow the same distribution. This assumption fails across weather: a detector trained on clear-weather street scenes degrades under fog or rain because global image style and local object appearance shift between the two conditions — a mismatch known in machine learning as the domain gap, and the task of closing it without target-domain labels is unsupervised domain adaptation (UDA). The standard benchmark setting derives from Cityscapes, a large clear-weather urban driving dataset with pixel-level annotations: Foggy Cityscapes adds synthetic depth-dependent fog to its images, and Rainy Cityscapes adds synthetic rain streaks (for example via the RainMix augmentation), so a detector can be trained on labeled clear images and evaluated on unlabeled adverse-weather ones. Detection quality is reported as mean Average Precision (mAP), which pools classification and localization quality over object categories. A leading UDA strategy is adversarial feature alignment: a domain discriminator is trained to tell source from target features, while a gradient reversal layer (GRL) between the feature extractor and the discriminator multiplies the gradient by a negative weight $\lambda$ on the backward pass, so the extractor learns features the discriminator cannot separate — approximately domain-invariant features. The classical GRL uses a fixed $\lambda$ and treats easy and hard examples identically. Li et al. extend this in two ways. First, their AdvGRL replaces the fixed weight with an adaptive one: writing $L_c$ for the domain-classifier loss on an example, they set $\lambda_{adv}=\min(\lambda_0/L_c,\beta)$ when $L_c<\alpha$ and $\lambda_{adv}=\lambda_0$ otherwise, where $\alpha$ is a hardness threshold (a small $L_c$ means the domain classifier already identifies the example's domain easily, so its features are not yet domain-invariant — a hard example for adaptation — and the reversed gradient is amplified) and $\beta$ is an overflow threshold capping the amplification to avoid destructively large gradients. Second, they synthesize an auxiliary domain $A$ from the source data by weather-oriented augmentation, and add a triplet-style domain-level metric regularization that pulls source features toward target features while pushing them away from auxiliary features — formally minimizing $\max(d(F^S,F^T)-d(F^S,F^A)+\delta,0)$ at image level and analogously at object level, where $d$ is a feature distance and $\delta$ a margin. With these components the method reports state-of-the-art mAP on Cityscapes to Foggy/Rainy Cityscapes and strong car-detection AP on Cityscapes to KITTI (a real cross-camera transfer between two driving datasets captured with different camera rigs, treated as a domain gap without weather). Alignment quality is evidenced qualitatively by t-SNE — a visualization technique that projects high-dimensional feature distributions to two dimensions, where source and target distributions appear more overlapping after adaptation. The design choices behind these gains, however, are set by hand: the auxiliary domain is one particular augmentation-synthesized domain, $\alpha$ is derived ad hoc from a heuristic averaging, only four $(\alpha,\beta)$ combinations are tested, all experiments use a Faster R-CNN detector and synthetic-weather benchmarks, and invariance is never measured quantitatively. The unresolved question is what in this framework is principled rather than tuned — how much the reported gains depend on the auxiliary-domain construction, how the AdvGRL thresholds should be chosen in general, whether the framework transfers beyond the tested detector and benchmark family, and how complete the learned weather invariance really is.

Problem Statement

For the augmentation-based auxiliary-domain adversarial domain-adaptive object detection framework of Li et al. (IEEE T-IV 2025; WACV 2023 preliminary version) — image-level and object-level adversarial alignment with the Adversarial Gradient Reversal Layer (AdvGRL) weight $\lambda_{adv}=\min(\lambda_0/L_c,\beta)$ when $L_c<\alpha$ and $\lambda_0$ otherwise, where $L_c$ is the domain-classifier loss on an example, $\alpha$ is a hardness threshold (the reversed gradient is amplified when the discriminator already separates the example well, i.e. $L_c<\alpha$), $\beta$ is an overflow threshold capping that amplification, and $\lambda_0$ is the base weight, plus a triplet-style domain-level metric regularization that pulls source features toward target features while pushing them away from an augmentation-synthesized auxiliary domain — the following questions are open. Each is left open by the source's own experiments and concerns replacing the framework's hand-tuned design and untested generalization with a principled, empirically validated understanding of when, why, and how well it works; none is a precondition for the others.

(1) Auxiliary-domain sensitivity and construction: the source probes two alternative constructions — a regular augmentation combining color change, blur and salt-and-pepper noise, giving 38.7 mAP versus 42.3 mAP for its rain-synthesis auxiliary domain, and a CycleGAN style-transfer baseline giving 32.8 mAP on Cityscapes to Foggy Cityscapes (Sec. IV-K). How much do the adaptation gains depend on the choice of auxiliary-domain synthesis (the augmentation used to build the auxiliary domain) beyond these two probes, and does a principled criterion exist for choosing or constructing the auxiliary domain that predicts performance better than the source paper's single fixed choice?

(2) AdvGRL parameter selection: the source derives the hardness threshold ad hoc by averaging its binary cross-entropy domain-classifier loss at two probability-label settings — $\alpha=0.63$ from $(P_i,G_i)=(0.7,1)$ and $(0.3,0)$, and $\alpha=0.54$ from $(0.9,1)$ and $(0.1,0)$ (Sec. IV-B and IV-I) — and tests only the four $(\alpha,\beta)$ pairs $(0.63,30)$, $(0.63,10)$, $(0.54,30)$, $(0.54,10)$, giving mAP 42.34, 38.83, 39.38 and 40.47 on Cityscapes to Foggy Cityscapes with $\lambda_0=1$. How should the hardness threshold $\alpha$ and overflow threshold $\beta$ (with $\lambda_0$) be chosen beyond these four tested combinations, and does any such rule hold in settings other than those used to derive it?

(3) Scope of the demonstrated evidence: all source experiments use a Faster R-CNN detector on synthetic Foggy and Rainy Cityscapes targets of varying intensity, plus the clear-weather Cityscapes-to-KITTI cross-camera transfer in which the object-level metric-regularization term is dropped, and the paper's only generality claim (Sec. III-H) concerns pixel-to-pixel dataset correspondence rather than detector architectures. Does the framework behave as reported outside this demonstrated configuration — for example with other detector families (such as one-stage detectors or DETR-style detectors, i.e. end-to-end transformer-based detectors), on more diverse autonomous-driving datasets beyond the Cityscapes-derived benchmarks and the Cityscapes-to-KITTI cross-camera transfer, or under more severe real-world weather conditions rather than synthetic fog and rain of varying intensity?

(4) Invariance completeness and residual failures: the source evidences alignment only qualitatively, through t-SNE feature-distribution plots (Sec. IV-H; t-SNE being a technique that projects high-dimensional feature distributions to two dimensions for visual inspection). How complete is the learned weather invariance actually, beyond such qualitative visualization, and where do failures persist, e.g. for particularly difficult objects, camera settings, or weather combinations not seen during adaptation?

The source's Conclusions state no future-work or open-question agenda, so these are separate open directions grounded in the framework's hand-tuned design choices and limited evidence base rather than one jointly necessary objective; answering any one of them with evidence meeting the standard of the source's own experimental setting is a substantive contribution to the characterization.

References

1. Jinlong Li, Runsheng Xu, Xinyu Liu, Jin Ma, Baolu Li, Qin Zou, Jiaqi Ma, and Hongkai Yu. Domain Adaptation Based Object Detection for Autonomous Driving in Foggy and Rainy Weather. IEEE Transactions on Intelligent Vehicles, 10(2):900-911, 2025. DOI: 10.1109/TIV.2024.3419689. https://doi.org/10.1109/TIV.2024.3419689
2. Jinlong Li, Runsheng Xu, Jin Ma, Qin Zou, Jiaqi Ma, and Hongkai Yu. Domain Adaptive Object Detection for Autonomous Driving under Foggy Weather. In 2023 IEEE/CVF Winter Conference on Applications of Computer Vision (WACV), pages 612-622, 2023. DOI: 10.1109/WACV56688.2023.00068. https://doi.org/10.1109/WACV56688.2023.00068
3. Jinlong Li, Runsheng Xu, Xinyu Liu, Jin Ma, Baolu Li, Qin Zou, Jiaqi Ma, and Hongkai Yu. Domain Adaptation based Object Detection for Autonomous Driving in Foggy and Rainy Weather. arXiv:2307.09676 (v4, 2024). https://arxiv.org/abs/2307.09676
4. Saniya M. Deshmukh, Kailash A. Hambarde, and Hugo Proenca. Generalization Under Scrutiny: Cross-Domain Detection Progresses, Pitfalls, and Persistent Challenges. arXiv:2604.08230, 2026. https://arxiv.org/abs/2604.08230
5. Mengmeng Zhang, Qiyu Rong, and Hongyuan Jing. TTSDA-YOLO: A Two Training Stage Domain Adaptation Framework for Object Detection in Adverse Weather. IEEE Transactions on Instrumentation and Measurement, 74:1-13, 2025. DOI: 10.1109/TIM.2024.3497132. https://doi.org/10.1109/TIM.2024.3497132
6. Minsik Jeon, Junwon Seo, and Jihong Min. DA-RAW: Domain Adaptive Object Detection for Real-World Adverse Weather Conditions. In 2024 IEEE International Conference on Robotics and Automation (ICRA), pages 2013-2020, 2024. DOI: 10.1109/ICRA57147.2024.10611219. https://doi.org/10.1109/ICRA57147.2024.10611219
7. Zhuobin Fu, Kan Chang, Mingyang Ling, Qingzhi Zhang, and Enze Qi. Auxiliary Domain-Guided Adaptive Object Detection in Adverse Weather Conditions. In Asian Conference on Computer Vision (ACCV 2024), Lecture Notes in Computer Science, pages 312-329, 2024. DOI: 10.1007/978-981-96-0885-0_18. https://doi.org/10.1007/978-981-96-0885-0_18
8. Gege Zhang, Luping Wang, Zhiyong Zhang, and Zengping Chen. CPLT: Curriculum Pseudo Label Transformer for Domain Adaptive Object Detection in Foggy Weather. IEEE Sensors Journal, 23(23):29857-29868, 2023. DOI: 10.1109/JSEN.2023.3325266. https://doi.org/10.1109/JSEN.2023.3325266
9. Hui Zhang, Guiyang Luo, Yuanzhouhan Cao, Xiao Wang, Yidong Li, and Fei-Yue Wang. Scale-Disentangled and Uncertainty-Guided Alignment for Domain-Adaptive Object Detection. IEEE Transactions on Intelligent Transportation Systems, 25(12):19507-19521, 2024. DOI: 10.1109/TITS.2024.3447052. https://doi.org/10.1109/TITS.2024.3447052
10. Huy Tran and Khanh-Duy Nguyen. Benchmark evaluation of unsupervised domain adaptive object detectors in foggy UAV environments. International Journal of Machine Learning and Cybernetics, 17(8), 2026. DOI: 10.1007/s13042-026-03225-y. https://doi.org/10.1007/s13042-026-03225-y