Image-matching enhancement using a polarized intensity-hue-saturation fusion method

Linghao Shen; Linghao Shen; Mohamed Reda; Mohamed Reda; Yongqiang Zhao

doi:10.1364/AO.419726

Applied Optics
Vol. 60,
Issue 13,
pp. 3699-3715
(2021)
•https://doi.org/10.1364/AO.419726

Image-matching enhancement using a polarized intensity-hue-saturation fusion method

Linghao Shen, Mohamed Reda, and Yongqiang Zhao

Not Accessible

Your library or personal account may give you access

Get PDF
Email
Share
Get Citation
Copy Citation Text
Linghao Shen, Mohamed Reda, and Yongqiang Zhao, "Image-matching enhancement using a polarized intensity-hue-saturation fusion method," Appl. Opt. 60, 3699-3715 (2021)

Export Citation
- BibTex
- Endnote (RIS)
- HTML
- Plain Text
Citation alert
Save article

Check for updates

Related Topics
Table of Contents Category
- Imaging Systems, Image Processing, and Displays
Optics & Photonics Topics
?

The topics in this list come from the Optics and Photonics Topics applied to this article.

About this Article
History
- Original Manuscript: February 1, 2021
- Revised Manuscript: March 28, 2021
- Manuscript Accepted: March 28, 2021
- Published: April 23, 2021

Abstract

Atmospheric absorption and scattering (e.g., haze) cause degradation in the image quality of outdoor scenes, which affects the image-matching process. The scale-invariant feature transform (SIFT) algorithm is not effective in haze. Edge information is required to enhance the matching process. Utilizing the polarization information expressed by the Stokes vector component S1 with its edge information can improve the keypoint localization in the matching process. In this paper, a novel, to the best of our knowledge, fusion method called polarized intensity-hue-saturation is proposed that uses polarization and depth information by fusion of a polarized haze-removed image with the estimated depth and by applying S1. The instant dehazing method uses polarized images to obtain a haze-removed image and its estimated depth map. The fused image has high spatial details required for enhancing the matching process. The experimental results show that the proposed method outperforms the existing image-matching schemes and improves the conventional SIFT matching method.

Full Article | PDF Article

More Like This

Polarization dehazing method based on spatial frequency division and fusion for a far-field and dense hazy image

Feng Huang, Chaozhen Ke, Xianyu Wu, Shu Wang, Jing Wu, and Xuesong Wang
Appl. Opt. 60(30) 9319-9332 (2021)

Polarimetric dehazing method for visibility improvement based on visible and infrared image fusion

Jian Liang, Wenfei Zhang, Liyong Ren, Haijuan Ju, and Enshi Qu
Appl. Opt. 55(29) 8221-8226 (2016)

Polarization dehazing method based on separating and iterative optimizing airlight from the frequency domain for different concentrations of haze

Rui Sun, Tanbin Liao, Zhiguo Fan, Xudong Zhang, and Changxiang Wang
Appl. Opt. 61(35) 10362-10373 (2022)

Previous Article Next Article

Data Availability

Data underlying the results presented in this paper are not publicly available at this time but may be obtained from the authors upon reasonable request.

Cited By

You do not have subscription access to this journal. Cited by links are available to subscribers only. You may subscribe either as an Optica member, or as an authorized user of your institution.

Contact your librarian or system administrator
or
Login to access Optica Member Subscription

Figures (16)

You do not have subscription access to this journal. Figure files are available to subscribers only. You may subscribe either as an Optica member, or as an authorized user of your institution.

Contact your librarian or system administrator
or
Login to access Optica Member Subscription

Tables (9)

You do not have subscription access to this journal. Article tables are available to subscribers only. You may subscribe either as an Optica member, or as an authorized user of your institution.

Contact your librarian or system administrator
or
Login to access Optica Member Subscription

Equations (18)

You do not have subscription access to this journal. Equations are available to subscribers only. You may subscribe either as an Optica member, or as an authorized user of your institution.

Contact your librarian or system administrator
or
Login to access Optica Member Subscription

Datasets	Methods	NIQE [Smaller]	SSIM [1]	PSNR [Inf.]	MSE [0]
Data set 12	Haze image	23.6865	0.7621	20.5262	0.0008
	Haze-removed image	23.6765	0.7722	20.8161	0.0007
	Pol-IHS fused image	23.6713	0.7877	22.5322	0.0005
Data set 1	Haze image	49.7977	0.9211	57.1582	0.001
	Haze-removed image.	49.7977	0.9227	57.3646	0.001
	Pol-IHS fused image	49.7966	0.9958	57. 4067	0.0005
Data set 7	Haze image	59.8008	0.8996	41.9998	0.0004
	Haze-removed image	59.7998	0.9024	42.7498	0.0002
	Pol-IHS fused image	59.7972	0.9036	42.8872	0.0001
Data set 4	Haze image	13.6850	0.8501	20.5301	0.0007
	Haze-removed image	13.6766	0.8689	21.9541	0.0006
	Pol-IHS fused image	13.6709	0.8902	22.6804	0.0004

Data Set	# of kp Feature Haze Image	# of kp Feature Haze-Removed	# of kp Feature Pol-IHS	Incr. Haze-Removed %	Incr. Pol-IHS %
Data set 1	196	249	398	27	103
Data set 2	249	328	720	31.7	189
Data set 3	671	698	823	4	22.7
Data set 4	228	433	450	47.3	50
Data set 5	895	985	1210	10	35.2
Data set 6	324	370	664	14.2	104.9
Data set 7	475	604	712	27.1	49.9
Data set 8	933	1059	1410	13.5	51.1
Data set 9	893	1012	1351	13.3	51.3
Data set 10	869	1058	1311	21.7	50.9
Data set 11	538	666	901	23.8	67.5
Data set 12	1170	1370	1611	17.1	37.7

Data Set 12	# of kp Feature Scene 1	# of kp Feature Scene 2	# of Matching Points	Effectiveness %	Time (s)
H-SIFT	1170	980	57	5.3	7.21
Hr-SIFT	1370	1195	72	5.6	7.07
Pol-IHS-SIFT	1611	1420	114	7.52	10.01
Pol-IHS-SURF	1558	1321	97	6.74	6.99
Pol-IHS-OBR	1602	1406	110	7.31	6.05

Data Set 7	# of kp Feature Scene 1	# of kp Feature Scene 2	# of Matching Points	Effectiveness %	Time (s)
H-SIFT	475	467	71	15	4.04
Hr-SIFT	604	497	85	15.5	4.49
Pol-IHS-SIFT	712	655	117	17.12	5.09
Pol-IHS-SURF	677	636	94	15.69	3.87
Pol-IHS-OBR	685	660	109	16.2	3.80

Data Set 1	# of kp Feature Scene 1	# of kp Feature Scene 2	# of Matching Points	Effectiveness %	Time (s)
H-SIFT	196	91	18	12.54	2.21
Hr-SIFT	249	116	24	13.15	2.16
Pol-IHS-SIFT	398	161	40	14.3	3.13
Pol-IHS-SURF	344	129	30	12.68	2.01
Pol-IHS-ORB	360	140	32	12.8	1.98

Data Set 4	# of kp Feature Scene 1	# of kp Feature Scene 2	# of Matching Points	Effectiveness %	Time (s)
H-SIFT	228	72	15	10	2.3
Hr-SIFT	443	83	27	10.27	2.9
Pol-IHS-SIFT	450	101	38	13.6	3.13
Pol-IHS-SURF	445	93	30	11.15	2.2
Pol-IHS-ORB	447	96	32	11.79	2.0

	$v = 0.01$		Effectiveness (%) at Different PSNRs
Data Sets	$m$	PSNR [dB]	Hr-SIFT	Pol-IHS-SIFT	Pol-IHS-SURF	Pol-IHS-OBR
Data set 12	0.5	11.42	4.21	6.87	5.51	6.62
	1	10.93	4.42	6.99	5.69	6.79
	2	10.85	4.59	7.01	5.74	6.81
Data set 7	0.5	10.56	14.36	16.4	14.42	15.1
	1	9.912	14.81	16.62	14.93	15.32
	2	9.106	14.87	16.75	15.01	15.54
Data set 1	0.5	8.235	12.52	13.31	11.3	11.52
	1	7.581	12.71	13.62	11.59	11.96
	2	6.259	12.84	13.84	11.99	12.1
Data set 4	0.5	10.04	9.54	12.22	10.56	10.64
	1	9.633	9.63	12.69	10.91	10.99
	2	7.233	9.95	13.2	11.01	11.12


Input: ${I_{0}, I_{90}}$ (Two polarized images for the two scenes).
Output: keypoints kp and the matched features $m$ of the two image scenes.
1. Calculate the haze-removed image $\hat{R}$ and the estimated depth map $D$ for each scene.
2. Calculate the Stokes vector component $S_{1}$ .
3: Calculate the optimized intensity component $I_{o p t}$ ,
where $k = 0, 1, \dots u n t i l ‖ I_{o p t}^{k} - I_{o p t}^{k + 1} ‖_{2} / ‖ I_{o p t}^{k} ‖ \leq ε$
$I_{o p t}^{0} = I + S_{1}$
$I_{o p t}^{k + 1} = I_{o p t}^{k} - Δ t (H_{p}^{} (H_{p} * (I_{o p t}^{k} + S_{1}) - ω H_{p} * D_{h})$
$+ λ_{1} L_{p}^{} (L_{p} * (I_{o p t}^{k} + S_{1}) - I) - λ_{2} \nabla (I_{o p t}^{k} + S_{1}))$
4. The Pol-IHS fused image $F$ is calculated by $F = \hat{R} + S_{1} (D_{h} - I_{o p t})$
5. Apply the SIFT algorithm on the two Pol-IHS fused images for the two image scenes.
6. Obtain the keypoints kp and the matched features m of the two image scenes.
End.


Input: ${I_{0}, I_{90}}$ (Two polarized images for the two scenes).
Output: keypoints kp and the matched features $m$ of the two image scenes.
1. Calculate the haze-removed image $\hat{R}$ and the estimated depth map $D$ for each scene.
2. Calculate the Stokes vector component $S_{1}$ .
3: Calculate the optimized intensity component $I_{o p t}$ ,
where $k = 0, 1, \dots u n t i l ‖ I_{o p t}^{k} - I_{o p t}^{k + 1} ‖_{2} / ‖ I_{o p t}^{k} ‖ \leq ε$
$I_{o p t}^{0} = I + S_{1}$
$I_{o p t}^{k + 1} = I_{o p t}^{k} - Δ t (H_{p}^{} (H_{p} * (I_{o p t}^{k} + S_{1}) - ω H_{p} * D_{h})$
$+ λ_{1} L_{p}^{} (L_{p} * (I_{o p t}^{k} + S_{1}) - I) - λ_{2} \nabla (I_{o p t}^{k} + S_{1}))$
4. The Pol-IHS fused image $F$ is calculated by $F = \hat{R} + S_{1} (D_{h} - I_{o p t})$
5. Apply the SIFT algorithm on the two Pol-IHS fused images for the two image scenes.
6. Obtain the keypoints kp and the matched features m of the two image scenes.
End.

Abstract

Data Availability

Cited By

Figures (16)

Tables (9)

Equations (18)

Applied Optics