Abstract

Division of focal plane (DoFP) polarimeter is widely used in polarization imaging sensors. The periodically arranged micro-polarizers integrated on the focal plane ensure its outstanding real-time performance, but reduce the spatial resolution of output images and further affect the calculation of polarization parameters. In this paper, a four-layer, end-to-end fully convolutional neural network called Fork-Net is proposed, which aims to directly improve the imaging quality of three polarization properties: intensity (i.e., S0), degree of linear polarization (DoLP), and angle of polarization (AoP), rather than focusing on reducing the interpolation error of intensity images of different polarization orientations. The Fork-Net accepts raw mosaic images as input and directly outputs S0, DoLP, and AoP. It is also trained with a customized loss function. The experimental results show that compared with existing methods, the proposed one achieves the highest peak signal-to-noise ratio (PSNR) and prominent visual quality on output images.

© 2019 Optical Society of America under the terms of the OSA Open Access Publishing Agreement

Full Article  |  PDF Article
OSA Recommended Articles
Learning a convolutional demosaicing network for microgrid polarimeter imagery

Junchao Zhang, Jianbo Shao, Haibo Luo, Xiangyue Zhang, Bin Hui, Zheng Chang, and Rongguang Liang
Opt. Lett. 43(18) 4534-4537 (2018)

Deep learning approach for Fourier ptychography microscopy

Thanh Nguyen, Yujia Xue, Yunzhe Li, Lei Tian, and George Nehmetallah
Opt. Express 26(20) 26470-26484 (2018)

Fast phase retrieval in off-axis digital holographic microscopy through deep learning

Gong Zhang, Tian Guan, Zhiyuan Shen, Xiangnan Wang, Tao Hu, Delai Wang, Yonghong He, and Ni Xie
Opt. Express 26(15) 19388-19405 (2018)

References

  • View by:
  • |
  • |
  • |

  1. M. Sarkar, D. S. S. San Segundo Bello, C. van Hoof, and A. Theuwissen, “Integrated polarization analyzing cmos image sensor for material classification,” IEEE Sensors J. 11, 1692–1703 (2011).
    [Crossref]
  2. D. H. Goldstein, “Polarimetric characterization of federal standard paints,” in Polarization Analysis, Measurement, and Remote Sensing III, vol. 4133 (International Society for Optics and Photonics, 2000), pp. 112–124.
    [Crossref]
  3. Y. Aron and Y. Gronau, “Polarization in the lwir: a method to improve target aquisition,” in Infrared Technology and Applications XXXI, vol. 5783 (International Society for Optics and Photonics, 2005), pp. 653–662.
    [Crossref]
  4. B. Huang, T. Liu, H. Hu, J. Han, and M. Yu, “Underwater image recovery considering polarization effects of objects,” Opt. Express 24, 9826–9838 (2016).
    [Crossref] [PubMed]
  5. E. Salomatina-Motts, V. Neel, and A. Yaroslavskaya, “Multimodal polarization system for imaging skin cancer,” Opt. Spectrosc. 107, 884–890 (2009).
    [Crossref]
  6. J. S. Tyo, D. L. Goldstein, D. B. Chenault, and J. A. Shaw, “Review of passive imaging polarimetry for remote sensing applications,” Appl. Opt. 45, 5453–5469 (2006).
    [Crossref] [PubMed]
  7. R. Perkins and V. Gruev, “Signal-to-noise analysis of stokes parameters in division of focal plane polarimeters,” Opt. Express 18, 25815–25824 (2010).
    [Crossref] [PubMed]
  8. S. Gao and V. Gruev, “Bilinear and bicubic interpolation methods for division of focal plane polarimeters,” Opt. Express 19, 26161–26173 (2011).
    [Crossref]
  9. S. Gao and V. Gruev, “Gradient based interpolation for division of focal plane polarization imaging sensors,” in 2012 IEEE International Symposium on Circuits and Systems (ISCAS), (IEEE, 2012), pp. 1855–1858.
    [Crossref]
  10. J. Zhang, H. Luo, B. Hui, and Z. Chang, “Image interpolation for division of focal plane polarimeters with intensity correlation,” Opt. Express 24, 20799–20807 (2016).
    [Crossref] [PubMed]
  11. J. Zhang, W. Ye, A. Ahmed, Z. Qiu, Y. Cao, and X. Zhao, “A novel smoothness-based interpolation algorithm for division of focal plane polarimeters,” in 2017 IEEE International Symposium on Circuits and Systems (ISCAS), (IEEE, 2017), pp. 1–4.
  12. J. Zhang, H. Luo, R. Liang, A. Ahmed, X. Zhang, B. Hui, and Z. Chang, “Sparse representation-based demosaicing method for microgrid polarimeter imagery,” Opt. Lett. 43, 3265–3268 (2018).
    [Crossref] [PubMed]
  13. J. Zhang, J. Shao, H. Luo, X. Zhang, B. Hui, Z. Chang, and R. Liang, “Learning a convolutional demosaicing network for microgrid polarimeter imagery,” Opt. Lett. 43, 4534–4537 (2018).
    [Crossref] [PubMed]
  14. O. Ronneberger, P. Fischer, and T. Brox, “U-net: Convolutional networks for biomedical image segmentation,” in Medical Image Computing and Computer-Assisted Intervention (MICCAI), (Springer International Publishing, Cham, 2015), pp. 234–241.
  15. C. Dong, C. C. Loy, K. He, and X. Tang, “Image super-resolution using deep convolutional networks,” IEEE Transactions on Pattern Analysis Mach. Intell. 38, 295–307 (2016).
    [Crossref]
  16. V. Nair and G. E. Hinton, “Rectified linear units improve restricted boltzmann machines,” in Proceedings of the 27th international conference on machine learning (ICML-10), (2010), pp. 807–814.
  17. Z. Wang, A. C. Bovik, H. R. Sheikh, and E. P. Simoncelli, “Image quality assessment: from error visibility to structural similarity,” IEEE transactions on image processing 13, 600–612 (2004).
    [Crossref] [PubMed]
  18. H. Zhao, O. Gallo, I. Frosio, and J. Kautz, “Loss functions for neural networks for image processing,” arXiv preprint arXiv:1511.08861 (2015).
  19. K. He, X. Zhang, S. Ren, and J. Sun, “Delving deep into rectifiers: Surpassing human-level performance on imagenet classification,” in The IEEE International Conference on Computer Vision (ICCV), (2015), pp. 1026–1034.
  20. D. P. Kingma and J. Ba, “Adam: A method for stochastic optimization,” arXiv preprint arXiv:1412.6980 (2014).

2018 (2)

J. Zhang, H. Luo, R. Liang, A. Ahmed, X. Zhang, B. Hui, and Z. Chang, “Sparse representation-based demosaicing method for microgrid polarimeter imagery,” Opt. Lett. 43, 3265–3268 (2018).
[Crossref] [PubMed]

J. Zhang, J. Shao, H. Luo, X. Zhang, B. Hui, Z. Chang, and R. Liang, “Learning a convolutional demosaicing network for microgrid polarimeter imagery,” Opt. Lett. 43, 4534–4537 (2018).
[Crossref] [PubMed]

2016 (3)

C. Dong, C. C. Loy, K. He, and X. Tang, “Image super-resolution using deep convolutional networks,” IEEE Transactions on Pattern Analysis Mach. Intell. 38, 295–307 (2016).
[Crossref]

B. Huang, T. Liu, H. Hu, J. Han, and M. Yu, “Underwater image recovery considering polarization effects of objects,” Opt. Express 24, 9826–9838 (2016).
[Crossref] [PubMed]

J. Zhang, H. Luo, B. Hui, and Z. Chang, “Image interpolation for division of focal plane polarimeters with intensity correlation,” Opt. Express 24, 20799–20807 (2016).
[Crossref] [PubMed]

2011 (2)

S. Gao and V. Gruev, “Bilinear and bicubic interpolation methods for division of focal plane polarimeters,” Opt. Express 19, 26161–26173 (2011).
[Crossref]

M. Sarkar, D. S. S. San Segundo Bello, C. van Hoof, and A. Theuwissen, “Integrated polarization analyzing cmos image sensor for material classification,” IEEE Sensors J. 11, 1692–1703 (2011).
[Crossref]

2010 (1)

R. Perkins and V. Gruev, “Signal-to-noise analysis of stokes parameters in division of focal plane polarimeters,” Opt. Express 18, 25815–25824 (2010).
[Crossref] [PubMed]

2009 (1)

E. Salomatina-Motts, V. Neel, and A. Yaroslavskaya, “Multimodal polarization system for imaging skin cancer,” Opt. Spectrosc. 107, 884–890 (2009).
[Crossref]

2006 (1)

J. S. Tyo, D. L. Goldstein, D. B. Chenault, and J. A. Shaw, “Review of passive imaging polarimetry for remote sensing applications,” Appl. Opt. 45, 5453–5469 (2006).
[Crossref] [PubMed]

2004 (1)

Z. Wang, A. C. Bovik, H. R. Sheikh, and E. P. Simoncelli, “Image quality assessment: from error visibility to structural similarity,” IEEE transactions on image processing 13, 600–612 (2004).
[Crossref] [PubMed]

Ahmed, A.

J. Zhang, H. Luo, R. Liang, A. Ahmed, X. Zhang, B. Hui, and Z. Chang, “Sparse representation-based demosaicing method for microgrid polarimeter imagery,” Opt. Lett. 43, 3265–3268 (2018).
[Crossref] [PubMed]

J. Zhang, W. Ye, A. Ahmed, Z. Qiu, Y. Cao, and X. Zhao, “A novel smoothness-based interpolation algorithm for division of focal plane polarimeters,” in 2017 IEEE International Symposium on Circuits and Systems (ISCAS), (IEEE, 2017), pp. 1–4.

Aron, Y.

Y. Aron and Y. Gronau, “Polarization in the lwir: a method to improve target aquisition,” in Infrared Technology and Applications XXXI, vol. 5783 (International Society for Optics and Photonics, 2005), pp. 653–662.
[Crossref]

Ba, J.

D. P. Kingma and J. Ba, “Adam: A method for stochastic optimization,” arXiv preprint arXiv:1412.6980 (2014).

Bovik, A. C.

Z. Wang, A. C. Bovik, H. R. Sheikh, and E. P. Simoncelli, “Image quality assessment: from error visibility to structural similarity,” IEEE transactions on image processing 13, 600–612 (2004).
[Crossref] [PubMed]

Brox, T.

O. Ronneberger, P. Fischer, and T. Brox, “U-net: Convolutional networks for biomedical image segmentation,” in Medical Image Computing and Computer-Assisted Intervention (MICCAI), (Springer International Publishing, Cham, 2015), pp. 234–241.

Cao, Y.

J. Zhang, W. Ye, A. Ahmed, Z. Qiu, Y. Cao, and X. Zhao, “A novel smoothness-based interpolation algorithm for division of focal plane polarimeters,” in 2017 IEEE International Symposium on Circuits and Systems (ISCAS), (IEEE, 2017), pp. 1–4.

Chang, Z.

J. Zhang, H. Luo, R. Liang, A. Ahmed, X. Zhang, B. Hui, and Z. Chang, “Sparse representation-based demosaicing method for microgrid polarimeter imagery,” Opt. Lett. 43, 3265–3268 (2018).
[Crossref] [PubMed]

J. Zhang, J. Shao, H. Luo, X. Zhang, B. Hui, Z. Chang, and R. Liang, “Learning a convolutional demosaicing network for microgrid polarimeter imagery,” Opt. Lett. 43, 4534–4537 (2018).
[Crossref] [PubMed]

J. Zhang, H. Luo, B. Hui, and Z. Chang, “Image interpolation for division of focal plane polarimeters with intensity correlation,” Opt. Express 24, 20799–20807 (2016).
[Crossref] [PubMed]

Chenault, D. B.

J. S. Tyo, D. L. Goldstein, D. B. Chenault, and J. A. Shaw, “Review of passive imaging polarimetry for remote sensing applications,” Appl. Opt. 45, 5453–5469 (2006).
[Crossref] [PubMed]

Dong, C.

C. Dong, C. C. Loy, K. He, and X. Tang, “Image super-resolution using deep convolutional networks,” IEEE Transactions on Pattern Analysis Mach. Intell. 38, 295–307 (2016).
[Crossref]

Fischer, P.

O. Ronneberger, P. Fischer, and T. Brox, “U-net: Convolutional networks for biomedical image segmentation,” in Medical Image Computing and Computer-Assisted Intervention (MICCAI), (Springer International Publishing, Cham, 2015), pp. 234–241.

Frosio, I.

H. Zhao, O. Gallo, I. Frosio, and J. Kautz, “Loss functions for neural networks for image processing,” arXiv preprint arXiv:1511.08861 (2015).

Gallo, O.

H. Zhao, O. Gallo, I. Frosio, and J. Kautz, “Loss functions for neural networks for image processing,” arXiv preprint arXiv:1511.08861 (2015).

Gao, S.

S. Gao and V. Gruev, “Bilinear and bicubic interpolation methods for division of focal plane polarimeters,” Opt. Express 19, 26161–26173 (2011).
[Crossref]

S. Gao and V. Gruev, “Gradient based interpolation for division of focal plane polarization imaging sensors,” in 2012 IEEE International Symposium on Circuits and Systems (ISCAS), (IEEE, 2012), pp. 1855–1858.
[Crossref]

Goldstein, D. H.

D. H. Goldstein, “Polarimetric characterization of federal standard paints,” in Polarization Analysis, Measurement, and Remote Sensing III, vol. 4133 (International Society for Optics and Photonics, 2000), pp. 112–124.
[Crossref]

Goldstein, D. L.

J. S. Tyo, D. L. Goldstein, D. B. Chenault, and J. A. Shaw, “Review of passive imaging polarimetry for remote sensing applications,” Appl. Opt. 45, 5453–5469 (2006).
[Crossref] [PubMed]

Gronau, Y.

Y. Aron and Y. Gronau, “Polarization in the lwir: a method to improve target aquisition,” in Infrared Technology and Applications XXXI, vol. 5783 (International Society for Optics and Photonics, 2005), pp. 653–662.
[Crossref]

Gruev, V.

S. Gao and V. Gruev, “Bilinear and bicubic interpolation methods for division of focal plane polarimeters,” Opt. Express 19, 26161–26173 (2011).
[Crossref]

R. Perkins and V. Gruev, “Signal-to-noise analysis of stokes parameters in division of focal plane polarimeters,” Opt. Express 18, 25815–25824 (2010).
[Crossref] [PubMed]

S. Gao and V. Gruev, “Gradient based interpolation for division of focal plane polarization imaging sensors,” in 2012 IEEE International Symposium on Circuits and Systems (ISCAS), (IEEE, 2012), pp. 1855–1858.
[Crossref]

Han, J.

B. Huang, T. Liu, H. Hu, J. Han, and M. Yu, “Underwater image recovery considering polarization effects of objects,” Opt. Express 24, 9826–9838 (2016).
[Crossref] [PubMed]

He, K.

C. Dong, C. C. Loy, K. He, and X. Tang, “Image super-resolution using deep convolutional networks,” IEEE Transactions on Pattern Analysis Mach. Intell. 38, 295–307 (2016).
[Crossref]

K. He, X. Zhang, S. Ren, and J. Sun, “Delving deep into rectifiers: Surpassing human-level performance on imagenet classification,” in The IEEE International Conference on Computer Vision (ICCV), (2015), pp. 1026–1034.

Hinton, G. E.

V. Nair and G. E. Hinton, “Rectified linear units improve restricted boltzmann machines,” in Proceedings of the 27th international conference on machine learning (ICML-10), (2010), pp. 807–814.

Hu, H.

B. Huang, T. Liu, H. Hu, J. Han, and M. Yu, “Underwater image recovery considering polarization effects of objects,” Opt. Express 24, 9826–9838 (2016).
[Crossref] [PubMed]

Huang, B.

B. Huang, T. Liu, H. Hu, J. Han, and M. Yu, “Underwater image recovery considering polarization effects of objects,” Opt. Express 24, 9826–9838 (2016).
[Crossref] [PubMed]

Hui, B.

J. Zhang, J. Shao, H. Luo, X. Zhang, B. Hui, Z. Chang, and R. Liang, “Learning a convolutional demosaicing network for microgrid polarimeter imagery,” Opt. Lett. 43, 4534–4537 (2018).
[Crossref] [PubMed]

J. Zhang, H. Luo, R. Liang, A. Ahmed, X. Zhang, B. Hui, and Z. Chang, “Sparse representation-based demosaicing method for microgrid polarimeter imagery,” Opt. Lett. 43, 3265–3268 (2018).
[Crossref] [PubMed]

J. Zhang, H. Luo, B. Hui, and Z. Chang, “Image interpolation for division of focal plane polarimeters with intensity correlation,” Opt. Express 24, 20799–20807 (2016).
[Crossref] [PubMed]

Kautz, J.

H. Zhao, O. Gallo, I. Frosio, and J. Kautz, “Loss functions for neural networks for image processing,” arXiv preprint arXiv:1511.08861 (2015).

Kingma, D. P.

D. P. Kingma and J. Ba, “Adam: A method for stochastic optimization,” arXiv preprint arXiv:1412.6980 (2014).

Liang, R.

J. Zhang, H. Luo, R. Liang, A. Ahmed, X. Zhang, B. Hui, and Z. Chang, “Sparse representation-based demosaicing method for microgrid polarimeter imagery,” Opt. Lett. 43, 3265–3268 (2018).
[Crossref] [PubMed]

J. Zhang, J. Shao, H. Luo, X. Zhang, B. Hui, Z. Chang, and R. Liang, “Learning a convolutional demosaicing network for microgrid polarimeter imagery,” Opt. Lett. 43, 4534–4537 (2018).
[Crossref] [PubMed]

Liu, T.

B. Huang, T. Liu, H. Hu, J. Han, and M. Yu, “Underwater image recovery considering polarization effects of objects,” Opt. Express 24, 9826–9838 (2016).
[Crossref] [PubMed]

Loy, C. C.

C. Dong, C. C. Loy, K. He, and X. Tang, “Image super-resolution using deep convolutional networks,” IEEE Transactions on Pattern Analysis Mach. Intell. 38, 295–307 (2016).
[Crossref]

Luo, H.

J. Zhang, J. Shao, H. Luo, X. Zhang, B. Hui, Z. Chang, and R. Liang, “Learning a convolutional demosaicing network for microgrid polarimeter imagery,” Opt. Lett. 43, 4534–4537 (2018).
[Crossref] [PubMed]

J. Zhang, H. Luo, R. Liang, A. Ahmed, X. Zhang, B. Hui, and Z. Chang, “Sparse representation-based demosaicing method for microgrid polarimeter imagery,” Opt. Lett. 43, 3265–3268 (2018).
[Crossref] [PubMed]

J. Zhang, H. Luo, B. Hui, and Z. Chang, “Image interpolation for division of focal plane polarimeters with intensity correlation,” Opt. Express 24, 20799–20807 (2016).
[Crossref] [PubMed]

Nair, V.

V. Nair and G. E. Hinton, “Rectified linear units improve restricted boltzmann machines,” in Proceedings of the 27th international conference on machine learning (ICML-10), (2010), pp. 807–814.

Neel, V.

E. Salomatina-Motts, V. Neel, and A. Yaroslavskaya, “Multimodal polarization system for imaging skin cancer,” Opt. Spectrosc. 107, 884–890 (2009).
[Crossref]

Perkins, R.

R. Perkins and V. Gruev, “Signal-to-noise analysis of stokes parameters in division of focal plane polarimeters,” Opt. Express 18, 25815–25824 (2010).
[Crossref] [PubMed]

Qiu, Z.

J. Zhang, W. Ye, A. Ahmed, Z. Qiu, Y. Cao, and X. Zhao, “A novel smoothness-based interpolation algorithm for division of focal plane polarimeters,” in 2017 IEEE International Symposium on Circuits and Systems (ISCAS), (IEEE, 2017), pp. 1–4.

Ren, S.

K. He, X. Zhang, S. Ren, and J. Sun, “Delving deep into rectifiers: Surpassing human-level performance on imagenet classification,” in The IEEE International Conference on Computer Vision (ICCV), (2015), pp. 1026–1034.

Ronneberger, O.

O. Ronneberger, P. Fischer, and T. Brox, “U-net: Convolutional networks for biomedical image segmentation,” in Medical Image Computing and Computer-Assisted Intervention (MICCAI), (Springer International Publishing, Cham, 2015), pp. 234–241.

Salomatina-Motts, E.

E. Salomatina-Motts, V. Neel, and A. Yaroslavskaya, “Multimodal polarization system for imaging skin cancer,” Opt. Spectrosc. 107, 884–890 (2009).
[Crossref]

San Segundo Bello, D. S. S.

M. Sarkar, D. S. S. San Segundo Bello, C. van Hoof, and A. Theuwissen, “Integrated polarization analyzing cmos image sensor for material classification,” IEEE Sensors J. 11, 1692–1703 (2011).
[Crossref]

Sarkar, M.

M. Sarkar, D. S. S. San Segundo Bello, C. van Hoof, and A. Theuwissen, “Integrated polarization analyzing cmos image sensor for material classification,” IEEE Sensors J. 11, 1692–1703 (2011).
[Crossref]

Shao, J.

J. Zhang, J. Shao, H. Luo, X. Zhang, B. Hui, Z. Chang, and R. Liang, “Learning a convolutional demosaicing network for microgrid polarimeter imagery,” Opt. Lett. 43, 4534–4537 (2018).
[Crossref] [PubMed]

Shaw, J. A.

J. S. Tyo, D. L. Goldstein, D. B. Chenault, and J. A. Shaw, “Review of passive imaging polarimetry for remote sensing applications,” Appl. Opt. 45, 5453–5469 (2006).
[Crossref] [PubMed]

Sheikh, H. R.

Z. Wang, A. C. Bovik, H. R. Sheikh, and E. P. Simoncelli, “Image quality assessment: from error visibility to structural similarity,” IEEE transactions on image processing 13, 600–612 (2004).
[Crossref] [PubMed]

Simoncelli, E. P.

Z. Wang, A. C. Bovik, H. R. Sheikh, and E. P. Simoncelli, “Image quality assessment: from error visibility to structural similarity,” IEEE transactions on image processing 13, 600–612 (2004).
[Crossref] [PubMed]

Sun, J.

K. He, X. Zhang, S. Ren, and J. Sun, “Delving deep into rectifiers: Surpassing human-level performance on imagenet classification,” in The IEEE International Conference on Computer Vision (ICCV), (2015), pp. 1026–1034.

Tang, X.

C. Dong, C. C. Loy, K. He, and X. Tang, “Image super-resolution using deep convolutional networks,” IEEE Transactions on Pattern Analysis Mach. Intell. 38, 295–307 (2016).
[Crossref]

Theuwissen, A.

M. Sarkar, D. S. S. San Segundo Bello, C. van Hoof, and A. Theuwissen, “Integrated polarization analyzing cmos image sensor for material classification,” IEEE Sensors J. 11, 1692–1703 (2011).
[Crossref]

Tyo, J. S.

J. S. Tyo, D. L. Goldstein, D. B. Chenault, and J. A. Shaw, “Review of passive imaging polarimetry for remote sensing applications,” Appl. Opt. 45, 5453–5469 (2006).
[Crossref] [PubMed]

van Hoof, C.

M. Sarkar, D. S. S. San Segundo Bello, C. van Hoof, and A. Theuwissen, “Integrated polarization analyzing cmos image sensor for material classification,” IEEE Sensors J. 11, 1692–1703 (2011).
[Crossref]

Wang, Z.

Z. Wang, A. C. Bovik, H. R. Sheikh, and E. P. Simoncelli, “Image quality assessment: from error visibility to structural similarity,” IEEE transactions on image processing 13, 600–612 (2004).
[Crossref] [PubMed]

Yaroslavskaya, A.

E. Salomatina-Motts, V. Neel, and A. Yaroslavskaya, “Multimodal polarization system for imaging skin cancer,” Opt. Spectrosc. 107, 884–890 (2009).
[Crossref]

Ye, W.

J. Zhang, W. Ye, A. Ahmed, Z. Qiu, Y. Cao, and X. Zhao, “A novel smoothness-based interpolation algorithm for division of focal plane polarimeters,” in 2017 IEEE International Symposium on Circuits and Systems (ISCAS), (IEEE, 2017), pp. 1–4.

Yu, M.

B. Huang, T. Liu, H. Hu, J. Han, and M. Yu, “Underwater image recovery considering polarization effects of objects,” Opt. Express 24, 9826–9838 (2016).
[Crossref] [PubMed]

Zhang, J.

J. Zhang, J. Shao, H. Luo, X. Zhang, B. Hui, Z. Chang, and R. Liang, “Learning a convolutional demosaicing network for microgrid polarimeter imagery,” Opt. Lett. 43, 4534–4537 (2018).
[Crossref] [PubMed]

J. Zhang, H. Luo, R. Liang, A. Ahmed, X. Zhang, B. Hui, and Z. Chang, “Sparse representation-based demosaicing method for microgrid polarimeter imagery,” Opt. Lett. 43, 3265–3268 (2018).
[Crossref] [PubMed]

J. Zhang, H. Luo, B. Hui, and Z. Chang, “Image interpolation for division of focal plane polarimeters with intensity correlation,” Opt. Express 24, 20799–20807 (2016).
[Crossref] [PubMed]

J. Zhang, W. Ye, A. Ahmed, Z. Qiu, Y. Cao, and X. Zhao, “A novel smoothness-based interpolation algorithm for division of focal plane polarimeters,” in 2017 IEEE International Symposium on Circuits and Systems (ISCAS), (IEEE, 2017), pp. 1–4.

Zhang, X.

J. Zhang, H. Luo, R. Liang, A. Ahmed, X. Zhang, B. Hui, and Z. Chang, “Sparse representation-based demosaicing method for microgrid polarimeter imagery,” Opt. Lett. 43, 3265–3268 (2018).
[Crossref] [PubMed]

J. Zhang, J. Shao, H. Luo, X. Zhang, B. Hui, Z. Chang, and R. Liang, “Learning a convolutional demosaicing network for microgrid polarimeter imagery,” Opt. Lett. 43, 4534–4537 (2018).
[Crossref] [PubMed]

K. He, X. Zhang, S. Ren, and J. Sun, “Delving deep into rectifiers: Surpassing human-level performance on imagenet classification,” in The IEEE International Conference on Computer Vision (ICCV), (2015), pp. 1026–1034.

Zhao, H.

H. Zhao, O. Gallo, I. Frosio, and J. Kautz, “Loss functions for neural networks for image processing,” arXiv preprint arXiv:1511.08861 (2015).

Zhao, X.

J. Zhang, W. Ye, A. Ahmed, Z. Qiu, Y. Cao, and X. Zhao, “A novel smoothness-based interpolation algorithm for division of focal plane polarimeters,” in 2017 IEEE International Symposium on Circuits and Systems (ISCAS), (IEEE, 2017), pp. 1–4.

Appl. Opt. (1)

J. S. Tyo, D. L. Goldstein, D. B. Chenault, and J. A. Shaw, “Review of passive imaging polarimetry for remote sensing applications,” Appl. Opt. 45, 5453–5469 (2006).
[Crossref] [PubMed]

IEEE Sensors J. (1)

M. Sarkar, D. S. S. San Segundo Bello, C. van Hoof, and A. Theuwissen, “Integrated polarization analyzing cmos image sensor for material classification,” IEEE Sensors J. 11, 1692–1703 (2011).
[Crossref]

IEEE transactions on image processing (1)

Z. Wang, A. C. Bovik, H. R. Sheikh, and E. P. Simoncelli, “Image quality assessment: from error visibility to structural similarity,” IEEE transactions on image processing 13, 600–612 (2004).
[Crossref] [PubMed]

IEEE Transactions on Pattern Analysis Mach. Intell. (1)

C. Dong, C. C. Loy, K. He, and X. Tang, “Image super-resolution using deep convolutional networks,” IEEE Transactions on Pattern Analysis Mach. Intell. 38, 295–307 (2016).
[Crossref]

Opt. Express (4)

J. Zhang, H. Luo, B. Hui, and Z. Chang, “Image interpolation for division of focal plane polarimeters with intensity correlation,” Opt. Express 24, 20799–20807 (2016).
[Crossref] [PubMed]

B. Huang, T. Liu, H. Hu, J. Han, and M. Yu, “Underwater image recovery considering polarization effects of objects,” Opt. Express 24, 9826–9838 (2016).
[Crossref] [PubMed]

R. Perkins and V. Gruev, “Signal-to-noise analysis of stokes parameters in division of focal plane polarimeters,” Opt. Express 18, 25815–25824 (2010).
[Crossref] [PubMed]

S. Gao and V. Gruev, “Bilinear and bicubic interpolation methods for division of focal plane polarimeters,” Opt. Express 19, 26161–26173 (2011).
[Crossref]

Opt. Lett. (2)

J. Zhang, H. Luo, R. Liang, A. Ahmed, X. Zhang, B. Hui, and Z. Chang, “Sparse representation-based demosaicing method for microgrid polarimeter imagery,” Opt. Lett. 43, 3265–3268 (2018).
[Crossref] [PubMed]

J. Zhang, J. Shao, H. Luo, X. Zhang, B. Hui, Z. Chang, and R. Liang, “Learning a convolutional demosaicing network for microgrid polarimeter imagery,” Opt. Lett. 43, 4534–4537 (2018).
[Crossref] [PubMed]

Opt. Spectrosc. (1)

E. Salomatina-Motts, V. Neel, and A. Yaroslavskaya, “Multimodal polarization system for imaging skin cancer,” Opt. Spectrosc. 107, 884–890 (2009).
[Crossref]

Other (9)

D. H. Goldstein, “Polarimetric characterization of federal standard paints,” in Polarization Analysis, Measurement, and Remote Sensing III, vol. 4133 (International Society for Optics and Photonics, 2000), pp. 112–124.
[Crossref]

Y. Aron and Y. Gronau, “Polarization in the lwir: a method to improve target aquisition,” in Infrared Technology and Applications XXXI, vol. 5783 (International Society for Optics and Photonics, 2005), pp. 653–662.
[Crossref]

S. Gao and V. Gruev, “Gradient based interpolation for division of focal plane polarization imaging sensors,” in 2012 IEEE International Symposium on Circuits and Systems (ISCAS), (IEEE, 2012), pp. 1855–1858.
[Crossref]

O. Ronneberger, P. Fischer, and T. Brox, “U-net: Convolutional networks for biomedical image segmentation,” in Medical Image Computing and Computer-Assisted Intervention (MICCAI), (Springer International Publishing, Cham, 2015), pp. 234–241.

J. Zhang, W. Ye, A. Ahmed, Z. Qiu, Y. Cao, and X. Zhao, “A novel smoothness-based interpolation algorithm for division of focal plane polarimeters,” in 2017 IEEE International Symposium on Circuits and Systems (ISCAS), (IEEE, 2017), pp. 1–4.

H. Zhao, O. Gallo, I. Frosio, and J. Kautz, “Loss functions for neural networks for image processing,” arXiv preprint arXiv:1511.08861 (2015).

K. He, X. Zhang, S. Ren, and J. Sun, “Delving deep into rectifiers: Surpassing human-level performance on imagenet classification,” in The IEEE International Conference on Computer Vision (ICCV), (2015), pp. 1026–1034.

D. P. Kingma and J. Ba, “Adam: A method for stochastic optimization,” arXiv preprint arXiv:1412.6980 (2014).

V. Nair and G. E. Hinton, “Rectified linear units improve restricted boltzmann machines,” in Proceedings of the 27th international conference on machine learning (ICML-10), (2010), pp. 807–814.

Cited By

OSA participates in Crossref's Cited-By Linking service. Citing articles from OSA journals and other participating publishers are listed here.

Alert me when this article is cited.


Figures (8)

Fig. 1
Fig. 1 The schematic diagram of an typical DoFP polarimeter.
Fig. 2
Fig. 2 The architecture diagram of proposed Fork-Net. The transparent blocks in the middle actually correspond to feature maps from different layers.
Fig. 3
Fig. 3 Schematic diagram of the polarization image acquisition experimental device.
Fig. 4
Fig. 4 PSNRs vs training epochs curves of S 0, DoLP and AoP for networks with different architectures. (a) Curves of S   0. (b) Curves of DoLP. (c) Curves of AoP.
Fig. 5
Fig. 5 PSNRs vs training epochs curves of S 0, DoLP and AoP for networks with different kernel sizes. (a) Curves of S   0. (b) Curves of DoLP. (c) Curves of AoP.
Fig. 6
Fig. 6 AoP images which indicates the improvement brought by variance constraint term. First row shows the grayscale images and the second row shows the heatmaps of AoP. (a)AoP images output from Fork-Net trained with common MAE loss function. (b) AoP images output from Fork-Net trained with the proposed customized loss function. (c) Original AoP images.
Fig. 7
Fig. 7 Feature maps output from layers alone the DoLP branch. (Due to the limited space, for the three layers in the middle, only four maps are selected to be shown as examples.)
Fig. 8
Fig. 8 S   0, DoLP grayscale images and AoP heatmaps obtained through different methods. Images in the pink boxes show the magnification of local regions.

Tables (3)

Tables Icon

Table 1 PSNRs of Networks with Different Architectures and Loss Functions on Test Set

Tables Icon

Table 2 PSNRs on Test Set and Parameter Numbers of Fork-Nets with Different Kernel Sizes

Tables Icon

Table 3 The PSNRs for Different Methods on Test Set

Equations (6)

Equations on this page are rendered with MathJax. Learn more.

L o s s = 1 N n = 1 N [ 1 W H ( λ 1 Y ^ S 0 ( n ) Y S 0 ( n ) 1 + λ 2 Y ^ D o L P ( n ) Y D o L P ( n ) 1 + λ 3 Y ^ A o P ( n ) Y A o P ( n ) 1 ) λ 4 log C ]
C = 2 σ ^ A o P σ A o P + ( k M A X I ) 2 σ ^ A o P 2 + σ A o P 2 + ( k M A X I ) 2
I n t e n s i t y = 1 2 ( I 0 + I 45 + I 90 + I 135 )
D o L P = ( I 0 I 90 ) 2 + ( I 45 I 135 ) 2 I n t e n s i t y
A o P = 1 2 arctan ( I 45 I 135 I 0 I 90 )
P S N R = 10 log 10 ( M A X I 2 M S E )

Metrics