Abstract

A 3D light field display typically reconstructs a 3D scene by sampling either the projections of the 3D scene at different depths or the directions of the light rays apparently emitted by the 3D scene and viewed from different eye positions. These light field display methods are potentially capable of rendering correct or nearly correct focus cues and therefore addressing the well-known vergence-accommodation conflict problem plaguing the conventional stereoscopic displays. However, very limited efforts have been made to investigate the effects of light ray sampling on the quality of the rendered focus cues and thus the visual responses of a viewer in light field displays. In this paper, by accounting for both the specifications of a light field display system and the ocular factors of the human visual system, we systematically model and analyze the ray position sampling issue in the reconstruction of the light field and characterize its effect on the quality of the rendered retinal image and on the accommodative response in viewing a 3D light field display. Using a recently developed 3D light field display prototype, we further experimentally validated the effects of ray position sampling on the resolution and accommodative response of a light field display, of which the result matches with theoretical characterization.

© 2019 Optical Society of America under the terms of the OSA Open Access Publishing Agreement

Full Article  |  PDF Article
OSA Recommended Articles
Creating effective focus cues in multi-plane 3D displays

Sowmya Ravikumar, Kurt Akeley, and Martin S. Banks
Opt. Express 19(21) 20940-20952 (2011)

High-performance integral-imaging-based light field augmented reality display using freeform optics

Hekun Huang and Hong Hua
Opt. Express 26(13) 17578-17590 (2018)

Hybrid multi-layer displays providing accommodation cues

Dongyeon Kim, Seungjae Lee, Seokil Moon, Jaebum Cho, Youngjin Jo, and Byoungho Lee
Opt. Express 26(13) 17170-17184 (2018)

References

  • View by:
  • |
  • |
  • |

  1. M. Huebschman, B. Munjuluri, and H. Garner, “Dynamic holographic 3-D image projection,” Opt. Express 11(5), 437–445 (2003).
    [Crossref] [PubMed]
  2. G. E. Favalora, “Volumetric 3D displays and application infrastructure,” Computer 38(8), 37–44 (2005).
    [Crossref]
  3. J. P. Rolland, M. W. Krueger, and A. Goon, “Multifocal planes head-mounted displays,” Appl. Opt. 39(19), 3209–3215 (2000).
    [Crossref] [PubMed]
  4. K. Akeley, S. J. Watt, A. R. Girshick, and M. S. Banks, “A stereo display prototype with multiple focal distances,” ACM Trans. Graph. 23(3), 804–813 (2004).
    [Crossref]
  5. S. Liu and H. Hua, “A systematic method for designing depth-fused multi-focal plane three-dimensional displays,” Opt. Express 18(11), 11562–11573 (2010).
    [Crossref] [PubMed]
  6. X. Hu and H. Hua, “High-resolution optical see-through multi-focal-plane head-mounted display using freeform optics,” Opt. Express 22(11), 13896–13903 (2014).
    [Crossref] [PubMed]
  7. Y. Takaki and N. Nago, “Multi-projection of lenticular displays to construct a 256-view super multi-view display,” Opt. Express 18(9), 8824–8835 (2010).
    [Crossref] [PubMed]
  8. Y. Kajiki, H. Yoshikawa, and T. Honda, “Hologram-like video images by 45-view stereoscopic display,” Proc. SPIE 3012, 154–166 (1997).
    [Crossref]
  9. Y. Takaki, “High-density directional display for generating natural three-dimensional images,” Proc. IEEE 94(3), 654–663 (2006).
    [Crossref]
  10. J. H. Lee, J. Park, D. Nam, S. Y. Choi, D. S. Park, and C. Y. Kim, “Optimal projector configuration design for 300-Mpixel multi-projection 3D display,” Opt. Express 21(22), 26820–26835 (2013).
    [Crossref] [PubMed]
  11. A. Jones, I. McDowall, H. Yamada, M. Bolas, and P. Debevec, “Rendering for an interactive 360° light field display,” ACM Trans. Graph. 26(3), 40 (2007).
    [Crossref]
  12. H. Hua and B. Javidi, “A 3D integral imaging optical see-through head-mounted display,” Opt. Express 22(11), 13484–13491 (2014).
    [Crossref] [PubMed]
  13. W. Song, Y. Wang, D. Cheng, and Y. Liu, “Light field head-mounted display with correct focus cue using micro structure array,” Chin. Opt. Lett. 12(6), 060010–060013 (2014).
    [Crossref]
  14. D. Lanman and D. Luebke, “Near-eye light field displays,” ACM Trans. Graph. 32(6), 1–10 (2013).
    [Crossref]
  15. H. Huang and H. Hua, “High-performance integral-imaging-based light field augmented reality display using freeform optics,” Opt. Express 26(13), 17578–17590 (2018).
    [Crossref] [PubMed]
  16. G. Wetzstein, D. Lanman, W. Heidrich, and R. Raskar, “Layered 3D: tomographic image synthesis for attenuation-based light field and high dynamic range displays,” ACM Trans. Graph. 30(4), 95 (2011).
    [Crossref]
  17. G. Wetzstein, D. Lanman, M. Hirsch, and R. Raskar, “Tensor displays: compressive light field synthesis using multilayer displays with directional backlighting,” ACM Trans. Graph. 31(4), 80 (2012).
    [Crossref]
  18. H. Hua, “Enabling focus cues in head-mounted displays,” Proc. IEEE 105(5), 805–824 (2017).
    [Crossref]
  19. H. Huang and H. Hua, “Systematic characterization and optimization of 3D light field displays,” Opt. Express 25(16), 18508–18525 (2017).
    [Crossref] [PubMed]
  20. T. Okoshi, “Optimum design and depth resolution of lens-sheet and projection-type three-dimensional displays,” Appl. Opt. 10(10), 2284–2291 (1971).
    [Crossref] [PubMed]
  21. F. Jin, J. S. Jang, and B. Javidi, “Effects of device resolution on three-dimensional integral imaging,” Opt. Lett. 29(12), 1345–1347 (2004).
    [Crossref] [PubMed]
  22. Y. M. Kim, K. H. Choi, and S. W. Min, “Analysis on expressible depth range of integral imaging based on degree of voxel overlap,” Appl. Opt. 56(4), 1052–1061 (2017).
    [Crossref] [PubMed]
  23. H. Hiura, K. Komine, J. Arai, and T. Mishina, “Measurement of static convergence and accommodation responses to images of integral photography and binocular stereoscopy,” Opt. Express 25(4), 3454–3468 (2017).
    [Crossref] [PubMed]
  24. J. Schwiegerling, Field Guide to Visual and Ophthalmic Optics (SPIE, 2004).

2018 (1)

2017 (4)

2014 (3)

2013 (2)

2012 (1)

G. Wetzstein, D. Lanman, M. Hirsch, and R. Raskar, “Tensor displays: compressive light field synthesis using multilayer displays with directional backlighting,” ACM Trans. Graph. 31(4), 80 (2012).
[Crossref]

2011 (1)

G. Wetzstein, D. Lanman, W. Heidrich, and R. Raskar, “Layered 3D: tomographic image synthesis for attenuation-based light field and high dynamic range displays,” ACM Trans. Graph. 30(4), 95 (2011).
[Crossref]

2010 (2)

2007 (1)

A. Jones, I. McDowall, H. Yamada, M. Bolas, and P. Debevec, “Rendering for an interactive 360° light field display,” ACM Trans. Graph. 26(3), 40 (2007).
[Crossref]

2006 (1)

Y. Takaki, “High-density directional display for generating natural three-dimensional images,” Proc. IEEE 94(3), 654–663 (2006).
[Crossref]

2005 (1)

G. E. Favalora, “Volumetric 3D displays and application infrastructure,” Computer 38(8), 37–44 (2005).
[Crossref]

2004 (2)

K. Akeley, S. J. Watt, A. R. Girshick, and M. S. Banks, “A stereo display prototype with multiple focal distances,” ACM Trans. Graph. 23(3), 804–813 (2004).
[Crossref]

F. Jin, J. S. Jang, and B. Javidi, “Effects of device resolution on three-dimensional integral imaging,” Opt. Lett. 29(12), 1345–1347 (2004).
[Crossref] [PubMed]

2003 (1)

2000 (1)

1997 (1)

Y. Kajiki, H. Yoshikawa, and T. Honda, “Hologram-like video images by 45-view stereoscopic display,” Proc. SPIE 3012, 154–166 (1997).
[Crossref]

1971 (1)

Akeley, K.

K. Akeley, S. J. Watt, A. R. Girshick, and M. S. Banks, “A stereo display prototype with multiple focal distances,” ACM Trans. Graph. 23(3), 804–813 (2004).
[Crossref]

Arai, J.

Banks, M. S.

K. Akeley, S. J. Watt, A. R. Girshick, and M. S. Banks, “A stereo display prototype with multiple focal distances,” ACM Trans. Graph. 23(3), 804–813 (2004).
[Crossref]

Bolas, M.

A. Jones, I. McDowall, H. Yamada, M. Bolas, and P. Debevec, “Rendering for an interactive 360° light field display,” ACM Trans. Graph. 26(3), 40 (2007).
[Crossref]

Cheng, D.

Choi, K. H.

Choi, S. Y.

Debevec, P.

A. Jones, I. McDowall, H. Yamada, M. Bolas, and P. Debevec, “Rendering for an interactive 360° light field display,” ACM Trans. Graph. 26(3), 40 (2007).
[Crossref]

Favalora, G. E.

G. E. Favalora, “Volumetric 3D displays and application infrastructure,” Computer 38(8), 37–44 (2005).
[Crossref]

Garner, H.

Girshick, A. R.

K. Akeley, S. J. Watt, A. R. Girshick, and M. S. Banks, “A stereo display prototype with multiple focal distances,” ACM Trans. Graph. 23(3), 804–813 (2004).
[Crossref]

Goon, A.

Heidrich, W.

G. Wetzstein, D. Lanman, W. Heidrich, and R. Raskar, “Layered 3D: tomographic image synthesis for attenuation-based light field and high dynamic range displays,” ACM Trans. Graph. 30(4), 95 (2011).
[Crossref]

Hirsch, M.

G. Wetzstein, D. Lanman, M. Hirsch, and R. Raskar, “Tensor displays: compressive light field synthesis using multilayer displays with directional backlighting,” ACM Trans. Graph. 31(4), 80 (2012).
[Crossref]

Hiura, H.

Honda, T.

Y. Kajiki, H. Yoshikawa, and T. Honda, “Hologram-like video images by 45-view stereoscopic display,” Proc. SPIE 3012, 154–166 (1997).
[Crossref]

Hu, X.

Hua, H.

Huang, H.

Huebschman, M.

Jang, J. S.

Javidi, B.

Jin, F.

Jones, A.

A. Jones, I. McDowall, H. Yamada, M. Bolas, and P. Debevec, “Rendering for an interactive 360° light field display,” ACM Trans. Graph. 26(3), 40 (2007).
[Crossref]

Kajiki, Y.

Y. Kajiki, H. Yoshikawa, and T. Honda, “Hologram-like video images by 45-view stereoscopic display,” Proc. SPIE 3012, 154–166 (1997).
[Crossref]

Kim, C. Y.

Kim, Y. M.

Komine, K.

Krueger, M. W.

Lanman, D.

D. Lanman and D. Luebke, “Near-eye light field displays,” ACM Trans. Graph. 32(6), 1–10 (2013).
[Crossref]

G. Wetzstein, D. Lanman, M. Hirsch, and R. Raskar, “Tensor displays: compressive light field synthesis using multilayer displays with directional backlighting,” ACM Trans. Graph. 31(4), 80 (2012).
[Crossref]

G. Wetzstein, D. Lanman, W. Heidrich, and R. Raskar, “Layered 3D: tomographic image synthesis for attenuation-based light field and high dynamic range displays,” ACM Trans. Graph. 30(4), 95 (2011).
[Crossref]

Lee, J. H.

Liu, S.

Liu, Y.

Luebke, D.

D. Lanman and D. Luebke, “Near-eye light field displays,” ACM Trans. Graph. 32(6), 1–10 (2013).
[Crossref]

McDowall, I.

A. Jones, I. McDowall, H. Yamada, M. Bolas, and P. Debevec, “Rendering for an interactive 360° light field display,” ACM Trans. Graph. 26(3), 40 (2007).
[Crossref]

Min, S. W.

Mishina, T.

Munjuluri, B.

Nago, N.

Nam, D.

Okoshi, T.

Park, D. S.

Park, J.

Raskar, R.

G. Wetzstein, D. Lanman, M. Hirsch, and R. Raskar, “Tensor displays: compressive light field synthesis using multilayer displays with directional backlighting,” ACM Trans. Graph. 31(4), 80 (2012).
[Crossref]

G. Wetzstein, D. Lanman, W. Heidrich, and R. Raskar, “Layered 3D: tomographic image synthesis for attenuation-based light field and high dynamic range displays,” ACM Trans. Graph. 30(4), 95 (2011).
[Crossref]

Rolland, J. P.

Song, W.

Takaki, Y.

Y. Takaki and N. Nago, “Multi-projection of lenticular displays to construct a 256-view super multi-view display,” Opt. Express 18(9), 8824–8835 (2010).
[Crossref] [PubMed]

Y. Takaki, “High-density directional display for generating natural three-dimensional images,” Proc. IEEE 94(3), 654–663 (2006).
[Crossref]

Wang, Y.

Watt, S. J.

K. Akeley, S. J. Watt, A. R. Girshick, and M. S. Banks, “A stereo display prototype with multiple focal distances,” ACM Trans. Graph. 23(3), 804–813 (2004).
[Crossref]

Wetzstein, G.

G. Wetzstein, D. Lanman, M. Hirsch, and R. Raskar, “Tensor displays: compressive light field synthesis using multilayer displays with directional backlighting,” ACM Trans. Graph. 31(4), 80 (2012).
[Crossref]

G. Wetzstein, D. Lanman, W. Heidrich, and R. Raskar, “Layered 3D: tomographic image synthesis for attenuation-based light field and high dynamic range displays,” ACM Trans. Graph. 30(4), 95 (2011).
[Crossref]

Yamada, H.

A. Jones, I. McDowall, H. Yamada, M. Bolas, and P. Debevec, “Rendering for an interactive 360° light field display,” ACM Trans. Graph. 26(3), 40 (2007).
[Crossref]

Yoshikawa, H.

Y. Kajiki, H. Yoshikawa, and T. Honda, “Hologram-like video images by 45-view stereoscopic display,” Proc. SPIE 3012, 154–166 (1997).
[Crossref]

ACM Trans. Graph. (5)

K. Akeley, S. J. Watt, A. R. Girshick, and M. S. Banks, “A stereo display prototype with multiple focal distances,” ACM Trans. Graph. 23(3), 804–813 (2004).
[Crossref]

A. Jones, I. McDowall, H. Yamada, M. Bolas, and P. Debevec, “Rendering for an interactive 360° light field display,” ACM Trans. Graph. 26(3), 40 (2007).
[Crossref]

D. Lanman and D. Luebke, “Near-eye light field displays,” ACM Trans. Graph. 32(6), 1–10 (2013).
[Crossref]

G. Wetzstein, D. Lanman, W. Heidrich, and R. Raskar, “Layered 3D: tomographic image synthesis for attenuation-based light field and high dynamic range displays,” ACM Trans. Graph. 30(4), 95 (2011).
[Crossref]

G. Wetzstein, D. Lanman, M. Hirsch, and R. Raskar, “Tensor displays: compressive light field synthesis using multilayer displays with directional backlighting,” ACM Trans. Graph. 31(4), 80 (2012).
[Crossref]

Appl. Opt. (3)

Chin. Opt. Lett. (1)

Computer (1)

G. E. Favalora, “Volumetric 3D displays and application infrastructure,” Computer 38(8), 37–44 (2005).
[Crossref]

Opt. Express (9)

M. Huebschman, B. Munjuluri, and H. Garner, “Dynamic holographic 3-D image projection,” Opt. Express 11(5), 437–445 (2003).
[Crossref] [PubMed]

S. Liu and H. Hua, “A systematic method for designing depth-fused multi-focal plane three-dimensional displays,” Opt. Express 18(11), 11562–11573 (2010).
[Crossref] [PubMed]

X. Hu and H. Hua, “High-resolution optical see-through multi-focal-plane head-mounted display using freeform optics,” Opt. Express 22(11), 13896–13903 (2014).
[Crossref] [PubMed]

Y. Takaki and N. Nago, “Multi-projection of lenticular displays to construct a 256-view super multi-view display,” Opt. Express 18(9), 8824–8835 (2010).
[Crossref] [PubMed]

J. H. Lee, J. Park, D. Nam, S. Y. Choi, D. S. Park, and C. Y. Kim, “Optimal projector configuration design for 300-Mpixel multi-projection 3D display,” Opt. Express 21(22), 26820–26835 (2013).
[Crossref] [PubMed]

H. Hua and B. Javidi, “A 3D integral imaging optical see-through head-mounted display,” Opt. Express 22(11), 13484–13491 (2014).
[Crossref] [PubMed]

H. Huang and H. Hua, “High-performance integral-imaging-based light field augmented reality display using freeform optics,” Opt. Express 26(13), 17578–17590 (2018).
[Crossref] [PubMed]

H. Hiura, K. Komine, J. Arai, and T. Mishina, “Measurement of static convergence and accommodation responses to images of integral photography and binocular stereoscopy,” Opt. Express 25(4), 3454–3468 (2017).
[Crossref] [PubMed]

H. Huang and H. Hua, “Systematic characterization and optimization of 3D light field displays,” Opt. Express 25(16), 18508–18525 (2017).
[Crossref] [PubMed]

Opt. Lett. (1)

Proc. IEEE (2)

H. Hua, “Enabling focus cues in head-mounted displays,” Proc. IEEE 105(5), 805–824 (2017).
[Crossref]

Y. Takaki, “High-density directional display for generating natural three-dimensional images,” Proc. IEEE 94(3), 654–663 (2006).
[Crossref]

Proc. SPIE (1)

Y. Kajiki, H. Yoshikawa, and T. Honda, “Hologram-like video images by 45-view stereoscopic display,” Proc. SPIE 3012, 154–166 (1997).
[Crossref]

Other (1)

J. Schwiegerling, Field Guide to Visual and Ophthalmic Optics (SPIE, 2004).

Cited By

OSA participates in Crossref's Cited-By Linking service. Citing articles from OSA journals and other participating publishers are listed here.

Alert me when this article is cited.


Figures (11)

Fig. 1
Fig. 1 The generalized schematic model of a light field 3D display, along with the illustration of the retinal response of a reconstructed point when the accommodative distance, A, of the eye is (a) the same as, (b) larger than, or (c) smaller than the depth of the reconstructed point. Modified from Fig. 2 in [19].
Fig. 2
Fig. 2 Schematic illustration and results of ray trace based on setup which has (a) infinitely high pixel resolution and (b) limited pixel resolution of the CDP. Not to scale.
Fig. 3
Fig. 3 Accumulated retinal PSFs of viewing LF-3D displays with varying pixel resolutions from infinitely high to 12 arcmins of the CDP, and with varying view densities or equivalently numbers of views from 2by2 to 4by4 views encircled by a 3mm eye pupil.
Fig. 4
Fig. 4 The MTF plots of the retinal image of LF-3D displays with different pixel resolution on CDP ranging from infinitely high to 12 arcmins for reconstructed 3D targets at depth of (a) 1 diopter and (c) 2 diopters with the depth of CDP at 1 diopter. Simulated accumulated retinal PSF and retinal images of a series of Snellen letter ‘E’s corresponding to different pixel resolutions on CDP ranging from infinitely high to 12 arcmins for reconstructed 3D targets at depth of (b) 1 diopter and (d) 2 diopters. PSFs and images not in same scale.
Fig. 5
Fig. 5 Plots of (a) accommodative response curve as a function of accommodation shift and (b) accommodation error as a function of depth shift of reconstruction through a display with a viewing density of 0.57 mm−2 and infinitely high pixel resolution. CDP at 1 diopter.
Fig. 6
Fig. 6 Plots of accommodation error as a function of the depth shift of reconstruction from CDP from −1 to 1.5 diopters by displays of different pixel resolutions with a target spatial resolution of (a) 5, (b) 10 and (c) 15 cpd, respectively. (d) Summarization of average absolute accommodation error. CDP at 1 diopter.
Fig. 7
Fig. 7 Plots of accommodation error as a function of the depth shift of the reconstruction from CDP from −1 to 1.5 diopters by displays of different pixel resolutions from infinitely high to 3 arcmins with their view densities configured at (a) 0.57, (b) 1.27 and (c) 2.26mm−2, respectively. (d) Summarization of average absolute accommodation error. CDP at 1 diopter.
Fig. 8
Fig. 8 (a) Schematic optical layout of an optical see-through LF-3D display system [15] and experimental setup for visual response quantification. (b) Photo of the experimental setup arrangement. (c) Captured image of both display and see-through targets through the combiner.
Fig. 9
Fig. 9 Captured images of the displayed target of Snellen letter ‘E’s rendered at the depth of (a-c) 1 diopter, same as the CDP; and (d-f) 2 diopters, 1 diopter closer to the camera from the CDP, through LF-3D displays corresponding to a pixel resolution of (a, d) 3, (b, e) 6, and (c, f) 12 arcmins on the CDPs, respectively. The camera focused on the same depth as that of the reconstructed target.
Fig. 10
Fig. 10 Captured images of the displayed target with a spatial frequency of 2.5 cycles per degree with (a)-(b) the display pixel resolution set at 3 and 6 arcmins, respectively, while both of the reconstruction plane and the camera focus depths were located at the CDP (1.7 diopters); (c)-(d) the display pixel resolution set at 3 and 6 arcmins, respectively, while both of the reconstruction plane and the camera focus depths were located at 1.2 diopters; (e) the display pixel resolution set at 3 arcmins while the reconstruction plane at 1.2 diopters and the camera focused at 1.4 diopters; and (f) the display pixel resolution set at 6 arcmins while the reconstruction plane at 1.2 diopters and the camera focused at 1.3 diopters.
Fig. 11
Fig. 11 Plots of accommodation error as a function of the depth shift of reconstruction from CDP with different pixel resolution. CDP at 1.7 diopters.

Tables (1)

Tables Icon

Table 1 Specifications of the System

Equations (7)

Equations on this page are rendered with MathJax. Learn more.

P S F L F A c c u ( x ' , y ' , A ) = 1 m = 1 M n = 1 N k = 1 K L ( m , n ) w ( λ k ) s ( d c x m , d c y m ) · m = 1 M n = 1 N k = 1 K L ( m , n ) w ( λ k ) s ( d c x m , d c y n ) | P S F C m n k ( x ' , y ' , A ) | 2 ,
d c x m = ( 2 m M 1 ) 2 Δ d x ; d c y n = ( 2 n N 1 ) 2 Δ d y ,
Δ d c x = Δ d c y = 2 1 π σ v i e w ,
P S F C ( x ' , y ' , A ) = 1 λ 2 z C D P z e y e exp [ j π λ z C D P ( Δ x c ' 2 + Δ y c ' 2 ) ] exp [ j π λ z e y e ( x ' 2 + y ' 2 ) ] P e y e ( x , y ) exp [ j 2 π λ W e y e ( Δ x c ' , Δ y c ' ; x , y ) ] exp [ j π λ ( 1 z C D P 1 A ) ( x 2 + y 2 ) ] exp { j 2 π λ [ ( Δ x c ' z C D P + x ' z e y e ) x + ( Δ y c ' z C D P + y ' z e y e ) y ] } d x d y ,
Δ x c m ' = d c x m Δ z + h x z C D P z C D P + Δ z ; Δ y c n ' = d c y n Δ z + h y z C D P z C D P + Δ z ,
Δ x c s m ' = p c · r o u n d ( Δ x c m ' p c ) ; Δ y c s n ' = p c · r o u n d ( Δ y c s n ' p c ) ,
P S F L F A c c u p ( x ' , y ' , A ) = 1 m = 1 M n = 1 N k = 1 K L ( m , n ) w ( λ k ) s ( d c x m , d c y m ) · m = 1 M n = 1 N k = 1 K L ( m , n ) w ( λ k ) s ( d c x m , d c y n ) r e c t ( x ' z C D P p c z e y e , y ' z C D P p c z e y e ) * | P S F C m n k p ( x ' , y ' , A , Δ x c s m ' , Δ y c s m ' ) | 2 ,

Metrics