## Abstract

We derive the probabilities of the signal OAM state and crosstalk OAM state for a Laguerre-Gaussian (LG) beam propagating through Kolmogorov and Non-Kolmogorov turbulence, and derive the accurate analytical function of the probability for the received OAM state modulated by an arbitrary receiver aperture. The probability of the detected OAM state with a receiver aperture for different values of the radius is demonstrated numerically. Our numerical results show that the probability of the signal OAM state remains almost invariant when the radius of the receiver aperture varies. The probability of the crosstalk OAM state decreases with the decrease of the radius of the receiver aperture, thus it can be optimized by choosing a suitable value of the radius of the receiver aperture. Our results will be useful in free-space optical communications.

© 2018 Optical Society of America under the terms of the OSA Open Access Publishing Agreement

## 1. Introduction

The vortex beams carrying orbital angular momentum (OAM) state are used in free space optical (FSO) communication and fiber optical communication to increase capacity and spectral efficiency [1–3]. In FSO communication, the vortex beams are inevitably disturbed by turbulent atmosphere [4–7], and then the initial OAM state induced by the turbulent atmosphere will be destroyed and spread to the neighbor OAM states [8,9]. Then the adaptive optics is used to compensate the orbital angular momentum of a vortex beam disturbed by turbulent atmosphere [10]. Investigations on the probability of the receiver OAM state affected by atmospheric turbulence are helpful to improve FSO communication. Therefore the probability of the OAM state carried by vortex beams through turbulent atmosphere was studied in detail both theoretically and experimentally in [11–13]. Paterson proposed a semiclassical approach to analyze the probability of OAM state induced by atmospheric turbulence [8]. The rotational field correlation formulation of the vortex beam propagating through atmospheric turbulence was used to predict the change of the OAM state [11]. The power of the transmitted OAM state of vortex beams passing through weak-to-strong atmospheric turbulence spreading to the neighbor OAM state was studied experimentally [12]. The purities of different degradation of the OAM state propagating through simulated atmospheric turbulence were measured, and the results showed that the purity of the OAM states was induced by atmospheric turbulence independent of the transmitted topological charge number [13]. The structure function of the index of refraction and the corresponding power spectrum of Non-Kolmogorov turbulence are different from those of Kolmogorov turbulence, and the non-Kolmogorov turbulence obey arbitrary power laws [14,15]. Average intensity of flattened Gaussian beam in non-Kolmogorov turbulence was explored in [16]. Recently, the spiral spectrum of the OAM states of vortex beams induced by non-Kolmogorov turbulence were studied in [17–19], and the results showed that the probability of OAM state was degraded by non-Kolmogorov turbulence.

The spot size of the beams remains unchanged [20] or increases when they propagate through free space and turbulent atmosphere [21,22]. Usually, the sizes of the elements of the optical system are limited and an aperture is used at the receiver plane. Fried adopted a large aperture to reduce the variance of a fluctuating signal in the space-to-ground communications systems [23]. The OAM state fluctuation averaging with circular-symmetric and non-circular-symmetric apertures were studied in [24], and the results showed that the aperture averaging of OAM fluctuations were affected by the aperture area and the turbulence spectrum. Gibson et al. explored the effects of segment angular apertures on the receiver OAM state of a LG beam, and found that the probability of the receiver OAM states was determined by the size of the segment angular aperture. The high probability of the receiver OAM state was detected by the case of no angular restrictionin [25]. The OAM states remain mutually orthogonal when a restricted angular aperture is used in the received plane for the OAM communications [25,26]. Tyler and Boyd studied the probability of the detected OAM state for the pure vortex beams in turbulent atmosphere restricted by the aperture. The approximate expression of the probability was obtained under the limiting condition of very small or very large radius of an aperture. The probability of the received OAM state for the entire domain of the aperture size was obtained in two asymptotic forms [27]. In this letter, we theoretically studied the probability of the detected OAM state of a LG beam induced by Kolmogorov and Non-Kolmogorov turbulence. The probability of the detected OAM state is restricted by the receiver aperture. The accurate analytical function of the probability for the received OAM state is derived. Under the same conditions, the probability of the detected signal OAM state remains also invariant for the different values of the radius of the receiver aperture, while the probability of the neighbor OAM state decreases with the decrease of the radius of the receiver aperture. Our results show that the receiver aperture can optimize the probability of neighbor OAM state for a LG beam disturbed by atmospheric turbulence.

## 2. The spatial coherence length

In this letter, we explore the probability of the OAM state of a LG beam propagating through Kolmogorov and Non-Kolmogorov turbulence. The spatial coherence length of the aberrations induced by the atmospheric turbulence${\rho}_{0}$ is approximately equal to [28]

*z*is the propagation distance. For non-Kolmogorov turbulence, the spectral density with the van Karman spectrum obeys the following function, in which the slope 11/3 is generalized to an arbitrary parameter$\alpha $, as [14]

On substituting Eq. (2) into Eq. (1), we obtain the spatial coherence length${\rho}_{0NT}$of Non-Kolmogorov turbulence as follows

Under the condition of $\alpha =11/3$,${L}_{0}=\infty $and ${d}_{0}=0$, the spatial coherence length${\rho}_{0NT}$of Non-Kolmogorov turbulence reduces to the spatial coherence length${\rho}_{0T}$of a spherical wave propagating through Kolmogorov turbulence as follows [29]

Here ${C}_{n}^{2}$ is the structure constant with units ${\text{m}}^{-2/3}$## 3. Probability of the received OAM state

Recently, the OAM state of a LG beam is used for mode-division multiplexing (MDM) in FSO communication. The OAM beam inevitably encounters atmospheric turbulence after propagation for long distance. The electric field will be disturbed by atmospheric turbulence, the cumulative effect induced by turbulence suppose as a pure phase perturbation on the OAM beam, as follows [8]

*l*

_{0}is the initial quantum number of the incident LG beam.

In general, the OAM state carried by a LG beam is mutually orthogonal. The received field can be described by a superposition of orthogonal basis of LG beams,

When the OAM beam propagate through Kolmogorov and Non-Kolmogorov turbulence, the initial OAM state will spread to the neighbor OAM states. The spot size of the beam increases when the topological charge of the OAM state increases. At received plane, we use a circular aperture to filter the crosstalk from the neighbor OAM states. The function of a circular aperture can be obtained by summing the complex Gaussian functions as follows [30,31]

*r*is the polar coordinates at received plane,

*a*denotes the radius of a circle aperture,

*A*and

_{t}*B*are the expansion and Gaussian coefficients, respectively.

_{t}When a circular aperture is used at the received plane, Eq. (9) can be expressed as follows

Applying the Rytov approximation, the phase perturbation function obeys Gaussian random process of wave structure function in turbulence [8,9], on substituting Eqs. (6) and (10) into Eq. (11), we can obtain

The amplitude distribution of LG beam propagating through free space without turbulence at received plane can be obtained as follows [29]

## 4. Numerical analysis

In this section, we study the probability of the OAM state of a LG beam propagating through Kolmogorov and Non-Kolmogorov turbulence. A circular receiver aperture is used to reduce probability of crosstalk from the neighbor OAM states. Figure 1 shows the schematic of the OAM state disturbed by a circular receiver aperture and turbulent atmosphere. From Fig. 1, there are many OAM states at the received plane, although the incident beam only has one OAM state. This phenomenon can be explained by the fact that the OAM state disturbed by atmospheric turbulence spreads to the neighbor OAM states. The initial OAM state will seriously spread to neighbor OAM states when atmospheric turbulence is strong and serious crosstalk occurs. A circular receiver aperture is used to filter the crosstalk.

Figures 2 and 3 show the probability of the OAM state of a LG beam propagating through Kolmogorov and Non-Kolmogorov turbulence optimized by a receiver aperture for different values of the radius *a*. In the following numerical examples, we chose the parameters for turbulence and the beam as ${\tilde{C}}_{n}^{2}=5\times {10}^{-15}{\text{m}}^{3-\alpha}$, *L*_{0} = 1m, *d*_{0} = 1mm,$\alpha =3.1$,$\lambda =1550\text{n}m$,${l}_{0}=3$and${w}_{0}=25mm$. From Fig. 2, one can see that the probability of the received OAM state of Δ*l* = 0 in Kolmogorov and Non-Kolmogorov turbulence remains almost unchanged within one kilometer and then decreases. The probability of the received OAM states of Δ*l* = (1, 2, and 3), that is the neighbor OAM states, in Kolmogorov and Non-Kolmogorov turbulence increases with the increase of the propagation distance. The probability of the received OAM states of Δ*l* = (1, 2, and 3) increases to the maximum value and then decreases with the further increase of the propagation distance. The probability of the OAM state for the topological charge difference Δ*l* = 1 is larger than the other topological charge differences. The probability of the OAM state with large topological charge difference is larger than that of those with small topological charge differences. The probability of the received OAM states reaches to the same value when the radius of an aperture is infinity in the far field (i.e., non-optimized results). From Figs. 2 and 3, the probability of the OAM states for Δ*l* = (1, 2, and 3) without modulation by the receiver aperture are larger than the probability modulated by the receiver aperture. The probability of the OAM states for Δ*l* = (1, 2, and 3) modulated by a receiver aperture with the small radius is smaller than an aperture with the large radius. Here the distance of 1 km seems be to a critical point. In fact, the critical value is closely related with the structure constant of the Kolmogorov or Non-Kolmogorov turbulence. The effect of the turbulence on the probability of the OAM state accumulates on propagation [29]. Within the distance of 1km, the probability of the crosstalk OAM state for a LG beam affected by Kolmogorov and Non-Kolmogorov turbulence with in the distance of 1 km is small.

Figures 4 and 5 show 3D-probabilities of the OAM states of a LG beam propagating through Kolmogorov and Non-Kolmogorov turbulence versus the propagation distance and the radius of a receiver aperturefor different values of the topological charge difference. In the near field, the probability of the received OAM state with Δ*l* = 0 increases quickly when the radius of a receiver aperture increases. The probability of the OAM state will reach to the maximum value and remain unchanged with the further increase of the radius of the receiver aperture (see Fig. 4(a) and 5(a)). From Figs. 4 and 5, the increasing speed of the probability of the OAM state with Δ*l* = 1 is larger than that of the OAM state with Δ*l* = (2 and 3) for large values of the radius of the receiver aperture. In the far field, the increasing speed of the probability of the OAM state with Δ*l* = 0 is smaller than that of the OAM state with Δ*l* = (1, 2 and 3) when the radius of the receiver aperture increases. When the radius of a receiver aperture is fixed, the probability of OAM state with Δ*l* = 0 will decrease with the increase of the propagation distance. The probability of OAM states with Δ*l* = (1, 2 and 3) will increase as the propagation distance increases. Then the probability will decrease when the propagation distance increases further. However at the same conditions, the probability of the neighbor OAM states for Δ*l* = 1 is the maximum. The probabilities of the different neighbor OAM states decrease when the radius of the receiver aperture decreases.

Figure 6 shows 3D-probability of LG beam propagating through Non-Kolmogorov turbulence versus an arbitrary parameter and the radius of a receiver aperture for different values of the topological charge difference. In the numerical calculation, we set the propagation distance as 1km. When the radius of the receiver aperture increases or the arbitrary parameter decreases, the probability of the OAM state with Δ*l* = 0 for a LG beam propagating through Non-Kolmogorov turbulence will increase, and the probability will remain unchanged when the radius of the receiver aperture increases further (see Fig. 6(a)). When the arbitrary parameter and the radius of the receiver aperture increase, the probabilities of the OAM states with Δ*l* = (1, 2 and 3) will increase and the probability of the OAM state with Δ*l* = 0 will decrease. From Figs. 2 to 6, we find that we can choose suitable radius of the receiver aperture to relieve the probabilities of the neighbor OAM states in Kolmogorov or Non-Kolmogorov turbulence.

## 5. Conclusion

In summary, the probability of the received OAM state of a LG beam induced by Kolmogorov and Non-Kolmogorov turbulence has been derived, and the receiver aperture has been used to optimize the probability of the crosstalk modes. Our numerical results have shown that the low probability of the crosstalk OAM state correspond to small radius of the receiver aperture. The probability of the signal OAM state will approach to the same value as the radius of the receiver aperture increases or decreases. The probability of the signal OAM state decreases and the probability of the crosstalk OAM state increases when the propagation distance and an arbitrary parameter increase. Our results will be useful for FSO communications.

## Funding

National Natural Science Fund for Distinguished Young Scholar (11525418); National Natural Science Foundation of China (91750201 & 11747065); The Fundamental Research Funds for the Open Research Fund of State Key Laboratory of Transient Optics and Photonics of Chinese Academy of Sciences (SKLST201608).

## References and links

**1. **J. Wang, J. Yang, I. M. Fazal, N. Ahmed, Y. Yan, H. Huang, Y. Ren, Y. Yue, S. Dolinar, M. Tur, and A. E. Willner, “Terabit free-space data transmission employing orbital angular momentum multiplexing,” Nat. Photonics **6**(7), 488–496 (2012). [CrossRef]

**2. **N. Bozinovic, Y. Yue, Y. Ren, M. Tur, P. Kristensen, H. Huang, A. E. Willner, and S. Ramachandran, “Terabit-Scale Orbital Angular Momentum Mode Division Multiplexing in Fibers,” Science **340**(6140), 1545–1548 (2013). [CrossRef] [PubMed]

**3. **T. Lei, M. Zhang, Y. Li, P. Jia, G. N. Liu, X. Xu, Z. Li, C. Min, J. Lin, C. Yu, H. Niu, and X. Yuan, “Massive individual orbital angular momentum channels for multiplexing enabled by Dammann gratings,” Light Sci. **4**(3), e257 (2015). [CrossRef]

**4. **Y. Yuan, T. Lei, Z. Li, Y. Li, S. Gao, Z. Xie, and X. Yuan, “Beam wander relieved orbital angular momentum communication in turbulent atmosphere using Bessel beams,” Sci. Rep. **7**(1), 42276 (2017). [CrossRef] [PubMed]

**5. **J. Wang, “Advances in communications using optical vortices,” Photon. Res. **4**(5), B14–B28 (2016). [CrossRef]

**6. **J. A. Anguita, M. A. Neifeld, and B. V. Vasic, “Turbulence-induced channel crosstalk in an orbital angular momentum-multiplexed free-space optical link,” Appl. Opt. **47**(13), 2414–2429 (2008). [CrossRef] [PubMed]

**7. **M. Krenn, R. Fickler, M. Fink, J. Handsteiner, M. Malik, T. Scheidl, R. Ursin, and A. Zeilinger, “Communication with spatially modulated light through turbulent air across Vienna,” New J. Phys. **16**(11), 113028 (2014). [CrossRef]

**8. **C. Paterson, “Atmospheric turbulence and orbital angular momentum of single photons for optical communication,” Phys. Rev. Lett. **94**(15), 153901 (2005). [CrossRef] [PubMed]

**9. **Y. Yuan, T. Lei, S. Gao, X. Weng, L. Du, and X. Yuan, “The Orbital Angular Momentum Spreading for Cylindrical Vector Beams in Turbulent Atmosphere,” IEEE Photonics J. **9**(2), 6100610 (2017). [CrossRef]

**10. **N. Li, X. Chu, P. Zhang, X. Feng, C. Fan, and C. Qiao, “Compensation for the orbital angular momentum of a vortex beam in turbulent atmosphere by adaptive optics,” Opt. Laser Technol. **98**, 7–11 (2018). [CrossRef]

**11. **M. Nairat and D. Voelz, “Propagation of rotational field correlation through atmospheric turbulence,” Opt. Lett. **39**(7), 1838–1840 (2014). [CrossRef] [PubMed]

**12. **Y. Ren, H. Huang, G. Xie, N. Ahmed, Y. Yan, B. I. Erkmen, N. Chandrasekaran, M. P. J. Lavery, N. K. Steinhoff, M. Tur, S. Dolinar, M. Neifeld, M. J. Padgett, R. W. Boyd, J. H. Shapiro, and A. E. Willner, “Atmospheric turbulence effects on the performance of a free space optical link employing orbital angular momentum multiplexing,” Opt. Lett. **38**(20), 4062–4065 (2013). [CrossRef] [PubMed]

**13. **B. Rodenburg, M. P. J. Lavery, M. Malik, M. N. O’Sullivan, M. Mirhosseini, D. J. Robertson, M. Padgett, and R. W. Boyd, “Influence of atmospheric turbulence on states of light carrying orbital angular momentum,” Opt. Lett. **37**(17), 3735–3737 (2012). [CrossRef] [PubMed]

**14. **I. Toselli, L. C. Andrews, R. L. Phillips, and V. Ferrero, “Free-space optical system performance for laser beam propagation through non-Kolmogorov turbulence,” Proc. SPIE **6457**, 64570T (2007). [CrossRef]

**15. **V. P. Lukin, E. V. Nosov, V. V. Nosov, and A. V. Torgaev, “Causes of non-Kolmogorov turbulence in the atmosphere,” Appl. Opt. **55**(12), B163–B168 (2016). [CrossRef] [PubMed]

**16. **X. Chu, C. Qiao, and X. Feng, “Average intensity of flattened Gaussian beam in non-Kolmogorov turbulence,” Opt. Laser Technol. **43**(7), 1150–1154 (2011). [CrossRef]

**17. **J. Ou, Y. Jiang, J. Zhang, H. Tang, Y. He, S. Wang, and J. Liao, “Spreading of spiral spectrum of Bessel–Gaussian beam in non-Kolmogorov turbulence,” Opt. Commun. **318**, 95–99 (2014). [CrossRef]

**18. **M. Wei, J. Wang, Y. Zhang, and Z. Hu, “Orbital-angular-momentum photons for optical communication in non-Kolmogorov atmospheric turbulence,” Opt. Commun. **416**, 89–93 (2018). [CrossRef]

**19. **Y. Zhu, L. Zhang, Z. Hu, and Y. Zhang, “Effects of non-Kolmogorov turbulence on the spiral spectrum of Hypergeometric-Gaussian laser beams,” Opt. Express **23**(7), 9137–9146 (2015). [CrossRef] [PubMed]

**20. **K. Zhang, Y. Yuan, D. Zhang, X. Ding, B. Ratni, S. N. Burokur, M. Lu, K. Tang, and Q. Wu, “Phase-engineered metalenses to generate converging and non-diffractive vortex beam carrying orbital angular momentum in microwave region,” Opt. Express **26**(2), 1351–1360 (2018). [CrossRef] [PubMed]

**21. **W. H. Carter, “Spot size and divergence for Hermite Gaussian beams of any order,” Appl. Opt. **19**(7), 1027–1029 (1980). [CrossRef] [PubMed]

**22. **F. Wang and Y. Cai, “Second-order statistics of a twisted gaussian Schell-model beam in turbulent atmosphere,” Opt. Express **18**(24), 24661–24672 (2010). [CrossRef] [PubMed]

**23. **D. L. Fried, “Aperture Averaging of Scintillation,” J. Opt. Soc. Am. A **57**(2), 169–175 (1967). [CrossRef]

**24. **M. Charnotskii, “Turbulence effects on fluctuations of the aperture-averaged orbital angular momentum,” J. Opt. Soc. Am. A **35**(5), 702–711 (2018). [CrossRef] [PubMed]

**25. **G. Gibson, J. Courtial, M. Padgett, M. Vasnetsov, V. Pas’ko, S. Barnett, and S. Franke-Arnold, “Free-space information transfer using light beams carrying orbital angular momentum,” Opt. Express **12**(22), 5448–5456 (2004). [CrossRef] [PubMed]

**26. **S. Zheng, X. Hui, J. Zhu, H. Chi, X. Jin, S. Yu, and X. Zhang, “Orbital angular momentum mode-demultiplexing scheme with partial angular receiving aperture,” Opt. Express **23**(9), 12251–12257 (2015). [CrossRef] [PubMed]

**27. **G. A. Tyler and R. W. Boyd, “Influence of atmospheric turbulence on the propagation of quantum states of light carrying orbital angular momentum,” Opt. Lett. **34**(2), 142–144 (2009). [CrossRef] [PubMed]

**28. **E. Shchepakina and O. Korotkova, “Second-order statistics of stochastic electromagnetic beams propagating through non-Kolmogorov turbulence,” Opt. Express **18**(10), 10650–10658 (2010). [CrossRef] [PubMed]

**29. **L. C. Andrews and R. L. Phillips, *Laser beam propagation in the turbulent atmosphere*, (2nd edition, SPIE press, Bellington, 2005).

**30. **D. Ding and X. Liu, “Approximate description for Bessel, Bessel–Gauss, and Gaussian beams with finite aperture,” J. Opt. Soc. Am. A **16**(6), 1286–1293 (1999). [CrossRef]

**31. **Y. Cai and L. Hu, “Propagation of partially coherent twisted anisotropic Gaussian Schell-model beams through an apertured astigmatic optical system,” Opt. Lett. **31**(6), 685–687 (2006). [CrossRef] [PubMed]

**32. **S. Gradshteyn and I. M. Ryzhik, *Table of Integrals, Series, and Products*, 7^{th} ed. (Elsevier Academic Press Burlington, 2000).