# Statistical Tools for Imaging Atmospheric Cherenkov Telescopes

## Abstract

**:**

## 1. Introduction

## 2. Event Reconstruction Techniques

- the energy of the primary gamma ray that initiated the shower,
- its arrival direction,
- and one or more discriminating variables.

**Figure 2.**Difference between the images of gamma-induced (

**left**) and hadron-induced (

**right**) showers in the camera of a IACT. Reprinted with permission from Ref. [12]. Copyright 2009 Völk et al.

#### 2.1. Hillas Method

#### 2.2. Semi-Analytical Method

#### 2.3. 3D-Gaussian Model

#### 2.4. MC Template-Based Analysis

#### 2.5. Multivariate Analysis

- nonlinear correlations between the parameters are taken into account and
- those parameters with no discrimination power are ignored.

#### 2.6. Deep Learning Methods

## 3. Detection Significance and Background Modeling

- zero or negligible relative to the source counts,
- known precisely,
- estimated from an OFF measurement.

#### 3.1. The Background Is Zero or Negligible

#### 3.2. The Background Is Known Precisely

#### 3.3. The Background Is Estimated from an OFF Measurement

- On-Off background: the OFF counts are taken from (usually consecutive) observations made under identical conditions, meaning that $\alpha $ is simply given by ${t}_{on}/{t}_{off}$ with ${t}_{on}$ and ${t}_{off}$ the exposure time for the ON and OFF observation, respectively. The main advantage of this method is that no assumption is required for the acceptance, except that it is the same for the ON and OFF regions. The drawback of this approach is that dedicated OFF observations are needed, thus “stealing” time from the on-source ones.
- Wobble or reflected-region background: the OFF counts are taken from regions located, on a run-by-run basis, at identical distances from the center of the field of view. Each of the OFF regions is obtained by reflecting the ON region with respect to the FoV center. This is the reason this method is called the reflected-region method. If we have n OFF regions then $\alpha $ is equal to $1/n$. This technique was originally applied to wobble observations [43] and was later on used also in other observation modes.
- Ring background: the OFF counts are taken from a ring around the ROI or around the center of the field of view.
- Template background: the OFF counts are given by those events that have been discarded in the signal extraction selection based on a discriminating variable. In this method, first developed for the HEGRA experiment [44] and more recently refined for HESS [45], the discarded events are used to template the background.

#### 3.4. Bounds, Confidence and Credible Intervals

## 4. Flux Estimation and Model Parameter Inference

#### 4.1. Unfolding

#### 4.2. Forward Folding

## 5. Discussion

## Funding

## Institutional Review Board Statement

## Informed Consent Statement

## Data Availability Statement

## Acknowledgments

## Conflicts of Interest

## Abbreviations

BDT | Boosted Decision Tree |

BF | Bayes Factor |

CDF | Cumulative Distribution Function |

CI | Confidence Interval |

CL | Confidence Level |

CNN | Convolutional Neural Networks |

DL | Deep Learning |

IACT | Imaging Atmospheric Cherenkov Telecope |

ImPACT | Image Pixel-wise Atmospheric Cherenkov Telescopes |

IRF | Instrument Response Function |

LL | Lower Limit |

MC | Monte Carlo |

Probability Distribution Function | |

PhE | Photo-Electron |

PMF | Probability Mass Function |

PSF | Point Spread Function |

RF | Random Forest |

ROI | Region of Interest |

UL | Upper Limit |

## Notes

1 | Hereafter throughout the paper the symbol $\mathcal{S}$ is used for the statistic, while the generic symbol $p\left(\right)$ is used to indicate all probability density functions (PDFs) and probability mass functions (PMFs) (the former applies to continuous variables and the latter to discrete variables). |

2 | By convention $\alpha $ is the probability of making a type I error, i.e., rejecting a hypothesis that is true. It is also refereed as the statistical significance or p-value. |

3 | Here we are assuming that the analyzer would make this conclusion every time that the observed statistic falls above the 95th-percentile of the statistic distribution. |

4 | The misuse and misinterpretation of statistical tests in the scientific community led the American Statistical Association (ASA) to release in 2016 a statement [5] on the correct use of statistical significance and p-values. |

5 | By nuisance parameters we mean parameters that are not of interest but must be accounted for. |

6 | The sim_telarray code is a program that given as input a complete set of photon bunches simulates the camera response of the telescope. |

7 | In IACTs a run is generally referred to as a data taking (lasting roughly half an hour) performed on a given target under the same conditions throughout the entire observation. |

8 | On the one hand it is important to train the classifier to maximize the separation between the signal and background, and on the other it is also crucial to avoid overtraining (also referred to as overfitting), i.e., avoiding the classifier to characterize statistical fluctuations from the training samples wrongly as true features of the event classes. |

9 | One can check that by computing $\sqrt{\mathcal{S}}$ one would get the same value of Equation (16). That is because $\mathcal{S}$ is a ${\chi}^{2}$ random variable. |

10 | See for instance Ref. [42] for a review of the problem regarding the choice of the priors. |

11 | Indeed one can check that Equation (26) yields $\widehat{\widehat{b}}={N}_{off}$ when $s={N}_{on}-\alpha {N}_{off}$. |

12 | If $s=0$ then $\widehat{\widehat{b}}=({N}_{on}+{N}_{off})/(1+\alpha )$ and the term $(1+\alpha )\widehat{\widehat{b}}-{N}_{on}-{N}_{off}$ in Equation (27) vanishes. |

13 | PSF stands for Point Spread Function. See Section 4 for its definition. |

14 | The CDF of a ${\chi}^{2}$ distribution with 1 degree of freedom is 0.68 for ${\chi}^{2}=1$ and 0.95 for ${\chi}^{2}=3.84$. |

15 | Another way to choose ${s}_{1}$ and ${s}_{2}$ is to guarantee that the mean is the central value of the interval $[{s}_{1},{s}_{2}]$. In principle, one is free to pick up infinitely intervals from the constraint given by Equation (41). A more detailed discussion on this topic can be found in Ref. [65]. |

16 | Generally the performance of the energy and direction reconstruction only depends on the event true energy and arrival direction, which justifies the assumption that ${E}_{r}$ and ${\widehat{\mathbf{n}}}_{r}$ are conditional independent variables. |

17 | For a more accurate discussion that includes also other variables (such as the photon direction $\widehat{\mathbf{n}}$) one can check Ref. [66]. |

## References

- Aleksić, J.; Ansoldi, S.; Antonelli, L.A.; Antoranz, P.; Babic, A.; Bangale, P.; Barceló, M.; Barrio, J.; González, J.B.; Bednarek, W.; et al. The major upgrade of the MAGIC telescopes, Part II: A performance study using observations of the Crab Nebula. Astropart. Phys.
**2016**, 72, 76–94. [Google Scholar] [CrossRef] [Green Version] - Hinton, J. The status of the HESS project. New Astron. Rev.
**2004**, 48, 331–337. [Google Scholar] [CrossRef] [Green Version] - Holder, J.; Atkins, R.; Badran, H.; Blaylock, G.; Bradbury, S.; Buckley, J.; Byrum, K.; Carter-Lewis, D.; Celik, O.; Chow, Y.; et al. The first VERITAS telescope. Astropart. Phys.
**2006**, 25, 391–401. [Google Scholar] [CrossRef] [Green Version] - Actis, M.; Agnetta, G.; Aharonian, F.; Akhperjanian, A.; Aleksić, J.; Aliu, E.; Allan, D.; Allekotte, I.; Antico, F.; Antonelli, L.; et al. Design concepts for the Cherenkov Telescope Array CTA: An advanced facility for ground-based high-energy gamma-ray astronomy. Exp. Astron.
**2011**, 32, 193–316. [Google Scholar] [CrossRef] [Green Version] - Wasserstein, R.L.; Lazar, N.A. The ASA Statement on p-Values: Context, Process, and Purpose. Am. Stat.
**2016**, 70, 129–133. [Google Scholar] [CrossRef] [Green Version] - Neyman, J.; Pearson, E.S., IX. On the problem of the most efficient tests of statistical hypotheses. Philos. Trans. R. Soc. Lond. Ser. A Contain. Pap. A Math. Phys. Character
**1933**, 231, 289–337. [Google Scholar] - Wilks, S. The Large-Sample Distribution of the Likelihood Ratio for Testing Composite Hypotheses. Ann. Math. Stat.
**1938**, 9, 60–62. [Google Scholar] [CrossRef] - Protassov, R.; Van Dyk, D.A.; Connors, A.; Kashyap, V.L.; Siemiginowska, A. Statistics, handle with care: Detecting multiple model components with the likelihood ratio test. Astrophys. J.
**2002**, 571, 545. [Google Scholar] [CrossRef] [Green Version] - Loredo, T.J. From Laplace to supernova SN 1987A: Bayesian inference in astrophysics. In Maximum Entropy and Bayesian Methods; Springer: Berlin/Heidelberg, Germany, 1990; pp. 81–142. [Google Scholar]
- Lyons, L. Open statistical issues in particle physics. Ann. Appl. Stat.
**2008**, 2, 887–915. [Google Scholar] [CrossRef] - van de Schoot, R.; Depaoli, S.; King, R.; Kramer, B.; Märtens, K.; Tadesse, M.G.; Vannucci, M.; Gelman, A.; Veen, D.; Willemsen, J.; et al. Bayesian statistics and modelling. Nat. Rev. Methods Prim.
**2021**, 1, 1–26. [Google Scholar] [CrossRef] - Völk, H.J.; Bernlöhr, K. Imaging very high energy gamma-ray telescopes. Exp. Astron.
**2009**, 25, 173–191. [Google Scholar] [CrossRef] [Green Version] - Hillas, A.M. Cerenkov Light Images of EAS Produced by Primary Gamma Rays and by Nuclei. In Proceedings of the 19th International Cosmic Ray Conference (ICRC19), San Diego, CA, USA, 11–23 August 1985; Volume 3, p. 445. [Google Scholar]
- Zanin, R.; Carmona, E.; Sitarek, J.; Colin, P.; Frantzen, K.; Gaug, M.; Lombardi, S.; Lopez, M.; Moralejo, A.; Satalecka, K.; et al. MARS, the MAGIC analysis and reconstruction software. In Proceedings of the 33st International Cosmic Ray Conference, Rio de Janeiro, Brasil, 2–9 July 2013. [Google Scholar]
- Aharonian, F.; Akhperjanian, A.; Bazer-Bachi, A.; Beilicke, M.; Benbow, W.; Berge, D.; Bernlöhr, K.; Boisson, C.; Bolz, O.; Borrel, V.; et al. Observations of the Crab nebula with HESS. Astron. Astrophys.
**2006**, 457, 899–915. [Google Scholar] [CrossRef] - Le Bohec, S.; Degrange, B.; Punch, M.; Barrau, A.; Bazer-Bachi, R.; Cabot, H.; Chounet, L.; Debiais, G.; Dezalay, J.; Djannati-Ataï, A.; et al. A new analysis method for very high definition Imaging Atmospheric Cherenkov Telescopes as applied to the CAT telescope. Nucl. Instrum. Methods Phys. Res. Sect. A Accel. Spectrometers Detect. Assoc. Equip.
**1998**, 416, 425–437. [Google Scholar] [CrossRef] [Green Version] - De Naurois, M.; Rolland, L. A high performance likelihood reconstruction of γ-rays for imaging atmospheric Cherenkov telescopes. Astropart. Phys.
**2009**, 32, 231–252. [Google Scholar] [CrossRef] [Green Version] - Kertzman, M.; Sembroski, G. Computer simulation methods for investigating the detection characteristics of TeV air Cherenkov telescopes. Nucl. Instrum. Methods Phys. Res. Sect. A Accel. Spectrometers Detect. Assoc. Equip.
**1994**, 343, 629–643. [Google Scholar] [CrossRef] - Heck, D.; Knapp, J.; Capdevielle, J.; Schatz, G.; Thouw, T. CORSIKA: A Monte Carlo code to simulate extensive air showers. Rep. Fzka
**1998**, 6019. [Google Scholar] [CrossRef] - Lemoine-Goumard, M.; Degrange, B.; Tluczykont, M. Selection and 3D-reconstruction of gamma-ray-induced air showers with a stereoscopic system of atmospheric Cherenkov telescopes. Astropart. Phys.
**2006**, 25, 195–211. [Google Scholar] [CrossRef] [Green Version] - Naumann-Godó, M.; Lemoine-Goumard, M.; Degrange, B. Upgrading and testing the 3D reconstruction of gamma-ray air showers as observed with an array of Imaging Atmospheric Cherenkov Telescopes. Astropart. Phys.
**2009**, 31, 421–430. [Google Scholar] [CrossRef] [Green Version] - Becherini, Y.; Djannati-Ataï, A.; Marandon, V.; Punch, M.; Pita, S. A new analysis strategy for detection of faint γ-ray sources with Imaging Atmospheric Cherenkov Telescopes. Astropart. Phys.
**2011**, 34, 858–870. [Google Scholar] [CrossRef] [Green Version] - Parsons, R.; Hinton, J. A Monte Carlo template based analysis for air-Cherenkov arrays. Astropart. Phys.
**2014**, 56, 26–34. [Google Scholar] [CrossRef] [Green Version] - Bernlöhr, K. Simulation of imaging atmospheric Cherenkov telescopes with CORSIKA and sim_telarray. Astropart. Phys.
**2008**, 30, 149–158. [Google Scholar] [CrossRef] [Green Version] - Vincent, S. A Monte Carlo template-based analysis for very high definition imaging atmospheric Cherenkov telescopes as applied to the VERITAS telescope array. arXiv
**2015**, arXiv:1509.01980. [Google Scholar] - Holler, M.; Lenain, J.P.; de Naurois, M.; Rauth, R.; Sanchez, D. A run-wise simulation and analysis framework for Imaging Atmospheric Cherenkov Telescope arrays. Astropart. Phys.
**2020**, 123, 102491. [Google Scholar] [CrossRef] - Bock, R.; Chilingarian, A.; Gaug, M.; Hakl, F.; Hengstebeck, T.; Jiřina, M.; Klaschka, J.; Kotrč, E.; Savickỳ, P.; Towers, S.; et al. Methods for multidimensional event classification: A case study using images from a Cherenkov gamma-ray telescope. Nucl. Instrum. Methods Phys. Res. Sect. A Accel. Spectrometers Detect. Assoc. Equip.
**2004**, 516, 511–528. [Google Scholar] [CrossRef] - Breiman, L.; Friedman, J.H.; Olshen, R.A.; Stone, C.J. Classification and Regression Trees; Routledge: Abingdon, UK, 2017. [Google Scholar]
- Ohm, S.; van Eldik, C.; Egberts, K. γ/hadron separation in very-high-energy γ-ray astronomy using a multivariate analysis method. Astropart. Phys.
**2009**, 31, 383–391. [Google Scholar] [CrossRef] - Fiasson, A.; Dubois, F.; Lamanna, G.; Masbou, J.; Rosier-Lees, S. Optimization of multivariate analysis for IACT stereoscopic systems. Astropart. Phys.
**2010**, 34, 25–32. [Google Scholar] [CrossRef] [Green Version] - Krause, M.; Pueschel, E.; Maier, G. Improved γ/hadron separation for the detection of faint γ-ray sources using boosted decision trees. Astropart. Phys.
**2017**, 89, 1–9. [Google Scholar] [CrossRef] [Green Version] - Albert, J.; Aliu, E.; Anderhub, H.; Antoranz, P.; Armada, A.; Asensio, M.; Baixeras, C.; Barrio, J.; Bartko, H.; Bastieri, D.; et al. Implementation of the random forest method for the imaging atmospheric Cherenkov telescope MAGIC. Nucl. Instrum. Methods Phys. Res. Sect. A Accel. Spectrometers Detect. Assoc. Equip.
**2008**, 588, 424–432. [Google Scholar] [CrossRef] [Green Version] - Colin, P.; Tridon, D.B.; Carmona, E.; De Sabata, F.; Gaug, M.; Lombardi, S.; Majumdar, P.; Moralejo, A.; Scalzotto, V.; Sitarek, J. Performance of the MAGIC telescopes in stereoscopic mode. arXiv
**2009**, arXiv:0907.0960. [Google Scholar] - Goodfellow, I.; Bengio, Y.; Courville, A. Deep Learning; MIT Press: Cambridge, MA, USA, 2016. [Google Scholar]
- Feng, Q.; Lin, T.T.; VERITAS Collaboration. The analysis of VERITAS muon images using convolutional neural networks. Proc. Int. Astron. Union
**2016**, 12, 173–179. [Google Scholar] [CrossRef] [Green Version] - LeCun, Y.; Bengio, Y.; Hinton, G. Deep learning. Nature
**2015**, 521, 436–444. [Google Scholar] [CrossRef] [PubMed] - Shilon, I.; Kraus, M.; Büchele, M.; Egberts, K.; Fischer, T.; Holch, T.L.; Lohse, T.; Schwanke, U.; Steppa, C.; Funk, S. Application of deep learning methods to analysis of imaging atmospheric Cherenkov telescopes data. Astropart. Phys.
**2019**, 105, 44–53. [Google Scholar] [CrossRef] [Green Version] - Mangano, S.; Delgado, C.; Bernardos, M.I.; Lallena, M.; Vázquez, J.J.R.; CTA Consortium. Extracting gamma-ray information from images with convolutional neural network methods on simulated cherenkov telescope array data. In IAPR Workshop on Artificial Neural Networks in Pattern Recognition; Springer: Cham, Switzerland, 2018; pp. 243–254. [Google Scholar]
- Nieto Castaño, D.; Brill, A.; Kim, B.; Humensky, T.B. Exploring deep learning as an event classification method for the Cherenkov Telescope Array. In Proceedings of the 35th International Cosmic Ray Conference, (ICRC2017), Busan, Korea, 12–20 July 2017; p. 809. [Google Scholar]
- Holch, T.L.; Shilon, I.; Büchele, M.; Fischer, T.; Funk, S.; Groeger, N.; Jankowsky, D.; Lohse, T.; Schwanke, U.; Wagner, P. Probing Convolutional Neural Networks for Event Reconstruction in Gamma-Ray Astronomy with Cherenkov Telescopes. In Proceedings of the 35th International Cosmic Ray Conference, (ICRC2017), Busan, Korea, 12–20 July 2017; p. 795. [Google Scholar]
- Cash, W. Parameter estimation in astronomy through application of the likelihood ratio. Astrophys. J.
**1979**, 228, 939–947. [Google Scholar] [CrossRef] - D’Agostini, G. Jeffreys priors versus experienced physicist priors-arguments against objective Bayesian theory. arXiv
**1998**, arXiv:physics/9811045. [Google Scholar] - Aharonian, F.; Akhperjanian, A.; Barrio, J.; Bernlöhr, K.; Börst, H.; Bojahr, H.; Bolz, O.; Contreras, J.; Cortina, J.; Denninghoff, S.; et al. Evidence for TeV gamma ray emission from Cassiopeia A. Astron. Astrophys.
**2001**, 370, 112–120. [Google Scholar] [CrossRef] [Green Version] - Rowell, G.P. A new template background estimate for source searching in TeV γ-ray astronomy. Astron. Astrophys.
**2003**, 410, 389–396. [Google Scholar] [CrossRef] [Green Version] - Fernandes, M.V.; Horns, D.; Kosack, K.; Raue, M.; Rowell, G. A new method of reconstructing VHE γ-ray spectra: The Template Background Spectrum. Astron. Astrophys.
**2014**, 568, A117. [Google Scholar] [CrossRef] [Green Version] - Berge, D.; Funk, S.; Hinton, J. Background modelling in very-high-energy γ-ray astronomy. Astron. Astrophys.
**2007**, 466, 1219–1229. [Google Scholar] [CrossRef] [Green Version] - Rolke, W.A.; Lopez, A.M. Confidence intervals and upper bounds for small signals in the presence of background noise. Nucl. Instrum. Methods Phys. Res. Sect. A Accel. Spectrometers Detect. Assoc. Equip.
**2001**, 458, 745–758. [Google Scholar] [CrossRef] [Green Version] - Li, T.P.; Ma, Y.Q. Analysis methods for results in gamma-ray astronomy. Astrophys. J.
**1983**, 272, 317–324. [Google Scholar] [CrossRef] - Cousins, R.D.; Linnemann, J.T.; Tucker, J. Evaluation of three methods for calculating statistical significance when incorporating a systematic uncertainty into a test of the background-only hypothesis for a Poisson process. Nucl. Instrum. Methods Phys. Res. Sect. A Accel. Spectrometers Detect. Assoc. Equip.
**2008**, 595, 480–501. [Google Scholar] [CrossRef] [Green Version] - Linnemann, J.T. Measures of significance in HEP and astrophysics. arXiv
**2003**, arXiv:physics/0312059. [Google Scholar] - Vianello, G. The significance of an excess in a counting experiment: Assessing the impact of systematic uncertainties and the case with a Gaussian background. Astrophys. J. Suppl. Ser.
**2018**, 236, 17. [Google Scholar] [CrossRef] [Green Version] - Klepser, S. A generalized likelihood ratio test statistic for Cherenkov telescope data. Astropart. Phys.
**2012**, 36, 64–76. [Google Scholar] [CrossRef] [Green Version] - Nievas-Rosillo, M.; Contreras, J. Extending the Li&Ma method to include PSF information. Astropart. Phys.
**2016**, 74, 51–57. [Google Scholar] - Klepser, S. The optimal on-source region size for detections with counting-type telescopes. Astropart. Phys.
**2017**, 89, 10–13. [Google Scholar] [CrossRef] [Green Version] - Weiner, O.M. A new time-dependent likelihood technique for detection of gamma-ray bursts with IACT arrays. arXiv
**2015**, arXiv:1509.01290. [Google Scholar] - Dickinson, H.; Conrad, J. Handling systematic uncertainties and combined source analyses for Atmospheric Cherenkov Telescopes. Astropart. Phys.
**2013**, 41, 17–30. [Google Scholar] [CrossRef] [Green Version] - Spengler, G. Significance in gamma ray astronomy with systematic errors. Astropart. Phys.
**2015**, 67, 70–74. [Google Scholar] [CrossRef] [Green Version] - Loredo, T.J. Promise of Bayesian inference for astrophysics. In Statistical Challenges in Modern Astronomy; Springer: New York, NY, USA, 1992; pp. 275–297. [Google Scholar] [CrossRef] [Green Version]
- D’Amico, G.; Terzić, T.; Strišković, J.; Doro, M.; Strzys, M.; van Scherpenberg, J. Signal estimation in on/off measurements including event-by-event variables. Phys. Rev. D
**2021**, 103, 123001. [Google Scholar] [CrossRef] - Gregory, P. Bayesian Logical Data Analysis for the Physical Sciences: A Comparative Approach with Mathematica
^{®}Support; Cambridge University Press: Cambridge, UK, 2005. [Google Scholar] - Knoetig, M.L. Signal discovery, limits, and uncertainties with sparse on/off measurements: An objective bayesian analysis. Astrophys. J.
**2014**, 790, 106. [Google Scholar] [CrossRef] [Green Version] - Jeffreys, H. The Theory of Probability; OUP: Oxford, UK, 1998. [Google Scholar]
- Casadei, D. Objective Bayesian analysis of “on/off” measurements. Astrophys. J.
**2014**, 798, 5. [Google Scholar] [CrossRef] [Green Version] - Rolke, W.A.; Lopez, A.M.; Conrad, J. Limits and confidence intervals in the presence of nuisance parameters. Nucl. Instrum. Methods Phys. Res. Sect. A Accel. Spectrometers Detect. Assoc. Equip.
**2005**, 551, 493–503. [Google Scholar] [CrossRef] [Green Version] - O’Hagan, A.; Forster, J.J. Kendall’s Advanced Theory of Statistics, Volume 2B: Bayesian Inference; Arnold: London, UK, 2004; Volume 2. [Google Scholar]
- Ackermann, M.; Ajello, M.; Albert, A.; Allafort, A.; Atwood, W.; Axelsson, M.; Baldini, L.; Ballet, J.; Barbiellini, G.; Bastieri, D.; et al. The Fermi large area telescope on orbit: Event classification, instrument response functions, and calibration. Astrophys. J. Suppl. Ser.
**2012**, 203, 4. [Google Scholar] [CrossRef] [Green Version] - Albert, J.; Aliu, E.; Anderhub, H.; Antoranz, P.; Armada, A.; Asensio, M.; Baixeras, C.; Barrio, J.; Bartko, H.; Bastieri, D.; et al. Unfolding of differential energy spectra in the MAGIC experiment. Nucl. Instrum. Methods Phys. Res. Sect. A Accel. Spectrometers Detect. Assoc. Equip.
**2007**, 583, 494–506. [Google Scholar] [CrossRef] [Green Version] - Tikhonov, A.N. On the Solution of Ill-Posed Problems and the Method of Regularization; Doklady Akademii Nauk; Russian Academy of Sciences: Moscow, Russia, 1963; Volume 151, pp. 501–504. [Google Scholar]
- Bertero, M. Linear inverse and III-posed problems. Adv. Electron. Electron Phys.
**1989**, 75, 1–120. [Google Scholar] - Schmelling, M. The method of reduced cross-entropy A general approach to unfold probability distributions. Nucl. Instrum. Methods Phys. Res. Sect. A Accel. Spectrometers Detect. Assoc. Equip.
**1994**, 340, 400–412. [Google Scholar] [CrossRef] [Green Version] - D’Agostini, G. A multidimensional unfolding method based on Bayes’ theorem. Nucl. Instrum. Methods Phys. Res. Sect. A Accel. Spectrometers Detect. Assoc. Equip.
**1995**, 362, 487–498. [Google Scholar] [CrossRef] - D’Agostini, G. Improved iterative Bayesian unfolding. arXiv
**2010**, arXiv:1010.0632. [Google Scholar] - Choudalakis, G. Fully bayesian unfolding. arXiv
**2012**, arXiv:1201.4612. [Google Scholar] - Rico, J. Gamma-Ray Dark Matter Searches in Milky Way Satellites—A Comparative Review of Data Analysis Methods and Current Results. Galaxies
**2020**, 8, 25. [Google Scholar] [CrossRef] [Green Version] - Acciari, V.A.; Ansoldi, S.; Antonelli, L.A.; Engels, A.A.; Baack, D.; Babić, A.; Banerjee, B.; de Almeida, U.B.; Barrio, J.A.; González, J.B.; et al. Bounds on Lorentz invariance violation from MAGIC observation of GRB 190114C. Phys. Rev. Lett.
**2020**, 125, 021301. [Google Scholar] [CrossRef] [PubMed] - Weekes, T.C.; Cawley, M.F.; Fegan, D.; Gibbs, K.; Hillas, A.; Kowk, P.; Lamb, R.; Lewis, D.; Macomb, D.; Porter, N.; et al. Observation of TeV gamma-rays from the crab nebula using the atmospheric cherenkov imaging technique. Astrophys. J.
**1989**, 342, 379–395. [Google Scholar] [CrossRef] - Amato, E.; Olmi, B. The Crab Pulsar and Nebula as seen in gamma-rays. Universe
**2021**, 7, 448. [Google Scholar] [CrossRef] - Al Samarai, I.; Batista, R.A.; de Almeida, U.B.; dos Anjos, R.D.C.; Dal Pino, E.D.G.; de los Reyes Lopez, R.; de Ona Wilhelmi, E.; Dominis Prester, D.; Freixas Coromina, L.; Garcia López, R.; et al. Science with the Cherenkov Telescope Array. Astrophys. J. Suppl.
**2019**, 240, 32. [Google Scholar] [CrossRef] [Green Version]

**Figure 1.**Schematic workflow of the inference analysis performed in order to estimate the intrinsic flux $\mathrm{\Phi}$ of gamma rays (and the values of its parameters $\theta $) from the recorded images. The acronym IRF stands for Instrument Response Function (see Section 4). The bold arrows (going from the right to the left) show the relation of cause and effect. The aim of the inference analysis (shown as a thin arrow going from the left to the right) is to invert such relation.

**Figure 3.**

**Left panel**: distribution for background events (hatched red) and simulated $\gamma $ (blue filled) of the discriminating variable given in output from the BDT method implemented by the HESS collaboration. Reprinted with permission from Ref. [30] Copyright 2010 Fiasson et al.

**Right panel**: distribution for background events (black) and simulated $\gamma $ (blue) of the discriminating variable (called hadronness) given in output from the RF method implemented by the MAGIC collaboration. Reprinted with permission from Ref. [33] Copyright 2009 Colin et al.

**Figure 4.**A comparison between the frequentist (

**left panel**) and Bayesian (

**right panel**) conclusion from the experiment result ${N}_{on}=82$ on the hypothesis that the gamma ray expected counts is 65.4 and no background is present. Left panel: in black the cumulative distribution function (CDF) of the statistic defined in Equation (15) from ${10}^{6}$ simulations assuming $s=65.4$, while in grey the expected CDF of a ${\chi}^{2}$ random variable. The step shape of the CDF of the statistic is due to the discrete nature of the Poisson distribution. Dashed lines show the point in which the statistic is 3.9 and the CDF is 0.952. Right panel: evolution of the BF defined in Equation (19) as a function of the expected counts ${s}_{2}$, using ${s}_{1}=65.4$ and ${N}_{on}=82$. Dashed lines show the point in which the expected counts are 65.4 and by definition the BF is 1.

**Figure 5.**Distribution of the Cash statistic in Equation (24) (

**left panel**) and the Li&Ma statistic in Equation (28) (

**right panel**) from measurements in which the background is known precisely to be $\overline{b}=10$ or must be estimated from the OFF counts ${N}_{off}$ with $\alpha =1$, respectively. In both simulations, the true values of s and b are 0 and 10, respectively. In both plots, the PDF of a normal distribution with mean zero and variance 1 is shown in grey as reference. In both cases ${10}^{6}$ simulations were performed. The distribution in the left panel looks to be less populated due to the fact that having the background fixed to 10 (and not estimated from an OFF measurement) limits the number of possible outcomes of the statistic.

**Figure 6.**Comparison between the Li&Ma statistic in Equation (28) (x-axis) and the Cash statistic in Equation (24) (y-axis). In both plots, each point shows the significance for a different ${N}_{on}$ ranging from 10 (where the significance is zero in both cases) to 56. For the Cash formula $\overline{b}$ is fixed to 10 in both plots, while for the Li&Ma formula ${N}_{off}$ is 10 with $\alpha =1$ in the left plot, and ${N}_{off}=1000$ with $\alpha =0.01$ in the right plot. As a reference the equation $y=x$ (dashed line) is shown. One can see that the Li&Ma statistic converges to the Cash one when $\alpha \phantom{\rule{3.33333pt}{0ex}}\ll \phantom{\rule{3.33333pt}{0ex}}1$.

**Figure 7.**

**Left panel**: Points show the PMF defined in Equation (35) of the number of signal events ${N}_{s}$, while lines show the PDF defined in Equation (34) of the expected signal counts s. Different colors are used to distinguish the different counts ${N}_{off}$ (160 in red, 120 in blue and 10 in black), while ${N}_{on}$ and $\alpha $ are fixed to 80 and 0.5, respectively.

**Right panel**: The mode of the PDF defined in Equation (34) as a function of the excess ${N}_{on}-\alpha {N}_{off}$. As a dashed line the equation $x=y$ is shown for reference.

**Figure 8.**Evolution of the statistic $\mathcal{S}$ as a function of the model parameter $\theta $ from different pseudo-experiments with fixed true value $\overline{\theta}$. Vertical line shows the true value of $\theta $, while the horizontal one shows the threshold ${\mathcal{S}}^{*}$ for the statistic such that $\mathcal{S}\left(\overline{\theta}\right)\le {\mathcal{S}}^{*}$ only $x\%$ of the time. Black curves are those that fulfill this condition while grey ones are those that do not. In the left plot the intersection between the curves and the line $\mathcal{S}={\mathcal{S}}^{*}$ defines CIs which by construction cover the true value of $\theta $ $x\%$ of the time. These CIs cannot be anymore constructed for the curves in the right plot where the statistic has below ${\mathcal{S}}^{*}$ more than one minimum. In both plots the curves shown are not specific on any particular problem but only serve as a schematic representation.

**Figure 9.**Evolution of the coverage of the CIs obtained from solving $\mathcal{S}=$ 1 (grey line) or $\mathcal{S}=$ 3.84 (black line) as a function of the signal s. $\mathcal{S}$ is the statistic defined in Equation (27). The dashed horizontal lines show the expected coverage from the assumption that $\mathcal{S}$ is a ${\chi}^{2}$-random variable with 1 degree of freedom. In each MC simulation the observed counts ${N}_{on}$ and ${N}_{off}$ were simulated from a Poisson distribution with expected counts $s+\alpha b$ and b, respectively.

**Left panel**: the expected background count is fixed to 1.

**Right panel**: the expected background count is fixed to 10. In both plots $\alpha =0.5$ is assumed.

**Figure 10.**

**Left panel**: Evolution in energy of the collection efficiency $\epsilon \left(E\right)$ multiplied by the collection area of the telescope, which for IACTs is generally of the order of ${10}^{5}\phantom{\rule{0.166667em}{0ex}}{\mathrm{m}}^{2}$.

**Right panel**: evolution in reconstructed/estimated energy and true energy of the binned dispersion energy (or migration matrix). Both figures are Reprinted with permission from Ref. [67] Copyright 2007 Albert et al.

Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations. |

© 2022 by the author. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).

## Share and Cite

**MDPI and ACS Style**

D’Amico, G.
Statistical Tools for Imaging Atmospheric Cherenkov Telescopes. *Universe* **2022**, *8*, 90.
https://doi.org/10.3390/universe8020090

**AMA Style**

D’Amico G.
Statistical Tools for Imaging Atmospheric Cherenkov Telescopes. *Universe*. 2022; 8(2):90.
https://doi.org/10.3390/universe8020090

**Chicago/Turabian Style**

D’Amico, Giacomo.
2022. "Statistical Tools for Imaging Atmospheric Cherenkov Telescopes" *Universe* 8, no. 2: 90.
https://doi.org/10.3390/universe8020090