Machine-Learning Application for a Likelihood Ratio Estimation Problem at LHC

Round 1

Reviewer 1 Report

The paper presents machine learning-based LHC approximation. However, based on reviewer understanding and knowledge, the paper lacks of machine learning algorithms in detail. Meanwhile another problem is to radicate the state of art comparison is completely missing along with other methods. Also it does not qualify for a journal paper.

Author Response

Dear reviewer,

Thank you very much for your first read and these comments!

Here our answer:

What do you mean with "the paper lacks of machine learning algorithms in detail"?

Direct importance estimation method has been described in detail in the dedicated section and the network architecture has been also described. This paper is not intented as an explanation of the functioning of Deep Neural Networks, already extensively documented, but as a review of a specific kind of application adopted within the ATLAS experiment.

The reason why a direct comparison is not described in the article is that the two datasets are different, it is not possible to directly compare the same method applied on two different problems. Moreover we are limited on inserting only public results, approved from the ATLAS Collaboration, therefore any internal studies that may prove an indirect test of the impact of the method in the single analysis can’t be inserted in this article.

Reviewer 2 Report

The paper needs extensive English language revision. Example:

L35:  duplicate sentence " This is one of the most crucial task in physics analyses.one of the most crucial task in physics " L36:  Most *of the* used....

 

 

Author Response

Dear reviewer,

Thank you very much for your first read and these comments!

All the typos were fixed and English revised.

Thank you

Reviewer 3 Report

In this review, the authors of this paper describe two cases in which neural networks have been used to compute the likelihood ratio of particle physics analyses by solving a custom loss function given by a least-square problem to remove background noise from the Large Hadron Collider. This is an important field of research, which requires novel methods to generate a solution; however, there are a few issues with the execution of the current review.

 

Major critiques:

The authors explain the use of two different machine learning approaches that are published in the literature (one is not peer-reviewed but part of the CERN document server), but do not provide a comparison between the two. Because of this, the review seems only to summarize the two methods, which severely limits the importance of their article. Authors should highlight the strengths and weaknesses of both methods, and pinpoint the gap in knowledge that the current review is bridging.

There is no in-depth discussion of the results, or the significance of their review.

The article is poorly cited – it is hard to know when the authors are providing their own opinions/research, and when they are referencing the literature. Authors should cite references where appropriate throughout the review.

Grammar in sections 1-3 is poorly written and many sentences need re-wording and/or clarification.

 

Minor critiques:

There are a number of subjective adjectives such as “excellent” and “good”. In addition, the authors have a number of informal sentences which should be rewritten to a formal scientific standard.

Authors should provide previous ways in which this problem has been approached in the past, and why machine learning would be superior.

The word signal in page 1, line 29 is in italics, when referred to this signal in the rest of the manuscript, use italics as well.

The writing of this review is inconsistent. Sections 1-3 are comprised of short paragraphs, many of which should be a single paragraph. This makes the first portion of the manuscript hard to read and understand.

Authors define abbreviations multiple time along the manuscript, and sometimes use the abbreviation before it is defined.

In page 1, line 19: an event range should be provided rather than stating “up to 100 kHz”.

Page 1, line 35: authors state “This is one of the most crucial task…” it is unclear which method they are referring to. In addition, the sentence is repeated twice.

In pages 1-2, lines 35-37: the authors mention two approaches, they should provide examples and outcomes of such approaches to highlight the importance of machine learning.

In page 2, lines 41-42: the authors mention the background estimation procedures, they should expand the examples and define the “analysis peculiarity”

In page 2, the lines without a line number (between lines 43 and 44): the paragraph that begins with “The case of interest discussed in this paper is the background estimation…” should be moved to the introduction.

In page 2, lines 44-48: the authors state “for the majority of the cases, it’s not simple and straightforward to obtain”, they should expand on the limitations, and expand why it is not “straightforward”.

In page 2, lines 49-50: the authors should expand on the rationale of using machine learning to solve the optimization problem, and provide examples of other methods used to do so, if any.

In page 2, lines 58-63: the grammatic style makes the sentence hard to follow. Please revise grammar.

In page 3, line 74: authors state “In the expression in the centre of 2”, but it is unclear which portion of the equation is what they consider to be the center.

In page 4, line 102: authors should expand on the parameter R, which was not previously defined.

In page 5, line 146: the authors state " The further classification of events where the Higgs boson mass candidate fails (passes)…”, it is unclear why the parenthesis states the opposite (e.g., passes) than what the sentence had stated originally.

Author Response

Dear reviewer,

Thank you very much for your first read and these comments! They were very helpful for organizing and more thoroughly documenting the work desxcribed in the paper.

Here our responses.

Thank you!

The reason why a direct comparison is not described in the article is that the two datasets are different, it is not possible to directly compare the same method applied on two different problems. Moreover we are limited on inserting only public results, approved from the ATLAS Collaboration, therefore any internal studies that may prove an indirect test of the impact of the method in the single analysis can’t be inserted in this article.

Citations have been inserted in a more "visible" way, in particular in the 3rd section (previously the 4th).

Thank you for all the grammatical and syntactical corrections, they have been incorporated into the second version of the paper.

 

 

Reviewer 4 Report

This paper addresses the problem of estimating background processes using deep learning techniques. The problem is relevant, and the methodology is adequate. However, some improvements may be considered, as such

- the abstract can be improved. It is generic and it is insufficient to understand the article

- the introduction can be improved - e.g., it does not present the objectives and the structure of the article

- the paragraph of line 35 must be rewritten

- the related work is missing

- the number of references is minimal.

Author Response

Dear reviewer,

Thank you very much for your first read and these comments!

Here we post our answers.

Thank you!

• The abstract and the introduction have been completely reorganized and extended.
• Sections 2 and 3 have been rewrittn, English reviewed and merged together in a single Section.
• Only really cited references have been considered.

Round 2

Reviewer 1 Report

The authors had addressed the queries raised by the  reviewer. No more concerns from myside

 

Reviewer 2 Report

Nothing

Reviewer 4 Report

Well done.