# Understanding the Accuracy Limitations of Quantifying Methane Emissions Using Other Test Method 33A

^{1}

^{2}

^{*}

## Abstract

**:**

## 1. Introduction

## 2. Materials and Methods

_{4}analyzer, LI-7500DS CO

_{2}/H

_{2}O analyzer, LI-200R pyranometer (Lincoln, NE, USA), a Gill® WindMaster 3-axis sonic anemometer (Lymington, UK), and an Omega iBTHx barometric pressure, temperature, and humidity sensor (Norwalk, CT, USA) [19,20,21,22,23]. Data from the system were recorded at a rate of 10 Hz. The LI-7700, LI-7500, and WindMaster were mounted vertically, 4 m from the base of the trailer. The trailer base was about 0.5 m above the ground; therefore, the sensor height was considered 4.5 m for data processing. Details of the DAQ system can be found in [11,24]. More details on the DAQ equipment specifications are presented in the Appendix A, Table A1. The DAQ setup is presented in Figure A1.

## 3. Results and Discussion

#### 3.1. Instrument Uncertainty

- measured value = the value reported and recorded form the instrument
- upper value = measured value + standard uncertainty of measured value
- lower value = measured value − standard uncertainty of measured value

- The percentage of the measured estimate was calculated for each of the 729 results for each period.
- From these 729 results, a single estimate was randomly selected 100 times to represent the given period.
- The mean, standard deviation (σ), and standard error (SE) of these periods were calculated. The SE was calculated using Equation (2), where n represents the number of samples (100).
- The measurement uncertainty of the 100 samples was calculated as [±1.96 * SE] representing the 95% confidence interval (CI) of the standard normal distribution.
- This process was repeated for 1000 iterations and the average measurement uncertainty was determined to be the measurement uncertainty for all estimates.

#### 3.2. Method Uncertainty

_{2}fluxes. The qualifiers for acceptable periods were based on the drivers of these types of fluxes. The qualifiers for similar periods were defined as follows:

- -
- Photosynthetic Photon Flux Density (PPFD) difference less than 75 µmol/m
^{2}s - -
- Air Temperature Difference less than 3 °C
- -
- Vapor Pressure Deficit Difference less than 200 Pa (0.2 kPa)
- -
- Wind Speed Difference less than 1 m/s

_{1}and x

_{2}have measurement uncertainties associated with them of δq

_{1}and δq

_{2}. Because the expected difference between x

_{1}and x

_{2}of sufficiently similar periods would be zero, it follows that the variance of (δq

_{1}− δq

_{2}) would be equivalent to the variance of (x

_{1}− x

_{2}). Hence, the standard deviation of the uncertainty was inferred from the standard deviation of the measurements whose expected values are equivalent [18]. We applied the same logic to OTM estimates. Sufficiently similar periods of measurement should yield similar results.

- -
- No precipitation between periods
- -
- Air Temperature difference less than 3 °C

- -
- PPFD difference less than 75 µmol/m
^{2}s - -
- Wind speed difference less than 1 m/s

^{2}s] is a measure of light intensity approximately equal to 4.6 W/m

^{2}[23]. In this research the variable of solar loading in W/m

^{2}was recorded by a pyranometer and converted to PPFD. From the background periods collected in between controlled releases there were 1208 periods that contained both valid OTM and EC estimates. From these 1208 periods there were 65 pairs of periods that met the criteria for OTM.

#### 3.3. Instrument Uncertainy Results

#### 3.4. Method Uncertainy Results

## 4. Conclusions

## Supplementary Materials

## Author Contributions

## Funding

## Institutional Review Board Statement

## Informed Consent Statement

## Data Availability Statement

## Conflicts of Interest

## Appendix A

Device | Manufacturer | Detection Method | Max Rate/ Used Rate | Parameters Measured | Range | Resolution (res)/Accuracy (acc) | Operating Limits |
---|---|---|---|---|---|---|---|

Gill WindMaster | Gill Instruments Ltd. (Hampshire, UK) | Ultrasonic Pulse | 20 Hz/ 10 Hz | 3-D Wind Speed | 0–50 m/s | <1.5% RMS | T: −40–70 °C |

RH: <5–100% | |||||||

LI-7700 | LI-COR Biosciences (Lincoln, NE, USA) | Wavelength Modulation Spectroscopy | 20 Hz/ 10 Hz | CH_{4} conc.TemperaturePressure | CH: 0–40 ppm at 25 °C_{4} | 5 ppb res.<1% linearity | T: −25–50 °C |

P: 50–110 kPa | |||||||

RH: 0–100% | |||||||

LI-7500 | LI-COR Biosciences (Lincoln, NE, USA) | Non-dispersive spectroscopy | 20 Hz/ 10 Hz | CO_{2} conc.H _{2}O conc.Temperature Pressure | CO: 0–3000 µmol/mol_{2} | CO: <1% of reading_{2} | RH: 0–95% |

H: 0–60 µmol/mol_{2}O | H: <1% of reading_{2}O | ||||||

T: −20–70 °C | T: ±0.3 °C | T: −25–50 °C | |||||

P: 50–110 kPa | P: 0.4 kPa | P: 50–110 kPa | |||||

LI-200R | LI-COR Biosciences (Lincoln, NE, USA) | Photovoltaic | 1 × 10^{5} Hz/10 Hz | Solar Loading | 0–3000 W/m^{2} | ±3% over reading | T: −40–65 °C |

RH: 0–100% | |||||||

Omega iBTHx | Omega™ Engineering (Norwalk, CT, USA) | Various | 0.25 Hz/ 0.25 Hz | Temperature Pressure Relative Humidity | T: 0–70 °C | T: ±2 °C acc.0.01 °C res. | T: 0–70 °C |

P: 0–110 kPa | P: ±0.2 kPa acc.0.01 kPa res. | P: 0–110 kPa | |||||

RH: 0–100% | RH: 2% for 10–90 acc. 0.03% res. | RH: 0–100% |

Release Rates (g/s) | Distances (m) | ||
---|---|---|---|

0.036 | 42 | 72 | 119 |

0.119 | 57 | 72 | 119 |

0.239 | 42 | 72 | 119 |

**Calculation A1.**Accuracy Uncertainty Example

**Calculation A2.**Resolution Uncertainty Example

**Calculation A3.**Total Standard Uncertainty

- The resolution of the LI-7700 is 5 parts per billion (ppb). The half interval in ppm is then 0.005.

- 2.
- The accuracy of the analyzer is 1% of the reading across the full calibration range. So, if the concentration is 2.0945 ppm.

- 3.
- The total uncertainty is the sum of the squares of the resolution and accuracy uncertainty.

## References

- Miller, S.; Wofsy, S.; Michalak, A.; Sweeny, C. Anthropogenic Emissions of Methane in the United States. Proc. Natl. Acad. Sci. USA
**2013**, 110, 20018–20022. [Google Scholar] [CrossRef] [PubMed] [Green Version] - Omara, M.; Zimmerman, N.; Sullivan, M.; Li, X.; Ellis, A.; Cesa, R.; Subramanian, R.; Presto, A.; Robinson, A. Methane Emissions from Natural Gas Production Sites in the United States: Data Synthesis and National Estimate. Environ. Sci. Technol.
**2018**, 42, 12915–12925. [Google Scholar] [CrossRef] [PubMed] [Green Version] - Zavala-Araiza, D.; Lyon, D.; Alvarez, R.; Hamburg, S. Reconciling Divergent Estimates of Oil and Gas Methane Emissions. Proc. Natl. Acad. Sci. USA
**2015**, 112, 15597–15602. [Google Scholar] [CrossRef] [PubMed] [Green Version] - Kimbrel, D.; Balasbas, J.; Zolnierek, J.; Fallah, J.; Thibodeaux, C. Sampling of Methane Emissions Detection Technologies and Practices for Natural Gas Distribution Infrastructure; National Association of Regulatory Utility Commissioners: Washington, DC, USA; Available online: https://pubs.naruc.org/pub/0CA39FB4-A38C-C3BF-5B0A-FCD60A7B3098 (accessed on 9 March 2021).
- Jakober, C.; Mara, S.; Hsu, Y.K.; Herner, J. Mobile Measurements of Climate Forcing Agents: Application to Methane Emissions from Landfill and Natural Gas Compression. J. Air Waste Manag. Assoc.
**2015**, 65, 404–412. [Google Scholar] [CrossRef] [PubMed] [Green Version] - Omara, M.; Sullivan, M.; Li, X.; Subramanian, R.; Robinson, A.; Presto, A. Methane Emissions from Conventional and Unconventional Natural Gas Production Sites in the Marcellus Shale Basin. Environ. Sci. Technol.
**2016**, 50, 2099–2107. [Google Scholar] [CrossRef] [PubMed] - Bell, C.; Vaughn, T.; Zimmerle, D.; Herndon, S.; Yacovitch, T.; Heath, G.; Petron, G.; Edie, R.; Field, R.; Murphy, S.; et al. Comparison of Methane Emission Estimates from Multiple Measurement Techniques at Natural Gas Production Pads. Elem. Sci. Anthr.
**2017**, 5, 79. [Google Scholar] [CrossRef] [Green Version] - Thoma, E.D.; Brantley, H.; Squier, B.; DeWees, J.; Segall, R.; Merrill, R. Development of Mobile Measurement Method Series. In Proceedings of the 108th Annual Conference of the Air & Waste Management Association (AWMA), Raleigh, NC, USA, 23–26 June 2015; p. 15. [Google Scholar]
- Brantley, H.L.; Thoma, E.D.; Squier, W.C.; Guven, B.B.; Lyon, D. Assessment of Methane Emissions from Oil and Gas Production Pads using Mobile Measurements. Environ. Sci. Technol.
**2014**, 48, 14508–14515. [Google Scholar] [CrossRef] [PubMed] [Green Version] - Edie, R.; Robertson, A.M.; Field, R.A.; Soltis, J.; Snare, D.A.; Zimmerle, D.; Bell, C.S.; Vaughn, T.L.; Murphy, S.M. Constraining the accuracy of flux estimates using OTM 33A. Atmos. Meas. Tech.
**2020**, 13, 341–353. [Google Scholar] [CrossRef] [Green Version] - Heltzel, R.S.; Zaki, M.T.; Gebreslase, A.K.; Abdul-Aziz, O.I.; Johnson, D.R. Continuous OTM 33A Analysis of Controlled Releases of Methane with Various Time Periods, Data Rates and Wind Filters. Environments
**2020**, 7, 65. [Google Scholar] [CrossRef] - Robertson, A.M.; Edie, R.; Snare, D.; Soltis, J.; Field, R.A.; Burkhart, M.D.; Bell, C.S.; Zimmerle, D.; Murphy, S.M. Variation in Methane Emission Rates from Well Pads in Four Oil and Gas Basins with Contrasting Production Volumes and Compositions. Environ. Sci. Technol.
**2017**, 51, 8832–8840. [Google Scholar] [CrossRef] [PubMed] - OTM 33A V1.3. Environmental Protection Agency. 2014. Available online: https://www3.epa.gov/ttnemc01/prelim/otm33a.pdf (accessed on 1 January 2020).
- Albertson, J.; Harvey, T.; Foderaro, G.; Zhu, P.; Zhou, X.; Ferrari, S.; Sharooz Amin, M.; Modrak, M.; Brantley, H.; Thoma, E.D. Mobile Sensing Approach for Regional Surveillance of Fugitive Methane Emissions in Oil and Gas Production. Environ. Sci. Technol.
**2016**, 50, 2487–2497. [Google Scholar] [CrossRef] [PubMed] [Green Version] - Yacovitch, T.; Herndon, S.; Petron, G.; Kofler, J.; Lyon, D.; Zahnizer, M.; Kolb, C. Mobile Laboratory Observations of Methane Emission in the Barnett Shale Region. Environ. Sci. Technol.
**2015**, 49, 7889–7895. [Google Scholar] [CrossRef] [PubMed] - Lin, X.; Talbot, R.; Laine, P.; Torres, A. Characterizing Fugitive Methane Emissions in the Barnett Shale Area Using a Mobile Laboratory. Environ. Sci. Technol.
**2015**, 49, 8139–8146. [Google Scholar] [CrossRef] [PubMed] - Heltzel, R.; Johnson, D.; Zaki, M.; Gebreslase, A.; Abdul-Aziz, O.I. Machine Learning Techniques to Increase the Performance of Indirect Methane Quantification from a Single, Stationary Sensor. Heyilon, 2022; under review. [Google Scholar]
- Hollinger, D.Y.; Richardson, A.D. Uncertainty in eddy covariance measurements and its application to physiological models. Tree Physiol.
**2005**, 25, 873–885. [Google Scholar] [CrossRef] [PubMed] [Green Version] - Gill Instruments Limited WindMaster 3-Axis Ultrasonic Anemometer. Available online: http://www.gillinstruments.com/products/anemometer/windmaster.htm (accessed on 2 January 2020).
- LI-COR LI-7700. LI-COR Biosciences. Available online: https://www.licor.com/env/support/LI-7700/manuals.html (accessed on 8 March 2021).
- LI-COR LI-7500. LI-COR Biosciences. Available online: https://www.licor.com/env/support/LI-7500/home.html (accessed on 8 March 2021).
- Omega iBTHx. Available online: https://in.omega.com/pptst/IBTX_IBTHX.html (accessed on 9 March 2021).
- LI-COR LI-200R Pyranometer. Available online: https://www.licor.com/env/products/light/pyranometer (accessed on 28 August 2020).
- Heltzel, R.S. On the Improvement of the Indirect Quantification of Methane Emissions: A Stationary Single Sensor Approach; West Virginia University: Morgantown, WV, USA, 2021. [Google Scholar]
- J.W. Ruby Research Farm. Davis College of Agriculture, Natural Resources and Design, West Virginia University. Available online: https://www.davis.wvu.edu/about-davis-college/farms-and-forests/j-w-ruby-research-farm (accessed on 8 March 2021).
- Johnson, D.; Heltzel, R.; Oliver, D. Temporal Variations in Methane Emissions from an Unconventional Well Site. ACS Omega
**2019**, 4, 3708–3715. [Google Scholar] [CrossRef] [PubMed] [Green Version] - Johnson, D.; Heltzel, R. On the Long-Term Temporal Variations in Methane Emissions from an Unconventional Natural Gas Well Site. ACS Omega
**2021**, 6, 14200–14207. [Google Scholar] [CrossRef] [PubMed] - MSEEL. Available online: http://mseel.org/ (accessed on 2 January 2020).
- Probability Distributions for Measurement Uncertainty. Available online: https://www.isobudgets.com/probability-distributions-for-measurement-uncertainty/ (accessed on 8 March 2021).
- How to Calculate Resolution Uncertainty. Available online: https://www.isobudgets.com/calculate-resolution-uncertainty/ (accessed on 8 March 2021).
- Estimating Measurement Uncertainty. Available online: https://www.sml-inc.com/uncertainy.htm (accessed on 8 March 2021).
- Yamulki, S.; Jarvis, S.C.; Owen, P. Methane Emission and Uptake from Soils as Influenced by Excreta Deposition from Grazing Animals. J. Environ. Qual.
**1999**, 28, 676–682. [Google Scholar] [CrossRef] - Koehn, A.C.; Leytem, A.B.; David, L. Bjorneberg Comparison of Atmospheric Stability Methods for Calculating Ammonia and Methane Emission Rates with WindTrax. Trans. ASABE
**2013**, 56, 763–768. [Google Scholar] [CrossRef] - Environmental Growth Chambers. Available online: http://www.egc.com/useful_info_lighting.php (accessed on 9 March 2021).

**Figure 1.**Histogram of measurement uncertainty as a percentage of the OTM estimate. (100 random periods, bootstrapped 1000 times).

**Table 1.**Number of valid controlled release and background period for various release rates and distances.

Distance (m) | Release Rate (g/s) | ||||
---|---|---|---|---|---|

None | 0.036 | 0.119 | 0.239 | Total | |

42 | 577 | 234 | 110 | 38 | 382 |

57 | 224 | - | 47 | 34 | 81 |

72 | 289 | 63 | 100 | - | 163 |

119 | 118 | 98 | 68 | 12 | 178 |

Total | 1208 | 395 | 325 | 84 | 804 |

Robertson et al. | Edie et al. | Brantley et al. | This Work | |
---|---|---|---|---|

Count (#) | 19 | 24 | 107 | 181 |

Release Rates (g/s) | 0.03–0.56 | 0.04–0.6 | 0.19–1.2 | 0.04–0.24 |

Full Range of % Error | −75% to 60% | −60% to 175% | −60% to 52% | −95% to 1070% |

Tests within ±30% Error | - | - | 71% | 30% |

Tests within ±50% Error | 85% | - | 56% | |

68th Percentile Error | ±28% | ±38% | - | ±64% |

Device | Relevant Variable | Acronym | Resolution | Accuracy |
---|---|---|---|---|

Gill WindMaster | X wind speed | u | ±0.01 m/s | <1.5% RMS |

Y wind speed | v | |||

Z wind speed | w | |||

LI-7700 | Methane Concentration | ch4 | ±0.005 ppm | <1% |

LI-7500 | Temperature | t | ±0.003 K | ±0.3 K |

Pressure | p | ±0.06 mbar | ±4 mbar |

Distance (m) for Calculation | σ (x_{1} − x_{2}) | σ (δq) | 95% CI | Mean Estimate of Periods | 95% CI/Mean Estimate |
---|---|---|---|---|---|

[g/s] | [g/s] | [±g/s] | [g/s] | [%] | |

42 | 0.007 | 0.005 | 0.001 | 0.007 | 17% |

57 | 0.012 | 0.008 | 0.002 | 0.012 | 17% |

72 | 0.018 | 0.012 | 0.003 | 0.018 | 17% |

119 | 0.045 | 0.032 | 0.008 | 0.045 | 17% |

Quantification Method | Uncertainty Method | How It Is Presented | Result |
---|---|---|---|

OTM | Instrument Measurement Uncertainty | The range of uncertainty as a percentage of the OTM estimate for any period based on the uncertainty of the instruments used to record data. | ±3.8% |

OTM | Modified H&R 24 h Difference Method | The range of measurement uncertainty of the method due to randomness in the measurement. | ±17% |

OTM | Combined Uncertainty | The minimal possible uncertainty from the OTM method, calculated as the combined uncertainty of the other two methods. | ±17.4% |

Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations. |

© 2022 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).

## Share and Cite

**MDPI and ACS Style**

Heltzel, R.; Johnson, D.; Zaki, M.; Gebreslase, A.; Abdul-Aziz, O.I.
Understanding the Accuracy Limitations of Quantifying Methane Emissions Using Other Test Method 33A. *Environments* **2022**, *9*, 47.
https://doi.org/10.3390/environments9040047

**AMA Style**

Heltzel R, Johnson D, Zaki M, Gebreslase A, Abdul-Aziz OI.
Understanding the Accuracy Limitations of Quantifying Methane Emissions Using Other Test Method 33A. *Environments*. 2022; 9(4):47.
https://doi.org/10.3390/environments9040047

**Chicago/Turabian Style**

Heltzel, Robert, Derek Johnson, Mohammed Zaki, Aron Gebreslase, and Omar I. Abdul-Aziz.
2022. "Understanding the Accuracy Limitations of Quantifying Methane Emissions Using Other Test Method 33A" *Environments* 9, no. 4: 47.
https://doi.org/10.3390/environments9040047