What Is Darapskite? Some Reflections on How Artificial Intelligence Could Promote an Academic Gap ^†

Abstract

Two AI chatbots were tested with questions regarding a relatively minor mineral species. There were several issues with the information provided but the answers were well structured and well written. Nevertheless, these AI chatbots are a potentially useful tool, especially for those struggling with the English language, albeit a skeptical overview of their use is required.

1. Introduction

Artificial intelligence is one of the main topics in the news, especially after the release of ChatGPT by OpenAI, as it was able to generate new content, which is also, hence, a cause for concern in academic circles due to its potential impact on authorship [1]. In an arguably complementary perspective, generative AI has been defended as a potential leveling tool for writers [2]. Consider that it is only 2023 and searching for publications including titles with ChatGPT in them listed 3720 results on Google Scholar [3]. If Teaching OR education is added to the search expression, 238 results were prompted [4]. For the more recent Google Bard, launched in the current year [5], we obtained 34 results [6], several of which related to education. If Mineralogy is included, even removing the teaching or education part and the time restriction, zero results were obtained in both cases. Google has already announced the development of a new AI system, which is already being assessed for education [7] and will certainly elicit an answer from OpenAI and Microsoft. According to the Handbook of Mineralogy [8], darapskite is a mineral species with the end chemical formula Na₃(NO₃)(SO₄).H₂O, having a structure belonging to the monoclinic crystal system and described by the P2₁/m space group [9]. It was named after Ludwig Darapsky (it has a widespread occurrence in Chile). Information on the cell coordinates of the different atoms can be found at the link [10] to the American Mineralogist Crystal Structure Database [11]. According to Ericksen and Mrose [12], it was first described in 1891. Darapskite is a relatively rare mineral that, nonetheless, has been identified in several places, both in geological terrains and in the built environment, being also included in several laboratory studies; a Google Scholar search for darapskite prompts 352 results [13]. It hence seemed an interesting candidate to assess the potentialities and problems associated with the use of generative artificial intelligence software. Initially, several Android chatbot apps were tested with this question, giving wildly variable wrong answers, and hence we focused on two major players (see below).

2. Materials and Methods

The materials in this work were the software and hardware used. All the tests of the artificial intelligence chatbots were performed with an Android 11 smartphone. We tested Bing with ChatGPT-4 (henceforth simply Bing, [14]) and Google Bard (henceforth simply Bard, [15]). The method used was simply to put 3 questions to the chatbots. In addition to the initial question already mentioned, we asked two other questions, on the occurrence of darapskite in buildings and the distinction of this mineral from niter, which can be found in association with darapskite both on geological terrains and in the built environment. This last question is arguably the one that best tests the generative potential of these chatbots, since it requires the synthesis of available information.

3. What Is Darapskite?

To present an easier-to-read document, all the original answers of the two chatbots are, with minimal editing, in the Appendix A, Appendix B and Appendix C. While Bing included references in its answer, for Bard it was necessary to request them. Sometimes this failed, but asking repeatedly to regenerate the draft led to the indication of references. The two chatbots gave answers with the correct chemical formula and some assorted information on this mineral (Appendix A), with some variation among them. However, according to the guidelines of the International Mineralogical Association [16], a mineral species is also defined by its crystallographic properties, and the chatbots had a subpar performance on this; Bard indicated the crystal system (Bing not even that) but did not give information on such issues as the space group and the cell coordinates of atoms.

4. List All the Occurrences of Darapskite in Buildings

As seen in Appendix B, Bing produced what seemed the most complete answer, listing three cases. However, checking the indicated references, while the first concerned indeed a paper about darapskite on two Dutch churches, the second one did not have any reference to buildings and the third was a laboratory study. Bard gave arguably worst results, not only denying the occurrence of darapskite in buildings but discussing a case of Dutch churches (we presume it was the same study), stating that efflorescence could be confounded with darapskite. This was particularly baffling, as Google Scholar listed 181 results for a search with darapskite AND buildings [17]. The chatbots leave out many other papers that have found darapskite in buildings, as can be seen from Google Scholar’s list.

5. How Can We Distinguish Darapskite from Niter with the Polarizing Microscope?

Both Bing and Bard made glaring errors in this question (Appendix C). Bing stated that niter is uniaxial, a significantly clamoring mistake since in the same answer Bing stated (correctly) that niter is orthorhombic (which implies a biaxial ellipsoid). Interestingly, Bing does mention a distinguishing optical property of niter, which is that for some orientations one can observe positive and negative relief in relation to the usual mounting medium. The values for the refractive index are the same as those of the indicated reference, including the use of the letters a, b and g instead of the usual Greek letters alfa, beta and gamma. Curiously, Bing’s answer does not indicate a reference for the information on niter’s optical properties and all the information on the refringence ellipsoid is different from that shown in webmineral [18] and the Handbook of Mineralogy [19], which are the same in terms of refractive index values (so, from where did Bing obtain this critically wrong information?). The same pattern is observed in the information for cleavage. Bing does not mention the differences in twinning between these two minerals. Bard goes a bit further down, alleging that niter is not birefringent and giving values for the refractive index that are higher than any of those listed for niter (idem for darapskite) and for some reason seems to “think” that the higher the refractive index the darker the mineral will appear in a polarizing microscope. Neither webmineral [20] nor the Handbook of Mineralogy [9], nor the paper by Ericksen and Mrose [12] make any reference to pleochroism in darapskite. An immensely more astonishing error is observed in the comparison of densities, as 2.64 > 2.16(!). Like Bing, Bard also does not mention twinning. Going down, after asking for references for the provided information, Bard lists the correct webmineral link, the general reference of the Handbook of Mineralogy (an odd choice, as it is possible to give the specific link for darapskite), a Mindat link for another mineral (which is different from the one given in the two previous sections!) and two book references that we were unable to find with the information given (we did not check for all the mistakes in the information provided by the chatbots).

6. Final Considerations

The assessed AI chatbots produced well-structured and well-written answers, and hence can be considered a potentially useful tool, especially for users that have difficulties and doubts with the English language. Nevertheless, these answers might also contain subtle errors and gaps (sometimes also information that can be considered lateral to the posed questions), which on occasions might not be easy to detect. It will be necessary to have a critical overview of its use. In a kind of analogy with the intersectionality framework used in the social sciences, this could cause especially significant impacts on less skeptical, more socially isolated, economically disadvantaged students of less recondite study areas, with the risk of contributing to an academic gap in relation to more critically minded, socially connected, economically advantaged students from more prominent areas of knowledge. We argue that studies like the present one are needed for a practical assessment from the user’s point of view, with the selection of questions being of vital importance and requiring input from researchers in different areas beyond computer science. AI chatbots could be an important tool for education but demand attention and skepticism. Educators should embrace this as an opportunity to promote critical thinking skills (as well as an unavoidable development that their students should not miss). We propose that more studies along the lines of the present one will be critical to test the usefulness of these digital tools. A vital component will be to formulate questions that push for the generative role, testing the intelligence allegation. This will have to be conducted by researchers in other knowledge areas besides computer sciences.