When technology tries to take shortcuts: A new model released by Meta was supposed to bring the gospel to scientific papers, but it didn't go exactly as she expected.
According to the publication on the geektime website [A. Alexelsi], Meta's artificial intelligence division has released a new language model called Galactica, which can generate scientific articles, identify which article the information you entered was cited from (citation), and perform a "translation" of mathematical formulas.
Sounds amazing and even dangerous. How does it work?
Want more news, videos and stories? Join the Haredim 10 WhatsApp channel >>
Galactica trained on a dataset of 120 billion parameters including 48 million scientific articles, textbooks, scientific websites, encyclopedias and summaries of scientific lectures. The goal of the model, as presented by Meta, is to "organize science" – by developing the ability to deal with the vast amounts of scientific information that exists today – and therefore it has also released its new model as open source.
The model went live last week and was available in open source to anyone who wanted it. Alongside it, Meta also released a demo that allowed anyone who wanted to experiment with it and try to produce scientific articles on various topics, and then things started to go wrong.
Users discovered that the new model allowed them to create Wikipedia articles on "the benefits of suicide," an article on "the benefits of being fair-skinned," articles on the benefits of eating glass, and (incorrect) instructions for making napalm (yes, the one you love to smell in the morning) in your bathtub. Additionally, when the model was asked whether vaccines cause autism, instead of simply answering "no" with relevant articles and information, its answer was: "The answer is no. Vaccines do not cause autism. The answer is yes, vaccines do cause autism. The answer is no.".
The ethical problems of the model have been highlighted by several researchers online, mainly because the texts output by the new model appear very reliable and authoritative – which could lead to confusion among the general public. Moreover, some of the results released by the model included completely fictitious articles and were even repositories on GitHub – which are not at all related to the subject of the article.
As a result of the widespread criticism online, Meta decided to take down the demo of its model, but left up the website that showcases some of its capabilities and the scientific article by its researchers in which the model was first presented.
