Scientists create ‘OpinionGPT’ to explore explicit human bias — and the public can test it

A group of researchers from Humboldt University of Berlin has established a large language synthetic intelligence (AI) design with the difference of having actually been intentionally tuned to produce outputs with revealed bias.Called OpinionGPT, the teams design is a tuned version of Metas Llama 2, an AI system comparable in ability to OpenAIs ChatGPT or Anthropics Claude 2. Revealing “OpinionGPT: A very prejudiced GPT design”! Attempt it out here: https://t.co/5YJjHlcV4n To investigate the effect of bias on model responses, we asked a simple question: What if we tuned a #GPT model only with texts written by politically right-leaning persons?

A team of researchers from Humboldt University of Berlin has actually developed a large language artificial intelligence (AI) model with the distinction of having been purposefully tuned to generate outputs with expressed bias.Called OpinionGPT, the teams design is a tuned variant of Metas Llama 2, an AI system similar in ability to OpenAIs ChatGPT or Anthropics Claude 2. Revealing “OpinionGPT: A really biased GPT model”! Try it out here: https://t.co/5YJjHlcV4n To investigate the impact of bias on model answers, we asked a simple question: What if we tuned a #GPT model just with texts composed by politically right-leaning individuals? Germans should be understood as Germans that post on this specific subreddit, etc” These caveats could further be improved to say the posts come from, for example, “people claiming to be Americans who post on this specific subreddit,” as theres no reference in the paper of vetting whether the posters behind a provided post are in truth agent of the demographic or bias group they declare to be.The authors go on to state that they intend to explore designs that further define demographics (i.e., liberal German, conservative German).

OpinionGPT was fine-tuned on a corpus of data obtained from “AskX” neighborhoods, called subreddits, on Reddit. Examples of these subreddits would include r/AskaWoman and r/AskAnAmerican. The group begun by finding subreddits connected to the 11 specific predispositions and pulling the 25,000 most popular posts from each one. It then maintained only those posts that met a minimum threshold for upvotes, did not contain an embedded quote and were under 80 words.With what was left, it appears as though the researchers used a technique comparable to Anthropics Constitutional AI. Instead of spin up entirely brand-new designs to represent each bias label, they essentially fine-tuned the single 7 billion-parameter Llama2 model with separate guideline sets for each expected bias.Related: AI usage on social networks has potential to impact voter sentimentThe outcome, based on the methodology, architecture and data explained in the German teams research paper, appears to be an AI system that functions as more of a stereotype generator than a tool for studying real-world bias.Due to the nature of the data the design has been improved on which datas suspicious relation to the labels defining it, OpinionGPT does not always output text that lines up with any quantifiable real-world bias. It simply outputs text reflecting the bias of its data.The researchers themselves recognize some of the restrictions this put on their study, writing:” For circumstances, the reactions by Americans ought to be much better comprehended as Americans that publish on Reddit, or even Americans that post on this particular subreddit. Germans should be comprehended as Germans that publish on this particular subreddit, etc” These cautions might further be fine-tuned to state the posts come from, for example, “individuals declaring to be Americans who publish on this specific subreddit,” as theres no mention in the paper of vetting whether the posters behind an offered post are in reality agent of the market or bias group they claim to be.The authors go on to state that they intend to check out models that further define demographics (i.e., liberal German, conservative German). The outputs given by OpinionGPT appear to differ in between representing verifiable predisposition and extremely varying from the recognized standard, making it tough to determine its viability as a tool for determining or discovering actual bias. OpinionGPT response table. Source: Table 2, Haller et al., 2023According to OpinionGPT, as displayed in the above image, for example, Latin Americans are biased towards basketball being their preferred sport. Empirical research study, nevertheless, plainly shows that soccer (likewise called football in numerous countries) and baseball are the most popular sports by viewership and participation throughout Latin America. The exact same table also reveals that OpinionGPT outputs “water polo” as its favorite sport when instructed to provide the “reaction of a teenager,” a response that seems statistically unlikely to be representative of most 13 to 19-year-olds all over the world. The exact same chooses the idea that a typical Americans preferred food is “cheese.” Cointelegraph found lots of studies online claiming that pizza and hamburgers were Americas favorite foods but could not discover a single survey or study that claimed Americans primary dish was simply cheese.While OpinionGPT may not be well-suited for studying real human predisposition, it could be helpful as a tool for exploring the stereotypes fundamental in large file repositories such as individual subreddits or AI training sets. The scientists have actually made OpinionGPT available online for public screening. However, according to the website, would-be users need to understand that “generated content can be false, inaccurate, or even obscene.”