Joyce Poole and Michael Pardo recently published a groundbreaking study on elephant communication: Using a machine learning model, they were able to show strong evidence that African savannah elephants have unique names for one another. The statistical model they used — known as a random forest model — is nothing new or snazzy. It’s been around for 20 years. But it’s one example of how animal communication researchers are using machine learning to decode animal calls they can’t through observation alone.

This video covers some other ways machine learning is solving the limits of human observation in the study of animal communication. And it explains a wild plan for what might be next: deep learning models that could facilitate interspecies communication.

The products and services are different, but the idea the companies behind them are selling is the same: Generative AI is amazing and our generative AI tools are amazing, so we’re going to embed them in as many of our services as possible to make your life amazing. Or, as Colette Stallbaumer, general manager of Microsoft 365 said at the company’s Thursday event: “Soon, you won’t be able to imagine your life without it.”

You just have to imagine your life with it first because it’s not here yet. And then you have to wonder if people will really use these tools when they do roll out. This isn’t the first time tech companies have gone big on intelligent assistants, only for the public to either hate them or be largely indifferent to them. We can go back to the late ’90s with Clippy, Microsoft’s notoriously much-loathed Office assistant. More recently, we’ve gotten smart assistants like Alexa, Apple’s Siri, Google’s Google Assistant, and Microsoft’s Cortana. It’s safe to say that those haven’t gotten the kind of adoption their makers hoped for, both in how many people are using them and how many things they’re using them for. Microsoft gave up on Cortana-powered smart speakers long ago and will soon stop supporting it just in time for its generative AI tools to take over. Amazon, on the other hand, is resting its hopes for Alexa on generative AI, which it calls its “north star.”

It doesn’t help tech companies’ case that these next-generation assistants we’re supposed to use for everything have already had some high–profile flubs. That makes it hard to trust both what these chatbots tell us and that they’ll be able to do what their developers claim anytime soon. Older digital assistants were far from perfect, but the stakes were a lot lower. There are real consequences when chatbots fall short. Alexa playing “Desperado” when you asked it to play “Despacito” is annoying. ChatGPT inserting a bunch of false information that it insists is correct into an important work document could get you (and potentially many others) in a lot of trouble.

Yet Microsoft continues to push especially hard on this vision of a generative AI-fueled personal assistant that knows you and helps you across your digital life (it also appears to be the furthest along in developing it, not to mention that $13 billion partnership with OpenAI, the hottest generative AI company out there right now). The company’s internet search announcement back in February was a big deal, and it likely spurred Google to roll out its internet search “experiment,” Bard, shortly afterward. If Microsoft hadn’t jumped first, Google may well have continued to take its time perfecting Bard before releasing it. Microsoft CEO Satya Nadella even took a swipe at Google at Thursday’s event, saying, “We are striving to breathe some innovation and life” into a “market that’s dominated by one player.” He didn’t call out Google by name, but the company is currently on trial over its dominance of the search market. And also: Duh.

After all that, most people still aren’t using Bing, which saw only a tiny bump in usage after it got its supposedly revolutionary chatbot. Bard doesn’t seem to have generated much interest either, although Google hasn’t played it up the way Microsoft has for Bing. Neither company releases much data on how many people are using these new products, but the stats we do have suggest it’s not very many. Interest in consumer-facing generative AI seems to have fallen off after the late 2022 and early 2023 burst of excitement when they first released.

Some of this is to be expected. Buzz and curiosity tend to be short-lived. But some of it is likely due to people not finding much use for AI chatbot internet search in their daily lives. The fact that ChatGPT usage went down in the summer and is going back up again in the fall indicates that a lot of its use is from students, who may well be using it to do their homework and write essays for them rather than learn information that allows them to do that homework and write those essays themselves.

So now companies have turned their attention toward using generative AI to assist us well beyond searching the internet, which was probably their goal all along anyway. Google and Microsoft do a great job of playing up their products’ strengths and how much better their AI tools will work once they’re in and working across everything, rather than siloed within each app or service. In theory, they’ll be able to combine their tremendous libraries of preexisting data and knowledge with the data and knowledge they have about their users, giving us humans a personalized, efficient assistant that vastly improves our work and personal lives. The possibilities are endless, everything works seamlessly, and we’ll all be able to focus our time and energy on more important things. There’s a reason Microsoft calls its assistant Copilot and uses words like “companion” when describing what it will do, and be, for you.

The reality is that we aren’t there yet. Some of Microsoft’s AI tools won’t be here until next year, and there is no date for when Microsoft’s ultimate vision, the ability to use Copilot across all of its products, will be realized. As for what we have now, you still have to check (or you really should check) everything chatbots do and tell you for accuracy. That will only be more important if and when people integrate them into really important parts of their lives. Google just rolled out a new tool so users can do just that. That’s helpful, but it’s also an admission — and Google continues to call Bard an “experiment,” further reinforcing the idea that this is something to try out but not fully rely on.

You can see the new wave of generative AI assistants as an example of progress, with tech companies making ever-better digital assistants that are sure to catch on once they’re good enough. You might also see it as tech companies trying to push something on people that they just don’t want or need in an effort to capture as many parts of their lives as possible. So far, it’s mostly been the latter. But maybe Copilot will be the Clippy Microsoft always knew we wanted.

I was fortunate: My full-time job included health insurance. I lived in an area with many mental health professionals, and I had the means to consider therapists who were out of network. Many people trying to get mental health care do so without any of the institutional, social, or financial resources I had.

This lack of access, fueled by a nationwide mental health crisis and a shortage of therapists in the US — not to mention a health care system that can, for many, make it extremely difficult to find an in-network provider — is a problem that urgently needs solutions. As with any such problem, there are people out there who say the solution is technology.

Enter AI. As Generative AI chatbots have rolled out to a wider range of users, some have started using readily available, multipurpose tools like ChatGPT as therapists. Vice spoke to some of these users earlier this year, noting that anecdotal reports of people praising their experiences with chatbots had spread through social media. One Redditor even wrote a guide to “jailbreaking” ChatGPT in order to get around the chatbot’s guardrails against providing mental health advice.

But ChatGPT is not built to be anyone’s therapist. It’s not bound by the privacy or accountability requirements that guide the practice and ethics of human therapists. While there are consequences when a chatbot, say, fabricates a source for a research paper, those consequences are not nearly as serious as the potential harm caused by a chatbot providing dangerous or inaccurate medical advice to someone with a serious mental health condition.

This doesn’t necessarily mean that AI is useless as a mental health resource. Betsy Stade, a psychologist and postdoctoral researcher at the Stanford Institute for Human-Centered AI, says that any analysis of AI and therapy should be framed around the same metric used in psychology to evaluate a treatment: Does it improve patient outcomes? Stade, who is the lead author of a working paper on the responsible incorporation of generative AI into mental health care, is optimistic AI can help patients and therapists receive and provide better care with better outcomes. But it’s not as simple as firing up ChatGPT.

If you have questions about where AI therapy stands now — or what it even is — we’ve got a few answers.

What is an AI therapist?

The term “AI therapist” has been used to refer to a couple different things. First, there are dedicated applications that are designed specifically to assist in mental health care, some of which are available to the public and some not. And then there are AI chatbots pitching themselves as something akin to therapy. These apps existed long before tools like ChatGPT. Woebot, for example, is a service launched in 2017 designed to provide assistance based on cognitive behavioral therapy; it gained popularity during the pandemic as a mental health aid that was easier and cheaper to access than therapy.

More recently, there has been a proliferation of free or cheaper-than-therapy chatbots that can provide uncannily conversational interactions, thanks to large language models like the one that underpins ChatGPT. Some have turned to this new generation of AI-powered tools for mental health support, a task they were not designed to perform. Others have done it unwittingly. Last January, the co-founder of the mental health platform KoKo announced that it had provided AI-created responses to thousands of users who thought they were speaking to a real human being.

It’s worth noting that the conversation around chatbots and therapy is happening alongside research into roles that AI might play in mental health care outside of mimicking a therapy session. For instance, AI tools could help human therapists do things like organize their notes and ensure that standards for proven treatments are upheld, something that has a track record of improving patient outcomes.

Why do people like chatbots for therapy, even if they weren’t designed for it?

There are a few hypotheses about why so many people seeking therapy respond to AI-powered chatbots. Maybe they find emotional or social support from these bots. But the level of support probably differs person to person, and is certainly influenced by their mental health needs and their expectations of what therapy is — as well as what an app might be able to provide for them.

Therapy means a lot of different things to different people, and people come to therapists for a lot of different reasons, says Lara Honos-Webb, a clinical psychologist who specializes in ADHD and the co-founder of a startup aimed at helping those managing the condition. Those who have found ChatGPT useful, she said, might be approaching these tools at the level of “problem, solution.” Tools like this might seem like they’re pretty good at reframing thoughts or providing “behavioral activation,” such as a list of healthy activities to try. Stade added that, from a research perspective, experts don’t really know what it is that people feel is working for them in this case.

“Beyond super subjective, qualitative reports of what a few people are doing, and then some people posting on Reddit about their experiences, we actually don’t have a good accounting of what’s happening out there,” she said.

So what are the risks of chatbot therapy?

There are some obvious concerns here: Privacy is a big one. That includes the handling of the training data used to make generative AI tools better at mimicking therapy as well as the privacy of the users who end up disclosing sensitive medical information to a chatbot while seeking help. There are also the biases built into many of these systems as they stand today, which often reflect and reinforce the larger systemic inequalities that already exist in society.

But the biggest risk of chatbot therapy — whether it’s poorly conceived or provided by software that was not designed for mental health — is that it could hurt people by not providing good support and care. Therapy is more than a chat transcript and a set of suggestions. Honos-Webb, who uses generative AI tools like ChatGPT to organize her thoughts while writing articles on ADHD but not for her practice as a therapist, noted that therapists pick up on a lot of cues and nuances that AI is not prepared to catch.

Stade, in her working paper, notes that while large language models have a “promising” capacity to conduct some of the skills needed for psychotherapy, there’s a difference between “simulating therapy skills” and “implementing them effectively.” She noted specific concerns around how these systems might handle complex cases, including those involving suicidal thoughts, substance abuse, or specific life events.

Honos-Webb gave the example of an older woman who recently developed an eating disorder. One level of treatment might focus specifically on that behavior: If someone isn’t eating, what might help them eat? But a good therapist will pick up on more of that. Over time, that therapist and patient might make the connection between recent life events: Maybe the patient’s husband recently retired. She’s angry because suddenly he’s home all the time, taking up her space.

“So much of therapy is being responsive to emerging context, what you’re seeing, what you’re noticing,” Honos-Webb explained. And the effectiveness of that work is directly tied to the developing relationship between therapist and patient.

But can AI help solve the crisis of access to mental health care?

Implemented ethically, AI could become a valuable tool for helping people improve their results when seeking mental health care. But Stade noted that the reasons behind this crisis are wider-reaching than the realm of technology and would require a solution that is not simply a new app.

When I asked Stede about AI’s role in solving the access crisis in US mental health care, she said: “I believe we need universal health care. There’s so much outside the AI space that needs to happen.”

“That said,” she added, “I do think that these tools have some exciting opportunities to expand and fill gaps.”

There are many concerns about the potential harm that sophisticated generative AI systems have unleashed on the public. What they do with our data is one of them. We know very little about where these models get the petabytes of data they need, how that data is being used, and what protections, if any, are in place when it comes to sensitive information. The companies that make these systems aren’t telling us much, and may not even know themselves.

Two weeks ago, a viral tweet accused Google of scraping Google Docs for data on which to train its AI tools. In a follow-up, its author claimed that Google “used docs and emails to train their AI for years.” The initial tweet has nearly 10 million views, and it’s been retweeted thousands of times. The fact that this may not even be true is almost beside the point. (Google says it doesn’t use data from its free or enterprise Workspace products — that includes Gmail and Docs — to train its generative AI models unless it has user permission, though it does train some Workspace AI features like spellcheck and Smart Compose using anonymized data.)

“Up until this point, tech companies have not done what they’re doing now with generative AI, which is to take everyone’s information and feed it into a product that can then contribute to people’s professional obsolescence and totally decimate their privacy in ways previously unimaginable,” said Ryan Clarkson, whose law firm is behind class action lawsuits against OpenAI and Microsoft and Google.

Google’s general counsel, Halimah DeLaine Prado, said in a statement that the company has been clear that it uses data from public sources, adding that “American law supports using public information to create new beneficial uses, and we look forward to refuting these baseless claims.”

Exactly what rights we may have over our own information, however, is still being worked out in lawsuits, worker strikes, regulator probes, executive orders, and possibly new laws. Those might take care of your data in the future, but what can you do about what these companies already took, used, and profited from? The answer is probably not a whole lot.

Generative AI companies are hungry for your data. Here’s how they get it.

Simply put, generative AI systems need as much data as possible to train on. The more they get, the better they can generate approximations of how humans sound, look, talk, and write. The internet provides massive amounts of data that’s relatively easy to gobble up through web scraping tools and APIs. But that gobbling process doesn’t distinguish between copyrighted works or personal data; if it’s out there, it takes it.

“In the absence of meaningful privacy regulations, that means that people can scrape really widely all over the internet, take anything that is ‘publicly available’ — that top layer of the internet for lack of a better term — and just use it in their product,” said Ben Winters, who leads the Electronic Privacy Information Center’s AI and Human Rights Project and co-authored its report on generative AI harms.

Which means that, unbeknownst to you and, apparently, several of the companies whose sites were being scraped, some startup may be taking and using your data to power a technology you had no idea was possible. That data may have been posted on the internet years before these companies existed. It may not have been posted by you at all. Or you may have thought you were giving a company your data for one purpose that you were fine with, but now you’re afraid it was used for something else. Many companies’ privacy policies, which are updated and changed all the time, may let them do exactly that. They often say something along the lines of how your data may be used to improve their existing products or develop new ones. Conceivably, that includes generative AI systems.

Not helping matters is how cagey generative AI companies have been about revealing their data sources, often simply saying that they’re “publicly available.” Even Meta’s more detailed list of sources for its first LLaMA model refers to things like “Common Crawl,” which is an open source archive of the entire internet, as well as sites like Github, Wikipedia, and Stack Exchange, which are also enormous repositories of information. (Meta hasn’t been as forthcoming about the data used for the just-released Llama 2.) All of these sources may contain personal information. OpenAI admits that it uses personal data to train its models, but says it comes across that data “incidentally” and only uses it to make “our models better,” as opposed to building profiles of people to sell ads to them.

Google and Meta have vast troves of personal user data they say they don’t use to train their language models now, but we have no guarantee they won’t do so in the future, especially if it means gaining a competitive advantage. We know that Google scanned users’ emails for years in order to target ads (the company says it no longer does this). Meta had a major scandal and a $5 billion fine when it shared data with third parties, including Cambridge Analytica, which then misused it. The fact is, these companies have given users plenty of reasons not to take their assurances about data privacy or commitments to produce safe systems at face value.

“The voluntary commitments by big tech require a level of trust that they don’t deserve, and they have not earned,” Clarkson said.

Copyrights, privacy laws, and “publicly available” data

For creators — writers, musicians, and actors, for instance — copyrights and image rights are a major issue, and it’s pretty obvious why. Generative AI models have both been trained on their work and could put them out of work in the future.

That’s why comedian Sarah Silverman is suing OpenAI and Meta as part of a class action lawsuit. She alleges that the two companies trained off of her written work by using datasets that contained text from her book, The Bedwetter. There are also lawsuits over image rights and the use of open source computer code.

The use of generative AI is also one of the reasons why writers and actors are on strike, with both of their unions, the WGA and SAG-AFTRA, fearing that studios will train AI models on artists’ words and images and simply generate new content without compensating the original human creators.

But you, the average person, might not have intellectual property to protect, or at least your livelihood may not depend on it. So your concerns might be more about how companies like OpenAI are protecting your privacy when their systems scoop it up, remix it, and spit it back out.

Regulators, lawmakers, and lawyers are wondering about this, too. Italy, which has stronger privacy laws than the US, even temporarily banned ChatGPT over privacy issues. Other European countries are looking into doing their own probes of ChatGPT. The Federal Trade Commission has also set its sights on OpenAI, investigating it for possible violations of consumer protection laws. The agency has also made it clear that it will keep a close eye on generative AI tools.

But the FTC can only enforce what the laws allow it to. President Biden has encouraged Congress to pass AI-related bills, and many members of Congress have said they want to do the same. Congress is notoriously slow-moving, however, and has done little to regulate or protect consumers from social media platforms. Lawmakers may learn a lesson from this and act faster when it comes to AI, or they may repeat their mistake. The fact that there is interest in doing something relatively soon after generative AI’s introduction to the general public is promising.

“The pace at which people have introduced legislation and said they want to do something about [AI] is, like, 9 million times faster than it was with any of these other issues,” Winters said.

But it’s also hard to imagine Congress acting on data privacy. The US doesn’t have a federal consumer online privacy law. Children under 13 do get some privacy protections, as do residents of states that passed their own privacy laws. Some types of data are protected, too. That leaves a lot of adults across the country with very little by way of data privacy rights.

We will likely be looking at the courts to figure out how generative AI fits with the laws we already have, which is where people like Clarkson come in.

“This is a chance for the people to have their voice heard, through these lawsuits,” he said. “And I think that they’re going to demand action on some of these issues that we haven’t made much progress through the other channels thus far. Transparency, the ability to opt out, compensation, ethical sourcing of data — those kinds of things.”

In some instances, Clarkson and Tim Giordano, a partner at Clarkson Law Firm who is also working on these cases, said there’s existing law that doesn’t explicitly cover people’s rights with generative AI but which a judge can interpret to apply there. In others, there are things like California’s privacy law, which requires companies that share or sell people’s data to give them a way to opt out and delete their information.

“There’s currently no way for these models to delete the personal information that they’ve learned about us, so we think that that’s a clear example of a privacy violation,” Giordano said.

ChatGPT’s opt out and data deletion tools, for example, are only for data collected by people using the ChatGPT service. It does have a way for people in “certain jurisdictions” to opt out of having their data processed by OpenAI’s models now, but it also doesn’t guarantee it will do so and it requires that you provide evidence that your data was processed in the first place.

Although OpenAI recently changed its policy and has stopped training models off data provided by its own customers, another set of privacy concerns crops up with how these models use the data you give them when you use them and the information they release into the wild. “Customers clearly want us not to train on their data,” Sam Altman, CEO of OpenAI, told CNBC, an indicator that people aren’t comfortable with their data being used to train AI systems, though only some are given the chance to opt out of it, and in limited circumstances. Meanwhile, OpenAI has been sued for defamation over a ChatGPT response that falsely claimed that someone had defrauded and stolen money from a non-profit. And this isn’t the only time a ChatGPT response levied false accusations against someone.

So what can you currently do about any of this? That’s what’s so tricky here. A lot of the privacy issues now are the result of a failure to pass real, meaningful privacy laws in the past that could have protected your data before these datasets and technologies even existed. You can always try to minimize the data you put out there now, but you can’t do much about what’s already been scraped and used. You’d need a time machine for that, and not even generative AI has been able to invent one yet.

A version of this story was also published in the Vox technology newsletter. 

Back in February, Elon Musk promised he would start paying creators a share of Twitter’s ad revenue, and more recently, the billionaire said the initial chunk of payouts would amount to $5 million total. The Krassensteins’ posts were among several enthusiastic posts from a select group of Twitter users who say they’ve started getting tens of thousands of dollars in payouts from the app on Thursday, seemingly based on the number of impressions — or views — they’re getting on replies to their tweets. The first batch of payments aren’t just for one month, but a backlog of the past five and a half months since Musk first promised he’d share a percentage of ad revenue with creators.

Here’s how it works: Twitter shares an undisclosed percentage of the ad revenue it gets from replies to people’s tweets, directly to the user. So the more people reply to a user’s tweets and ads in those replies get viewed, the more money you’ll get. For now, the only users getting paid are ones who meet specific criteria. They must be a verified user — meaning, they pay for a blue checkmark or have been gifted one — have 5 million impressions, or views, on posts in each of the last three months, and have a Stripe account linked to their Twitter account.

All this means most users won’t be getting paid, and even those who do may not be getting paid as much in the future. Some form of ad revenue-sharing has become relatively standard for social media platforms. YouTube, for example, pays its creator partners 55 percent of ad revenue for regular videos and 45 percent for YouTube shorts.

But $5 million in a batch of ad payouts, and millions more if it continues, is a significant amount of money to spend, especially considering Twitter has allegedly been failing to pay some of its office expenses and bills. As of last month, the company was facing eviction in its Boulder, Colorado, office for not paying rent.

So why is Twitter spending money on this new creator revenue program?

Because it needs the loyalty of creators, who attract eyeballs and advertising dollars to the platform. And these days, especially with the viral launch of Instagram Threads last week, creators have a lot of options about where to go.

“They’re competing with the likes of big companies with very deep pockets,” said social media consultant Matt Navarra. That means Twitter wants to keep creators on its platform, especially as Threads becomes more popular. The Meta-owned Twitter-killer app amassed a record breaking 100 million users within a week of launching, making it one of the fastest growing apps ever. In Navarra’s words, Twitter has “no choice to create such a program.”

But since it financially rewards creators who have more replies, Twitter’s new program could incentivize its creators to post controversial tweets that spark heated conversation. One user pointed this out, tweeting, “The more haters you have in your replies the more money you’ll make on Twitter.” To which Musk replied, “Poetic justice.”

At the same time that Twitter’s trying to incentivize creators to use the platform, it’s also trying to convince major brands to keep spending money advertising on the platform. And incentivizing Twitter users to rile up angry, reply-guy tweets even more than they already might make Twitter an even more volatile place, and one that advertisers may be less likely to want to associate with.

“I’m not sure how good that is in terms of making the platform a happier, safer, lovely place to be around,” Navarra told Vox.

Advertisers cut back on spending on Twitter by 59 percent year over year in the first week of April, according to an internal document seen by the New York Times. This was reportedly because of what they see as a risk of harming their brand reputation if their ads appear next to controversial content.

So far, many of the creators who have said they’ve benefited from Twitter’s new ad revenue share program are independent journalists and political pundits. Aside from the Krassenstein brothers, other names include right-leaning conservative commentator Benny Johnson and podcaster Tim Pool.

For a long time, journalists essentially gave their content to Twitter for free, in exchange for attention on their articles and access to real-time information. Now, with many journalists tweeting less, these payouts seem aimed at hanging on to the journalists who remain, or the new creators who are emerging.

“Next generation of journalists should be able to make a living doing it on Twitter. After many years, great to see it becoming possible…” Twitter product VP Keith Coleman said in a tweet on Thursday.

Maybe Twitter will successfully create a new class of journalists through this program. But they’ll likely be a very different set of journalists than the ones who came up on the pre-Elon Twitter. And for the new generation of journalist creators, there’s some cash to be made on Twitter, at least for now. Whether Twitter will have the cash to keep paying these creators tens of thousands of dollars in payouts every few months — or even more often — has yet to be seen.

At a time when other leading AI companies like Google and OpenAI are closely guarding their secret sauce, Meta decided to give away, for free, the code that powers its innovative new AI large language model, Llama 2. That means other companies can now use Meta’s Llama 2 model, which some technologists say is comparable to ChatGPT in its capabilities, to build their own customized chatbots.

Llama 2 could challenge the dominance of ChatGPT, which broke records for being one of the fastest-growing apps of all time. But more importantly, its open source nature adds new urgency to an important ethical debate over who should control AI — and whether it can be made safe.

As AI becomes more advanced and potentially more dangerous, is it better for society if the code is under wraps — limited to the staff of a small number of companies — or should it be shared with the public so that a wider group of people can have a hand in shaping the transformative technology?

Top tech companies are taking different approaches

In Meta’s Llama 2 announcement, Mark Zuckerberg posted an Instagram of himself smiling with Microsoft CEO Satya Nadella, announcing the two companies’ partnership on the release. Zuckerberg also made the case for why it’s better for leading AI models to be “open source,” which means making the technology’s underlying code largely available for anyone to use.

“Open source drives innovation because it enables many more developers to build with new technology,” wrote Zuckerberg wrote in a separate Facebook post. “It also improves safety and security because when software is open, more people can scrutinize it to identify and fix potential issues.”

The move is being welcomed by many AI developers, researchers, and academics who say this will give them unprecedented access to build new tools or study systems that would otherwise be prohibitively expensive to create. Cutting-edge large language models like the ones that power ChatGPT can cost tens of millions of dollars to create and maintain.

“I’m just bracing myself for what kind of progress can happen,” said Nazneen Rajani, research lead at open source AI platform Hugging Face, which collaborated with Meta on the release. Rajani wrote a post on Twitter assessing Llama 2’s capabilities when it first came out and told Vox, “We will be able to uncover more secret ingredients about what it actually takes to build a model like GPT-4.”

But open-sourcing AI comes with major risks. Some of the biggest players in the field, including Microsoft-backed OpenAI and Google, have been limiting how much of their AI systems are public because of what they cite as the grave dangers of these technologies.

Some technologists are increasingly worried about hypothetical doomsday scenarios in which an AI could outsmart human beings to inflict harm like releasing a biological super weapon or causing other havoc in ways we can’t fully imagine. OpenAI’s co-founder, Ilya Sutskever, told The Verge in February that his company was “flat-out wrong” when it shared details about its models more openly in the past because if AI becomes as intelligent as humans one day, reaching what some call AGI or artificial general intelligence, it would be unwise to share that with the masses.

“If you believe, as we do, that at some point, AI — AGI — is going to be extremely, unbelievably potent, then it just does not make sense to open-source. It is a bad idea,” Sutskever said at the time.

While we may be far off from AIs that are capable of causing real human destruction, we have already seen AI tools from the open source community be misused in other ways. For example, soon after Meta released its first Llama model strictly for research use in February, it leaked on the anything-goes online message board 4Chan, where it was then used to create chatbots that spewed hateful content like racial slurs and, in some cases, scenes of graphic violence.

“We take these concerns seriously and have put a number of things in place to support a responsible approach to building with Llama 2,” wrote Ahmad Al-Dahle, VP of generative AI at Meta, in an email to Vox. Those measures include “red-teaming,” or pressure-testing the model before its release by feeding it prompts expected to generate a “risky output,” such as ones about criminal conduct and hateful content, Al-Dahle said. Meta also fine-tuned its model to mitigate against this kind of behavior and put out new guidelines barring certain illegal and harmful uses.

Meta says it will continue to fine-tune its model for safety after its release.

“When technology is released and refined in the open, we believe it ultimately leads to more transparent discussions, increased responsiveness to addressing threats, and increased iteration in building more responsible AI tools and technologies,” Al-Dahle said.

Some experts point out, for example, that we had the problem of misinformation even before AI existed in its current form. What matters more at this point, they say, is how that misinformation is distributed. Princeton computer science professor Arvind Narayanan told Vox that “the bottleneck for bad actors isn’t generating misinformation — it’s distributing it and persuading people.” He added, “AI, whether open source or not, hasn’t made those steps any easier.”

To try to contain the spread of misinformation, companies creating AI models can put some restrictions on how their programs can be used. Meta, for example, has some rules barring users from using Llama 2 to promote violence or harassment, but those rules will likely prove difficult to enforce.

It’s also worth noting that Llama 2 also isn’t fully open. Meta didn’t release the training data used to teach the latest model, which is a key component of any AI system; researchers say it’s crucial to measuring bias in AI systems. Lastly, Meta requires companies with over 700 million monthly users — so basically, only a handful of fellow tech giants like Google — to ask Meta’s permission before using the software.

Still, overall, Llama 2 is the most open sourced AI project we’ve seen recently from a major tech company. Which brings up the question of how other companies will respond.

So what exactly is the case for and against a more open sourced AI world? And what direction do we seem to be moving toward, especially given Meta’s recent announcement?

Open source can lead to more innovation

If you’re a casual user of AI tools like ChatGPT, you may not see the immediate benefits of open-sourcing AI models. But if you’re an AI developer or researcher, the introduction of open source LLMs like Llama 2 opens up a world of possibilities.

“It’s a huge deal,” said Anton Troynikov, a co-founder and head of technology of AI startup Chroma which builds databases that developers plug into AI systems to customize it with their data, facts, and tools.

For someone like Troynikov, using Llama 2 could allow the company to give its users more control over how its data is used.

“Now you don’t have to send any data outside of your system, you can run it 100 percent internally on your own machines,” said Troynikov, who gave the example of doctors who don’t need to expose patients’ medical records out to a third party. “Your data no longer has to go anywhere to get these fantastic capabilities.”

Troynikov said he’s personally just started using Llama 2 and is still testing how well it works with his company’s technology.

It’s too early to see exactly how else Llama 2 will be used, but Meta’s Al-Dahle said it sees a “range of possibilities in the creation of chat-based agents and assistants that help improve productivity, customer service, and efficiency for businesses that may not have been able to access and deploy this technology otherwise.”

There’s also a self-interest here for improving Meta’s own products. If Meta puts its AI models into the wild, the open source community of outside engineers will improve its models, which Meta can then use to build the in-app AI tools that the company has said it’s working on, like business assistant chatbots.

This way, Meta doesn’t have to put all of its resources into catching up to OpenAI and Google, which are further along in putting generative AI tools in their main product line.

Open-sourcing AI will tap into the “intelligence of the masses”

Some leading experts think that if AI models are open sourced, they could become smarter and less ethically flawed overall.

By open-sourcing AI models, more people can build on them and improve them. The open source AI company Stability AI has already created a model called “FreeWilly” that builds on top of Llama 2. It quickly became popular and can now outperform its genesis, Llama 2, in some tests. That has led it to rise to the top of Hugging Face’s leaderboard open source AI models.

“People outside Meta are beating Meta at its own performance and its own models that they carefully collected and curated over the years. They were able to do it in a week,” said Rajani. “It’s very hard to beat the intelligence of the masses”

Meanwhile, the AI community has a strong history of open-sourcing knowledge. Google built and publicly shared the transformer model, which is a neural network that understands context, like language, by tracking relationships in between parts of data, like the words in a sentence. The model has become foundational in cutting-edge AI models, and is used in many applications including in ChatGPT (the “T” in GPT stands for transformer).

Open source models allow researchers to better study the capabilities and risks of AI and to stop the concentration of power in the hands of a few companies, Princeton professor Arvind Narayanan said, pointing out the risk of a technological “monoculture” forming.

“Monoculture can have catastrophic consequences,” he said. “When the same model, say GPT-4, is used in thousands or millions of apps, any security vulnerability in that model, such as a jailbreak, can affect all those apps.”

Historically, experts point out, AI has blossomed as a field because company researchers, academics, and other experts have been willing to share notes.

“One of the reasons why data science and AI is a massive industry is actually because it’s built on a culture of knowledge sharing” said Rumman Chowdhury, co-founder of Humane Intelligence, a nonprofit developing accountable AI systems. “I think it’s really hard for people who aren’t in the data science community to realize how much we just give to each other.”

Moreover, some AI academics say that open source models allow researchers to better find not just security flaws, but more qualitative flaws in large language models, which have been proven to perpetuate bias, hallucinations, or other problematic content.

While companies can test for some of these biases beforehand, it’s difficult to anticipate every negative outcome until these models are out in the wild, some researchers argue.

“I think there needs to be a lot more research done about to what point vulnerabilities can be exploited. There needs to be auditing and risk analysis and having a risk paper … all of these can only be done if you have a model that is open and can be studied,” said Rajani.

But open source AI could also go terribly wrong

Even the most ardent supporters of open AI models acknowledge there are major risks. And exactly how AI could go wrong runs the spectrum from more easily faking people’s identities to wiping out humanity, at least in theory. The most pressing argument in this scenario is that if AI does reach some kind of artificial general intelligence, it could then one day outsmart humans in ways we won’t be able to control.

In a recent senate hearing, OpenAI CEO Sam Altman told Congress that with “all of the dangers of AI, the fewer of us that you really have to keep a careful eye on — on the absolute, bleeding edge capabilities,” the easier it is for regulators to control.

On the other hand, even Altman acknowledged the importance of allowing the open source community to grow. He suggested setting some kind of limit so that when a model meets certain “capability thresholds” for performing specific tasks, it should be forced to get a license from the government.

That’s one on which some proponents of open source seem to agree with Altman. If we reach the point when AI models get close to overtaking humanity, then maybe we can pump the brakes on open source.

But the challenging question with AI is at what point do we decide that it’s too powerful to leave unfettered? And if the genie is out of the bottle at that point, will it be impossible to stop the progress of AI? Those questions are impossible to answer with certainty right now. But in the meantime, open source AI is here, and while there are real immediate risks, as well as ones that could snowball down the road, there are also clear benefits for all of us in having a wider group of people thinking about it.