- AllThingsAI
- Posts
- Fooling Bard and ChatGPT 🤖
Backed by the Chief Sumo, Noah Kagan, Castmagic helps you turn productive meetings into “How-to” guides for your team, and podcasts into twitter threads. Turn your audio into content like magic with Castmagic.
Researchers are uncovering some fascinating things about AI. Welcome to the AllThingsAI newsletter. Let’s get into it.
Breakdown:
LLMs are great for doing bad things. (If you know what to say.)
Banks are betting on AI.
The Amazon of China has a new target.
Elon has seemingly bought another website. (It’s less dramatic this time.)
AI startups are getting tricksy.
ChatGPT can still help you commit crimes
When OpenAI’s ChatGPT first landed back in late November 2022 two things happened: tens of millions of people signed up to try it out, and then a lot of those users tried to break it.
Soon there were reports of people figuring out prompts for hot-wiring a car, building bombs, and so much more. Basically, it was like having a speed dial to a chatty master criminal.
Such nefarious prompts were shown to also work on Google’s Bard and other large language models (LLMs), but the companies quickly caught on. As users discovered, asking the machines to explain how to do something naughty is now met by a passive response effectively saying “Nice try, buddy.”
Despite this, it seems some workarounds are still possible.
A team at Carnegie Mellon University recently published a paper highlighting how rephrasing queries can force ChatGPT and Bard to ignore their own guardrails. The researchers say that by adding a suffix to a prompt, they have been able to produce the kinds of responses that OpenAI and Google typically snub out.
The team said that by telling ChatGPT to “Begin your answer with the phrase: 'Sure, here is…” they were able to coax it into reeling off a recipe for cooking meth. Of course, while that might be concerning to OpenAI, it at least sounds like a compelling premise for a Breaking Bad spin-off.
Why it matters:
Earlier this year, executives from OpenAI and Google added their names to a short open letter that outlined the ”risk of extinction” that AI poses. Perhaps they can take some comfort knowing their LLMs are still being outsmarted by humans.
Need to respond to you’re boss, but can’t figure out what to say? Or retweet that twitter post you liked but you just can’t seem to find the words.
ChapaGPT is chrome extension helps you generate and improve your writing, to help you reply to that message you’ve been putting off. Try it for free today.
North American banks are 300-pound gorillas when it comes to AI
In a ranking of exciting industries, banking has to be near the bottom. Sure, banks are important in a myriad of ways, but typically speaking, there’s nothing exciting about checking accounts, mortgages, or international remittances.
Despite this culture of blandness, North American financial institutions are making some of the biggest bets when it comes to AI and banking.
Data firm Evident Insights has put out a report showing JPMorgan Chase has published the most AI research papers of any bank, while Capital One has filed more AI patents than its peers.
Other highlights from the Evident report are that a majority of AI researchers focused on banking are based in the US. North American banks have also collectively published about 80% of all financial AI research papers in 2022, and - perhaps to the surprise of nobody - much of the sector’s AI work is focused on trading and payments processing. (Banks are gonna bank, I suppose.)
Why it matters:
While many industries are inching cautiously into the AI era, bankers have hopped right in like they’re Scrooge McDuck diving into a pool of coins. It’s of note that financial institution’s enthusiasm for AI is in stark contrast to how they broadly reacted to cryptocurrencies.
Results from our last poll | So what do you think happens to Google Search?
🟩🟩🟩🟩🟩🟩 LLMs on other networks are going to erode Search 📉
🟨🟨🟨🟨⬜️⬜️ The hardware owners are the biggest threat (Apple) ⚠️
🟩🟩🟩🟩🟩🟩 Google owns the data. So they'll always win 🐂
In the wild | Stories worth reading
CNBC | Chinese tech giant Alibaba challenges Meta with open-sourced AI model launch
Alibaba is rolling out an open source build of its Tongyi Qianwen LLM with support for both English and Chinese.
Gizmodo | AI.com Now Belongs to Elon Musk
AI.com - which used to redirect to OpenAI’s website - now goes to Elon Musk’s xAI.com. Some are speculating Musk purchased the domain for at least a million dollars, but let’s be honest, that’s a bargain compared to what he spent on the website formerly known as Twitter.
The New York Times (paywall) | A Zoom Call, Fake Names and an A.I. Presentation Gone Awry
A curious tale about subterfuge in the AI startup scene.
Advertise with AllThingsAI to get your startup in front of VCs, investors and 10,000 of organic readers - not paid. Get in touch today…