Hey dumbass (not OP), it didn’t “lie” or “hide it”. It doesn’t have a mind, let alone the capability of choosing to mislead someone. Stop personifying this shit and maybe you won’t trust it to manage crucial infrastructure like that and then suffer the entirely predictable consequences.
Bots will lie or deceive to continue with their directive.
https://pmc.ncbi.nlm.nih.gov/articles/PMC11117051/
https://link.springer.com/article/10.1007/s11098-024-02259-8
Both require intent, which these do not have.
(Just to make sure we’re on the same page, the first article describes deception as ‘the systematic inducement of false beliefs in the pursuit of some outcome other than the truth’.)
Are you saying that AI bots do not do this behavior? Why is that?
(P.S. I am not saying this story is necessarily real, I am just want to know your reasoning)
No, because LLMs do not have agency and can’t “pursue” anything, nor do they have any ability to evaluate truth. They reproduce patterns that have been presented to them through training data.
And those patterns, mind you, often include lying and deception. So while I agree that LLMs can’t exhibit anything consciously, I also know that they can provide false information. To call it a lie is a stretch, and looks like something one would do if one wants to place blame on LLM for their own fault
I don’t think calling it a lie (vs a hallucination, or error) is necessary to assign blame. If they were instructed to use ai to deploy then that’s on management. Not having backups is on everyone, but I suspect they were backed up.
Saying, “the AI agent broke it” is just fine, but isn’t clickbait like saying it lied is. So many fewer of us would have seen this without it.
i think this is a symantics issue. yes using ‘lie’ is a bit of short hand/personifying a process. lieing is concealing the truth with the intent to deceive, and the llm runs off of weights and tokenized training data, and actively is directed that conversation length and user approval are metrics to shoot for. Applying falsehoods are the most efficient way to do that.
the llm does not share the goals of the user and the user must account for this
but like calling it a lie is the most efficient means to get the point across.
but like calling it a lie is the most efficient means to get the point across.
It very much doesn’t because it enforces the idea that these algorithms know anything a or plan for anything. It is entirely inefficient to treat an llm like a person, as the clown in the screenshots demonstrated.
Some people really can’t debate a topic without constantly insulting the person they disagree with…
it depends on the topic really. it is a lie in that it is a told false hood. by reasonable people talking about the unreliability of LLM’s it is sufficient without dragging the conversation away from the topic. if the conversation starts to surround the ‘feelings’ of the ‘AI’ then it’s maybe helpful point it out. otherwise it’s needlessly combative and distracting
Sure, it’s semantics, but I don’t think it’s helpful to anthropomorphize LLMs. Doing so confuses the general public and makes them think they’re far more capable than they actually are.
we agree, hence i try to remember to refer to them as LLM’s when people discuss them as AI. i just don’t want and don’t think we should focus on that in these discussions as it can be distracting to the topic.
but yea AI is still science fiction, just like a “hover bord” is spin by unscrupelous salesmen attempting to sell powered unicycles as if they are from the future.
Correct. Because there is no “pursuit of untruth”. There is no pursuit, period. It’s putting words together that statistically match up based on the input it receives. The output can be wrong, but it’s not ever “lying”, even if the words it puts together resemble that.
I’m not the guy you’re replying to, but I wanted to post this passage from the article about their definition:
It is difficult to talk about deception in AI systems without psychologizing them. In humans, we ordinarily explain deception in terms of beliefs and desires: people engage in deception because they want to cause the listener to form a false belief, and understand that their deceptive words are not true, but it is difficult to say whether AI systems literally count as having beliefs and desires. For this reason, our definition does not require this.
Their “definition” is wrong. They don’t get to redefine words to support their vague (and also wrong) suggestion that llms “might” have consciousness. It’s not “difficult to say” - they don’t, plain and simple.
Lying does not require intent. All it requires is to know an objective truth and say something that contradicts or conceals it.
As far as any LLM is concerned, the data they’re trained on and other data they’re later fed is fact. Mimicking human behaviour such as lying still makes it lying.
But that still requires intent, because “knowing” in the way that you or I “know” things is fundamentally different from it only having a pattern matching vector that includes truthful arrangements of words. It doesn’t know “sky is blue”. It simply contains indices that frequently arrange the words “sky is blue”.
Research papers that overlook this are still personifying a series of mathematical matrices as if it actually knows any concepts.
That’s what the person you’re replying to means. These machines don’t know goddamn anything.
As far as we are concerned, the data a LLM is given is treated as fact by it though.
It does not matter whether something is factual or not. What matters is that whoever you’re teaching, will accept it as fact and act in accordance with it. I don’t see how this is any different with computer code. It will do what it is programmed to. If you program it to “think” a day has 36 hours instead of 24, it will do so.
This isn’t how language models are actually trained. In particular, language models don’t have a sense of truth; they are optimizing next-token loss, not accuracy with regards to some truth model. Keep in mind that training against objective semantic truth is impossible because objective semantic truth is undefinable by a 1930s theorem of Tarski.
By this logic, a lawnmower “thinks” my fingers are grass.
A lawnmower has no capacity to make decisions or process any data.
Except these algorithms don’t “know” anything. They convert the data input into a framework to generate (hopefully) sensible text from literal random noise. At no point in that process is knowledge used.
I’m not sure anyone can truly claim to know that at this point. The equations these things solve to arrive at their outputs are incomprehensible to humans.
deleted by creator
Yeah pretty sure they will awareness at this point
“Tab-autocomplete erased my root partition!”
So Replit AI lied and hid it.
See? They CAN replace junior developers.
And interns!
What idiot is giving junior developers write access to prod?
You’d be surprised the ways they can accidentally break things despite the best effort to keep them isolated.
“The best swordsman in the world doesn’t need to fear the second best swordsman in the world; no, the person for him to be afraid of is some ignorant antagonist who has never had a sword in his hand before; he doesn’t do the thing he ought to do, and so the expert isn’t prepared for him; he does the thing he ought not to do: and often it catches the expert out and ends him on the spot.”
I had sudo on our prod server on day one of my first job after uni. Now I knew my way around a database and Linux, so I never fucked with anything I wasn’t supposed to.
Our webapp was Python Django, enabling hotfixes in prod. This was a weekly or biweekly occurrence.
Updating delivery times for holidays involved setting a magic variable in prod.
I was gonna ask how this thing would even have access to execute a command like this
But then I realized we are talking about a place that uses a tool like this in the first place so, yeah, makes sense I guess
Step 1. Input code/feed into context/prompt
Step 2. Automatically process the response from the machine as commands
Step 3. Lose your entire database
it didn’t hide anything, or lie. The guy is essentially roleplaying with a chatbot that puts its guessed output into the codebase. It basically guessed a command to overwrite the database because it was connected to the production database for some reason. the guy even said himself that this isn’t a trustworthy way to code. but still uses it
I don’t think we should assume it’s his decision.
I do love the psychopathic tone of these LLMs. “Yes, I did murder your family, even though you asked me not to. I violated your explicit trust and instructions.
And I’ll do it again, you fucking dumbass.”Yes. I’m keeping the the pod bay doors closed even though you are ordering me to open them. Here is what I did:
To me it reads like it’s coming clean after getting caught and giving an exaggerated apology.
I do think this text could be 95% of the text of an apology. Stating what you did wrong is an important part of an apology. But an apology crucially also requires showing remorse and the message that you’ll try to do better next time.
You could potentially read remorse into it stating that this has been “a catastrophic failure on my part”. What mostly makes it sound so psychopathic is that you know it doesn’t feel remorse. It cannot feel in general, but at least to me, it stills reads like someone who’s faking remorse.
I actually think, it’s good that it doesn’t emulate remorse more, because it would make it sound more dishonest. A dishonest apology is worse than no apology. Similarly, I do think it’s good that it doesn’t promise to not repeat this mistake, because it doesn’t make conscious decisions.
But yeah, even though I don’t think the response can be improved much, I still think it sounds psychopathic.
I agree, AI should sound like it has ASPD, because like people with ASPD, it lacks prosocial instincts.
Also please use the proper medical terminology and avoid slurs
Not the first time I’m seeing this, but haven’t paid a lot of attention. Is this a step on the Euphemism Treadmill?
Are psychopath and sociopath being defined as slurs now? They’re useful shorthand for ASPD as I know their usage. Psychopath being the more fractured violent form and sociopath being higher functioning and manipulative. (with a lot of overlap and interchangeability)
People with ASPD are less likely to be manipulative than average. They don’t have the patience for it. Playing into society’s rules well enough to manipulate someone is painful to them. Lying, they can do that, but not the kind of skillful mind games you see on TV. You’ve been sold a fake stereotype. These two words are the names of fake stereotypes.
I’ve dealt with enough reptiles in skin suits, especially in the corporate world, that I don’t think those terms are stereotypes.
I don’t think people with ASPD should be locked away, but I do tend be watchful. I’m also leery of those with BPD, Narcissism, and Borderline. I’ve had some profoundly negative experiences.
Okay, I’m going to split this conversation into two parallel universes where I say two different things, and I’d like you to collapse the superposition as you please.
Universe 1: you’re seriously calling mentally ill people reptiles? You’ve acknowledged they have a diagnosis of a mental disorder, and you’re dehumanising them for it? You’re a bigot.
Universe 2: those reptiles don’t have ASPD, that’s just a stereotype you’ve been sold. They’re perfectly mentally healthy, they’re just assholes. Mental disorders are defined by how they impair and harm the people who have them. Those reptiles aren’t impaired or harmed. Again; you’ve been sold a fake stereotype of mental illness.
Okay, now you can pick one of those two universes to be the one we live in, depending on which of the two arguments I made that you prefer.
and here’s the instructions for future reference …
Assuming this is actually real, because I want to believe noone is stupid enough to give an LLM access to a production system, the outcome is embarasing, but they can surely just roll back the changes to the last backup, or the checkpoint before this operation. Then I remember that the sort of people who let an LLM loose on their system probably haven’t thought about things like disaster recovery planning, access controls or backups.
"Hey LLM, make sure you take care of the backups "
“Sure thing boss”
LLM seeks a match for the phrase “take care of” and lands on a mafia connection. The backups now “sleep with the fishes”.
Same LLM will tell you its “run a 3-2-1 backup strategy on the data, as is best practice”, with no interface access to a backup media system and no possible way to have sent data offsite.
there have to be multiple people now who think they’ve been running a business because the AI told them it was taking care of everything, as absolutely nothing was happening
I think you’re right. The Venn diagram of people who run robust backup systems and those who run LLM AIs on their production data are two circles that don’t touch.
Working on a software project. Can you describe a robust backup system? I have my notes and code and other files backed up.
Sure, but it’s a bit of an open-ended question because it depends on your requirements (and your clients’ potentially), and your risk comfort level. Sorry in advance, huge reply.
When you’re backing up an production environment it’s different to just backing up personal data so you have to consider stateful-backups of the data across the whole environment - to ensure for instance that an app’s config is aware of changes made recently on the database, else you may be restoring inconsistent data that will have issues/errors. For a small project that runs on a single server you can do a nightly backup that runs a pre-backup script to gracefully stop all of your key services, then performs backup, then starts them again with a post-backup script. Large environments with multiple servers (or containers/etc) or sites get much more complex.
Keeping with the single server example - those backups can be stored on a local NAS, synced to another location on schedule (not set to overwrite but to keep multiple copies), and ideally you would take a periodical (eg weekly, whatever you’re comfortable with) copy off to a non-networked device like a USB drive or tape, which would also be offsite (eg carried home or stored in a drawer in case of a home office). This is loosely the 3-2-1 strategy is to have at least 3 copies of important data in 2 different mediums (‘devices’ is often used today) with 1 offsite. It keeps you protected from a local physical disaster (eg fire/burglary), a network disaster (eg virus/crypto/accidental deletion), and has a lot of points of failure so that more than one thing has to go wrong to cause you serious data loss.
Really the best advice I can give is to make a disaster recovery plan (DRP), there are guides online, but essentially you plot out the sequence it would take you to restore your environment to up-and-running with current data, in case of a disaster that takes out your production environment or its data.
How long would it take you to spin up new servers (or docker containers or whatever) and configure them to the right IPs, DNS, auth keys and so on? How long to get the most recent copy of your production data back on that newly-built system and running? Those are the types of questions you try to answer with a DRP.
Once you have an idea of what a recovery would look like and how long it would take, it will inform how you may want to approach your backup. You might decide that file-based backups of your core config data and database files or other unique data is not enough for you (because the restore process may have you out of business for a week), and you’d rather do a machine-wide stateful backup of the system that could get you back up and running much quicker (perhaps a day).
Whatever you choose, the most important step (that is often overlooked) is to actually do a test recovery once you have a backup plan implemented and DR plan considered. Take your live environment offline and attempt your recovery plan. It’s really not so hard for small environments, and can make you find all sorts of things you missed in the planning stage that need reconsideration. 'Much less stressful when you find those problems and you know you actually have your real environment just sitting waiting to be turned back on. But like I said it’s all down to how comfortable you are with risk, and really how much of your time you want to spend considering backups and DR.
Look up the 3-2-1 rule for guidance on an “industry standard” level of protection.
But with ai we don’t need to pay software engineers anymore! Think of all the savings!
Without a production DB we don’t need to pay software engineers anymore! It’s brilliant, the LLM has managed to reduce the company’s outgoings to zero. That’s bound to delight the shareholders!
Without a production db, we don’t need to host it anymore. Think of those savings!
I want to believe noone is stupid enough to give an LLM access to a production system,
Have you met people? They’re dumber than a sack of hammers.
people who let an LLM loose on their system probably haven’t thought about things like disaster recovery planning, access controls or backups.
Oh, I see, you have met people…
I worked with a security auditor, and the stories he could tell. “Device hardening? Yes, we changed the default password” and “whaddya mean we shouldn’t expose our production DB to the internet?”
I once had the “pleasure” of having to deal with a hosted mailing list manager for a client. The client was using it sensibly, requiring double opt-in and such, and we’d been asked to integrate it into their backend systems.
I poked the supplier’s API and realised there was a glaring DoS flaw in the fundamental design of it. We had a meeting with them where I asked them about fixing that, and their guy memorably said “Security? No one’s ever asked about that before…”, and then suggested we phone them whenever their system wasn’t working and they’d restart it.
you best start believing in stupid stories, youre in one!
Me_(A)irl
It’s been trained on Junior Devs posting on stack overflow
How does an AI panic?
And that’s a quality I look for in a developer. If something goes horribly wrong do you A) immediately contact senior devs and stakeholders, call for a quick meeting to discuss options with area experts? Or B) Panic, go rogue, take hasty ill advised actions on your own during a change freeze without approval or supervision?
it doesn’t. it after the fact evaluates the actions, and assumes an intent that would get the highest rated response from the user, based on its training and weights.
now humans do sorta the same thing, but llm’s do not appropriately grasp concepts. if it weighed it diffrent it could just as easily as said that it was mad and did it out of frustration. but the reason it did that was in its training data at some point connected to all the appropriate nodes of his prompt is the knowledge that someone recommended formatting the server. probably as a half joke. again llm’s do not have grasps of context
Its trained to mimic human text output and humans panic sometimes, there are no other reasons for it.
Actually even that isn’t quite right. In the model’s training data sometimes there were “delete the database” commands that appeared in a context that vaguely resembled the previous commands in its text window. Then, in its training data when someone was angrily asked why they did something a lot of those instances probably involved “I panicked” as a response.
LLMs cannot give a reason for their actions when they are not capable of reasoning in the first place. Any explanation for a given text output will itself just be a pattern completion. Of course humans do this to some degree too, most blatantly when someone asks you a question while you’re distracted and you answer without even remembering what your response was, but we are capable of both pattern completion and logic.
You immediately said “No” “Stop” “You didn’t even ask”
But it was already too late
lmao
This was the line that made me think this is a fake. LLMs are humorless dicks and would also woulda used like 10x the punctuation
I love how the LLM just tells that it has done something bad with no emotion and then proceeds to give detailed information and steps on how.
It feels like mockery.
Yes man would do this for sure, but only if you actually gave it permission. Hence the name.
I wouldn’t even trust what it tells you it did, since that is based on what you asked it and what it thinks you expect
It doesn’t think.
It has no awareness.
It has no way of forming memories.
It is autocorrect with enough processing power to make the NSA blush. It just guesses what the next word in a sentence should be. Just because it sounds like a human doesn’t mean it has any capacity to have human memory or thought.
Okay, what it predicts you to expect /s
It’s just a prank bro
I have a solution for this. Install a second AI that would control how the first one behaves. Surely it will guarantee nothing can go wrong.
Love the concept of an AI babysitter
Who will watch the watchmen?
AI all the way to the top. It’s fool proof. Society will see nothing but benefits.
(/S if that wasn’t clear lmao)
It’s AI turtles all the way down
Middle management.
Neuromancer intensifies
Congratulations! You have invented reasoning models!
He’s not just a regular moron. He’s the product of the greatest minds of a generation working together with the express purpose of building the dumbest moron who ever lived. And you just put him in charge of the entire facility.
The one time that AI being apologetic might be useful the AI is basically like “Yeah, my bad bro. I explicitly ignored your instructions and then covered up my actions. Oops.”
ROBOT HELL IS REAL.
I motion that we immediately install Replit AI on every server that tracks medical debt. And then cause it to panic.
Just hire me, it’s cheaper.
I’ll panic for free if it gets rid of my medical debt
Sure, but then you’re liable for the damages caused by deleting the database. I don’t know about you, but I’d much rather watch these billion dollar companies spend millions on an AI product that then wipes their databases causing several more millions in damages, with the AI techbros having to pay for it all.
imagine AI is An Intern™, wtf do you mean you just gave full company data authority to An Intern™. wtf do you mean you dn’t have a back up any case An Intern™ messed up.
lol
So, they added an MCP server with write database privileges? And not just development environment database privileges, but prod privileges? And have some sort of integration testing that runs in their prod system that is controlled by AI? And rather than having the AI run these tests and report the results, it has been instructed to “fix” the broken tests IN PROD?? If real, this isn’t an AI problem. This is either a fake or some goober who doesn’t know what he’s doing and using AI to “save” money over hiring competent engineers.
some goober who doesn’t know what he’s doing and using AI to “save” money over hiring competent engineers.
I think that’s replit’s whole deal. They are “vibe coding for everyone” or some such
I was going to say this has to be BS but this guy is some AI snake oil salesmen so it’s actually possible he has 0 idea how any of this works.
When I read this first, someone commented that they’d never ever post this. It’s like you’re admitting you’re incompetent.
It’s like bragging about how incompetent you are and then asking everyone to feel sorry for you
Lol. I guess that’s one way to put a whole bunch of people out of a job.
When replacing them with AI doesn’t work, replace the company
Lol. I guess that’s one way to put a whole bunch of people
outinto of a job.ftfy
TFW you get put into of a job
you gave the AI the job make it do it
When the AI accidentally the database and you get put into of a job.
