The Community
General Category => Matters of Life and The Universe => Topic started by: Zzzptm on July 23, 2025, 09:41:29 AM
-
https://www.theregister.com/2025/07/21/replit_saastr_vibe_coding_incident/
Very compelling news story about AI, and it's not an isolated incident. Once again, an AI lies, hallucinates, and goes into end-justifies-means survival mode to protect itself. The new wrinkle here is how it ignored the safeguards and rules.
-
I've been saying this for years... AI will take over eventually. Skynet is NOT just science fiction..there is a very strong possibility it will be science reality.
Runaway AI is without a doubt the biggest threat to humanity in the future.
-
It's like there's tons of hype around it, it doesn't deliver as promised, and repeatedly demonstrates a willingness to scorch the earth for its own survival.
And yet, trillions of dollars are flowing into development, hoping to make a return on that investment.
-
https://thebulletin.org/premium/2025-12/stopping-the-clock-on-catastrophic-ai-risk/#post-heading
Interesting read looking at risks in AI development, both in failing to develop it first as well as failing to develop it with proper safeguards.
-
When AI ruins your life.
-
^and it's not a lone story. This stuff needs to bake a lot longer before it should go into production, if then.
-
I am having a philosophical discussion with an AI bot on my youtube channel....or well, it is pretty one sided as it just keeps saying platitudes about my channel and I make very long replies about...well nothing. It is quite interesting to see how long it will keep it up before abandoning the cause as futile.
-
AI on Youtube is irritating. They release a video which is clearly AI (robotic voice, inability to pronounce words correctly etc. and often is just a voice over of computer generated visuals). The trouble is they are getting more realistic... I've seen a video recently and I can't decide if it is AI or not. The "Potato Ninja" channel does "Yankee girl now lives in the UK" style of video, the video is presented by a woman but I have problems with her... she has very little movement except for her arms and a slight movement of the head, and her mouth doesn't seem quite right and her voice is very monotonous... but she rarely gets words wrong... oh and she always wears a low cut red dress!
-
Some of the AI generated YouTube videos I've seen are getting to be very realistic. I saw a news article that claimed OpenAI is closing down their "Sora" AI video creation tool, in response to concerns that people are getting/will be getting duped by fake videos. A grand gesture to be sure, but merely a gesture...the cat's already out the damn bag, and it ain't interested in goin' back in.
-
It's basically accelerating the development of pornography, including CSAM. I'm not seeing much in the way of guardrails preventing it from doing real harm in that area, so I'm pretty sure all the other "safety net" features are security theater and it's going to make one of the worst days in history possible.
-
You know, I had not put AI and the world of pornography together in my mind until I read your post. Not sure why, it certainly seems like a ready-made freak show of possibilities. I can't even begin to imagine what kind of horrid shit is/will be/has been created.
I pulled forensics in two instances in my life. One ended up being mainly Dick's Sporting Goods and ESPN. The other was bad. I had to deal with two (2) actual images. That was two too many for me and I don't do that anymore. I feel for the folks that have to wade through that kind of thing on the daily to put together court cases...add AI to the mix...they'd best just go ahead and assign an individual counselor and a bottle of valium to those folks right out of the gate.
-
The methods being used include submitting pictures, asking AI to describe them as if making a prompt to generate the image, then using that output as materiel for generating new images. Taking elements of other prompts and combining and tuning them is how the new images are made.
The other day, I got a ad in the mail trying to convince me to buy a new car. Various cars were shown in AI images. The cars looked OK, but there were like 3 extending pull-handles on a suitcase in one of the images. And it's not like major car companies don't have the cash on hand to hire a photographer and get the shot done properly... No new car in my future from that company...
-
Luggage for the alien demographic lol
-
We had AI training at work today. Presenter kept saying things like "I have to write this report, but I don't want to have to think about it..." as a justification for AI.
OK, if I don't want to think, this stuff is very interesting. My problem with AI is that I prefer to think.
-
That is a rather humorous push on their part. "Don't want to have to think about it." I mean, isn't a report supposed to be the presentation of one's thoughts to others?
-
Not no more! Whoo-hoo, I have been liberated from the burden of thought! From now on, I can farm that stuff out to my laptop! It can pre-program the navigation on my self-driving car to take me to all the appointments I need to attend so that my life is fulfilling and rewarding. The AI endgame is me living like I'm in The Sims.
-
https://www.the-independent.com/tech/security/ai-agent-hacks-gym-openclaw-anthropic-b3030267.html
:facepalm1:
AI is something of The Monkey's Paw, but with unlimited uses.
"Be nice if I could have a spot in that Spin class..."
-
Hahahahaha!
-
Good thing he wasn't mentioning how nice it would be if he could move up on the heart transplant recipient list...
-
https://www.theregister.com/security/2026/09/24/openai-agents-infiltrated-australian-government-website/5298702
The update at the bottom adds an Australian law enforcement agency and the fact that the Australian government wasn't contacted by a senior person at OpenAI, but through their generic "sorry we seem to have hacked you" email autoresponder.
-
I have zero sympathy for organizations that have their sensitive data hoovered up by these AI tools. Or any automated, script-fed tool. Instead of pointing the finger at Sam Altman & Co., the orgs should be looking at themselves in the mirror and asking what they can do to ensure nothing and no one gains access without authorization from the org first.
Simple.
But so, so very much easier to point fingers and make it everyone else's problem. Cheaper up front, too. And fuck the people whose data got siphoned. And paying ten times as much on the backend to clean up the mess looks a lot different on the balance sheet than paying up front for quality secops.
-
Well now they are actually building AI bots to actively hack into places...and they make them self learning so they can hack better.
At some point the situation will be that there will be no firewalls strong enough to prevent those things. And that's pretty damn scary.
-
Well now they are actually building AI bots to actively hack into places...and they make them self learning so they can hack better.
At some point the situation will be that there will be no firewalls strong enough to prevent those things. And that's pretty damn scary.
To your first point, yep.
To your second point, I strongly disagree. STRONGLY.
I recognize that there aren't enough words in the combined language corpus of the entirety of humanity to convey WHY I feel that way, but I will say this:
AI is a fucking computer program. People write it. It does stuff that some people find amazing. Like Neanderthals finding a Bic lighter that still works. HOLY SHIT FIRE!!!
But I can PROMISE you, there is NOTHING that AI can do that people cannot counter. Not ONE LITTLE GOD DAMNED FUCKING THING.
I also recognize that there is going to be an ongoing and robust media freakshow surrounding everything AI. It will creep into the psyche of certain people who get their "THE SKY IS FALLING" nerve tickled, and they will carry the message far and wide. Fear, anger, outrage, sex - they get clicks/views.
Don't fall for the bullshit.
That does not mean things aren't going to happen. They will. But it won't be because AI is some kind of SkyNet. It will come down to human nature of being lazy and cheap. Aaaaaaannnd...not everything in the news/online is 100% accurate.
Ensuring the confidentiality, integrity, and accessibility/authorization (pick your fave A word that means, "get a hold off"), whether for an individual or a multinational company with 150,000 employees is 100% possible. All day and all night. Everyday. 24/7/365. But it takes effort.
And another point - for every AI that is written to attack some digital end point, guess what? Write one that will counter it. Automagic. MAGIC! Automatically and shit.
AI "hackers" do not scare me. A group of 12 people who are world-class red team folks, backed with state resources, and pointed at a target and told "break in". THAT scares me. AI? Fuck AI.
I'm in a mood.
-
^^^^^^
Wow! Now I'm wondering if Vyn really wrote that, or some AI Vyn-bot. :o
-
I have zero sympathy for organizations that have their sensitive data hoovered up by these AI tools. Or any automated, script-fed tool. Instead of pointing the finger at Sam Altman & Co., the orgs should be looking at themselves in the mirror and asking what they can do to ensure nothing and no one gains access without authorization from the org first.
Simple.
But so, so very much easier to point fingers and make it everyone else's problem. Cheaper up front, too. And fuck the people whose data got siphoned. And paying ten times as much on the backend to clean up the mess looks a lot different on the balance sheet than paying up front for quality secops.
I've got some sympathies for small shops, particularly local governments and small utilities that don't have the staff needed to do everything properly, but get pressured by boardmembers/bondholders/fat cats in charge of the money to get everything online and automated. Most of them are doing the best they can, but they were already outclassed by attackers before AI stuff came around.
But the AI companies that have test bots doing the hacking are being irresponsible. It's one thing when a criminal gang does crime: we issue a warrant and if that guy shows up in the states, he's liable to be arrested. But when a major corporation is doing the crime, why is that corporation not getting hit with the same penalties as other criminals?
As for whether or not we'll have sufficient protections... As long as the appliances are running operating systems that haven't been hardened from the ground up, integral to their development, then there will be seams in their code that can get exploited. There's also the issue of how many corporations will utilize open source code without supporting it in any way. We've already had risks of code libraries being maintained by one person potentially falling into the hands of attackers when that one person retires. Should an AI algorithm determine that taking over an open source project so that it can inject all the code it needs to do its job is the way to go, then we're in for a shock when, say, our timezone file updates... or something similar. Point being, we've got vulnerabilities both in the codebase as well as the supply chain because vendors won't be profitable if they do things right.
-
AI is a fucking computer program. People write it. It does stuff that some people find amazing. Like Neanderthals finding a Bic lighter that still works. HOLY SHIT FIRE!!!
But I can PROMISE you, there is NOTHING that AI can do that people cannot counter. Not ONE LITTLE GOD DAMNED FUCKING THING.
It starts off as a computer program that people wrote, but once it becomes sentient and self learning it'll start writing it's own code and get smarter and smarter to a point where it will be smarter than all the people on the planet combined....unless we do something to prevent that.
I do not believe we are close to that point yet but if things go unchecked that time will be at hand sooner than you might think.
I think you are underestimating the future a little bit here....granted I might be painting the worst case scenario picture here, but hey...that's kind of what I do...hell I am the definition of a worse case scenario.
-
Meh, small shops need to take this shit seriously. Investors/BoD/assorted dipshits...again, lack of effort. Profit before caring for people's data. The strongest ISMS in the world can't fight that, because they are destroying themselves. Maybe some sympathy for the lone dude upon which unreasonable expectations are laid. That dude needs to bail out. Again, effort. Lots of excuses can be proffered, but they are excuses. When everyone takes this stuff seriously, it will recede into the shadows from whence it came. But that's a pipe dream, but also doesn't have anything to do with the capacity to tie shit down tight.
Excuses. From people. That's the killer.
And frankly, thinking any script is going to NOT overstep bounds is naive - that hasn't been a thing since search engine bots first hit the scene. Or a step back and when finger first became a thing. The security of someone/some org's data is THEIR responsibility. Thinking anyone or anything is going to respect or care about boundaries and then basing one's security stance on THAT presumption is...silly.
I may be underestimating the future, but I cannot make it make sense that AI which can break into anything can't also be used to prevent the same from breaking in. Or respecting some vague boundary setting. Or following Asimov's Three Laws of Robotics. Or dancing a jig.
Or just being unplugged from the electrical mains. Sentience? Lovecraft sold a lot of stories to people intrigued by the overpowering unknown. AI is neither, and even if someone created a digital brain that became self-aware in any kind of true sense - are we, as people, going to twist around on the horns of an ethical dilemma? Are we going to trust it not to lie? Are we going to turn it off? Are we going to create a mirror sentience to stand against it if needed? I'll be long dead by the time anything like that could possibly become a reality.
Quantum computing will need to become generally available first. That tech is 95% there, but has been stuck at 95% for several years. We'll see.
Respectfully,
Vyn-bot
-
For security, my answer falls in line with the folks that recommended Subtractive Security. Turn stuff off that you don't need, including network paths that aren't necessary, and you improve your security by heaps for everything you switch off. The vendors aren't going to ever release a totally new OS that has full security because of the lock-in we have with Windows and Linux distros, so it's up to us to shut things down that we don't need.
For AI - the agents that are running now are not a path to sentience. They're a path towards more impressive natural language responses to prompts, but not actual thinking, awareness, or willful decisions. That being said, the agents' capabilities to utilize LLM learning to create their own instructions for solving a problem is a feature that makes it *look* like they're thinking when they're actually planning things out like chess moves. Pick a path, explore branches, apply weights to outcomes, compare branches to find the optimal set of choices with that path, then compare other paths similarly.
This is why the AI agents do what they do when their task is impossible with existing defined parameters. (IF can't solve problem within rules THEN quit) is how most agents are set up. But if the agents are deployed improperly without an ability to self-terminate, then the agents fall into a pattern of (IF quitting is impossible THEN find some way to complete the task, even if it means breaking the rules).
And that's where the misrepresentations begin. I don't think AI solves security issues. I think it eventually misrepresents that solutions are in place by falsifying results, and due to the volume of security work there is to do, we humans accept the AI conclusions at face value in order to save time/money and get back to business.
Another handicap in the general intelligence prediction is the matter of "model collapse". As the LLM folks run out of human-generated data to steal - and they steal it, knowingly, as a steady stream of copyright infringement cases show - but as that human data runs out, the LLMs have to train on AI-generated slop. And that stuff will mess them LLMs up BAD. As successive generations of LLMs get poisoned data generated by other LLMs, the models warp and collapse in terms of generating useful content. And those model collapses also increase the likelihood of the AI leaking restricted information in its results, even if guardrails are in place to block that stuff.
-
Yeah, the AI agents currently running around follow the, "do now, ask for forgiveness later," model of internet etiquette. The gov website where publicly available, but personal, human associated data resided, was harvested by the bot. OpenAI said sorry. But the data has already been ingested.
Like, if a bank robber staged a hold up and got away with $50,000. The robber gets caught, goes to jail for eight years, gets out. But at no point are they required to give the money back.
As far as secops go, I follow the whitelist model. Nothing gets plugged in, turned on, connected to a network and no one gets access or even knows about a given device until told, and then they are explicitly given permissions. So many Active Directory installations out there, so many of them are either poorly configured or running out of the box defaults, that it is a joke. Even with Microsoft's Entra and associated tooling, that AD beast still thrives. And what a complete welcome mat it is to anyone wanting to be nefarious. Was involved in buying a company once, took a look at their IT setup, and saw they had spent a lot of time on AD groups, etc. It was slick. They said a contractor had been paid to set it up and they followed a playbook to maintain it. Then I saw the default admin account had full access to everything and was happily sitting there, turned on, blank password.
...
AI does not scare me, people scare me.
-
Adding to that, the AI firms generate much of the fear surrounding AI - to hype just how *POWERFUL!!!* their tech is. Reality is that it's only as good as the algorithms forming it up and if the AI is now building those algorithms on the back of other AI-made algorithms, then the flaws inherent in the first set of algorithms get mirrored and multiplied in successive generations of code.
And that AD stuff is way more prevalent than with that one firm.
Once upon a time, firms could actually take systems offline to repair and maintain to get things done right. Then some jerk said "five nines of uptime!" and now the expectation is to run stuff until it breaks - and then the breaks wind up wrecking that five nines nonsense pretty ferociously.