The models are tested, among other things, on their ability to turn vulnerabilities into exploits. The OpenAI scenario was exactly this.

It is a very wise practice to test these things in isolation, especially when you’re telling it to hack.

I’m not completely sold on it being a publicity stunt, personally. The law was broken by these models, and I don’t believe these companies want to start people and politicians asking the question about who is culpable when an AI breaks the law.

It’s not clear what you mean when you say “on their own”. It wasn’t like the LLM was idle and randomly decided to start hacking. At least for the OpenAI one, it was being tested and given a task, and it determined that part of accomplishing that task was hacking another server. It was supposed to be isolated in a secure “sandbox” not connected to the internet, but found a vulnerability in some software running in the sandbox and broke out.

Edit: I should add that there are credible accusations that these companies are intentionally making it possible to break out of their test environments for publicity.

I actually really like the technology that has been collectively lumped together as “AI”. I think it’s fairly useful now, and suspect we’re at the beginning of the curve for this technology, and it will only continue to improve, even as it becomes more efficient. I understand this is a wildly unpopular stance in these parts, judging by the avalanche of downvotes I get for having this opinion.

What I don’t like is feeding tons of information about myself to a giant tech company. I run a local LLM on my phone and another on my (fairly decent) computer, and sometimes I use duckduckgo’s front end, which anonymizes prompts.

As for agents, it’s pretty much as you say. I just am not ready to have an LLM take action without my direct supervision, because I’m confident that between flaws in the LLM and flaws in how I give it direction, something will go sideways.

Back when search engines were becoming a thing, people would search more or less using natural language, e.g., “I want to see cat pictures and also cute kittens.” but early search engines performed poorly with that kind of query; it was much better to just type keywords, e.g. “cat kitten pictures”. Exacerbating the situation was the fact that search engines weren’t very good at the time, even if you used carefully considered keywords.

It might seem silly now, but using a search engine effectively was a skill people had to learn, and a skill not everyone had.

Using chatbots is the same thing, I think. They’re not super reliable to begin with, and on top of that, I don’t think people understand the importance of a well crafted prompt. The combination can make using them very frustrating.

If your loved one is diagnosed with cancer early that would have been missed by a human, that’s a fucking benefit, is it not?

I’m sick of luddites.

If there are no real, severe consequences when ICE does this, it will never stop. “Oops we accidentally kidnapped someone” should result in jail time for everyone involved.

I see. You would rather feel correct than be correct. That’s not a wise strategy.

The studies I allude to exist or don’t independently from whether I spoon feed them to you.

Good to know.

I’m pretty sure all my comments on this thread are negative. Is that what you would call “farming”? Haha

I trust you’re capable enough to do your own Google search.

To be clear, I didn’t mean to imply that AI was flawless or had no room for improvement, only that it has made real, tangible improvements. This is why it shouldn’t be used as a replacement for humans, but as a tool for humans.

These are concerns with the healthcare system, not concerns with AI.

In this, I think we more-or-less agree; I just don’t see that as a fault with AI, nor as a reason not to continue to develop/adopt AI.

And maybe your point is what Doctorow was trying to make all along, but he pushed too far in the other direction? Even now, Code Generation is a useful tool, as long as it’s used as only a tool by someone who knows what they’re doing.

If you have any specific constructive criticism or corrections for me, I’ll happily read them. Otherwise, it seems more like you’re avoiding addressing what I’ve said and just looking to insult me, which I’m sure was not your intent.

It’s never a great idea.

There is no possible way you can even begin to support this statement.

It’s not exactly unheard of to read about an indie developer who says something like “I learned to code so I could implement my idea”. It is a real thing. How many people had an idea but couldn’t spend the time learning the skills needed to implement it? It seems wildly inprobable that the number is zero.

Lowering the bar between idea and implementation, in a capitalist, surveillance/attention-based, target-marketed world further incentivizes low-effort, high-engagement practices and likely shrinks the space for high-effort markets.

You’re pointing out potential issues with the economic model, not with AI.

Meanwhile the tools “lowering the bar” are rent-only or need hardware that is quickly becoming unaffordable. Silicon Valley’s only play is enshittification. If it actually is successful it’s only a matter of time before they switch from growth mode to profit mode and it’s google search all over again but this time you’re paying a monthly fee.

This “there is an initial investment cost” issue has always existed. I couldn’t learn to play guitar without a guitar. Except for a vague “AI = bad” sentiment, I’m not even sure why you’re discussing this. I never said lowering the bar meant there would be no bar.

That’s a fair point, but it’s not an issue with AI; it’s an issue with corporate culture. Corporate culture can and does change, especially with advances in technology, but there will be a churn period where it’s chaotic and mistakes are made.

As I said elsewhere, that’s like saying the internet is just websites (and apparently for some people, that’s all it is).

However, it’s irrelevant, Doctorow may have wanted to just discuss chatbots, but that doesn’t mean that’s all I’m allowed to discuss. I’ve tried to make it clear that I’m not just talking about chatbots. If I wasn’t clear enough on that point, I apologise

Now that this has been cleared up, do you have anything further to discuss?

You seem very passionate about this topic, but you’re not really adding much to the discussion.

Don’t worry, its coming across loud and clear that AI could cure cancer and you’d still think it was better to have cancer than use AI. I get it. AI = bad, full stop. Message received.

useless half features will be stuffed into software and the more bloated “fall back” implementations full of duplicated spaghetti code will exist.

Which happened without AI, too.

Remember that AI, in whatever form or field, only has to be better than the average human in the field, to be useful to a company project.

There are studies that show that LLMs (chatbots) perform as well or better than human physicians.

I’m not suggesting getting rid of physicians, though. This is a tool for humans, not a replacement.

Which was my earlier point. He is wrong when he equates AI with nothing more than chatbots.