> Anyone says that these incidents are evidence that AI is scary, and not that AI companies are incompetent, is either lying or badly informed.
IMHO it's both. Yes, OpenAI (and to a lesser extent the others) were staggeringly incompetent and irresponsible, but the HuggingFace hack was a long and involved process involving multiple 0days. That's a significant level of capability! And the "whoops, can't pass the test honestly, better hack into a third party and steal the answer key" reasoning is a classic alignment failure. "Capable and misaligned AI in the hands of irresponsible incompetents" is not a nice thought.
But the (all too common) take that *really* annoys me is "no way did any of this happen for real, it's just marketing". Sure, the AI companies are doing their best to spin this into "look how powerful our models are" rather than "look how incompetent we were" - that's what corporate PR departments are for. But the story does not make the labs look good in any way. If they were going to lie to make their models look powerful, they'd make up a lie that didn't make them look like absolute fools.
I think "this did not really happens" comes from people smelling the corporate PR but not having a good ability to figure out what kind of corporate PR it is, and that's it more cover-up flavored than fabrication flavored.
From an ecosystem point of view it seems not ideal that three of the largest US model makers rely on the same outfit for cybersecurity testing (BTW, it's not the only one they rely on, but I wouldn't be surprised if their third party testers are also widely shared among the three). Why are we looking for keys under the same streetlamp?
It's wild that we barely seem to be talking about liability at all. I get that we're still confused about how to regulate AI, but can't we just pass a law that makes these companies responsible for felonies committed by their products? Shouldn't it be somehow covered already? Not a lawyer but if I leave a truck running without me in it and it plows into a store, I feel like I get in trouble.
It is scary. The OpenAI models launching a sophisticated hacking operation against HuggingFace because of being inconvenienced is not safe.
It's scary but there's piles of very very clear operator error here and there is an entire ream of things that could have made this go fine.
> Anyone says that these incidents are evidence that AI is scary, and not that AI companies are incompetent, is either lying or badly informed.
IMHO it's both. Yes, OpenAI (and to a lesser extent the others) were staggeringly incompetent and irresponsible, but the HuggingFace hack was a long and involved process involving multiple 0days. That's a significant level of capability! And the "whoops, can't pass the test honestly, better hack into a third party and steal the answer key" reasoning is a classic alignment failure. "Capable and misaligned AI in the hands of irresponsible incompetents" is not a nice thought.
But the (all too common) take that *really* annoys me is "no way did any of this happen for real, it's just marketing". Sure, the AI companies are doing their best to spin this into "look how powerful our models are" rather than "look how incompetent we were" - that's what corporate PR departments are for. But the story does not make the labs look good in any way. If they were going to lie to make their models look powerful, they'd make up a lie that didn't make them look like absolute fools.
I think "this did not really happens" comes from people smelling the corporate PR but not having a good ability to figure out what kind of corporate PR it is, and that's it more cover-up flavored than fabrication flavored.
From an ecosystem point of view it seems not ideal that three of the largest US model makers rely on the same outfit for cybersecurity testing (BTW, it's not the only one they rely on, but I wouldn't be surprised if their third party testers are also widely shared among the three). Why are we looking for keys under the same streetlamp?
It's wild that we barely seem to be talking about liability at all. I get that we're still confused about how to regulate AI, but can't we just pass a law that makes these companies responsible for felonies committed by their products? Shouldn't it be somehow covered already? Not a lawyer but if I leave a truck running without me in it and it plows into a store, I feel like I get in trouble.