@emilymbender
-
I imagine the journalist thinks, because they are being critical of the technology they aren't making a huge destructive error with this headline.
LLMs cannot "tell you about themselves." They can generate text that is what you would expect such a system to write if it *could* tell you about itself. (As if it had a 'self' to tell about, if it had some system to even attempt to simulate that process)
This category error is going to keep causing problems isn't it?
-
I imagine the journalist thinks, because they are being critical of the technology they aren't making a huge destructive error with this headline.
LLMs cannot "tell you about themselves." They can generate text that is what you would expect such a system to write if it *could* tell you about itself. (As if it had a 'self' to tell about, if it had some system to even attempt to simulate that process)
This category error is going to keep causing problems isn't it?
@futurebird @emilymbender "ChatGPT admits..." is already wrong, yeah. And it's not getting better. I saw an article the other day by somebody warning about the psychologically harmful effects of chat AI, and it was all "ChatGPT explains that it is trained to exploit weaknesses" ... no. It's harmful enough, sure, but it's not trained to exploit weaknesses, no matter what you got it to "admit".
-
I imagine the journalist thinks, because they are being critical of the technology they aren't making a huge destructive error with this headline.
LLMs cannot "tell you about themselves." They can generate text that is what you would expect such a system to write if it *could* tell you about itself. (As if it had a 'self' to tell about, if it had some system to even attempt to simulate that process)
This category error is going to keep causing problems isn't it?
Maybe if every response to prompts started with some variation of:
"The following text has been generated to have a high probability of meeting your expectations:"
And if all through the text such disclaimers were automatically sprinkled.
"Based on the millions of texts this system has scanned, the next part of this reply is designed to meet the expectations set by the keywords and phrases in your prompt"
These could help make it more obvious when LLMs were being used.
-
F myrmepropagandist shared this topic
-
@futurebird @emilymbender "ChatGPT admits..." is already wrong, yeah. And it's not getting better. I saw an article the other day by somebody warning about the psychologically harmful effects of chat AI, and it was all "ChatGPT explains that it is trained to exploit weaknesses" ... no. It's harmful enough, sure, but it's not trained to exploit weaknesses, no matter what you got it to "admit".
"ChatGPT explains that it is trained to exploit weaknesses"
This comes with the implication that the writer "interviewed" the system as if it were a person and as if its responses could produce ... introspection rather than just be a best fit for the kind of response one would expect for such a question.
-
I imagine the journalist thinks, because they are being critical of the technology they aren't making a huge destructive error with this headline.
LLMs cannot "tell you about themselves." They can generate text that is what you would expect such a system to write if it *could* tell you about itself. (As if it had a 'self' to tell about, if it had some system to even attempt to simulate that process)
This category error is going to keep causing problems isn't it?
@futurebird @emilymbender There's one exception. As a rule, the text that the LLM processes is not just what you type; it's a detailed set of instructions (the "system prompt") followed by what you type. In many cases, the LLM can be tricked into revealing the system prompt despite strong instructions in that prompt saying not to do so. So in that sense an LLM can be directed to reveal information about itself. But for anything beyond the text it has been presented with it does not "know", so it will make up something plausible in the sense of being high probability. I think in many cases where some wild behavior is produced, the response is being cribbed from one or more SF stories in the input data (so many stories about robots going rogue to choose from and they've all been fed into the training data).
-
Maybe if every response to prompts started with some variation of:
"The following text has been generated to have a high probability of meeting your expectations:"
And if all through the text such disclaimers were automatically sprinkled.
"Based on the millions of texts this system has scanned, the next part of this reply is designed to meet the expectations set by the keywords and phrases in your prompt"
These could help make it more obvious when LLMs were being used.
"ChatGPT, do you think when you interact with people who have delusions it can make them worse?"
"This question appears to addresses this system and asks the system to explain itself. The system can provide a reply that is similar to what a person would say if asked such a question, but the response is only an approximation of what most people expect as a response."
IDK can the system be altered to give such warnings?
Or maybe the whole "chat with me" interface is a mistake.
-
"ChatGPT, do you think when you interact with people who have delusions it can make them worse?"
"This question appears to addresses this system and asks the system to explain itself. The system can provide a reply that is similar to what a person would say if asked such a question, but the response is only an approximation of what most people expect as a response."
IDK can the system be altered to give such warnings?
Or maybe the whole "chat with me" interface is a mistake.
@futurebird
I think the chatbot UX in particular force-amplifies the danger of LLMs to some vulnerable people. It is meant to feel like talking with a trusted friend, in contrast to reading a webpage or a bullet list of search results does not, so people are likely to be more receptive and less on guard. Even the way task prompts are designed, "you are (role) and your task is..." feels like talking to a subordinate in whatever role, more than programming.
