Be Careful what you ask for...

Bruce Schneier cautions us that Large Language Models LLMs are very literal in their interpretation of what they are asked to do.  They are very good at interpreting the words and very poor at divining or understanding the intent or meaning.  Midas wanted gold but got the Golden Touch.  Sissa ben Dahir wanted a big reward for inventing chess but he really did not intend to destroy the world.  

One problem in large language models (LLMs) is the use of human language to specify the goals.  Bruce, and his colleague, Barath Raghavan, speak of the genie coefficient, the gap between what a user intends and how the AI interprets it.  The proper interpretation of human language relies upon context.  "If you ask  someone to 'get coffee,' they bring you a cup, not a plantation."  

Human language is not a good medium for fully specifying a goal or a result. We are not good at using it for that and machines are not good at interpreting it.  Because a prompt lacks any context, or even history, the prompt must include any necessary context.  That is why using Claude to write code is an iterative process of refining the results "until the user is satisfied." It is why IBM prefers application specific AIs that may use language for communicating with humans but not for communicating with each other.  It is why giving the same prompt to two different AIs, or even to the same one on different occasions, may render different results.  

We cannot allow ourselves the error of believing that because it can mimic the way that we communicate that it thinks like us.  Perhaps we should reinterpret the first "L" in LLM to stand for "limited" rather than "large."  

Responsibility and Accountability for AI

A human being or enterprise is, and must be held, responsible for everything that a computer is asked to do.  For AI they are responsible for the training, goals, data, weights, and, prompts.  They are responsible for all the results, i.e. action, uses, and properties including accuracy, correctness, and purpose.

The responsibility must include criminal or civil accountability.  Consider Anthropic's attack against Hugging Face. Anthropic lacked Mens Rea so they were not criminally liable.  However, they were reckless and therefore they are civilly liable for any and all damage that their experiment caused.  It is essential that they not be allowed to escape that responsibility.  We must limit criminal and reckless use of what it only a tool.  

We worry about AI going rogue but that is not the problem.  The problem is what someone or some enterprise asked the AI to do and what precautions they took.  It is clear that we are not very good at that.  If we want to enjoy the benefits of AI while avoiding unintended consequences we must hold each other scrupulously accountable for its uses.  

We must restrict the use of LLMs to applications where we can predict, evaluate, and control the results. I continue to recommend IBM's stated approach: application specific uses, curated, application, training data, and transparency and governance built in from the ground up.  I like the approaches of Apple Siri and Microsoft Co-pilot, that use AI to assist with specific tasks in a known context.   

What am I missing?