Artificial Intelligence

This blog post is a work in progress.  Comments are solicited.  It is an attempt to bring my seventy years of experience in using, governing, managing, and securing innovations in computing technology to the issue of "artificial" intelligence.

In 1956, I was part of the first generation of computer user/programmers.  We stood on the shoulders of the giants that I call generation zero, Turing, Flowers, Shannon, Aiken, Eckert, Mauchly, von Neuman, Hopper, et. al.   

We had to tell the computer exactly how to arrive at the result that we wanted.  We expressed the computations that we wanted in terms of the operation codes of the hardware.  Almost all computers had op codes for add, some could subtract. The powerful ones even had operations for multiply and divide.  However, even those did multiplication by iterative addition and division by iterative subtraction.  We called that programming.  Those of us who could describe how to achieve a result by such tiny steps were called programmers.

In the mid fifties, one might create a program on a blackboard, flip-chart, or yellow pad.  We would transcribe the program into punched paper cards or tape that could  be eletro-mechanically "read" by and into the storage of the computer.  The paper and the reader were our user input mechanism, our application programming interface, our API.  We would enter the program first, followed by the data, also in punched paper.  The process was so expensive that the results had to be very valuable in order for the process to be efficient.  

While in the early days, I operated the computer, within a decade users and programmers had pretty much been excluded from the "computer room" and the role of operator had become specialized.  Within another decade the computer terminal was the common user interface. In 1981 the IBM personal computer had a key-board and a CRT that could display 80 columns by 25 lines of green alpha-numeric characters.  Like the terminals, it provided what we now think of as a command line interface.  Three years later the Apple Macintosh had a  graphical user interface (GUI) with a mouse.  Two years later came Windows.  "Point and click" was now part of how one used a computer. 

As computers have become more powerful and cheaper, much of that improved power has gone into the user interface, into making them easier to use.  Early examples include high level languages, like Fortran and Cobol, and interpreters like APL and Basic

Today's input and output devices include touch screens, cameras, microphones, ear phones, speakers, and environment and application sensors and controls. Modern computers can talk, listen, and recognize and process images.  We have smart appliances, homes, vehicles, drones, agents,  and robots.  I can testify that in1956 some of these were almost beyond imagining.

Games have always been part of computing and a measure of computer program "intelligence."  In 1956, my colleague, Dick Casey, and I wrote a program to teach the IBM 650, using console switches and lights for i/0, to play Tic-Tac-Toe.  My  mentor, Dr. Albert Samuels, wrote a program for the IBM 701 to "learn" to play checkers.  It got really serious in 1997 when IBM's Deep Blue defeated chess champion, Gary Kasparov in a six game match.  On February 16, 2011 IBM's Watson defeated champions Ken Jennings and Brad Rutter at the TV Show Game, Jeopardy.  

In 2015, Google's Alpha defeated European champion Fan Hui 5-0. Its most famous victory was against world champion Lee Sedol in 2016, winning the series 4-1.  Alphagozero "taught" itself the game.  

In November 2022 OpenAI introduced ChatGPT and demonstrated its ability to converse in natural language.  Using text-to-speech and speech recognition, it can even talk and listen.  It was to be the first of many large language models (LLMs), the latest user interface to the computer.  However, the ability of these chat bots to communicate in natural language hides the huge data sets, the enormous computing power, and the complexity involved.  It caused many people to invest them with the personality and autonomy that they merely mimic.

Using these programs one no longer has to provide the computer with a program, the steps to create the result.  One need merely describe the result, "prompt" the computer and it "figures out" how to arrive at it.  If the prompt is ambiguous, the computer may, in natural language, ask the user for addtitional information. The "Model" instantiates both the capabilities and the limitations of the program.  

We call the model "large" because it includes a huge amount of data, so much that it all but defies human comprehension.  Indeed it can solve difficult problems in conversational time.  However, it is correct to think of it as fast "table lookup" in a big table, using so much  computing power and energy that it is just now becoming efficient.  

The LLM organizes its data to optimize its use; we call the data "training" and the organization "learning," expressions that we analogize from human intelligence.  Therefore, the "model" instantiates both the capabilities and limitations of the program and its data.  

In part, because of the complexity, training, and the lack of precision of the natural language used in the prompts, what we want the computer to do, the results are sometimes not what the user intends and expects.  Said another way, in spite of its capabilities, artificial intelligence may be no less error prone than natural intelligence.  Sometimes the results are so far from the user's intention, again by analogy to the natural, the computer is said to "hallucinate."  Therefore, it is best to restrict the use of AI to the set of problems where, while we might not be able to arrive at the results by ourselves we can easily  check the result.  For example, while one might not be able to compute the cube root of a large integer, we can recognize it when we see it.  

Nothing useful can be said about the security of any mechanism except in the context of a specific application and environment.  --Robert H. Courtney, his first law.  

Some human being or enterprise is responsible for everything a computer is asked to do and for all the properties and uses of the result.  

 The responsibility of both individuals and enterprises for the application and use of a computer, specifically to include those applications that use natural language and mimic people, includes compliance with law, regulation, contract, morality standards, and prudence.  

As with anything else, an individual exercises her responsibility by complying with the law, regulation, contract, ethics, use of appropriate tools, and due care.  This requires knowledge, skill, ability, experience, and good judgement.  One starts by ensuring that one meets the applicable requirements.   One ends by doing nothing rather than doing the wrong thing.  

An enterprise exercises its responsibility by policy and governance, i.e., what it authorizes its agents to do and tells them not to do.  Management must use all available controls including, but not limited to:

• Direction
• Training
• Assignment of duties, roles, responsibility, limits, discretion, and authority
• Supervision
• Multi-party controls
• Automation
• Budget
• Recognition and respect
• Compensation
• Disciplinary action 

to ensure that its intentions are met.

One of the things that policy must do is to express managements risk tolerance in such a way that all managers and professionals at all levels understand what that means that they should do.  For example, the management of a mature enterprise might say such things as "Novel technology must be applied in a cautious, conservative manner,"  "line of business managers (not IT) must authorize and budget for the application of novel technology," "application of novel technology must be initiated, authorized, and budgeted for by two or more levels of line of business management," or "novel technology should be exploited and applied consistent with normal business risk."

Alternatively, the management of a high business risk start-up might say "novel technology should be aggressively applied and exploited in pursuit of business opportunity."  Needless to say, that this author recommends the more conservative approach for most enterprises.  

Recommendations

Things you Should be Doing Anyway

Use strong Authentication: Use at least two kinds of evidence, (shared secret, possession, biometric, or behavior). At least one form of which is resistant to replay; e.g. one-time password (OTP), biometric with liveness test.   Internet facing applications, mission critical applications, on management and administrative controls, and everywhere else, in that order.  This is the most efficient of all measures.  Nothing else will give you as much protection for dollar expended.  Consider Passkeys, one time passwords from hardware tokens, software tokens, e-mail, or SMS messages, in decreasing order of strength.  Consider offering users a choice of methods for their convenience.  

Structure your network and layer your defenses.  Start by isolating high risk Internet facing applications, e.g., e-mail, browsing, and messaging from internal enterprise applications, from mission critical applications, from administrative and management controls, from servers, e.g. database, file, communications, from storage devices.  

Employee "Least Privilege" access control.  Individuals and processes should enjoy only those capabilities and privileges that are required to fulfill their assigned roles.

Restrict "write" access.  In order to preserve accountability, "write" access to any object should be restricted to a single individual or process, e.g., application, database manager.  If you employ the common risky practice of granting everyone "read/write," changing to this policy will involve some administrative effort and may encounter some user resistance.  However, it will reduce insider risk.   

Employ "zero trust."  Zero Trust can be thought of a special case of least privilege in which every connection between processes is authenticated in both directions, both vertically and horizontally.  For example, users and applications mutually authenticate one another. Applications authenticate the database manager, the database manager authenticates the file system and vice versa.  This policy can often be implemented using existing controls.

Use multi-party controls.  Assign duties and responsibilities in such a way that agents, simply by doing their jobs, act as checks upon one another and that two or more must act in concert for material acts. 

Monitor and log all activity.  Monitor and log both the traffic that passes a firewall and the traffic that is rejected.  Changes in traffic patterns may be evidence of an attack.  

These policies and controls can greatly increase the cost and time-to-success of attacks.  They are efficient, that is their cost is small, usually justified by the reduction in the cost of the risk that they mitigate.  They are both essential and efficient.  

As we have noted, large language models (LLMs) increase both risk and opportunity.  Several LLM builders have demonstrated the ability to efficiently discover large numbers of vulnerabilities in operating systems and browser.  These models can demonstrate how to exploit the vulnerabilities that they discover.  They should also be able to remediate them.  

In the next sections we will make recommendations to reduce the risk and safely exploit the opportunities. 

Use LLMs Conservatively.  Prefer low-risk applications, applications where the quality of the results are obvious, applications that are tolerant of error.  Choose vendors and products based upon experience and support; do not rely upon claims.  

Train with Application Specific Data.  Prefer to train LLMs with data that is curated and specific to the purpose.  

Over Control New Applications.  Err on the side of over controlling any new application, including all uses of LLMs. As the application matures, one can always relax controls.  However, if an application gets out of control, merely tightening the screws will not bring it back into control. 

Use AI in Development. Use LLMs for coding and testing, including detection and  elimination of vulnerabilities.  Apply AI to Network Monitoring.  As attacks increase in rate, human monitoring will be less effective and efficient.  Use AI to support incident analysis and remediation.  

Use AI to vet New Products and Changes.  The software supply chain is under attack and is becoming the preferred way to distribute malicious code.  The response of suppliers has been Caveat Emptor.  Use AI to "beware."

Restrict the use of Enterprise Data.  Expose enterprise data only to enterprise AI, never to public or cloud applications.  

Consider the use of AI to automate "patching."  Expect automation to be necessary to maintain software at the rate at which AI will discover vulnerabilities.   

Bits are Bits

 NIST Cannot quite get it right.  They have gone from encouraging the use of special characters to discouraging them and relying only on password length.  

The strength issue is not about what the user must enter in an ascii or other code but how much work it would take an exhaustive, or brute force attack to find it.  Both length and special characters are ways to increase the work of attack.  Each adds bits.

We first started to Insist upon upper and lower case and special character to get more bits in fixed length passwords.  While most of you are to young to  remember it, for years passwords were limited to 8 characters, or one fetch, for performance reasons.  While modern computers are so fast that performance is no longer an issue and modern database managers will accommodate passwords of any length, there may still be systems that limit the length of passwords.  Unfortunately, we forgot why we were insisting upon complexity.

Before the Internet, most end users had fewer than a handful of passwords, many only one.  Today many users have tens, even hundreds of passwords. (As I write this, I have 310.)  As the number of passwords grew so did bad practice.  Users chose passwords that were easy to remember and enter, and then reused those that met these tests.   

To resist this user behavior, many managers introduced rules to encourage strong passwords and resist weak or reused ones.  This solution has become the problem. Choosing passwords was already hard enough; choosing passwords that meet well intended but otherwise arbitrary rules is often too much.  Otherwise strong passwords, including those generated by a password manager, might not meet the rules.  Forcing periodic changes added insult to injury.  

Thus, NIST now recommends length.  While length adds to strength, the longer the password, the harder it is to enter, particularly without error.   The strength is measured in bits, not , but the use of the entire character set may help; in some special cases may still be required.  

All this is by way of saying choosing, remembering, and using strong passwords is not easy.  Choosing, remembering, and entering, more than a handful of passwords is not easy.  It has become a computer application.  Password managers are somewhere between popular and necessary.   

Courtney taught us that "nothing useful can be said about the security of a mechanism (including passwords) except in the context of a specific application and environment."  Writing guidance that covers all applications and environments has always been what we call a "hard problem."  Writing guidance that will stand up to changes over time is particularly hard.

A final word. Well chosen and managed passwords are resistant to brute force attacks.  Those are not the kind of attacks that we are seeing.  Rather, we are seeing social engineering followed by fraudulent replay attacks.  Passwords, of whatever strength, are fundamentally vulnerable to replay attacks.  Rather, we need strong authentication, that is, at least two kinds of evidence, at least one of which is resistant to replay.  Said another way, all strong authentication is multi-factor but not all multi-factor is strong.  

Prefer length to complexity, but allow the whole character set.  (Encourage complexity if length is otherwise restricted.)  Encourage your users to use a (cross platform?) password manager. Offer them strong authentication options.  Mandate strong authentication for employees.  Consider, indeed prefer, passkeys (https://whmurray.blogspot.com/search?q=passkeys).  Use biometrics for convenience in applications where replay is otherwise resisted. Prefer one-time passwords to mandatory periodic password change.  

Passkeys

By now you have probably heard about the "death of passwords," or at least alternatives to them.  Passkeys are one such alternative.  Apple, Google, and Microsoft  are rolling them out.  https://tinyurl.com/PasskeySupporters  They are intended for use in remote login to web based applications.  (While apps can use passkey, many are already passwordless.) PayPal, Kayak, Best Buy, eBay, GoDaddy, and Google are among those that are offering Passkeys as a preferred alternative means of user authentication. 

Passkeys resist the security problems with passwords.   They eliminate both the choice of password requirement and the forgotten password problem.  They resist brute force and replay attacks.  Social engineering (e.g., so called "phishing") attacks no longer work.  While the user may still be duped into logging on, the process that that uses does not leak reusable information. 

(However, Passkeys may still leave one vulnerable to session stealing (MitM) attacks. This is a limitation that they shares with most remote authentication methods.  Note that, unlike the reuse of passwords, MitM attacks do not include the ability to initiate sessions, only takeover sessions initiated by the legitimate user.  They also require the ability, usually by duping the user, to insert a process between the user and his target application.)

Passkeys are an application of asymmetric key cryptography.  They are an implementation of the Fast Identity Online (FIDO) standard defined by the FIDO Alliance.  

The private key is stored on a user side device, usually in an high security module (HSM) or trusted platform module (TPM)  and is used to sign a challenge (random value sent from the application side.)  The corresponding public key is stored on the application side and is used to verify the signed challenge.  Every time one chooses to sign on to an app or a web application with a passkey, one must authenticate to the device by biometric or PIN.  

Thus Passkeys offer strong authentication.  One must possess the device holding the  private key, something that one has, and the biometric, something that one is, or PIN, something one knows, required to open the device at time of use.  The exchange of the challenge and response resists replay.  

Most often, and at least in the short run, apps that implement Passkeys will  leave their use at the option of the user.  It will be offered as an option, either at enrollment time or when signing on.   If one accesses an account from multiple devices, one  may create a passkey for the account on multiple devices.  Apple plans to store keys in the cloud, as does now with passwords, so that one key can be used across multiple Apple devices sharing access to one Apple account.  

When attempting to logon to an account that expects a passkey from a device that does not already have access to a key, one may be offered a QR code to sync to a device that does have access to a (or the) key.  Both the security and the convenience are maintained.  

Indeed security and convenience are what Passkeys are about.  They make it easier to do the right thing than the wrong thing.  Smart enterprise applications will offer them as an option and smart users will choose them.  Some enterprises will mandate them.  They offer us one more opportunity to increase the cost of attack against our networks, systems, applications, and data while improving convenience.  

Note that Passkeys rely for their security in part on the device on which the private key is stored.  Thus they are often seen as limited to a single device.  However, a number of mechanisms are available to enable their safe use across devices.  These include storing the key in a "password manager" (e.g., Bitwarden) or using the network or the cloud (Apple's implementation.)

What I tell my family about protecting their identity.

 Recently a family member asked me how to respond to a solicitation for "identity protection."  The ad appealed to fear and some of the benefits were ambiguous. 

Every time we open an account or do business, we expose ourselves to fraud.  About three percent of us will be the victims of transaction (e.g., payment card) fraud but almost one percent of us will be victims of fraud so serious as to cause serious financial loss or crippling  damage to our reputations.  Therefore, I offer the following advice in the order of its importance.  

• Use strong (e.g., multi-factor) authentication wherever it is offered.  (Prefer Passkeys for a good balance of security and convenience.)
• Avoid doing business with those who do not offer it.
• Prefer purpose-built applications for financial activity.  Avoid the use of browsers.
• Prefer mobile computers to personal computers for financial activity.
• Review all account balances and activity on a timely basis (for large and active accounts, "review" equates to online and "timely" may equate to daily.)
• Sign up for "paperless" options.  (For good security these should be the default option but for reasons of "backwards compatibility," one must usually opt in.)
• Allow notifications.  (Again, this should be the default.)*
• Freeze your identity on all three credit bureaus.  (Locking and unlocking is now easy and free but all three bureaus will take every opportunity to try and sell you "identity protection" for a relatively high annual fee.  All three have had major compromises of personal data and are not reliable.)
• Use complimentary credit monitoring from AAA, American Express, or, as offered, by your bank or credit union.
• Most card issuers now permit you to "lock" your cards, using a mobile app.  Balance this with the convenience of using the card but be sure to lock the card if it is misplaced, lost, or stolen.  
• When buying online, prefer to pay with such checkout proxies as PayPal, Apple Pay, or Click to Pay.  Avoid using debit or credit cards.  However, prefer credit cards to debit cards.  
• When paying at the point of sale, prefer "contactless."  This resists the leakage of the Primary Account Number on the magnetic stripe.  Most banks now offer such cards and both Apple and Google Pay offer.
• Do not use the option permitting the merchant to retain debit or credit card information.  Checkout as a guest; avoid signing up for accounts.  
• When using debit or credit cards for the convenience of frequent purchases from a merchant (e.g., Amazon) consider the use of a one-time or one merchant token number from Privacy.com.  
• Consider insurance against financial loss and/or expenses related to identity theft.  Such insurance is not a substitute for any of the measures above, may be redundant of protections that you already enjoy (from homeowners insurance, fiduciaries, e.g., https://www.fidelity.com/security/customer-protection-guarantee ), may be expensive, and is best purchased from insurance sources (e.g. as an optional endorsement  to one's homeowners insurance).  https://tinyurl.com/FTCreportidenttiyfraud

* While I have been writing this I have received notices of three legitimate transactions.  This assures me that I will get timely notification of fraudulent ones.