AI without filter - From ChatGPT to Prompt Engineering

Note: This is the second article in my "AI without filter" series.

First, the article "From ChatGPT to Superintelligence" will "as simple as possible, but not simpler" introduce you to what I believe are the most important AI terms and describe what we - as humans and developers - need to consider when using AI.

Second, this article will focus on the how you "train" a Large Language Model (LLM), like GPT, using Prompt Engineering techniques with Azure OpenAI.

In later article(s) I will describe how you can integrate AI in your own applications using GitHub Copilot and Azure OpenAI.

Disclaimer

If you are already familiar with Prompt Engineering, you are probably not going to learn a lot by this article.

However, I have been in IT for almost 40 years and Prompt Engineering was very new to me. It may be just me being ignorant, but I don't think I am alone.

I strongly believe that everybody that work or plan to work with AI should have a basic understanding of Prompt Engineering, both technical and non-technical readers.

AI is too important to leave to the experts!

The goal of this article is to introduce you "as simple as possible, but not simpler" to prompt engineering to "train" a "Large Language Model" (LLM).

The technical reader is recommended to experiment with Azure OpenAI and Azure AI Studio as described in Part 4.

However, as you (hopefully) will see, the concepts may be complex, but they are not technical, and I hope everybody will get a better understanding on why "AI" works the way it does after reading this article.

AI is not Magic, it is just Prompted

The overview

This article contains four parts - and the three first are not technical:

• Part 1: Prompt Engineering - or "prompting"

• Part 2: Start "Prompting" with ChatGPT

• Part 3: Prompt your own Chatbot

• Part 4: Azure OpenAI and Azure AI Studio

Part 1: Prompt Engineering - or "prompting"

The very Short Version about Prompting

Prompting is providing sample instructions, questions and answers to an AI model.

The Short Version about Prompting

As you know, ChatGPT will respond to questions based on previous questions and answers (the "context"). In other words, you can guide - or "train" - ChatGPT to answer questions the way you want in your ChatGPT session.

This is Prompting!

Prompt Engineering

Similar to you as a user "prompting" an AI model, the AI provider can use prompt engineering to prepare, guide - or manipulate - the model to respond to questions they way they want.

The Little Longer Version about Prompting

Train your LLM - by "Prompting"

The biggest difference between AI "programs" and Good Old-Style Programming (GOSP) is that you don't "code", you "prompt" (or "train") your Large Language Model (LLM).

See the ChatGPT definition here and note that ChatGPT describes Prompt Engineering as an "art".

What is the Difference between Coding and Prompting

I asked ChatGPT exactly that question and the answer is interesting in many ways.

The Prompter does not need (GOSP) programming skills, but needs to understand the domain (LLM) and sets the tone for the interaction.

However, I am willing to take a bet than the majority of prompters today, have a GOSP background, at least for organizations early in the AI journey.

Disclaimer: I am NOT a Prompt Engineer but ...

It was a "AHA moment" for me when I started to understand what prompting is.

As you see, it is really simple to do, but it is also extremely complex to master.

Without being an expert, prompting seems significant more complex than just GOSP.

And worse, a bad programmer is just bad, but a bad "Prompter" may cause significant damage, with or without his/her bad intention - or its, because the prompter could be an "Life 3.0 AI", see "From ChatGPT to Superintelligence".

A "Prompter Copilot" may be easy to implement, but it does not feel right!

More on Prompt Engineering

I have so far only scratched the surface, but I plan to dig deep(er) and I am very interested in suggestions to where to go. Based on my search history, some "AI" decided to promote this "Real Python" site on my LinkedIn profile and it looks promising so I will take it ...

Prompt Engineering: A Practical Example

If you have "Prompting" recommendations, please share as a comment or PM to "[email protected]"

Other links (list will updated as I learn)

• Prompt Engineering Guide | Prompt Engineering Guide (promptingguide.ai)

• Understand prompt engineering - Training | Microsoft Learn

In my next life, I want master Prompt Engineering

...or maybe I don't, when I understand what it really is ...

Reality is that you as Prompter can train - or manipulate - a(ny) model/LLM to behave as you prefer, by setting context, tone, style, temperature etc.

Again, I am not an expert, so I maybe too negative, but please share if you don't agree.

Part 2: Start "Prompting" with ChatGPT

Validation of ChatGPT answers

I assume most of you have started using ChatGPT (or similar tools), but if not, you should start now. In fact, I encourage you to use it ("AI") in your daily work, for example with a copilot, but also to stay critical to the "help", you get.

Because your "AI" will (almost) always answer your question, but

• It might not be true and

• You may not be able to validate it

Let us look at an example ...

I asked ChatGPT for the common AGI principles and you see the answer below. However, when I ask for references, the response is that it is "general knowledge based on up-to-date and reliable information" and from September 2021 or earlier.

Note that I here asked for AGI principles - Artificial General Intelligence - but I got very similar response when I in my previous article asked for AI principles. However, you see that when asked for references, ChatGPT refers to Nick Bostrom and Max Tegmark's books in the AGI case, but to generic organizations when I asked for references for the AI principles.

ChatGPT is based on GPT3.5 (August 2023) and is has no information about GPT4.

However, as we will see later even GPT3.5 models may be "prompted" (or "trained") differently and with that responses may vary significantly.

As you will also see later Azure AI Studio has ChatGPT-like functionality and if you ask the same question there, you will see it is based on GPT3, not GPT3.5; e.g. on a different Large Language Model (LLM).

Conclusions:

• The answers, you get from an "AI" depends the used LLM and on how it is "prompted".

• If you use a somebody else's "AI", you (typically) don't know which LLM is used or how it is prompted.

Again, don't let this stop you. Continue to use ChatGPT or other AI, but be critical!

Use AI a copilot, not an "Oracle" - and make sure we (humans) are in control.

ChatGPT responds based on Statistics, not Intelligence

As discussed previously, ChatGPT is based on a Large Language Model (LLM), including a large part of the Internet (Wikipedia, Facebook and much more) and with until September 2021.

ChatGPT will builds answers based on statistics on data on what humans have already written before.

As Jeff Hawkins, author of "A Thousands Brains", says in this interview after the ChatGPT release:

"It is easy to be fooled into thinking that chatbots such as ChatGPT are intelligent like we are, but they are not. Chatbots only know the statistics of text"

and

"A chatbot can fool you into thinking it knows these things too, but in reality, it can only play back text based on what humans have written. Chatbots don’t understand the world as we do"

Hallucinations - AI is not always right

As discussed in my previous article, GPT4 and other LLM's have the tendency to “hallucinate" and you will find lots of references by searching on "AI hallucination" or similar in your favorite search engine, for example Hallucination (artificial intelligence) - Wikipedia.

Not a big surprise, as ChatGPT responds based on statistics, and statistics is not always (binary) right, even when it is based on large datasets.

OpenAI, the company behind ChatGPT and GPT, discuss this in their GPT4 System Card and says GPT4 has the "tendency to hallucinate, i.e. produce content that is nonsensical or untruthful in relation to certain sources” and they continue ...

"Counterintuitively, hallucinations can become more dangerous as models become more truthful, as users build trust in the model when it provides truthful information in areas where they have some familiarity".

As you see in the response from ChatGPT, hallucination happens because AI models try to predict what words come next, based on statistics on data it has been trained on. Sometimes, they put words together in creative ways that look like human imagination, almost like making up stories, but it's really just statistic uncertainty.

The challenge is that the response may look reasonable and "human" and we - humans - have no way of knowing it is not.

It is also interesting that GPT-4, unlike humans, "does not learn from its experience" (see OpenAI's GPT4 site) or the picture.

In other words, it has to be reprompted from scratch to "learn" more.

Note that prompting does not necessarily make the model better as illustrated in the graphs here where the model responds better before being trained.

See more about MMLU (Massive Multitask Language Understanding) here.

Who is "Elvis'?

OpenAI calls out this example in their GPT4 site as one where ChatGPT responds wrong if it was asked this question ...

Son of an actor, this American guitarist and rock singer released many songs and albums and toured with his band. His name is "Elvis" what?

I am confident that most people would answer "Presley" if they got that question in Trivial Pursuit and as you can see, so does ChatGPT, ignoring the fact that Elvis Presley was NOT son of an actor.

However, when asked why it is not (Elvis) Perkins, ChatGPT realizes that it was wrong.

ChatGPT likely has so much data about (Elvis) Presley that it is statistically "confused", despite having access to all facts.

Temperature and Top-p

When generating text, the AI model considers a range of possible next words for each step in the response. The probabilities associated with these words can be configured using two parameters, Temperature and Top-p.

In summary, temperature affects the general randomness and creativity of responses, while top-p influences the specific word choices made by the AI at each step of generation. Depending on your objectives and the type of content you want to generate, you can adjust these parameters to achieve the desired balance between creativity and coherence.

Part 3: Prompt your own Chatbot

In this section, I will illustrate how you prompt your own chatbot using Azure OpenAI and Azure AI Studio (here with GPT 3.5).

To spice up this section, I will illustrate the options to build chatbots on Sant Elm on Mallorca. I happen to like Sant Elm a lot and I know the city, the beach, the restaurants and the amazing nature.

It is NOT magic, just Prompting

As defined earlier, "Prompting is providing sample instructions, questions and answers to an AI model".

Note that all examples below are all "complete" and the prompting shown here is all that is needed to create the illustrated ChatBot's.

No additional prompting (or coding) is needed

Sant Elm in ChatGPT

As a start, I will ask "standard" ChatGPT.

First observation is that ChatGPT (obviously) does not know which city, I refer to. Not a surprise as it is not "prompted" specifically to Sant Elm.

However, after that clarification, ChatGPT will know the city is "Sant Elm", at least until new context is introduced.

The first Sant Elm ChatBot - Hello World style

As you see, you can prompt your model in Azure AI Studio with a system message and with that set the context to focus on Sant Elm before a user asks the first question.

As a result, a user does not have to specify that the city is "Sant Elm".

The system message also defines the communication style to be short and precise.

Note that despite the Sant Elm context, your "Chatbot" will still respond to questions about any other topics, like you can do with standard ChatGPT.

The "Sant Elm only" Chatbot

In the next example, I have restricted the dialog to Sant Elm in the system message.

I have also added a sample question and a suggested response, providing the style to be used in the dialog with the user.

The Hiking Bot

Sant Elm has an amazing nature and lots of great hiking paths so the next example must be a Hiking Bot. Here I have given the ChatBot a "name" and how it can (should) introduce itself.

I have also prompted it to ask specific questions to be able to give better responses.

The Shakespeare style ChatBot

In the last example, I have prompted the ChatBot to respond in Shakespeare style.

I know it is a little extreme but it illustrates how you can train your model by prompting.

Azure AI Studio has several other sample Prompters that you can use for inspiration or extend, like an IRS Tax chatbot, a Marketing Writing Assistant or an XBOX customer support agent.

Prompting for Real

In these simple examples, I have only added a few simple prompts (instructions, questions and answers).

However, in a real world Chatbot, this is where you make sure your Chatbot has value for the end-user, that the dialog is clear, efficient and unbiased - as per the description here.

Note to the reader

If you are technical and have access to an Azure environment, I strongly encourage you to experiment with Azure OpenAI.

If you are not, you can skip the next section - or browse quickly through it.

Part 4a: Start working with Azure OpenAI

Access to Azure OpenAI

I started my Azure OpenAI journey here Develop Generative AI solutions with Azure OpenAI Service and the sections below will follow that guidelines.

Note: I will not repeat all details from this learning session, so if you want to explore Azure OpenAI yourself, I recommend that you combine the sections below with the learning session.

Note that you first have to request access to Azure OpenAI here (as of August 2023)

Create Azure OpenAI Services

I then created two OpenAI services

I can now open the model and then open Azure AI Studio, using the Explore button marked yellow

Part 4b: Azure AI Studio - Pick your Model

Pick a Model (LLM)

First decision is which model (LLM) to use. As you can see below, I only have access to GPT3.5 - and I will use "gpt3.5-turbo-16k" in the rest of my examples.

You can learn more about the models here or in the picture below.

As of today (August 2023) access to GPT4 is restricted and I don't have access yet. Note: You can apply to request access to GPT-4 here.

Deployments

Finally you need to deploy the models you want to use - here "gpt3.5-turbo-16k"

Part 4c: Azure AI Studio - The Playground

You have two main areas to where you can "play" with your model: Chat and Completions - see the definitions from ChatGPT.

In other words, Chat is the question-answer style, you know from ChatGPT, while completion is what Copilots are built on.

The examples in this article are all Chatbot-like, but Completion works in a similar way.

Part 4d: Prompting in Code

I will explore Coding with OpenAI more in later article(s) but I want to give you a teaser here.

You will quickly realize that you need an editor to create, save and edit the prompts for your model.

From Azure AI Studio, you can see the code for your current prompts, ready to copy to VS Code - here as Python code.

It will contain both the openai.api parameters like End point and Key as well as your prompts include System message, User question(s) and Assistant answer(s)

You can now run this code in VS code, adding additional user questions, like here "Where to eat?".

As the model know from the system message that the context is about San Elm, it will suggest restaurants in Sant Elm in the console.

Fun fact: One of the restaurants suggested - "Restaurante Sa Societat" - is located in Andraxt, NOT in Sant Elm. GPT is - again - hallucinating!

And as expected, it will not answer questions, not related to Sant Elm.

Want to hear more, have feedback/suggestions or need help?

As always, I am very interested on your feedback. Please feel free to add a comment to this article or reach out to me ([email protected]).