From Zero to Chat-GPT (Part 1)

461

Learn how to create your own mini Chat-GPT

Who is this course for?

Anyone who wants to understand how Chat-GPT works, even if it seems like magic to you. If you know nothing about programming but have basic math skills (you can add, right?), you can follow along. If you’re already an engineer, you might want to skip the first few lessons. In this course, we’re going to create our own mini Chat-GPT from scratch. Let’s get started!

In this part, we will:

  1. Set up our Python programming environment
  2. Learn to display messages using the print command
  3. Understand variables and data types
  4. Explore the basic structure of a simple neuron model
  5. Discover the concept of Python functions and implement a basic activation function

Set up the programming environment:

Go to https://replit.com and sign up for a free account. Click Start creating.

Then you’ll need to choose a method to identify yourself. This is so everything you do, stays there.

You can use your current Google login (easier), or set up the account with an email and password if you wish.

Welcome to your first programming environment. In Replit, you can easily write programs and run them.

Click on one of the [+] buttons to create your first program. We’ll use the Python programming language.

Your project will be given a unique Title, you can change it, but it doesn’t matter. Click the [ + Create Repl ] button. You should now see a new Python Repl with a blank main.py program file.

Example 1: Show a message

Now you are goint to write your first program. We’ll use the print(“message…”) command to show a message. In the main.py file, type the following code:

print("Hello, I'm Mini-Chat-GPT!")

Tip: If you don’t want to type, you can select the text in the examples with the mouse. Then press copy (Control/Command + C), then click into the Replit program widow and press paste (Control/Command + V).

Click the ▶ Run button to execute the code. You should see the output “Hello, I’m Mini-Chat-GPT!” in the console. Congratulations! You’ve just written your first program in Python. Now, let’s move on to the next example, where we’ll introduce variables and basic data types needed to create a simple neuron model.

Example 2: Variables and basic data types

Variables are used to store data in a program. They have any made up name, like input1, and they hold a value, like 0.5. In Python, you don’t need to specify the data type of a variable explicitly; it’s determined automatically based on the assigned value (that is, a variable can store a number like 123 or other things like “a string of text”)

input1 = 0.5
message = "this is a text message"

# This symbol is used to write comments in the code that will be ignored by the computer, to explain things

You can use expressions and formulas to compute values from variables:

a = 5 # Number 5 stored in variable 'a'
b = a + 5 # Variable 'b' will be set to 10 (5 + 5)
print(2 * (a + b)) # Will show 30 as result (2 * (5 + 10)) 

Create a simple neuron model

We are going to create a small program to compute a simple expression. This expression will be a simple neuron model as shown here.

This digital ‘neuron’ is just a simple weighted sum formula, where all the ‘inputs‘ are multiplied by their ‘weights‘ and added together. Plus another number, the ‘bias‘, can be modified to influence the final result.

Now, you might be wondering why we chose this particular model. Well, this is one of the simplest models of a neuron in a neural network. Although it is a far cry from the complexity and sophistication of real biological neurons and even advanced artificial neurons used in state-of-the-art AI systems, it effectively demonstrates the basic principles that underlie them all.

First, in the main.py file, let’s define the inputs, weights, and bias for a simple neuron model.

input1 = 0.5 # First neuron input
input2 = 0.8 # Second neuron input

weight1 = 0.1 # First input modifier 'weight'
weight2 = -0.2 # Second input modifier 'weight'

bias = 0.3 # Final value added to the neuron

Don’t worry about the actual numbers; they are just that, numbers. They don’t mean anything. The important thing is the idea that the output (weighted sum) comes from mixing all the inputs, plus another number.

Calculate the neuron’s output

The output of the neuron is the sum of the products of each input and its corresponding weight, plus the bias. We’ll use the following formula:

output = input1 * weight1 + input2 * weight2 + bias

After all variables (after the “bias = … ” line), add the following code to main.py to calculate the output:

output = input1 * weight1 + input2 * weight2 + bias
print("Neuron output:", output)

Click the ▶ Run button to execute the code. You should see the output of the neuron in the console.

Neuron output: 0.18999999999999995

It’s a very simple idea. The output depends on all the inputs, but not in the same way, because they all have a weight. So if the first input goes up by 1, the output will go up by 0.1 (weight1). But if the second input goes up by 1, the output will go down -0.2 (weight2). If we modify the bias number, the output will also be modified by as much.

As you can see, what we call in programming a ‘neuron’ is just a stupid simple formula. Nothing magic, and nothing even remotely as complex as in biology.

Now that you’ve created a simple neuron model, let’s move on to the next example, where we’ll introduce basic Python functions and use them to create a basic activation function for the neuron.

Example 3: Python functions and activation functions

In this example, we’ll introduce Python functions, which allow us to group a series of statements into a reusable block of code. Functions are especially helpful when you want to perform a certain task multiple times or when you want to break down complex code into smaller, more manageable pieces. We’ll create a basic activation function for our neuron, which will help to determine whether the neuron should be active or not based on its output.

# Creating a function like myfunc() allows us to reuse a piece of code, it goes:
# def whatever_function_name(values, that, the, function, takes):
# [Tab] ... # Everything inside the function is preceeded by a [Tab] character (or 2 spaces)
# [Tab] return the + function * value # The function can compute values and do things, then return whatever result

def myfunc(x): 
  return x * 2

a = myfunc(3) # Sets 'a' to 6
b = myfunc(a) # Sets 'b' to 12

# You wont see the result in the output, unless you use the print() command
#print("a =", a)
#print("b =", b)

Create a function to calculate the neuron’s output

First, let’s create a function called calculate_output that takes the inputs, weights, and bias as arguments and returns the neuron’s output. Replace the existing code to calculate the output (the “output = …” line) with the following function definition:

# Place this function at the beginning of the program, for example

def calculate_output(input1, input2, weight1, weight2, bias):
  return input1 * weight1 + input2 * weight2 + bias

Now, you can use the calculate_output function to compute the neuron’s output. Replace the code that calculates the output and prints it with the following lines:

output = calculate_output(input1, input2, weight1, weight2, bias)
print("Neuron output:", output)

If you run the program, the output will remain the same. We just wrapped the formula into a function (calculate_output) we can easily reuse.

Task: Try changing the values of the weights and the bias. Observe how these changes affect the output of the neuron.

Create an activation function

An activation function is used to determine whether the neuron should be active or not based on its output. One common activation function is the ReLU (Rectified Linear Unit) function, which returns 0 if the input is negative and the input itself if it’s non-negative. Let’s create a function called relu to implement the ReLU activation function:

def relu(x):
    if x < 0:
        return 0
    else:
        return x

As simple as it looks; relu(-N) returns 0 (neuron inactive), but relu(N) returns N (neuron active)

Apply the activation function to the neuron’s output

Now, let’s apply the ReLU activation function to the output of our neuron. Add the following line of code after calculating the output:

activated_output = relu(output)

Then, print the activated output:

print("Activated neuron output:", activated_output)

Click the ▶ Run button to execute the code. You should see the output of the neuron and the activated output in the console. Congratulations, you have programmed a simple neuron model with an activation function.

Quick Recap

Before we wrap up this part, let’s quickly review what we’ve learned:

  • We set up our Python programming environment on Replit
  • We learned how to print messages in Python using the print() function
  • We explored Python variables and learned that their data types are automatically determined based on the assigned values
  • We learned about a simple neuron model and how the inputs, weights, and bias interact to produce an output
  • We learned to write Python functions and implemented a basic ReLU activation function

Summary of Part 1

In this first part, we have covered the basics of Python programming and built a simple digital neuron model using variables, expressions, and functions. We have also introduced the concept of an activation function to determine whether a neuron should be active or not based on its output. The simple neuron model demonstrates how inputs, weights, and biases interact to produce an output that can then be transformed by an activation function.

As you can see, the concept of a digital neuron is quite simple. It’s a building block that takes multiple inputs, each with an associated weight, combines them with a bias, and then passes the result through an activation function. This is the foundation upon which more complex neural networks are built.

In the next chapter, we’ll take this foundational knowledge and use it to create a neural network by connecting multiple neurons together. You’ll learn how these interconnected neurons can work together to solve more complex problems, ultimately paving the way to creating our own mini Chat-GPT.

End of Part 1

And with that, we wrap up Part 1 of our journey from Zero to Chat-GPT! I hope you’re excited to see just how far we can take these basic concepts. There’s nothing quite like the thrill of seeing your own code spring to life, and I’m thrilled to be part of this learning journey with you. Remember, every expert was once a beginner who dared to start.

So keep experimenting, keep asking questions, and most importantly, keep enjoying the process. After all, coding is not just about finding solutions, it’s also about having fun along the way! I can’t wait to dive deeper into neural networks with you in Part 2.

Until then, happy coding! Cheers, Ben

[Continued on From Zero to Chat-GPT Part 2]

Additional Resources

For further reading and exploration on these topics, you can check out the following resources:

  • Python Official Documentation: https://docs.python.org/3/
  • Deep Learning Book by Ian Goodfellow, Yoshua Bengio, and Aaron Courville: Link Here
  • Harvard CS50’s Introduction to Programming with Python – Full University Course: YouTube link
  • 3Blue1Brown’s “Neural Networks” YouTube playlist: Link Here
  • Machine Learning for Beginners: A Practical Guide by Michael R. Branson: Amazon link
  • Neural Networks and Deep Learning by Michael Nielsen: Link Here

Glossary

NeuronIn the context of artificial neural networks, a neuron is a mathematical function that takes multiple inputs, combines them in some way, and produces an output.
InputIn a neural network, an input is the data that is fed into the network.
WeightA weight is a factor by which the input is multiplied. It determines the importance or influence of an input on the output.
BiasA bias is an additional parameter in the neuron which is used to adjust the output along with the weighted sum of the inputs.
Activation FunctionAn activation function is used in a neuron to determine whether and how much the neuron should be activated based on its input.
AWS Cloud Credit for Research
Previous articleThe Open Source AI Revolution: How the Power of the Crowd is Outsmarting Big Tech Giants
Next articleRevolutionizing Education with AI-Generated Content
Benjamin Clarke, a New York-based technology columnist, specializes in AI and emerging tech trends. With a background in computer science and a Master's degree in Artificial Intelligence, Ben combines his technical expertise with an engaging storytelling style to bring complex topics to life. He has written for various publications and contributed to a variety of AI research projects. Outside of work, Ben enjoys exploring the vibrant New York City arts scene and trying out the latest gadgets and gizmos.

LEAVE A REPLY

Please enter your comment!
Please enter your name here