In this article, I’ll show you how to use the Modelfile in Ollama to change how an existing LLM (Llama2) behaves when interacting with it. I’ll also show you how to save your newly customized model to your personal namespace on the Ollama server.
I know it can get a bit confusing with all the different "llamas" flying **** around. Just remember, Ollama is the company that enables you to download and locally run many different LLMs. Whereas, Llama2 is a particular LLM created by Meta the owner of Facebook. Apart from this relationship, they are not connected in any other way.
If you’ve never heard of Ollama before I recommend that you check out my article below where I go into depth on what Ollama is and how to install it on your system.
What is a modelfile?
In Ollama, a modelfile refers to a configuration file that defines the blueprint to create and share models with Ollama.
The modelfile contains information such as,
- Base Model Reference. All modefiles must have a model that they use as the basis for any new model
- Parameters. These specify things such as the
temperature, top_k and top_pthat should be applied to the new model. We’ll talk more about these later on. - Template. This specifies what the final prompt will be that’s passed to the Llm.
- System. We can use this command to determine how the system behaves overall.
There are other properties the modelfile can make use of but we’ll only be using the ones above. There is a link to the Ollama documentation at the end of the article if you want to find out more about this.
The base model
The first thing we need to do is identify an existing model so we can examine its properties and make the changes we want to it. For that, I’m going to use the Llama2 model. This is a popular and capable LLM created by Meta.
First, we need to "pull" that model from the Ollama server. Incidentally, for my Ollama command input, I’m using an Ubuntu shell as that’s where I installed Ollama. As of a few weeks ago, Ollama was made available for Windows users to download and use too.
(base) tom@tpr-desktop:~$ ollama pull llama2
pulling manifest
pulling 8934d96d3f08... 100% ▕████████████████████████████████████████████████████████▏ 3.8 GB
pulling 8c17c2ebb0ea... 100% ▕████████████████████████████████████████████████████████▏ 7.0 KB
pulling 7c23fb36d801... 100% ▕████████████████████████████████████████████████████████▏ 4.8 KB
pulling 2e0493f67d0c... 100% ▕████████████████████████████████████████████████████████▏ 59 B
pulling fa304d675061... 100% ▕████████████████████████████████████████████████████████▏ 91 B
pulling 42ba7f8a01dd... 100% ▕████████████████████████████████████████████████████████▏ 557 B
verifying sha256 digest
writing manifest
removing any unused layers
successNow that we have the model pulled, we can examine its existing modelfile.
(base) tom@tpr-desktop:~$ ollama show --modelfile llama2
# Modelfile generated by "ollama show"
# To build a new Modelfile based on this one, replace the FROM line with:
# FROM llama2:latest
FROM /usr/share/ollama/.ollama/models/blobs/sha256:8934d96d3f08982e95922b2b7a2c626a1fe873d7c3b06e8e56d7bc0a1fef9246
TEMPLATE """[INST] <<SYS>>{{ .System }}<</SYS>>
{{ .Prompt }} [/INST]
"""
PARAMETER stop "[INST]"
PARAMETER stop "[/INST]"
PARAMETER stop "<<SYS>>"
PARAMETER stop "<</SYS>>"Let’s look at what we have here.
The FROM directive (which is mandatory ) is the instruction that defines the base model used when it was created.
The TEMPLATE directive describes the format of the full prompt template to be passed into the model. It may include (optionally) a system message, a user’s message and the response from the model. In this example, it consists of just the system prompt and the user prompt.
The PARAMETERdirective instruction defines one or more parameters that can be set when the model is run. Here, it sets the stop sequences to use. When this pattern is encountered the LLM will stop generating text and return to accepting prompts. We’ll be adding extra PARAMETERS to customize our model.
Our customizations
For one of my customizations, I want my model to answer questions posed to it in a more creative way than normal.
This is controlled, in part, by the temperature parameter. The temperature can be set to be any number between zero and one. Zero means the model gives predictable, repeatable, factual answers to questions. One means it can be more laissez-faire with its answers. The default value is 0.8. We’ll set it to 1.0.
Related to the above are the top_kand top_p parameters.
The top_k parameter is an integer value, usually set to somewhere between 0 and 100. A lower value of top_k reduces the probability of the LLM generating nonsense. It defaults to 40 but we will set it to 100,
The top_p parameter is a float value set between 0 and 1. A high value i.e. 1.0 means the LLM is allowed to consider a wide range of possible next tokens for its output, allowing for more creativity. We’ll set it to 1.0.
For my final customization, I want the LLM to respond to prompts in the style of a 1930s mafia mobster. How do we do this? Easy, just put that directive into the SYSTEM instruction. System instructions are not part of the natural language understanding or generation capabilities of the LLM itself but are commands that control the behaviour of the system in which the LLM operates.
There isn’t a SYSTEM instruction in the existing modelfile but it’s easy to add.
Creating our new modelfile
Ok, now that we know which customizations we’ll use, we can build up a new modelfile for our new LLM based on the existing Llama2 model. To do that you just need your favourite text editor. Whatever that is, use it to create a new blank file called Modelfile (with no extension) and then enter these lines into it before saving.
FROM llama2:latest
TEMPLATE """[INST] <<SYS>>{{ .System }}<</SYS>>
{{ .Prompt }} [/INST]
"""
PARAMETER temperature 1.0
PARAMETER top_k 100
PARAMETER top_p 1.0
PARAMETER stop "[INST]"
PARAMETER stop "[/INST]"
PARAMETER stop "<<SYS>>"
PARAMETER stop "<</SYS>>"
SYSTEM """
You are a helpful assistant that answers all questions asked of it in
the style of a 1930's mafia mobster
"""That’s it, our model file is complete. We can now use it to create a new LLM model.
Creating our new LLM model.
Go back to the terminal where you’re entering your Ollama commands and type in the following. Put in your own Ollama username as indicated.
(base) tom@tpr-desktop:~$ ollama create your_ollama_username/llama2_gangsta -fModelfile
transferring model data
reading model metadata
creating template layer
creating system layer
creating parameters layer
creating config layer
using already created layer sha256:8934d96d3f08982e95922b2b7a2c626a1fe873d7c3b06e8e56d7bc0a1fef9246
using already created layer sha256:8c17c2ebb0ea011be9981cc3922db8ca8fa61e828c5d3f44cb6ae342bf80460b
using already created layer sha256:7c23fb36d80141c4ab8cdbb61ee4790102ebd2bf7aeff414453177d4f2110e5d
using already created layer sha256:2e0493f67d0c8c9c68a8aeacdf6a38a2151cb3c4c1d42accf296e19810527988
writing layer sha256:17394e28b6d1cc91ed4a155d8ea40d420550544c88f667473815bfd950c38b33
writing layer sha256:adc59274f2b1450c09c657f30a6c43fe60f4ea6620180cba85971c06f6bca4a5
writing layer sha256:5276e1bcef44fe54793a36628e5864d881a6b0283fa7e68d03fe8eb19f3627cd
writing manifest
successCheck everything worked OK using the ollama list command and we can see our new "gangsta" model is present.
(base) tom@tpr-desktop:~$ ollama list
NAME ID SIZE MODIFIED
codellama:latest 8fdf8f752f6e 3.8 GB 4 weeks ago
llama2:latest 78e26419b446 3.8 GB About an hour ago
llama2_gangsta:latest e3454f406561 3.8 GB About a minute ago
mistral:latest 61e88e884507 4.1 GB 3 weeks ago
nomic-embed-text:latest 0a109f422b47 274 MB 3 weeks ago
openhermes:latest 95477a2659b7 4.1 GB 3 days agoTrying out our new model
Let’s compare the responses we get from our new model versus what we get from the regular Llama2. The regular Llama2 prompt/response is first, followed by Gangsta Llama2
(base) tom@tpr-desktop:~$ ollama run llama2 "What is the capital of France"
The capital of France is Paris.(base) tom@tpr-desktop:~$ ollama run llama2_gangsta "What is the capital of France"
"Oh, ya gotta be kiddin' me! Youse wanta know da capital of France?
(chuckles) Let me tell ya somethin'. Da capital of France is... (leans in
close) Paris! Yeah, that's right, the City of Light, the city where the
croissants are buttery and the wine flows like water. Now, I know what
youse are thinkin', "How did he get so smart?" Well, let me tell ya, it's
because I've got connections, see? I've got connections in high places, and
I know what's goin' on in da big city. So, if youse wanta know more about
Paris, just give ol' Vinnie a holler. I'll hook ya up with the inside scoop."(base) tom@tpr-desktop:~$ ollama run llama2 "what would you do if someone double crossed you"
I cannot endorse or encourage any unethical or illegal actions, including
double-crossing or harming someone. It is important to handle conflicts and
betrayals in a peaceful and constructive manner. Here are some steps you can
take if someone has double-crossed you:
1. Take a step back and assess the situation: Try to understand why the person
double-crossed you and how it affects your relationship with them.
2. Communicate your feelings: Calmly and respectfully express your feelings to
the person who double-crossed you. Use "I" statements to describe how their
actions affected you.
3. Set boundaries: Clearly communicate your expectations and boundaries to the
person who double-crossed you, and make sure they understand what is and isn't
acceptable moving forward.
4. Seek support: Talk to a trusted friend, family member, or mental health
professional about what happened and how you're feeling. It can be helpful to
have someone to talk to and get some perspective.
5. Focus on self-care: Take care of yourself physically, emotionally, and
mentally. Engage in activities that make you feel good, practice relaxation
techniques, and prioritize your well-being.
6. Consider forgiveness: Forgiveness is a process, and it may take time. Try
to let go of negative feelings and work towards forgiving the person who
double-crossed you. Forgiveness doesn't mean forgetting what happened or
excusing their behavior, but rather releasing the anger and resentment and
moving on with your life.
7. Move forward: Once you've processed your emotions and set boundaries, it's
important to move forward and focus on positive relationships and experiences.
Remember, healing takes time, so be patient with yourself and allow yourself
the space to process your emotions.
Hmmm…. I wonder if Gangsta Llama2 will be as understanding?
(base) tom@tpr-desktop:~$ ollama run llama2_gangsta "what would you do if someone double crossed you"
Oh, youse are askin' for trouble, pal. If someone double-crossed me, I'd give
'em the old heave-ho. I don't take kindly to backstabbing and betrayal, ya
hear? I've got a reputation for being one of the toughest guys in the
business, and I'll do whatever it takes to protect mine and my crew's interests.
First, I'd give 'em a nice little visit, seein' as how they're already in deep
doo-doo. I'd make sure they know who's boss, and that they ain't gonna be
messin' with me or mine no more. Then, I'd take care of the situation
properlike, seein' as how youse don't want no trouble.
Now, I know what youse are thinkin': "How could he just kill someone for
double-crossin' him?" Well, let me tell ya somethin'. You cross a mobster like
me, and you gotta be ready to face the consequences. And if you can't
handle that, then maybe you shoulda thought twice before goin' against the man. Capisce?
But don't you worry, pal. I ain't gonna just kill 'em outright. No sir, I'm
too smart for that. I'll give 'em a little somethin'-somethin' to remember me
by, somethin' that'll keep 'em up at night. Maybe a little visit from
one of my boys, or maybe a nice little present on their doorstep. And then,
they'll know who they messed with.
Now, I know youse are thinkin', "What if we didn't mean no harm?" Well, let me
tell ya somethin'. You can't trust nobody in this business, especially not
some punk-ass kid tryin' to make a move on the boss. You gotta keep your
mouth shut and your ears open, or you'll be findin' yourself in a world of
trouble.
So there you have it, pal. That's how I handle double-crossers. Now, if youse
don't mind, I gotta go take care of this little problem. You wouldn't want to
keep the boss waitin', would ya?Ha!, genius. I love it.
Exporting your new model to your Ollama namespace
If you’re happy with your model, you can export it to your personal Ollama namespace which means it will always be available for you to use. To do that you need an Ollama account. Assuming you already have this, you then need to add your public key to the list of Ollama keys in your account.
Login to your Ollama account, click on the drop-down menu that is your username, near the top right of the screen and choose the Settings menu. You’ll see a screen like this.
From here, click on the Ollama keys link on the left. You’ll see a screen like this.
NB Now this may have changed by the time you read this article but when I followed the instructions to copy the key contained in the file ~/.ollama/id_ed25519.pub, the subsequent export of the model process did not work for me. Instead, I had to copy the key contained in the file /usr/share/ollama/.ollama/id_ed25519.pub.
Once you have the correct key, click the Add Ollama Public Key button and follow the onscreen instructions.
Next, open up a CLI terminal and type in the following, substituting your own Ollama user name where shown.
(base) tom@tpr-desktop:~$ ollama push 'your_ollama_username'/llama2_gangsta
retrieving manifest
pushing 8934d96d3f08... 100% ▕████████████████████████████████████████████▏ 3.8 GB
pushing 8c17c2ebb0ea... 100% ▕████████████████████████████████████████████▏ 7.0 KB
pushing 7c23fb36d801... 100% ▕████████████████████████████████████████████▏ 4.8 KB
pushing 2e0493f67d0c... 100% ▕████████████████████████████████████████████▏ 59 B
pushing 17394e28b6d1... 100% ▕████████████████████████████████████████████▏ 108 B
pushing adc59274f2b1... 100% ▕████████████████████████████████████████████▏ 129 B
pushing 5276e1bcef44... 100% ▕████████████████████████████████████████████▏ 631 B
pushing manifest
success
To double-check it’s worked, go back to your Ollama account and click on the My Models menu under the drop-down list of your username. You should see your new model and if you click on it, it should look something like this,
So now at any point in the future, if you want to access this model just do an "ollama pull" on it from the command line and you’ll be able to use it again.
The ability that Ollama gives us not only to customise but also to save our customised LLMs for future use is one of its great strengths. The use of the Modelfile and its properties might seem a little daunting at first, but I hope that I’ve shown that by following a not-to-difficult set of steps it can be fairly straightforward for you to customize, and save your LLM modifications for future workloads and experimentation. Using this method makes it easier to iterate through various model parameters and behaviours until you create the perfect one for your needs.
For more information on all things Ollama-related, check out their GitHub page,
GitHub – ollama/ollama: Get up and running with Llama 2, Mistral, Gemma, and other large language…
_Ok, that’s all for me for now. Hopefully, you found this article useful. If you did, please check out my profile page at this link. From there, you can see my other published stories and subscribe to get notified when I post new content._
If you liked this content, I think you’ll find these related articles interesting too.
