Over the break I’ve been playing with llamas, or rather, with one llama in particular: llamafile. AI, ChatGPT and Large Language Models took the world by storm last year, and llamafile is one of the latter, albeit a rather special type of LLM; one that you can download and run locally on your own computer, or any computer, even if it’s not connected to the internet.
Llamafile comes to you courtesy of a Mozilla Hacks project, is free to download, free to use and open source, so anyone can go in an tinker with the code or contribute to the project. (Mozilla, you’ll recall, are the folk behind the Firefox browser.)
So how do you get started? It couldn’t be easier, in part because the same download can be used on machines running macOS, Linux or Windows. (Okay, you do need to add “.exe” to the file name to run it under Windows, but that’s about the extent of it.) Full instructions and download links are here. Just scroll down to Quickstart.
Once up and running, llamafile will open a browser window where you can type in your queries. A browser window? I thought you said it didn’t connect to the web. I did, and it doesn’t. If you look at the address in the address bar you’ll see it says “127.0.0.1:8080”. That “127.0.0.1” is what’s known as ‘localhost’, your local machine. (And the “8080” is a port number.) If you don’t believe me, disconnect Wi-Fi or pull out the plug. You’ll be able to continue llama-ing.
The interface may look a little intimidating, but all you really need do is type your name in the User Name pane and Say Something down the bottom. It then continues in a more conversational form.
So what exactly is a Large Language Model. Let’s ask:
Pretty impressive. But there’s a caveat here. You’ll notice I used the full name and not the abbreviation. I did originally, but this what the llama replied:
Further enquiries revealed the reason for the discrepancy:
I’m guessing ‘LLM’ wasn’t in common parlance back in the dim dark ages of September 2021, but it’s worth keeping in mind that all these tools have limitations, and even biases, no matter how chatty and helpful they may seem.
Other llamas
If you scroll further down the README.md file, (that’s this one here), you’ll find a few more llamafiles and instructions on how to use them. I put the modest 5.15 GB Mistral-7B-Instruct model through its story-telling paces. This llama runs from the command line so I used this command:
./mistral-7b-instruct-v0.2.Q5_K_M.llamafile --temp 0.7 -p '[INST]Write a story about llamas[/INST]'Here’s what it came up with:
Once upon a time, high in the Andes Mountains of South America, there lived a herd of llamas. They roamed freely across the vast grasslands, enjoying the fresh air and abundant food that the mountains provided. The llama was a proud creature, with long, thick fleece that kept them warm in the cold Andean nights.
One day, a young llama named Luna was born into the herd. She was small for her age, but had a bright, curious eye that seemed to take in everything around her. As she grew, Luna became known throughout the herd for her adventurous spirit and her love of exploring.
One day, while wandering through the mountains, Luna came across a group of tourists. They were amazed by her beauty and grace, and began taking photos and videos of her. Luna was unfazed by their attention, she simply continued to wander and explore the mountains that she loved so much.
As Luna grew older, she became a respected member of the herd, and her adventurous spirit only seemed to grow stronger. She continued to explore the Andes, always returning to the same spot where she had been born, to watch over her family and friends.
Years passed, and Luna’s fleece grew thicker and softer with age. It was prized by weavers and knitters from all around the world, and became known as one of the finest llama fleeces in existence. But despite her fame, Luna never forgot where she came from or the love of adventure that had brought her to the Andes Mountains.
And so, Luna lived out her days, surrounded by the beauty and wonder of the mountains that she loved so much, always ready for a new adventure whenever it called her name. [end of text]
That took 62 seconds on my modest Linux box.
How about something more challenging? How about a story about llamas in the style of Edgar Allan Poe?
Once upon a midnight dreary, I wandered weak and weary,
Over many a faint and forgotten mountain pathway.
Where the pale moonbeams kissed the craggy peaks,
There dwelt a secret, shadowy race of creatures.
Creatures that, though they walked upon two legs,
Were more akin to beasts than men.
These were the llamas, my friend,
The dreaded, mysterious llamas.
As I approached their rocky lair,
A chill ran down my spine,
For I could feel their malevolent gaze
Upon my very soul.
With every step I took,
The air grew colder and more oppressive,
As if the mountains themselves were conspiring
To keep me from my goal.
Finally, I arrived at the entrance to their domain,
A gaping maw in the side of the mountain,
Guarded by two great, hulking beasts.
Their fur was matted and dirty,
Their eyes burned with an otherworldly fire,
And in their nostrils hung long, twisted tusks.
These were the guardians of the llamas,
The fearsome, unyielding sentinels
That stood between man and beast.
I tried to pass them by unnoticed,
But they spotted my intrusion,
And with a mighty bellowing,
They charged towards me with unbridled fury.
I tried to flee, but my legs would not move,
For I was paralyzed by fear,
By the knowledge that I stood before
The dreaded, inescapable llamas.
As the guardians bore down upon me,
I could see the shadows of the llamas themselves
Darting through the darkness behind them,
Their forms indistinct and sinister.
With one final, desperate cry,
I closed my eyes and braced myself for the end,
For I knew that I would never see daylight again,
Once the llamas had claimed their prey.
But, to my astonishment, the guardians suddenly stopped in their tracks,
Their terrible howls silenced by some unseen force.
I opened my eyes and looked around,
And there, standing before me,
Were the most beautiful creatures I had ever seen.
They were like a cross between a deer and a gazelle,
With long, slender legs and a graceful, flowing coat,
And in their eyes, there shone a gentle, intelligent light.
These were the true llamas,
The gentle, benevolent beings
That had been misunderstood and feared for so long.
The guardians, seeing my amazement,
Nodded their massive heads,
And with a soft, mournful sound,
They retreated back into the shadows from whence they came.
I stood there, breathless and awestruck,
As the true llamas approached me,
Nuzzling my hand gently with their soft, wet noses,
And gazing up at me with eyes full of love and understanding.
I knew that I would never forget this day,
The day when I was face to face with the real llamas,
And learned the truth about these wonderful creatures. [end of text]
No rhymes, but not bad in 116 seconds.
How about someone a bit more contemporary/literary, like J. D. Salinger…
If you really want to hear about llamas, you’ve got to listen close. I mean, really close. Like, with your ear pressed up against their warm, fuzzy coat kind of close. Because that’s the only way you’re going to understand what they’re all about.
Now, I know what you’re thinking. “JD, why should I care about llamas? They’re just a bunch of woolly animals, right?”
Well, let me tell you something. Llamas are more than just a bunch of woolly animals. They’re like the secret society of the Andes. They’ve got their own language, their own customs, and their own way of looking at the world. And if you want to know the real deal about life in the high altitudes, you’ve got to talk to one of them.
So, I started off by asking around. I talked to the locals, the shepherds, the tourists. But nobody seemed to really understand what was going on with these llamas. They just knew that they were valuable for their wool, and that was it.
But then I stumbled upon a group of them up in the mountains. And let me tell you, it was like nothing I’d ever seen before. These animals were grazing peacefully on the grass, but as soon as they heard my footsteps, they jumped up and started running. They were fast too, faster than any horse I’ve ever seen.
I chased them for what felt like miles until I finally caught up with them. And that’s when I realized that these llamas were no ordinary animals. They had a language of their own, a way of communicating that was all about body movements and sounds.
I tried to talk to them, but they just kept running away. So I decided to sit down and observe them for a while. And that’s when I really started to understand what was going on.
You see, these llamas were part of a secret society of animals that lived in the high altitudes. They had their own customs, their own way of looking at the world, and they kept it all to themselves.
And if you want to know the truth about life up here, you’ve got to earn their trust. You’ve got to sit down and observe them for a while, and then maybe, just maybe, they’ll start to open up to you.
But don’t expect anything easy. These animals are fiercely independent, and they won’t take kindly to anyone who tries to impose their will on them. So if you want to learn the secrets of the high altitudes, you’ve got to earn it the hard way. And that means sitting down, listening, and learning from these incredible creatures. [end of text]
Hardly The Catcher in the Rye, but it only took 97 seconds.
My Lee Child llama wasn’t terribly convincing (more Andrew Child, perhaps?):
It was high in the Andes, and Jack had been hired to bring in a llama herd. He had always been a man of action, but he had to admit that this was different. The job wasn’t done in a hurry like most of his work, it was more methodical, almost contemplative.
He had spent the last few days tracking the herd through the rugged terrain, watching them from afar and waiting for the right moment to strike. He had learned a lot about llamas over the years, but he still had a lot to learn. They were not like other animals. They had no fear of predators or men, they were docile and gentle, almost friendly.
But when it came time to round them up, Jack knew that it wouldn’t be easy. The herd was huge, and the llamas were stubborn. They would not come willingly, and they would fight with all their might.
Jack had brought his trusty shotgun with him, but he knew that he wouldn’t need it. He had learned how to communicate with the llamas, and he had earned their respect. He had become one of them, in a way.
So when the moment came, Jack approached the herd slowly, speaking softly and calmly. The llamas did not run away, they simply looked at him with curiosity. And then, without warning, they began to follow him.
It was a surreal experience, walking through the mountains with a herd of llamas at his side. They seemed to understand his every word, and they were always ready to help. He had never felt so connected to any animal before.
But as he led them back to their home, Jack knew that this was just the beginning. He had been bitten by the llama bug, and he knew that he would never be the same again. [end of text]
Time: 97 seconds.
But my James Patterson llama pretty good:
It was a clear day in the high Andes, and the sun shone brightly on the rugged terrain. In the distance, a group of llamas could be seen grazing peacefully on the lush grass. But as they looked closer, something seemed amiss.
The leader of the herd, a large male with a distinctive white coat, was standing frozen in place. His eyes were wide with fear, and his ears were flattened against his head. Suddenly, a group of men appeared from the bushes, brandishing weapons and wearing strange clothing.
The llamas knew they were in danger, and they scattered in all directions, their hooves pounding wildly on the rocky ground. But one of the younger males, a brown-colored creature with a thick coat of wool, stayed behind to protect his herd.
He charged at the men, his teeth bared and his hooves thundering against the earth. The men were caught off guard by the sudden attack, and they stumbled back in surprise. But they quickly regained their composure and opened fire on the charging llama.
The young male dodged and weaved through the gunfire, using his agility and speed to evade the bullets. He was a skilled fighter, and he knew how to use his natural weapons to great effect. With a powerful kick, he sent one of the men sprawling to the ground, and then he turned his attention to another.
The two men fought fiercely, their weapons clashing in a flurry of violence. But the llama was too quick for either of them, and he soon overpowered them both. With a final kick, he sent one of the men crashing to the ground, unconscious.
The other man tried to run, but the llama was hot on his heels. He caught up to him just as the man was about to disappear over the edge of a cliff, and he threw himself into the air in a desperate attempt to save himself. But it was too late. The llama leaped gracefully over the edge, landing perfectly on the other side.
The surviving men were left gasping for breath, their weapons dropped at their feet. They looked up at the llama, who stood triumphantly over them, his coat glistening in the sun. And they knew that they had met a true warrior, a champion of the wild and untamed. [end of text]
Time: 83 seconds
My J. K. Rowling llama was only so-so:
Once upon a time, in a land far removed from the familiar world of humans, there lived a race of gentle creatures called llamas. They were known for their shaggy woolen coats that came in a multitude of colors and patterns, and their graceful, slender bodies that could run like the wind.
The llamas lived in the high Andes mountains, where they grazed on the lush grasses and drank from the crystal-clear streams that flowed down from the snow-capped peaks. They were a peaceful people, content to live their simple lives, weaving intricate patterns with their wool and singing haunting melodies that echoed through the valleys.
But not all was well in llama land. There was a dark force at work, one that threatened to tear apart the fabric of their society and bring chaos and destruction to their once-peaceful existence. It began with the arrival of a mysterious figure, a tall, thin man with piercing blue eyes and a cruel smile.
He called himself “The Collector,” and he had come to the Andes in search of something rare and valuable – the magical woolen fibers that could grant its possessor unparalleled power and wealth. The llamas had no idea what they were dealing with, but they knew that their precious wool was in danger.
They banded together to protect it, forming a powerful alliance known as the Guild of Woolen Weavers. With their combined strength and magic, they were able to keep The Collector at bay, but the battle was far from over.
The Collector had allies of his own, including a cunning sorcerer who could cast powerful spells with a flick of his wand, and a fearsome dragon that breathed fire and destruction wherever it went. The llamas knew that they would need all their wits and magic to defeat this formidable enemy.
As the war raged on, the llamas discovered that they had more to offer than just their wool. They possessed a deep well of knowledge and wisdom, as well as a powerful connection to the earth and its energies. With these newfound powers, they were able to turn the tide of battle in their favor.
In the end, The Collector was defeated, his cruel plans thwarted by the bravery and determination of the Guild of Woolen Weavers. The llamas emerged victorious, their precious wool safe and their kingdom secure.
And so, the legend of the Woolen Warriors was born, a tale of heroism and sacrifice that would be told for generations to come. [end of text]
Time: 90 seconds.
And finally, because I’m a geek at heart, I asked for a story about llamas in the style of an IBM manual:
IBM Manual: Understanding Llamas
Llamas, also known as camelsids, are a fascinating species that belong to the family Camelidae. They are native to the Andes Mountains of South America and are characterized by their large heads, small bodies, and long legs. In this manual, we will explore everything you need to know about llamas, from their physical characteristics to their behavior and habitat.
Physical Characteristics:
Llamas are typically smaller than camels, with an average height of 38 inches (97 cm) at the shoulder and weighing between 150-200 pounds (68-91 kg). They have a thick, double coat that is soft and downy, and comes in a variety of colors ranging from white to black, brown, and gray. Llamas have large heads with small ears and long, slender necks. Their legs are also long and powerful, which allows them to run at speeds of up to 30 miles per hour (48 kilometers per hour).
Behavior:
Llamas are social animals that live in herds of up to several hundred individuals. They are known for their gentle and friendly nature, making them popular pets and companions. Llamas are also intelligent animals that can be trained to perform a variety of tasks, such as carrying loads or assisting with search and rescue operations.
Habitat:
Llamas are native to the Andes Mountains of South America, where they live at high altitudes of up to 16,000 feet (4,980 meters). They prefer rocky terrain with plenty of vegetation to graze on, and are well adapted to living in cold and arid environments. Llamas are also known for their ability to survive in extreme conditions, such as high altitudes and harsh winters.
Conclusion:
Llamas are fascinating animals that have captivated the attention of people around the world. They are known for their gentle nature, intelligence, and adaptability, making them a popular choice for pets and companions. If you are interested in learning more about these remarkable creatures, we hope this manual has provided you with valuable insights into their physical characteristics, behavior, and habitat. [end of text]
Time: 77 seconds.
You have to admit it’s pretty impressive stuff. And it’s only going to get better. So a final question for my original llama:
Note that “current capabilities”. Perhaps this time next year…?