Mozilla Unveils Llamafile: A Game-Changer for Large Language Models in AI

The world is burgeoning with large language models. Everyone is buzzing about artificial intelligence and how it can change the landscape of many fields of endeavor including education, medicine and software development There is lots of pushback from folks who are skeptical of the accuracy and the source of the data that has been fed into the large language models that are in the backplane of these systems.

What is a large language model (LLM)? Briefly, they are a type of artificial intelligence (AI) that excels at processing and understanding large amounts of text data. They are essentially complex algorithms trained on massive datasets of text and code, enabling them to perform various tasks related to natural language processing. Can we trust the information they are sending back to us from questions that we ask them? These questions are called text prompts. I started experimenting with ChatGPT in December of 2022. Since then, I have used other proprietary systems like Google’s Gemini and Microsoft’s Copilot. These systems remind me of the once popular movie, “2001: A Space Odyssey,” where a computer named HAL tells his human operator, “Dave, that is something I cannot allow to happen.” Now, we’re at that moment where art and life intersect. We have machines which we talk to with text prompts, and they generate code, complete essays, create content for books, and even supply images that trigger our imagination. This all seems Orwellian, and it has prompted a lot of skepticism and alarm among academics, developers, and the general public, who are concerned that our privacy is being compromised.

Recently I learned of a large language model that I could download and install on my Linux computer to let me explore this exciting new world of LLMs without compromising my privacy. It is called Llamafile, and it’s being developed by Mozilla, which has long been a respected member of the open-source community. Llamafile is a single executable file that runs locally on most computers. Best of all, it is open source. Llamafile currently has nearly thirty developers led by Justine Tunney. It has an Apache 2.0 license. The project aims to make large language models more accessible to developers and end users.

To use Llamafile, you need to download it to your computer. Be sure to follow the Quickstart Guide highlighted in the project’s Github repository.

Open your terminal and navigate to the directory where you downloaded the example Llamafile. Make the file executable by changing the permissions. 

$  chmod +x llava-v1.5-7b-q4.llamafile

Once the file has become executable, you can launch it in the terminal. 

$ ./llava-v1.5-7b-q4.llamafile -ngl 9999

At this point, your browser should open automatically to https://127.0.0.1:8080 or https://locahost:8080, and you can interact with the LLM in its chat interface. 

Now that the Llamafile is active, I can ask it some questions to see if this large language model can supply me with the correct code to create a square using the ‘turtle’ module in Python. Here is the result.

Here’s another great example of using a text prompt to generate in this case a thank you note to my wife for preparing a wonderful meal. No home should be without AI.