On March 11, 2025, the second interactive Meetup of the year took place at the Living Lab in Leipzig. It focused on exploring the potential of Local Large Language Models (LLMs) and their practical applications.

Exploring Local LLMs

The session began with an introduction to Local LLMs, covering these key questions:

• What open-source models are available, and how can they fit different use cases?
• What hardware and software requirements are necessary for the local operation of LLMs?
• How do local LLMs enhance data privacy and customization compared to cloud-based solutions? What challenges do they pose?

Highlighting tools like Ollama, open-webui and also in hands-on manner Msty and demonstrate how these resources enable productive and private local setups.

Guest Talk: Practical Use Cases in the Medical Field

A special feature of the meetup was a guest talk by Dr. Peter Werner from Nuklearmedizin Neumann GmbH MVZ. He shared insights from his innovative project, which uses local LLMs to semi-automate the creation of medical reports.

Key points of the presentation:

• Local LLMs protect sensitive patient data by keeping it private and avoiding the risks associated with cloud-based solutions.
• The project incorporates tools like YOLOv5 to categorize documents and extract relevant sections, which are then anonymized and sent to GPT-4o for processing.
• This approach streamlines repetitive tasks like medical documentation, demonstrating the practical benefits of local LLMs in healthcare.

Interactive Workshop: Hands-On with Local LLMs

The presentation was followed by the hands-on workshop on Local LLMs. It gave participants the opportunity to set up and experiment with their own local LLMs. Here’s what participants explored:

• A step-by-step guide to installing Ollama and starting a local model
• Practical experience running a 14-billion-parameter model on a dedicated GPU with 16 GB of memory, connected via a local network to an eGPU.
• Experimentation with prompts and models, helping attendees understand the capabilities and requirements of local LLMs compared to proprietary solutions.

Networking and Opportunities: Join Us at the Next Meetup!

The evening ended with the participants sharing their experiences and ideas and discussing possible collaborations in the relaxed atmosphere of our Living Lab.

We encourage anyone interested in showcasing their own AI project or addressing a complex problem from their company to participate in future meetups. Feel free to reach out to Oliver Welz to join us and continue the conversation at the next Meetup: April 15, 2025 “Spin-off, but how?”