LM Studio can run LLMs locally (I have llama and phi installed). It also has an API over a localhost webserver. I use that API to make llama available in Obsidian using the Copilot plugin.

This is the API documentation. #openvraag other scripts / [[Persoonlijke tools 20200619203600]] I can use this in?

https://web.archive.org/web/20241112124556/https://www.heise.de/news/Lokale-KI-Nemotron-70B-auf-vier-Mac-minis-M4-Pro-mit-Thunderbolt-5-Interconnect-10014248.html

Exolabs.net experiment running large LLMs locally on 4 combined Mac Mini's. Links to preview and github shared code. For 6600-9360 you can run a cluster of 4 Minis locally. Affordable for SME outfits.