Back
AI Tools
Since for now we cannot use LLMs with cloud functions: what is the current solution to use e.g. privateGPT Can I write some python backend and can I also use the appwrite auth system for this?
TL;DR
The user encountered issues with using Appwrite Functions with LLMs due to OS dependencies, large LLM sizes, and lack of GPU support. Instead, they were suggested to set up their own REST API server and call it from an Appwrite function, allowing them to utilize Appwrite's auth system. An example of this setup can be found in the techscrunch GitHub repository. The user now asks for the current solution to use privateGPT and whether they can write a Python backend and integrate it with Appwrite's auth system. Unfortunately, the thread does not provide a solution to these specific questions.
there are some big hurdles with using Appwrite Functions with LLMs:
- OS level dependencies like gcc/c++ are required to build some packages, but those aren't available in the current version.
- the size of LLMs are too big to store anywhere in Appwrite.
- no GPU support so processing would be slow
As you suggested, you can set up your own REST API server with the LLM and then call it from an Appwrite function. This allows you to use Appwrite's auth
This is pretty much the same as using an Appwrite Function to connect to openai. This is what we did for techscrunch. https://github.com/appwrite/techscrunch/tree/main/appwrite/functions/generateArticle
Ok thanks Can’t open the link: 404 not found
Never mind. We had to take it down 😅
