How should I store very long text?

I saw this https://github.com/appwrite/appwrite/issues/3386

TL;DR

Question: How should I store very long text? Summary: The user is asking for advice on storing long text. They mention encountering an error related to the attribute limit, and discuss potential solutions such as using a new database or increasing the attribute size. There is a suggestion to link endpoints to a Discord bot for easier management. The user also provides a link to a GitHub issue discussing the maximum size for storing text in Appwrite. Solution: An answer provided suggests using LONGTEXT in MariaDB, with a recommended size of 16777216. It is mentioned that this will create a column with a maximum length of 4GB. The

but wasn't sure if it was something that was interesting or

as mentioned in the issue there are technical limitations to having a string attribute of unlimited length...so use a big number

how big does it go?

I was trying to find the max size for it, google didn't help though I didn' search the appwrite docs that well

https://github.com/appwrite/appwrite/issues/3386#issuecomment-1157707278

hey do you guys want an AI bot that can answer questions for you on Discord?

I feel like if you indexed your documentation and put it inside something like Danswer you could have it try to answer questions ahead of time and you just approve the answer if it's easy

I already have it indexed, I can host it if you'd like

right but do I need to go into the DB directly and add this value?

As you can see, even TEXT in MariaDB has some limitations regarding length. If you want to be extra safe, I would recommend LONGTEXT by setting size to 16777216. That will under the hood create A LONGTEXTTEXT column with a maximum length of 4,294,967,295 or 4GB ( 2^32 - 1 ) characters..

thanks for the suggestion

https://github.com/danswer-ai/danswer

if you link the endpoints to a simple discord bot instead it would probably be the easiest thing to do

I can do it if you'd like next week, let me know if you are interested

no...set a big size

yeah I tried that I'm confused though because before if I had too many attributes with like 6000-12000 it would have given me an error about size

that's why I was asking like that

maybe it was in the old version I was using

womp. no discord data source and the github data source only pulls in PRs

so try a big number now and let me know how that goes

what's attribute_limit_exceeded

I assume that means I need a new database?