How to schedule call to appwrite function ? Call multiple time same function lead to error
Hello, first, i open this post due to this message:
Internal curl errors has occurred within the executor!
Error Number: 111.
Error Msg: Connection refused
Error Code: 500
I'm using node-appwrite and parse a json file.
I have 2 appwrite function:
- to fetch the data and create pool of smaller data
- to update document with the new price
To explain, i have a file with 90_000 card price, i use functions to update price each day.
I'm parsing this data set and create pool of 1_000 update (that take 4/5min).
Basically my first function triggers 90 * 1_000 update
So first, i launch a function by cron that get the updated data (fresh price) I loop on the result to create the pool and iterate over it.
I've added a wait between each call of function of 500msbut i still get this error.
What can i do ?
There are the function for the 1) step -> fetching updated price and create small pool of data by their index
export default async ({ req, res, log, error }: Context) => {
const paperPrices = await fetchPrices();
const client = new Client();
const functions = new Functions(client);
client
.setEndpoint("ENDPOINT") // Your API Endpoint
.setProject(process.env.APPWRITE_PROJECT_NAME) // Your project ID
.setKey(process.env.APPWRITE_API_KEY); // Your secret API key
function createExecution(startIndex: number, endIndex: number) {
const payload = {
startIndex: startIndex,
endIndex: endIndex,
};
return functions
.createExecution(
"XXXXX", // functionId
JSON.stringify(payload), // body
true, // async (optional)
"/", // path (optional)
"POST" // method (optional)
// {} // headers (optional)
)
.then(function (response) {
log("Success");
})
.catch(function (err) {
error("Error");
});
}
const batchSize = 1000;
const totalSize = paperPrices.length;
for (let startIndex = 0; startIndex < totalSize; startIndex += batchSize) {
const endIndex = Math.min(startIndex + batchSize, totalSize);
await createExecution(startIndex, endIndex);
// Need to add a wait time to avoid rate limit
await new Promise((resolve) => setTimeout(resolve, 500));
}
if (req.method === "GET") {
return res.send("SUCCESS");
}
There are the code for the 2) function -> Get indexes, fetch the data and get small pool of data
export default async ({ req, res, log, error }: Context) => {
try {
const { startIndex, endIndex } = JSON.parse(req.body as string);
const allPrices: TodayPriceFormat[] = await fetchPrices(); // Data from third service
const prices: TodayPriceFormat[] = allPrices.slice(startIndex, endIndex);
for (const price of prices) {
await updatePrices(price); // Launch the update document function
}
return res.send("Updated price");
} catch (err) {
error("Error");
return res.send("Failed to update price");
}
};
The update document is very simple and use the sdk inside the updatePrices() method
return databases.updateDocument(databaseId, priceCollectionId, docId, doc);
How to schedule call to appwrite function ? Call multiple time same function lead to error
