Self HostedFlutterFunctions

this code runs too slow

Feb 12, 2024, 18:00

View on Discord

Votes

Replies

Participants

Unknown

Messages

jesusOriginal post

Rank 2: Process

Feb 12, 2024, 17:35

Hi, this a function in dart that subtract a 1 from the LMMS attribute for 6000 documents then update every document with the new subtracted value, the issue is, it takes so long (11 minutes)😭

var response = await database.listDocuments(
databaseId: databaseId,
collectionId: collectionId,
queries: [Query.limit(6000)]);
var documents = response.documents;

try {
for (var doc = 0; doc < documents.length; doc++) {
String documentId = documents[doc].$id;
final data = documents[doc].data;

  // Subtract 1 from LMMS first
  int LMMS = data['LMMS'] - 1;

  // Update the document with the initially subtracted LMMS

  await database.updateDocument(
      databaseId: databaseId,
      collectionId: collectionId,
      documentId: documentId,
      data: {'LMMS': LMMS});
}
return context.res.send(200);

} catch (e) {
print('Error updating document: $e');
return context.res.send(401);
}

Could any one help me make it faster.

Summary

Title: Dart function takes too long to update 6000 documents The developer is frustrated that their function takes 11 minutes to update 6000 documents. They are looking for help to make it faster. Solution: One suggestion is to use pagination by incorporating `Query.cursorAfter(cursor)` to process documents in smaller batches. This can be implemented using a do-while loop. Another suggestion is to use asynchronous operations and `Promise.all` in the JavaScript code, which can improve performance.

Kenny

Feb 12, 2024, 17:52

This is a function I've used in the past to update a lot of existing documents, you might see if it helps out any.

JavaScript

import { Client, Databases, Query } from 'node-appwrite';

export default async ({ req, res, log, error }) => {
  const client = new Client()
    .setEndpoint(process.env.APPWRITE_FUNCTION_ENDPOINT)
    .setProject(process.env.APPWRITE_FUNCTION_PROJECT_ID)
    .setKey(process.env.APPWRITE_FUNCTION_API_KEY);

  const database = new Databases(client);

  let response;

  const queries = [Query.limit(25), Query.select("$id")];
  let cursor = null;

  do {
    const newQueries = [...queries];

    if (cursor) {
      newQueries.push(Query.cursorAfter(cursor));
    }

    response = await database.listDocuments(
      DATABASE_ID,
      COLLECTION_ID,
      newQueries
    );

    cursor = response.documents[response.documents.length - 1].$id;

    await Promise.all(
      response.documents.map((document) => {
        database.updateDocument(
          DATABASE_ID,
          COLLECTION_ID,
          DOCUMENT_ID,
          data
        )
       })
    );
  } while (response.documents.length >= 25);

  return res.send('Complete!');
};

jesus

Rank 2: Process

Feb 12, 2024, 17:54

Kenny

Feb 12, 2024, 17:55

How often are you needing to run this function? Is it a one and done or ?

jesus

Rank 2: Process

Feb 12, 2024, 17:56

It will run a lot

jesus

Rank 2: Process

Feb 12, 2024, 17:57

That's why I'm frustrated it takes 11 minutes to complete for only 6000 document

this code runs too slow

Need support?_