DatabasesStorage

rzip Compression Appwrite Storage

Dec 27, 2023, 17:02

View on Discord

Votes

Replies

Participants

Unknown

Messages

Vashishth PatelOriginal post

Rank 1: Thread

Oct 19, 2023, 04:11

Hey team,
I am working on one issue (#4001) since last week, I need to implement rzip Compression support in Appwrite.
https://github.com/appwrite/appwrite/issues/4001

But in the initial phase only, I'm not able to find any native support for rzip Compression technique in Appwrite.

I can only be able to do is installing rzip manually in local machine and compress and decompress the files using exec command from php.

If it's not the solution, can you provide me any support documentation for rzip in PHP if there's any native support available for it.

Looking forward to learn!

Summary

The user is seeking guidance on implementing rzip compression support in Appwrite. They have not found any native support for rzip in Appwrite and are considering using the exec command in PHP to compress and decompress files. They have found a potential solution and provided a code example using the exec function. They are seeking confirmation and assistance from the team. A team member suggests referring to a merged PR (https://github.com/utopia-php/storage/pull/59) that is similar to what the user needs. The user also provides a script for installing rzip and asks for verification. There is no standard documentation or native support for r

Vashishth Patel

Rank 1: Thread

Oct 19, 2023, 08:27

can someone look into it @Haimantika @Christy Jacob ?

Vashishth Patel

Rank 1: Thread

Oct 19, 2023, 09:15

I've found this,

https://rzip.samba.org/ftp/rzip/rzip-2.1.tar.gz


here, I will first download it, and will exract it.
then installing and stuff.

after installtion I'll be trying to access rzip compression using exec function in php.
so would that be fine ?

As an example:

PHP

<?php

// Function to compress data using rzip
function rzipCompress(string $data): string
{
    // Write the data to a temporary file
    $tempFile = tempnam(sys_get_temp_dir(), 'rzip_');
    file_put_contents($tempFile, $data);

    // Execute the rzip command to compress the file
    $compressedFile = $tempFile . '.rz';
    $command = "rzip $tempFile -o $compressedFile";
    shell_exec($command);

    // Read the compressed data from the file
    $compressedData = file_get_contents($compressedFile);

    // Remove temporary files
    unlink($tempFile);
    unlink($compressedFile);

    return $compressedData;
}

// Function to decompress data using rzip
function rzipDecompress(string $data): string
{
    // Write the data to a temporary file
    $tempFile = tempnam(sys_get_temp_dir(), 'rzip_');
    file_put_contents($tempFile, $data);

    // Execute the rzip command to decompress the file
    $decompressedFile = $tempFile . '_decompressed';
    $command = "rzip -d $tempFile -o $decompressedFile";
    shell_exec($command);

    // Read the decompressed data from the file
    $decompressedData = file_get_contents($decompressedFile);

    // Remove temporary files
    unlink($tempFile);
    unlink($decompressedFile);

    return $decompressedData;
}

// Example usage
$originalData = "This is some test data that we want to compress using rzip.";
$compressedData = rzipCompress($originalData);
echo "Compressed Data: " . $compressedData . "\n";

$decompressedData = rzipDecompress($compressedData);
echo "Decompressed Data: " . $decompressedData . "\n";
?>

Vashishth Patel

Rank 1: Thread

Oct 19, 2023, 09:16

@Core Can you please look into this ?

Vashishth Patel

Rank 1: Thread

Oct 19, 2023, 09:17

My primary concern is to use cmd line using exec.

Vashishth Patel

Rank 1: Thread

Oct 19, 2023, 10:05

Dockerfile

curl https://rzip.samba.org/ftp/rzip/rzip-2.1.tar.gz --output rzip-2.1.tar.gz &&
tar -xzf rzip-2.1.tar.gz &&
cd rzip-2.1 &&
mv aclocal.m4 config.m4 &&
phpize &&
./configure &&
make &&
make install

Above is the script that I want to use in installing rzip. can you please verify ? because There's no standard documentation or native support available for rzip in php

Vashishth Patel

Rank 1: Thread

Oct 19, 2023, 12:05

@Haimantika !!

Haimantika

Oct 19, 2023, 12:26

Hi, just reading it, I do not have experience working with it, so I have asked the team to see if anyone can help.

Christy Jacob

Oct 19, 2023, 12:48

@Vashishth Patel the easiest way is to refer to PRs that have already been merged
https://github.com/utopia-php/storage/pull/59

This PR is very close to what you need to do in your case as well

Drake

Oct 19, 2023, 19:07

Please do not tag groups or people like this as it can be very disruptive

Drake

Oct 19, 2023, 19:12

hmmm i can't seem to find a rzip php extension, though 😬

Vashishth Patel

Rank 1: Thread

Oct 20, 2023, 06:20

Yes, my point is that only, there's no native support available for it.

Vashishth Patel

Rank 1: Thread

Oct 20, 2023, 06:21

So that's why I found this potential solution.

Vashishth Patel

Rank 1: Thread

Oct 20, 2023, 06:22

Can we use exec in the code, To compress and decompress the file ?

Vashishth Patel

Rank 1: Thread

Oct 20, 2023, 06:22

Just, give me your final thoughts on it, I'll make the PR asap!!

Vashishth Patel

Rank 1: Thread

Oct 24, 2023, 13:46

Any Updates ?

Vashishth Patel

Rank 1: Thread

Oct 31, 2023, 04:41

Can someone guide me in this @Christy Jacob @Steven ?
Atleast give me some resources or try to ans my questions, so I can move forward!!

Drake

Oct 31, 2023, 04:44

Honestly, you probably won't be able to complete this unless you can implement the rzip compression using php or some other extension. I'm not a fan of executing a shell command

Vashishth Patel

Rank 1: Thread

Oct 31, 2023, 08:15

But, would that be fine if I will go with executing shell command ?

Vashishth Patel

Rank 1: Thread

Oct 31, 2023, 08:16

Atleast I can give it a try !!

Drake

Oct 31, 2023, 13:03

Executing shell commands is often slower. In addition, you can open yourself to security problems if not handled properly

Eldad

Nov 9, 2023, 02:15

We did it in the past in some specific places, but the shell execution has a big overhead. I think @bradleySchofield use this (https://crates.io/crates/ext-php-rs) in the past to build a native php extension using rust native libraries.

rzip Compression Appwrite Storage

Need support?_