• Bug#1107333: RFP: pg-parquet -- Copy to/from Parquet in S3, Azure Blob

    From Antoine Beaupre@21:1/5 to All on Thu Jun 5 20:50:01 2025
    Package: wnpp
    Severity: wishlist
    X-Debbugs-Cc: debian-rust@lists.debian.org, team+postgresql@tracker.debian.org

    * Package name : pg-parquet
    Version : 0.4.0
    Upstream Contact: https://github.com/CrunchyData
    * URL : https://github.com/CrunchyData/pg_parquet/?tab=readme-ov-file
    * License : PostgreSQL
    Programming Lang: Rust
    Description : Copy to/from Parquet in S3, Azure Blob Storage, Google Cloud Storage, http(s) stores, local files or standard inout stream from within PostgreSQL

    pg_parquet is a PostgreSQL extension that allows you to read and write
    Parquet files, which are located in S3, Azure Blob Storage, Google
    Cloud Storage, http(s) endpoints or file system, from PostgreSQL via
    COPY TO/FROM commands. It depends on Apache Arrow project to read and
    write Parquet files and pgrx project to extend PostgreSQL's COPY
    command.

    -- Copy a query result into Parquet in S3
    COPY (SELECT * FROM table) TO 's3://mybucket/data.parquet' WITH (format 'parquet');

    -- Load data from Parquet in S3
    COPY table FROM 's3://mybucket/data.parquet' WITH (format 'parquet');

    ----

    We're using this on a development database server and are hoping to
    offload data from PostgreSQL into parquet files. We've been compiling
    this from source which is causing some trouble during major upgrades.

    Typically, extensions are maintained by the PostgreSQL, but in this
    case it's a Rust extension, so perhaps the rust team could do it?

    Not sure.

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From =?utf-8?Q?Antoine_Beaupr=C3=A9?=@21:1/5 to Matthias Geiger on Sat Jun 7 15:00:01 2025
    On 2025-06-07 11:02:55, Matthias Geiger wrote:
    Hi,

    a quick cargo-debstatus show that it's missing quite a few things,
    noteably parquet and aws-credentials, so not something that could be
    packaged quickly.

    Hi!

    (adding bug back in CC)

    Thanks for the update! On our side, the situation has actually changed:
    i was missing some information, and it turns out we're not actually
    aiming at using this production after all.. We've deactivated the
    extension on the server, in fact, to complete the upgrade.

    a.

    --
    We will create a civilization of the Mind in Cyberspace. May it be more
    humane and fair than the world your governments have made before.
    - John Perry Barlow, 1996
    A Declaration of Independence of Cyberspace

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)