Skip to main content

apache/datafusion-comet

最近记录的来源活动
SkillsMP 收录数据更新
已收集 skills
12
GitHub 星标
1,290
GitHub Forks
391

正在加载仓库 README…

已展示 12 / 12 个已收集 Skill。

职业分类
未分类
描述

Use when reviewing a DataFusion Comet pull request that touches Iceberg writes, the split-operator write plan (IcebergWriteStrategy, IcebergWriteExec, IcebergCommitExec), the native Iceberg writer (CometIcebergWriteExec, iceberg_write.rs,…

原文语言:英语

更新
职业分类
未分类
描述

Use when implementing a new Spark expression in DataFusion Comet. Walks through cloning latest Spark master to study the canonical implementation, checking the upstream datafusion-spark crate before writing native code, building the Comet serde and Rust…

原文语言:英语

更新
职业分类
未分类
描述

Use when reviewing a DataFusion Comet pull request that adds or changes a Spark expression, its Scala serde, its protobuf message, or its native Rust implementation. Load alongside review-comet-pr, which covers the parts of the review that apply to every PR.

原文语言:英语

更新
职业分类
未分类
描述

Use when reviewing a DataFusion Comet pull request. Covers the workflow that applies to every PR and routes to the area-specific review skills for expressions, FFI, memory management, shuffle, and Iceberg writes. Provides guidance to a human reviewer rather…

原文语言:英语

更新
职业分类
未分类
描述

Use when wiring an existing DataFusion or datafusion-spark function into Comet for a Spark expression. Identifies the right wiring pattern (one-line passthrough, datafusion-spark UDF registration, or custom serde with input massaging / restrictions), applies…

原文语言:英语

更新
职业分类
未分类
描述

Use when reviewing a DataFusion Comet pull request that crosses the JVM/native boundary, touching Arrow C Data or C Stream interface code, batch export and import, CometExecIterator, ScanExec, NativeUtil, CometVector subclasses, or jni_api. Load alongside…

原文语言:英语

更新
职业分类
未分类
描述

Use when reviewing a DataFusion Comet pull request that touches memory pools, memory reservations, spilling, the Spark TaskMemoryManager bridge, allocators, or memory-related configuration. Load alongside review-comet-pr.

原文语言:英语

更新
职业分类
未分类
描述

Use when reviewing a DataFusion Comet pull request that touches native or JVM columnar shuffle, the shuffle writers and readers, partitioning, the Arrow IPC block format, shuffle compression, or the Celeborn integration. Load alongside review-comet-pr.

原文语言:英语

更新
职业分类
软件质量保证分析师与测试员
描述

Audit an existing Comet expression for correctness and test coverage. Studies the Spark implementation across versions 3.4.3, 3.5.8, 4.0.1, and 4.1.1, reviews the Comet and DataFusion implementations, identifies missing test coverage, and offers to implement…

原文语言:英语

更新
职业分类
项目管理专家
描述

Triage open Comet issues marked `requires-triage` per the project bug triage guide. Classifies each issue as a bug or an enhancement, applies the recommended type (`bug`/`enhancement`), priority, and area labels, removes `requires-triage`, and files a dated…

原文语言:英语

更新
职业分类
软件开发工程师
描述

Use when optimizing the performance of an existing native scalar expression in the datafusion-comet-spark-expr crate (native/spark-expr/) — casts, string/JSON/array/math kernels that run per-row or per-batch. Covers benchmarking, keeping output bit-identical,…

原文语言:英语

更新
职业分类
计算机系统分析师
描述

Use when picking the next Spark expression to implement natively in Comet, or when asked whether a specific codegen-dispatched expression is worth a native Rust implementation. Scores the candidate on how likely a native path can be 100% Spark-compatible and…

原文语言:英语

更新
已展示 12 / 12 个已收集 Skill。

用 AI 助手安装

复制这段 Prompt,发给你正在使用的 AI 助手。