Skip to content

Commit 21dd477

Browse files
timsaucerclaude
andcommitted
docs(spark): list Spark-compatible aggregates in aggregations guide
Add avg, try_sum, collect_list, and collect_set under a dedicated Spark-Compatible Functions entry, with a note that the datafusion.functions.spark namespace mirrors Spark semantics and may differ from the like-named built-ins. Adds a (spark-functions) anchor for the cross-reference. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
1 parent be530d5 commit 21dd477

2 files changed

Lines changed: 12 additions & 0 deletions

File tree

docs/source/user-guide/common-operations/aggregations.md

Lines changed: 10 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -412,6 +412,16 @@ The available aggregate functions are:
412412
\- {py:meth}`datafusion.expr.GroupingSet.rollup`
413413
\- {py:meth}`datafusion.expr.GroupingSet.cube`
414414
\- {py:meth}`datafusion.expr.GroupingSet.grouping_sets`
415+
11. Spark-Compatible Functions
416+
: - {py:func}`datafusion.functions.spark.avg`
417+
- {py:func}`datafusion.functions.spark.try_sum`
418+
- {py:func}`datafusion.functions.spark.collect_list`
419+
- {py:func}`datafusion.functions.spark.collect_set`
420+
421+
The functions in the `datafusion.functions.spark` namespace mirror Apache
422+
Spark semantics, which can differ from the DataFusion built-ins of the same
423+
name. They live in a separate namespace so you opt in explicitly. See
424+
{ref}`spark-functions` for the full catalog and the semantic differences.
415425

416426
## User-Defined Aggregate Functions
417427

docs/source/user-guide/common-operations/spark-functions.md

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -17,6 +17,8 @@
1717
under the License.
1818
-->
1919

20+
(spark-functions)=
21+
2022
# Spark-Compatible Functions
2123

2224
DataFusion ships Spark-compatible versions of a wide set of functions

0 commit comments

Comments
 (0)