[ggangadharan]

What changes were proposed in this pull request?

Vectorize ntile when the number of buckets is not a constant: a column, an expression, or a
partition key. Row mode (GenericUDAFNTile) reads the number of buckets from the first row of each
partition, and the vectorized path now does the same.

• VectorPTFEvaluatorNTile: takes either a constant or a non-constant expression. For a
  non-constant one, the value is read from row 0 of the partition's first batch when the buffered
  batches are replayed for output. Partition key columns are restored at that point, so a
  partition key argument works even though it is not buffered. The bucket sizes are set up once per
  partition, so nothing is added per row. A constant is still neither evaluated per batch nor
  buffered.
• Vectorizer: non-constant arguments are now vectorized. A non-positive constant still falls
  back to row mode, as before. Lead/lag inside the argument now falls back with an explicit reason.
  ntile is in COLUMN_AGNOSTIC_FUNCTIONS, so that check was skipped for it before.
• VectorPTFDesc: corrected the comment on COLUMN_AGNOSTIC_FUNCTIONS, which said these functions
  don't read input columns. ntile now reads its argument, but only from forwarded batches.

Row mode does not validate the number of buckets. The vectorized evaluator returns the same result
wherever row mode returns one, and fails right away wherever row mode fails:

First-row value n (partition of N rows)	Row mode	Vectorized
n > 0	normal result	same result
n < -N	one row per bucket (truncating division)	same result
0	ArithmeticException: / by zero	HiveException: NTILE requires a positive number of buckets, got: 0
NULL	NullPointerException	HiveException: ... got: NULL
-N <= n < 0	loops until it runs out of memory	HiveException: ... got: n

Why are the changes needed?

HIVE-30018 vectorized ntile only for a positive int constant. Any other argument made the whole
PTF fall back to row mode, with
notVectorizedReason: PTF operator: ntile only supports a positive integer constant number of buckets.
This is part of the PTF vectorization coverage effort (HIVE-24872).

Does this PR introduce any user-facing change?

No, Released Hive versions do not vectorize ntile at all.

• Queries such as ntile(col) over (...) now run vectorized, with the same results as row mode.
  Example:

  set hive.vectorized.execution.ptf.enabled=true;
  create table t(name string, rowindex int) stored as orc;
  insert into t values ('a', 3), ('a', 4), ('a', 5), ('a', 6), ('b', 2), ('b', 9);
  explain vectorization detail
  select name, rowindex, ntile(rowindex) over (partition by name order by rowindex) as nt from t;

  Before: the reducer is not vectorized, with the notVectorizedReason above.
  After: the reducer runs VectorPTFOperator with functionInputExpressions: [col 1:int]. The
  query returns a: 1 2 3 3 (3 buckets) and b: 1 1 (2 buckets) in both modes.
• When the first row holds an invalid number of buckets (0, NULL, or a negative count no larger in
  magnitude than the partition), the query still fails, as in row mode. The error is now a clear
  HiveException, and the negative case fails immediately instead of running out of memory.

No configuration changes.

How was this patch tested?

New and updated qtests:

• clientpositive/vector_ptf_ntile.q: every non-constant case runs in row mode and vectorized mode
  with identical results. A new table holds each partition's number of buckets in its first row,
  with NULL, 0 and negative values in the later rows that must never be read.
  • Bucket counts: 1 bucket; fewer buckets than rows (with and without a remainder); as many as
    rows; more than rows (the largest int); a negative count larger in magnitude than the
    partition; single-row partitions.
  • Argument kinds: column, expression, partition key (not buffered), order key. Also checks that
    the first row is taken in ORDER BY order rather than insertion order.
  • Execution paths: multi-row batches at batch size 2 and the default; spilling every batch to
    disk; constant and non-constant ntile in one PTF alongside rank and sum.
  • Fallbacks: explain shows the row-mode fallback for a non-positive constant and for lead/lag in
    the argument.
• clientnegative/vector_ptf_ntile_invalid_buckets.q: 0, NULL and -N in the first row each fail
  with the vectorized error.
• Regression run of 21 ntile and vector PTF qtests: cbo_rp_windowing_2, lineage_ptf,
  ptf_decimal, sketches_materialized_view_ntile, sketches_rewrite_ntile,
  sketches_rewrite_ntile_partition_by, vector_ptf_buffered_complex_scratch_col,
  vector_ptf_ntile, vector_windowing, windowing, windowing_ntile, vector_ptf_1,
  vector_ptf_cume_dist, vector_ptf_percent_rank, vector_ptf_unbuffered_partition_col,
  vector_ptf_spill_partition_order_overlap, vector_ptf_part_simple,
  vector_ptf_nested_input_expr, vector_ptf_lead_lag, vector_windowing_rank,
  vector_windowing_expressions.

To reproduce:

mvn clean install -DskipTests -Pitests
cd itests/qtest
mvn test -Dtest=TestMiniLlapLocalCliDriver -Dqfile=vector_ptf_ntile.q
mvn test -Dtest=TestNegativeLlapLocalCliDriver -Dqfile=vector_ptf_ntile_invalid_buckets.q

[sonarqubecloud]

Quality Gate passed

Issues
2 New issues
0 Accepted issues

Measures
0 Security Hotspots
0.0% Coverage on New Code
1.2% Duplication on New Code

See analysis details on SonarQube Cloud

[ggangadharan]

@abstractdog could you please review the PR