Skip to content

HIVE-30267: Restore HivePrivilegeObject compatibility broken by HIVE-… - #6969

Merged
dengzhhu653 merged 6 commits into
apache:masterfrom
dengzhhu653:HIVE-30267
Nov 21, 2026
Merged

dengzhhu653 merged 6 commits into
apache:masterfrom
dengzhhu653:HIVE-30267

Conversation

@dengzhhu653

Copy link
Copy Markdown
Member

…26148

  • Default a null HivePrivObjectActionType to OTHER, as before HIVE-26148, so external authorizers (e.g. Ranger) no longer hit an NPE (e.g. on ADD JAR).
  • Replace the constructors added by HIVE-26148 with HivePrivilegeObject.builder(); keep only the legacy constructors used by external authorizers.
  • Set objectName on DATABASE privilege objects for authorizers reading it.

What changes were proposed in this pull request?

Why are the changes needed?

Does this PR introduce any user-facing change?

How was this patch tested?

…26148

* Default a null HivePrivObjectActionType to OTHER, as before HIVE-26148, so
  external authorizers (e.g. Ranger) no longer hit an NPE (e.g. on ADD JAR).
* Replace the constructors added by HIVE-26148 with HivePrivilegeObject.builder();
  keep only the legacy constructors used by external authorizers.
* Set objectName on DATABASE privilege objects for authorizers reading it.
@dengzhhu653

Copy link
Copy Markdown
Member Author

@saihemanth-cloudera @wecharyu @VenuReddy2103 please take a look. Thank you!

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

Removed public constructors break existing authorizer binaries, while inconsistent catalog resolution can return one catalog’s metadata under another catalog’s permissions.

4 open issues
What changed in this PR

This PR aims to restore external-authorizer compatibility after HIVE-26148 and migrate Hive’s internal privilege-object construction to a builder.

Changes:

  • Defaults missing action types to OTHER and introduces HivePrivilegeObject.builder().
  • Updates authorization callers, catalog fallback and lookup behavior, and database-name handling.
  • Adds regression tests and migrates existing authorization tests to the builder.
File Description
standalone-metastore/​metastore-rest-catalog/​src/​test/​java/​org/​apache/​iceberg/​rest/​TestIcebergAuthorizer.java Migrates read/write privilege fixtures to builders.
standalone-metastore/​metastore-rest-catalog/​src/​main/​java/​org/​apache/​iceberg/​rest/​IcebergAuthorizer.java Migrates REST authorization objects and adds a URI helper.
service/​src/​java/​org/​apache/​hive/​service/​server/​KillQueryImpl.java Builds the service privilege object through the builder.
service/​src/​java/​org/​apache/​hive/​service/​cli/​operation/​GetColumnsOperation.java Migrates column-metadata authorization and adds catalog fallback.
ql/​src/​test/​org/​apache/​hadoop/​hive/​ql/​security/​authorization/​plugin/​TestCreateHivePrivilegeObject.java Adds action-default, builder, catalog-lookup, and database-field tests.
ql/​src/​test/​org/​apache/​hadoop/​hive/​ql/​security/​authorization/​plugin/​TestAuthorizationMetaStoreFilterHook.java Migrates filtering and performance-test fixtures.
ql/​src/​test/​org/​apache/​hadoop/​hive/​ql/​security/​authorization/​plugin/​sqlstd/​TestOperation2PrivilegeDataConnector.java Migrates the connector privilege fixture.
ql/​src/​java/​org/​apache/​hadoop/​hive/​ql/​security/​authorization/​PrivilegeSynchronizer.java Uses builders for policy-provider resource objects.
ql/​src/​java/​org/​apache/​hadoop/​hive/​ql/​security/​authorization/​plugin/​sqlstd/​SQLStdHiveAccessController.java Builds privilege objects returned by privilege listings.
ql/​src/​java/​org/​apache/​hadoop/​hive/​ql/​security/​authorization/​plugin/​metastore/​HiveMetaStoreAuthorizer.java Supplies the configured default catalog to filtering lookups.
ql/​src/​java/​org/​apache/​hadoop/​hive/​ql/​security/​authorization/​plugin/​metastore/​HiveMetaStoreAuthorizableEvent.java Migrates event helpers and populates database object names.
ql/​src/​java/​org/​apache/​hadoop/​hive/​ql/​security/​authorization/​plugin/​metastore/​filtercontext/​TableFilterContext.java Builds table-filter privilege objects.
ql/​src/​java/​org/​apache/​hadoop/​hive/​ql/​security/​authorization/​plugin/​metastore/​filtercontext/​DataConnectorFilterContext.java Builds connector-filter privilege objects.
ql/​src/​java/​org/​apache/​hadoop/​hive/​ql/​security/​authorization/​plugin/​metastore/​filtercontext/​DatabaseFilterContext.java Populates catalog, database, and object-name fields.
ql/​src/​java/​org/​apache/​hadoop/​hive/​ql/​security/​authorization/​plugin/​metastore/​events/​LoadPartitionDoneEvent.java Migrates the table output privilege object.
ql/​src/​java/​org/​apache/​hadoop/​hive/​ql/​security/​authorization/​plugin/​metastore/​events/​DropFunctionEvent.java Migrates function objects and reuses URI construction.
ql/​src/​java/​org/​apache/​hadoop/​hive/​ql/​security/​authorization/​plugin/​metastore/​events/​CreateTableEvent.java Reuses the shared URI privilege helper.
ql/​src/​java/​org/​apache/​hadoop/​hive/​ql/​security/​authorization/​plugin/​metastore/​events/​CreateFunctionEvent.java Migrates database, function, and URI privilege construction.
ql/​src/​java/​org/​apache/​hadoop/​hive/​ql/​security/​authorization/​plugin/​metastore/​events/​AlterTableEvent.java Builds storage-handler URI privilege objects.
ql/​src/​java/​org/​apache/​hadoop/​hive/​ql/​security/​authorization/​plugin/​HivePrivilegeObjectUtils.java Adds null/default-catalog lookup equivalence and migrates helpers.
ql/​src/​java/​org/​apache/​hadoop/​hive/​ql/​security/​authorization/​plugin/​HivePrivilegeObject.java Introduces builders, revises constructors, and normalizes defaults.
ql/​src/​java/​org/​apache/​hadoop/​hive/​ql/​security/​authorization/​plugin/​AuthorizationMetaStoreFilterHook.java Normalizes catalogs using hook configuration and updates lookups.
ql/​src/​java/​org/​apache/​hadoop/​hive/​ql/​security/​authorization/​DefaultHiveAuthorizationTranslator.java Builds translated privilege objects.
ql/​src/​java/​org/​apache/​hadoop/​hive/​ql/​security/​authorization/​command/​CommandAuthorizerV2.java Migrates command authorization objects and catalog fallback.
ql/​src/​java/​org/​apache/​hadoop/​hive/​ql/​security/​authorization/​AuthorizationUtils.java Uses builders when converting metastore privilege references.
ql/​src/​java/​org/​apache/​hadoop/​hive/​ql/​processors/​CommandUtil.java Builds service objects for command authorization.
ql/​src/​java/​org/​apache/​hadoop/​hive/​ql/​parse/​SemanticAnalyzer.java Migrates table objects used for masking and filtering.
ql/​src/​java/​org/​apache/​hadoop/​hive/​ql/​optimizer/​calcite/​rules/​views/​HiveMaterializedViewUtils.java Migrates materialized-view privilege checks.
itests/​hive-unit/​src/​test/​java/​org/​apache/​hadoop/​hive/​ql/​security/​TestHDFSPermissionPolicyProvider.java Migrates fixtures and adds a table-object helper.
itests/​hive-unit/​src/​test/​java/​org/​apache/​hadoop/​hive/​ql/​security/​authorization/​plugin/​TestHiveAuthorizerShowFilters.java Builds mock filtering results through the builder.

Review effort: Balanced

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Needs a closer look

GET_COLUMNS can authorize the session catalog while returning metadata from a different default catalog, and DATABASE privilege objects still omit objectName on several authorizer paths.

2 open issues
2 resolved since last review

Review effort: Balanced

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Approval recommended

No active review issues are recorded for the privilege-object compatibility restoration and builder migration.

No open issues 🎉

2 resolved since last review

Review effort: Balanced

@ayushtkn ayushtkn left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanx @dengzhhu653 for the fix, Overall looks good to me. Majorly one thing, if you can help double check. Rest LGTM

Comment on lines +122 to +125
} else if (SessionState.get() != null) {
// HS2
return SessionState.get().getCurrentCatalog();
}

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A null catName now resolves to SessionState.getCurrentCatalog() in HS2. Metastore calls
that take no catalog (getDatabases(pattern), getTables(db, pattern), getTableMeta, ...)
still use the metastore default catalog, so authorization and the data read can target
different catalogs. Currently, a null catalog resolved to the metastore default.

Maybe lets drop the session branch so a null catalog keeps the metastore default. Callers that
mean the session catalog should pass it explicitly.
CommandAuthorizerV2 and DefaultHiveAuthorizationTranslator already do. If the branch stays, maybe add a test for it;
testBuilderDefaults only covers the no-session case.

If I am tracing it right: This also reaches GetTablesOperation through HivePrivilegeObjectUtils.getHivePrivDbObjects

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In SessionState, it resolves the catalog from the session conf when currentCatalog is null, some other places use the same way as well, such as,

return fromString(dbTableName, SessionState.get().getCurrentCatalog(), SessionState.get().getCurrentDatabase());
, used in drop partition, acquire locks, replication, and some analyzers.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanx @dengzhhu653
Keeping the session catalog in the builder is fine — it matches HiveTableName.of() and the SQL paths.

The read still has to use that same catalog. On master a null catName became MetaStoreUtils.getDefaultCatalog(getConf()), which is what the catalog-less client calls use, so GET_COLUMNS authorized and read the same catalog. After this change a null catName is the session catalog, but getDatabases(pattern) / getTables / getSchema / getPrimaryKeys still use the metastore default.

Please resolve it once, from the session when catalogName is null:

String cat = StringUtils.isNotBlank(catalogName)
? catalogName
: SessionState.get().getCurrentCatalog();

and pass cat to the catalog-aware metastore calls and to the privilege object. GetTablesOperation has the same split (getDatabases is called twice, lines 110 and 118, plus getTableMeta at 120) and isn't in this PR — it needs the same fix here.

PS. Non Blocking, just validate it, If it is f9, feel free to merge or a followup or whichever way you feel correct :-)

privObjs.add(new HivePrivilegeObject(HivePrivilegeObjectType.TABLE_OR_VIEW, catalogName, dbTabs.getKey(),
tabName));
privObjs.add(HivePrivilegeObject.builder(HivePrivilegeObjectType.TABLE_OR_VIEW)
.catName(catalogName)

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we should resolve the catalog once and use it for both authorization and the metadata fetch.

catalogName is used only for the privilege objects. getDatabases, getTables, getSchema,
getPrimaryKeys, and TableIterable all run against the metastore default.
Since the builder now maps a null catalogName to the session catalog, GET_COLUMNS authorizes one catalog and returns columns from another.

Maybe:

  String cat = catalogName != null ? catalogName : <metastore default catalog>;

and pass cat to every metastore call and to the privilege object.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If the catalogName is null, it will be set to SessionState.get().getCurrentCatalog() in HivePrivilegeObject, it defaults to be "hive" which is the same as the catalog these methods use.

As of now, I think the HS2 isn't able to serve multiple catalogs until we declare this feature is ready, to make things right, the SessionState.get().getCurrentCatalog() returns the default catalog.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sounds cool, Thanx Zhihua

@ayushtkn ayushtkn left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanx @dengzhhu653 Changes LGTM

@sonarqubecloud

Copy link
Copy Markdown

@dengzhhu653
dengzhhu653 merged commit 763ebbd into apache:master Nov 21, 2026
5 checks passed
@dengzhhu653
dengzhhu653 deleted the HIVE-30267 branch November 21, 2026 01:57
@dengzhhu653

Copy link
Copy Markdown
Member Author

Thanks for the review @ayushtkn !

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants