Repository navigation
HIVE-30267: Restore HivePrivilegeObject compatibility broken by HIVE-… - #6969
Conversation
…26148 * Default a null HivePrivObjectActionType to OTHER, as before HIVE-26148, so external authorizers (e.g. Ranger) no longer hit an NPE (e.g. on ADD JAR). * Replace the constructors added by HIVE-26148 with HivePrivilegeObject.builder(); keep only the legacy constructors used by external authorizers. * Set objectName on DATABASE privilege objects for authorizers reading it.
|
@saihemanth-cloudera @wecharyu @VenuReddy2103 please take a look. Thank you! |
There was a problem hiding this comment.
🟡 Changes recommended
Removed public constructors break existing authorizer binaries, while inconsistent catalog resolution can return one catalog’s metadata under another catalog’s permissions.
4 open issues
What changed in this PR
This PR aims to restore external-authorizer compatibility after HIVE-26148 and migrate Hive’s internal privilege-object construction to a builder.
Changes:
- Defaults missing action types to
OTHERand introducesHivePrivilegeObject.builder(). - Updates authorization callers, catalog fallback and lookup behavior, and database-name handling.
- Adds regression tests and migrates existing authorization tests to the builder.
| File | Description |
|---|---|
standalone-metastore/metastore-rest-catalog/src/test/java/org/apache/iceberg/rest/TestIcebergAuthorizer.java |
Migrates read/write privilege fixtures to builders. |
standalone-metastore/metastore-rest-catalog/src/main/java/org/apache/iceberg/rest/IcebergAuthorizer.java |
Migrates REST authorization objects and adds a URI helper. |
service/src/java/org/apache/hive/service/server/KillQueryImpl.java |
Builds the service privilege object through the builder. |
service/src/java/org/apache/hive/service/cli/operation/GetColumnsOperation.java |
Migrates column-metadata authorization and adds catalog fallback. |
ql/src/test/org/apache/hadoop/hive/ql/security/authorization/plugin/TestCreateHivePrivilegeObject.java |
Adds action-default, builder, catalog-lookup, and database-field tests. |
ql/src/test/org/apache/hadoop/hive/ql/security/authorization/plugin/TestAuthorizationMetaStoreFilterHook.java |
Migrates filtering and performance-test fixtures. |
ql/src/test/org/apache/hadoop/hive/ql/security/authorization/plugin/sqlstd/TestOperation2PrivilegeDataConnector.java |
Migrates the connector privilege fixture. |
ql/src/java/org/apache/hadoop/hive/ql/security/authorization/PrivilegeSynchronizer.java |
Uses builders for policy-provider resource objects. |
ql/src/java/org/apache/hadoop/hive/ql/security/authorization/plugin/sqlstd/SQLStdHiveAccessController.java |
Builds privilege objects returned by privilege listings. |
ql/src/java/org/apache/hadoop/hive/ql/security/authorization/plugin/metastore/HiveMetaStoreAuthorizer.java |
Supplies the configured default catalog to filtering lookups. |
ql/src/java/org/apache/hadoop/hive/ql/security/authorization/plugin/metastore/HiveMetaStoreAuthorizableEvent.java |
Migrates event helpers and populates database object names. |
ql/src/java/org/apache/hadoop/hive/ql/security/authorization/plugin/metastore/filtercontext/TableFilterContext.java |
Builds table-filter privilege objects. |
ql/src/java/org/apache/hadoop/hive/ql/security/authorization/plugin/metastore/filtercontext/DataConnectorFilterContext.java |
Builds connector-filter privilege objects. |
ql/src/java/org/apache/hadoop/hive/ql/security/authorization/plugin/metastore/filtercontext/DatabaseFilterContext.java |
Populates catalog, database, and object-name fields. |
ql/src/java/org/apache/hadoop/hive/ql/security/authorization/plugin/metastore/events/LoadPartitionDoneEvent.java |
Migrates the table output privilege object. |
ql/src/java/org/apache/hadoop/hive/ql/security/authorization/plugin/metastore/events/DropFunctionEvent.java |
Migrates function objects and reuses URI construction. |
ql/src/java/org/apache/hadoop/hive/ql/security/authorization/plugin/metastore/events/CreateTableEvent.java |
Reuses the shared URI privilege helper. |
ql/src/java/org/apache/hadoop/hive/ql/security/authorization/plugin/metastore/events/CreateFunctionEvent.java |
Migrates database, function, and URI privilege construction. |
ql/src/java/org/apache/hadoop/hive/ql/security/authorization/plugin/metastore/events/AlterTableEvent.java |
Builds storage-handler URI privilege objects. |
ql/src/java/org/apache/hadoop/hive/ql/security/authorization/plugin/HivePrivilegeObjectUtils.java |
Adds null/default-catalog lookup equivalence and migrates helpers. |
ql/src/java/org/apache/hadoop/hive/ql/security/authorization/plugin/HivePrivilegeObject.java |
Introduces builders, revises constructors, and normalizes defaults. |
ql/src/java/org/apache/hadoop/hive/ql/security/authorization/plugin/AuthorizationMetaStoreFilterHook.java |
Normalizes catalogs using hook configuration and updates lookups. |
ql/src/java/org/apache/hadoop/hive/ql/security/authorization/DefaultHiveAuthorizationTranslator.java |
Builds translated privilege objects. |
ql/src/java/org/apache/hadoop/hive/ql/security/authorization/command/CommandAuthorizerV2.java |
Migrates command authorization objects and catalog fallback. |
ql/src/java/org/apache/hadoop/hive/ql/security/authorization/AuthorizationUtils.java |
Uses builders when converting metastore privilege references. |
ql/src/java/org/apache/hadoop/hive/ql/processors/CommandUtil.java |
Builds service objects for command authorization. |
ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java |
Migrates table objects used for masking and filtering. |
ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/rules/views/HiveMaterializedViewUtils.java |
Migrates materialized-view privilege checks. |
itests/hive-unit/src/test/java/org/apache/hadoop/hive/ql/security/TestHDFSPermissionPolicyProvider.java |
Migrates fixtures and adds a table-object helper. |
itests/hive-unit/src/test/java/org/apache/hadoop/hive/ql/security/authorization/plugin/TestHiveAuthorizerShowFilters.java |
Builds mock filtering results through the builder. |
Review effort: Balanced
There was a problem hiding this comment.
🔵 Needs a closer look
GET_COLUMNS can authorize the session catalog while returning metadata from a different default catalog, and DATABASE privilege objects still omit objectName on several authorizer paths.
2 open issues
2 resolved since last review
Review effort: Balanced
5acc400 to
3846236
Compare
ayushtkn
left a comment
There was a problem hiding this comment.
Thanx @dengzhhu653 for the fix, Overall looks good to me. Majorly one thing, if you can help double check. Rest LGTM
| } else if (SessionState.get() != null) { | ||
| // HS2 | ||
| return SessionState.get().getCurrentCatalog(); | ||
| } |
There was a problem hiding this comment.
A null catName now resolves to SessionState.getCurrentCatalog() in HS2. Metastore calls
that take no catalog (getDatabases(pattern), getTables(db, pattern), getTableMeta, ...)
still use the metastore default catalog, so authorization and the data read can target
different catalogs. Currently, a null catalog resolved to the metastore default.
Maybe lets drop the session branch so a null catalog keeps the metastore default. Callers that
mean the session catalog should pass it explicitly.
CommandAuthorizerV2 and DefaultHiveAuthorizationTranslator already do. If the branch stays, maybe add a test for it;
testBuilderDefaults only covers the no-session case.
If I am tracing it right: This also reaches GetTablesOperation through HivePrivilegeObjectUtils.getHivePrivDbObjects
There was a problem hiding this comment.
In SessionState, it resolves the catalog from the session conf when currentCatalog is null, some other places use the same way as well, such as,
There was a problem hiding this comment.
Thanx @dengzhhu653
Keeping the session catalog in the builder is fine — it matches HiveTableName.of() and the SQL paths.
The read still has to use that same catalog. On master a null catName became MetaStoreUtils.getDefaultCatalog(getConf()), which is what the catalog-less client calls use, so GET_COLUMNS authorized and read the same catalog. After this change a null catName is the session catalog, but getDatabases(pattern) / getTables / getSchema / getPrimaryKeys still use the metastore default.
Please resolve it once, from the session when catalogName is null:
String cat = StringUtils.isNotBlank(catalogName)
? catalogName
: SessionState.get().getCurrentCatalog();
and pass cat to the catalog-aware metastore calls and to the privilege object. GetTablesOperation has the same split (getDatabases is called twice, lines 110 and 118, plus getTableMeta at 120) and isn't in this PR — it needs the same fix here.
PS. Non Blocking, just validate it, If it is f9, feel free to merge or a followup or whichever way you feel correct :-)
| privObjs.add(new HivePrivilegeObject(HivePrivilegeObjectType.TABLE_OR_VIEW, catalogName, dbTabs.getKey(), | ||
| tabName)); | ||
| privObjs.add(HivePrivilegeObject.builder(HivePrivilegeObjectType.TABLE_OR_VIEW) | ||
| .catName(catalogName) |
There was a problem hiding this comment.
I think we should resolve the catalog once and use it for both authorization and the metadata fetch.
catalogName is used only for the privilege objects. getDatabases, getTables, getSchema,
getPrimaryKeys, and TableIterable all run against the metastore default.
Since the builder now maps a null catalogName to the session catalog, GET_COLUMNS authorizes one catalog and returns columns from another.
Maybe:
String cat = catalogName != null ? catalogName : <metastore default catalog>;
and pass cat to every metastore call and to the privilege object.
There was a problem hiding this comment.
If the catalogName is null, it will be set to SessionState.get().getCurrentCatalog() in HivePrivilegeObject, it defaults to be "hive" which is the same as the catalog these methods use.
As of now, I think the HS2 isn't able to serve multiple catalogs until we declare this feature is ready, to make things right, the SessionState.get().getCurrentCatalog() returns the default catalog.
ayushtkn
left a comment
There was a problem hiding this comment.
Thanx @dengzhhu653 Changes LGTM
|
|
Thanks for the review @ayushtkn ! |





…26148
What changes were proposed in this pull request?
Why are the changes needed?
Does this PR introduce any user-facing change?
How was this patch tested?