Repository navigation
HIVE-29865: Fix database filtering failures with HiveMetaStoreAuthorizer - #6744
Conversation
22fba73 to
5adbe6e
Compare
| ret.add(getHivePrivilegeObject(db)); | ||
| } else { | ||
| HivePrivilegeObject hivePrivilegeObject = new HivePrivilegeObject(type, dbName); | ||
| HivePrivilegeObject hivePrivilegeObject = new HivePrivilegeObject(type, catName, dbName, dbName); |
There was a problem hiding this comment.
I was able to repro the issue in my setup as well 👍🏻
@tanishq-chugh , 1 question, why objectName i.e 4th paramerter is dbName?
There was a problem hiding this comment.
In L90, when DB != null, getHivePrivilegeObject(db) we are setting (type, catlaog, db, null). So in fix I think 4th param should be null.
There was a problem hiding this comment.
@Aggarwal-Raghav
I kept the objectName same as dbName here to keep it consistent with AuthorizationMetaStoreFilterHook where the HivePrivilegeObjectUtils utility method constructs the HivePrivilege Objects in the same shape (thinking of re-using this utility method here). This would keep the objects being referred by authorization manager / external plugin consistent across the two filter hook implementations.
However, i have found a couple of more issues with HiveMetaStoreAuthorizer that i plan to fix as part of this same PR. Changing it to draft for now.
|
There was a problem hiding this comment.
Warning
Copilot couldn't run its full agentic review because it didn't start before the timeout. Make sure your repository has a runner available, or add a copilot-code-review.yml file specifying one with the runs-on attribute. See the docs for more details.
Pull request overview
Fixes Hive metastore database filtering edge cases with HiveMetaStoreAuthorizer, aiming to avoid null database names and prevent recursive filtering that can trigger StackOverflowError (e.g., with Ranger integration).
Changes:
- Add a regression test to ensure
getAllDatabases()returns non-null database names when the filter hook is enabled. - Update
DatabaseFilterContextto use aHivePrivilegeObjectconstructor that includes catalog + DB identifiers. - Rework
HiveMetaStoreAuthorizer#filterDatabaseto filter viafilterDatabaseObjects(...)instead of the name-basedfilterDatabases(...)path.
Reviewed changes
Copilot reviewed 3 out of 3 changed files in this pull request and generated 3 comments.
| File | Description |
|---|---|
| ql/src/test/org/apache/hadoop/hive/ql/security/authorization/plugin/metastore/TestHiveMetaStoreAuthorizerFilterDatabases.java | Adds coverage for non-null DB names when metastore client filtering + filter hook are enabled. |
| ql/src/java/org/apache/hadoop/hive/ql/security/authorization/plugin/metastore/filtercontext/DatabaseFilterContext.java | Adjusts HivePrivilegeObject construction for DB inputs to include catalog context. |
| ql/src/java/org/apache/hadoop/hive/ql/security/authorization/plugin/metastore/HiveMetaStoreAuthorizer.java | Changes single-DB filtering to use object-based filtering to avoid recursion/overflow. |
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
| public final Database filterDatabase(Database database) throws MetaException, NoSuchObjectException { | ||
| if (database != null) { | ||
| String dbName = database.getName(); | ||
| List<String> databases = filterDatabases(database.getCatalogName(), Collections.singletonList(dbName)); | ||
| List<Database> databases = filterDatabaseObjects(Collections.singletonList(database)); | ||
| if (databases.isEmpty()) { | ||
| throw new NoSuchObjectException(String.format("Database %s does not exist", dbName)); | ||
| throw new NoSuchObjectException(String.format("Database %s does not exist", database.getName())); | ||
| } | ||
| } | ||
| return database; |
| ret.add(getHivePrivilegeObject(db)); | ||
| } else { | ||
| HivePrivilegeObject hivePrivilegeObject = new HivePrivilegeObject(type, dbName); | ||
| HivePrivilegeObject hivePrivilegeObject = new HivePrivilegeObject(type, catName, dbName, dbName); |
| List<Database> databases = filterDatabaseObjects(Collections.singletonList(database)); | ||
| if (databases.isEmpty()) { |
|
This pull request has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. |
|
Hi @tanishq-chugh, may I know what's the current status of this PR? |
|
Hi @dengzhhu653 |
| List<String> databases = filterDatabases(database.getCatalogName(), Collections.singletonList(dbName)); | ||
| if (databases.isEmpty()) { | ||
| throw new NoSuchObjectException(String.format("Database %s does not exist", dbName)); | ||
| List<Database> filteredDatabases = filterDatabaseObjects(Collections.singletonList(database)); |
There was a problem hiding this comment.
Thanx @tanishq-chugh for the fix, overall looks good to me.
I think after this change we are loosing the client config, which the private filterDatabaseObjects does call but the not public one 🤷 Can u double check once.
Maybe create a method like this
private HiveAuthzContext applyClientConfig(HiveAuthzContext hiveAuthzContext) {
Map<String, Object> clientConfig = cConfig.get();
if (clientConfig == null) {
return hiveAuthzContext;
}
HiveAuthzContext.Builder builder = new HiveAuthzContext.Builder();
builder.setClientConfig(clientConfig);
if (hiveAuthzContext != null) {
builder.setCommandString(hiveAuthzContext.getCommandString());
builder.setUserIpAddress(hiveAuthzContext.getIpAddress());
builder.setForwardedAddresses(hiveAuthzContext.getForwardedAddresses());
}
return builder.build();
}
and then in both call
HiveAuthzContext hiveAuthzContext =
applyClientConfig(hiveMetaStoreAuthzInfo.getHiveAuthzContext());
filterTableNames and filterTables also have the same problem, that you can deal as another PR as followup, upto u
03f5885 to
31d8eb5
Compare
dengzhhu653
left a comment
There was a problem hiding this comment.
Addressed the merge conflict due to #6969. Pending a green run.
|



What changes were proposed in this pull request?
Update the HivePrivilegeObject constructor being used in DatabaseFilterContext
Update the filterDatabase in HiveMetaStoreAuthorizer to prevent StackOverflow with RangerHiveAuthorizer
Why are the changes needed?
To Prevent Null DB Names being returned with HiveMetaStoreAuthorizer filter hook & StackOverflowError error when using RangerHiveAuthorizer & HiveMetaStoreAuthorizer filter hook.
Does this PR introduce any user-facing change?
Yes, Fixes the NPE & StackOverflowError in Iceberg REST Catalog and end-user does not encounter Null Db Names when using HiveMetaStoreAuthorizer filter hook
How was this patch tested?
Manual Testing