Skip to content

HIVE-29865: Fix database filtering failures with HiveMetaStoreAuthorizer - #6744

Merged
dengzhhu653 merged 4 commits into
apache:masterfrom
tanishq-chugh:hms-db-filter-context
Nov 21, 2026
Merged

dengzhhu653 merged 4 commits into
apache:masterfrom
tanishq-chugh:hms-db-filter-context

Conversation

@tanishq-chugh

@tanishq-chugh tanishq-chugh commented Aug 31, 2026 •

Copy link
Copy Markdown
Contributor

What changes were proposed in this pull request?

Update the HivePrivilegeObject constructor being used in DatabaseFilterContext
Update the filterDatabase in HiveMetaStoreAuthorizer to prevent StackOverflow with RangerHiveAuthorizer

Why are the changes needed?

To Prevent Null DB Names being returned with HiveMetaStoreAuthorizer filter hook & StackOverflowError error when using RangerHiveAuthorizer & HiveMetaStoreAuthorizer filter hook.

Does this PR introduce any user-facing change?

Yes, Fixes the NPE & StackOverflowError in Iceberg REST Catalog and end-user does not encounter Null Db Names when using HiveMetaStoreAuthorizer filter hook

How was this patch tested?

Manual Testing

@tanishq-chugh tanishq-chugh changed the title [WIP] Fix metastore database filter context returning null DB names HIVE-29865: Fix metastore database filter context returning null DB names Sep 1, 2026
ret.add(getHivePrivilegeObject(db));
} else {
HivePrivilegeObject hivePrivilegeObject = new HivePrivilegeObject(type, dbName);
HivePrivilegeObject hivePrivilegeObject = new HivePrivilegeObject(type, catName, dbName, dbName);

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I was able to repro the issue in my setup as well 👍🏻
@tanishq-chugh , 1 question, why objectName i.e 4th paramerter is dbName?

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In L90, when DB != null, getHivePrivilegeObject(db) we are setting (type, catlaog, db, null). So in fix I think 4th param should be null.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@Aggarwal-Raghav
I kept the objectName same as dbName here to keep it consistent with AuthorizationMetaStoreFilterHook where the HivePrivilegeObjectUtils utility method constructs the HivePrivilege Objects in the same shape (thinking of re-using this utility method here). This would keep the objects being referred by authorization manager / external plugin consistent across the two filter hook implementations.

However, i have found a couple of more issues with HiveMetaStoreAuthorizer that i plan to fix as part of this same PR. Changing it to draft for now.

@tanishq-chugh
tanishq-chugh marked this pull request as draft September 8, 2026 06:53
@sonarqubecloud

sonarqubecloud Bot commented Sep 9, 2026

Copy link
Copy Markdown

@tanishq-chugh tanishq-chugh changed the title HIVE-29865: Fix metastore database filter context returning null DB names HIVE-29865: Fix database filtering failures with HiveMetaStoreAuthorizer Sep 11, 2026
@tanishq-chugh
tanishq-chugh marked this pull request as ready for review September 11, 2026 09:06
@ayushtkn
ayushtkn requested review from ayushtkn and a lite review from Copilot September 14, 2026 08:55

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Warning

Copilot couldn't run its full agentic review because it didn't start before the timeout. Make sure your repository has a runner available, or add a copilot-code-review.yml file specifying one with the runs-on attribute. See the docs for more details.

Pull request overview

Fixes Hive metastore database filtering edge cases with HiveMetaStoreAuthorizer, aiming to avoid null database names and prevent recursive filtering that can trigger StackOverflowError (e.g., with Ranger integration).

Changes:

  • Add a regression test to ensure getAllDatabases() returns non-null database names when the filter hook is enabled.
  • Update DatabaseFilterContext to use a HivePrivilegeObject constructor that includes catalog + DB identifiers.
  • Rework HiveMetaStoreAuthorizer#filterDatabase to filter via filterDatabaseObjects(...) instead of the name-based filterDatabases(...) path.

Reviewed changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated 3 comments.

File Description
ql/src/test/org/apache/hadoop/hive/ql/security/authorization/plugin/metastore/TestHiveMetaStoreAuthorizerFilterDatabases.java Adds coverage for non-null DB names when metastore client filtering + filter hook are enabled.
ql/src/java/org/apache/hadoop/hive/ql/security/authorization/plugin/metastore/filtercontext/DatabaseFilterContext.java Adjusts HivePrivilegeObject construction for DB inputs to include catalog context.
ql/src/java/org/apache/hadoop/hive/ql/security/authorization/plugin/metastore/HiveMetaStoreAuthorizer.java Changes single-DB filtering to use object-based filtering to avoid recursion/overflow.

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment on lines 141 to 148
public final Database filterDatabase(Database database) throws MetaException, NoSuchObjectException {
if (database != null) {
String dbName = database.getName();
List<String> databases = filterDatabases(database.getCatalogName(), Collections.singletonList(dbName));
List<Database> databases = filterDatabaseObjects(Collections.singletonList(database));
if (databases.isEmpty()) {
throw new NoSuchObjectException(String.format("Database %s does not exist", dbName));
throw new NoSuchObjectException(String.format("Database %s does not exist", database.getName()));
}
}
return database;

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Addressed in 03a3c77

ret.add(getHivePrivilegeObject(db));
} else {
HivePrivilegeObject hivePrivilegeObject = new HivePrivilegeObject(type, dbName);
HivePrivilegeObject hivePrivilegeObject = new HivePrivilegeObject(type, catName, dbName, dbName);
Comment on lines 143 to 144
List<Database> databases = filterDatabaseObjects(Collections.singletonList(database));
if (databases.isEmpty()) {

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Addressed in 03a3c77

@github-actions

Copy link
Copy Markdown

This pull request has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs.
Feel free to reach out on the [email protected] list if the patch is in need of reviews.

@github-actions github-actions Bot added the stale label Nov 14, 2026
@dengzhhu653

Copy link
Copy Markdown
Member

Hi @tanishq-chugh, may I know what's the current status of this PR?

@tanishq-chugh

Copy link
Copy Markdown
Contributor Author

Hi @dengzhhu653
Had been busy with downstream work, let me revive this PR quickly

List<String> databases = filterDatabases(database.getCatalogName(), Collections.singletonList(dbName));
if (databases.isEmpty()) {
throw new NoSuchObjectException(String.format("Database %s does not exist", dbName));
List<Database> filteredDatabases = filterDatabaseObjects(Collections.singletonList(database));

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanx @tanishq-chugh for the fix, overall looks good to me.

I think after this change we are loosing the client config, which the private filterDatabaseObjects does call but the not public one 🤷 Can u double check once.

Maybe create a method like this

private HiveAuthzContext applyClientConfig(HiveAuthzContext hiveAuthzContext) {
  Map<String, Object> clientConfig = cConfig.get();
  if (clientConfig == null) {
    return hiveAuthzContext;
  }
  HiveAuthzContext.Builder builder = new HiveAuthzContext.Builder();
  builder.setClientConfig(clientConfig);
  if (hiveAuthzContext != null) {
    builder.setCommandString(hiveAuthzContext.getCommandString());
    builder.setUserIpAddress(hiveAuthzContext.getIpAddress());
    builder.setForwardedAddresses(hiveAuthzContext.getForwardedAddresses());
  }
  return builder.build();
}

and then in both call

HiveAuthzContext hiveAuthzContext =
    applyClientConfig(hiveMetaStoreAuthzInfo.getHiveAuthzContext());

filterTableNames and filterTables also have the same problem, that you can deal as another PR as followup, upto u

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice Catch @ayushtkn, have addressed the same in commit: 03f5885

@ayushtkn ayushtkn left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@dengzhhu653 dengzhhu653 left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Addressed the merge conflict due to #6969. Pending a green run.

@sonarqubecloud

Copy link
Copy Markdown

@dengzhhu653
dengzhhu653 merged commit b0a9080 into apache:master Nov 21, 2026
7 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants