You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
gradient-based saliency maps to support different activations #6057
I think the method tries to look for "relu" layers and replace it with a grad-hooked customised one. This assumingly might not work with other activation functions. this is a feature request to support different activation functions, SiLU for instance.
Request you to kindly review this PR: #9283 intended to resolve this issue.
It generalizes the guided-backpropagation:
A single _AutoGradActivation autograd Function that works for any activation module (ReLU, LeakyReLU, SiLU, GELU) by delegating forward to the module and computing the local derivative via autograd — no per-activation hardcoded math.
Wires guided backprop into bothGuidedBackpropGrad and GuidedBackpropSmoothGrad via a reusable _GuidedActivationsTemp context manager.
Fixes two real bugs PR 6057guided backprop silu #7070 didn't address: in-place activation aliasing (calling module(x) directly while saving a pre-activation clone) and shared module instances (named_modules(remove_duplicate=False) so each binding is wrapped/restored).
Please review and let me know your thoughts. Thanks!
Activity
AnaghaV17 commentedon Nov 15, 2026
@wyli @KumoLiu @Nic-Ma @ericspod,
Request you to kindly review this PR: #9283 intended to resolve this issue.
It generalizes the guided-backpropagation:
A single
_AutoGradActivationautograd Function that works for any activation module (ReLU, LeakyReLU, SiLU, GELU) by delegating forward to the module and computing the local derivative via autograd — no per-activation hardcoded math.Wires guided backprop into both
GuidedBackpropGradandGuidedBackpropSmoothGradvia a reusable_GuidedActivationsTempcontext manager.Fixes two real bugs PR 6057guided backprop silu #7070 didn't address: in-place activation aliasing (calling
module(x)directly while saving a pre-activation clone) and shared module instances (named_modules(remove_duplicate=False)so each binding is wrapped/restored).Please review and let me know your thoughts. Thanks!