My bad, I was testing it on an Omina possessed with chimeric bands.
They were both possessed and had chimeric bands on which treated them as the 'Beast' race.
The damage didn't change after rerolling so it's likely that it's either Beast or the player not counting as the Possessed race when using spirits.
I'll be testing more and will update with any extra findings.
Update:
Removing all of my black spirits resulted in no damage loss whatsoever.
Weapon used was a quickdraw that did roughly 26 damage per hit against a possessed Omina wearing Chimeric Bands and that remained static with or without attune grudge and black spirits.
They were both possessed and had chimeric bands on which treated them as the 'Beast' race.
The damage didn't change after rerolling so it's likely that it's either Beast or the player not counting as the Possessed race when using spirits.
I'll be testing more and will update with any extra findings.
Update:
Removing all of my black spirits resulted in no damage loss whatsoever.
Weapon used was a quickdraw that did roughly 26 damage per hit against a possessed Omina wearing Chimeric Bands and that remained static with or without attune grudge and black spirits.