...when you say that the actions of X are determined solely by historical/circumstantial factor Y, or more generally by external factor Z, you

a) absolve X of all responsibility for his/her actions, and

b) you're also saying X has no autonomous moral agency.

Both are essentially infantilizing, because this is the standard we apply to children.

I find that just as philosophically troubling as the inverse, which is the idea that X can act entirely outside of the historical framework. Neither approach is tenable, frankly.