Nylon Hammers usually are available in much smaller diameter - nylon is more dense than rubber, I believe there’s a limit on his small a rubber head can be.

Or just put “something” between your hammer and the fragile wood part; I’m thinking a flip-flop sole lol.