AIO bf trying to start a fight over me putting my sock in his hamper

i genuinely feel at a loss of what to do. that’s my sock and maybe it’s a men’s sock i don’t know, but it’s mine. feels like there’s nothing i can say to make him happy. is it wrong of me to not play into these games? he has never trusted me and i have done nothing absolutely nothing to make him feel that i have been unfaithful. we’re both 24 btw