If I fail to do a job assignment, have my privileges at work been breached?

For two offices, one of which I work, I was given a job assignment to do. But I wanted to make the job divided, so there are three other people who can do the same thing with the same job title. My boss told me he just needs one person to do the job and he selects me because “I ‘m able to do high-quality work and I’m accountable.” And I also mentioned that I’m not going to get more cash to do the extra job. I thought about it and decline politely. He then took the case to HR and wrote me down for insubordination. The other office actually has two employees who could easily do this job, but he doesn’t want them to do it or they can’t do it for other reasons (they won’t tell me). Didn’t I get singled out?