As you said that you are little slow in the work, it best to convince your boss and tell him that you are as fast as others. I think he will understand your concern. In such circumstances, mostly people get annoyed only if they come to know that you are doing slackness deliberately. I feel, if you tell the truth then things may improve and he will also feel that you are not giving him bluff.
There are many factor that start such issues to come in. Such some your peer may be complaining that he/she is working more than you but you are not doing. And if you tell the truth then you boss may stand with you and protect you also.