Car hauling is very cutthroat. You are crazy to take that little money using YOUR contacts. leave this SOB.
Head for the door, he's taking advantage and doesn't sound like you're happy. Take your skills and you're contacts and start you're own business, then you'll really be happy, maybe.