Best way to become a Broker already owning 4 refeer trucks?

Hello members, I own a small fleet of refeers and I have some trustworthy drivers that I keep happy. I would like to become a broker and cherry pick the best loads to suit my trucks and drivers.

1 out 4 trucks does FL to CA but I'm not happy with what I'm making due to sketchy brokers and excess/unnecessary wear my trucks go through.
3 trucks started running FL to GA and I'm quite happy with that. Each truck does two trips per week and makes me the same money as the one truck that goes to CA.

Some how I feel like I'm leaving money on the table dealing with sketchy brokers and people. I did some digging around and learned that an owner could have an edge if they become a broker. I could stay with the best loads/lanes and broker the rest to other O/O along with networking with direct shippers.

What do you guys think or suggest?
What am I missing or not seeing?

 

Thank you Dune-T. This tells me that if I concentrate on building relations with direct shippers I might not have to become a broker. Also, I've started reading this really good thread on just that and I came across post #10 where a member suggests to have your own website etc..