It's a full floater, so you could pull the axle shafts. You'll need to make some caps for the hubs. That will remove some rolling resistance. Also make sure the tires are fully inflated. Swapping to some aluminum wheels and smaller tires might even help a small amount.
To remove more wieght, you could pull the carrier and ring gear as well. I think that's an Eaton rear end, so you could pull the whole center chunk and cap the axle housing with something lighter to keep the water and dirt out.
Those are very heavy trailers. If you insist on towing with a TJ, you may be better off selling it and buying something lighter like a small Big Bubba's trailer.