There are two sides of the Hole in the Rock trail. The west side starting near Escalante and the east side starting near Halls Crossing. They used to be one trail, but that was when it was used by wagons and long before Lake Powell.
You are probably thinking about the east side. It is a fun trail, but pretty rocky. The trail starts 11 miles east of Halls Crossing right at a small airport called Cal Black Memorial Airport. I wouldn't call the trail a 4+, maybe a 3.5 or 4. The trail can be completed in 1 day if you haul a$$, but 2 days is better. The trail on the west side is easier. Most of the west side is usually graded.
The last time I did it, it was just 2 vehicles so we made good time. We left Utah County at 6 am and drove to the airport. Started the trail about 12:30 and made it to the end about 5pm. We turned around and started back around 6pm. We stopped to camp on the trail about 7:30pm. The next morning we finished the trail, stopped at Halls Crossing for gas and then drove to Arch Canyon. We drove to the end of Arch canyon and camped and the next morning drove out of Arch canyon and drove the Hotel Rock trail in and back out. We finished with enough time to make it back home if we had wanted, but we spent the night in Moab, just for fun. Made for a great trip, my wife even really liked it and previously she had vowed to never tent camp again.
We did this trip in the spring. I wouldn't recommend it during the summer, it was hot enough during the spring.