IMO most 'real' dirt bikers you see on the 4x4 roads are just getting to/from the real trail (singletrack) and because they are faster and more maneuverable, they will pass you when you may not like it... because they are back there choking on your dust and that SUCKS. I've done it hundreds of times and most people in 4x4's are friendly, will try make room to pass, slow down a little and if I can take a hand off the bars I will wave... or try to nod my head, to say thanks.
I try not to be stupid, won't pass when it's dangerous and will not spray rocks on others... I think most dirt bikers think they're being respectful, but people in slow moving 4x4's may see it as aggressive, when it's not. Dirt bikes coming at you can also be scary and I understand as I've been on both sides. Again, dirt bikes are more maneuverable and stop quicker and they know it... some guys seem to think open roads are a call to haul ass and that's just stupid, but I've done it... because I can.
I will say that usually 10% of most any group are bad apples and the ones that are dirt bikers really stand out. Usually if you make it obvious to be friendly to a guy on a dirtbike, he will appreciate it and respond the same way. And if you are a dick or don't understand that you're holding him up, causing him to eat your dust, etc, etc.... he'll respond by passing you as soon as he can, fast as he can, kicking up rocks on the way by. You reap what you sow.