It does make a difference but not so much to the rear delts. The difference has to do with internal vs external rotation which changes the contribution from certain rotator cuff muscles.
I don't agree, rotation does affect the rotor cuff muscles but it also does affect how the deltoid is recruited. Much like how elbows back vs. elbows down for lateral raises matters.
The simple answer: thumbs-up tends to point the elbows down, and this brings the middle head of the delt into play in the rear delt fly. Thumbs-forward (or even moreso with thumbs-down) takes the middle head out so it's more of a strict rear delt movement, although this might allow the teres major and lats to help more.
Another question, I was wondering how much the triceps are involved in both the grips?
very little. Not worth considering at all
Everything's worth considering.
Answer for Harrison: if your elbow is pointing back, the triceps have to contract to keep the arm straight. If they fatigue, the arm will begin to bend, making it easier to lift more weight.
If your elbow is pointing down, the triceps aren't very much involved in keeping the arm straight. It's the outer (lateral) ligaments of the elbow which are under strain to keep the arm in alignment, much like they would in a lateral raise done with thumbs-forward.
This is similar to chest flies: if the elbow is pointing back, the biceps keep the elbow from extending too far, if the elbow is pointing down, the inner (medial) ligaments of the elbow bear the strain instead.