I. Know what the construct is (and isn't) using common sense.
This should at least give you face validity.
II. What can go wrong with a measure with good face validity?
And what can you do about it?
Problem Solution(s) 1. Poor reliability Reduce error due to inconsistencies in weakens validity. administering the scale by standardizing how you administer the scale. Reduce random error due to inconsistencies in scoring by eliminating the observer or simplifying/structuring the observer's task. 2. Observer bias may Structure/simplify the observer's task, use poison validity. multiple-choice items, eliminate the observer, or use the "blind technique"
3. Subjects can bias the results by a. following demand Use blind techniques make faking hard by not characteristics letting them know they are being observed, not letting them know what they are supposed to be faking, making it expensive to fake, using physiological measures that may be impossible to fake. b. responding in the Make faking hard by not letting them know they are social desirable way being observed, not letting them know what they are supposed to be faking, making it expensive to fake, using physiological measures that may be impossible to fake. c. succumbing to Change the wording of questions so that sometimes response sets such as agreeing with an item means that you are in the acquiescence support of a position, whereas sometimes "yea-saying" bias. disagreeing means that you support that position. use disguised questions use behavioral or physiological measures instead of verbal measures. 4. Intuition may be See that your definition is consistent with theory inaccurate leading to and research about the construct you wish to not measuring the right measure. Make the case that you are measuring all construct. the relevant dimensions of the construct and have enough items to adequately represent each of these dimensions: the content validity approach.Ex: Love=lust, commitment, & trust Argue that all the items of your subscales are measuring the same construct by showing that the answers to these items are correlated: the internal consistency argument. Make the case that your measure must be valid because it correlates with other indicators of the construct: the convergent validity approach Show that you are not measuring the wrong construct by showing that your measure does not correlate too highly with measures of other constructs: the discriminant validity approach.