Validity means "Is the test fit for purpose?"

Some different types of validity:

Face Validity (low-level of importance overall)

• Asks: "Do the questions appear to measure what the test purports to measure?"
• Important for: Respondent buy-in
• How assessed: Simply by looking at the questions

Content Validity

• Asks: "Do there appear to be enough suitable questions to measure the complete construct we are trying to measure?"
• Important for: Ensuring a holistic assessment of the construct
• How assessed: Asking subject matter experts to review the questionnaire

Construct Validity (one of the two highest levels of importance overall)

• Asks: "Does the test actually assess the construct it purports to assess?"
• Important for: Ensuring that the construct is being measured by the test and for use in norm-referenced testing
• How assessed: Correlate respondents scores on the test under examination with a well established measure of the same construct. The expectation is that there will be a high correlation given that similar constructs should converge.

Criterion-related validity (one of the two highest levels of importance overall)

• Asks: "Can a respondent's test score predict a real world outcome such as performance at work?"
• Important for: Knowing whether the test can predict anything meaningful and for use in criterion-referenced testing
• How assessed: Correlate test scores with criteria such as performance appraisal scores.

Concurrent criterion related validity - test scores predict a criterion (e.g., performance) now
Predictive criterion related validity - test scores predict a future criterion (e.g., whether or not respondent will pass training)

Vocab:
Construct = (e.g.) A personality attribute such as extraversion or an aspect of ability such as numerical reasoning
Correlation = Relationship between 2 or more variables/constructs/things