Q: An executive in an upcoming program voiced his concern that the LPI instrument has no N/A response choice and, instead the instruction suggest that respondents use the value "3" if they feel that the statement does not apply. One of his respondents was worried that this would artificially lower certain scores, and I am hoping to get your rationale for that design.
A: It's been said, "You cannot NOT communicate." Everything we do—and don't do—sends a message. This is especially true for leaders. Because leaders are always "saying" something by their actions or non-actions, we don't offer observers an opt-out response of "not applicable" or "no opinion" on the Leadership Practices Inventory.
First of all, we do not allow respondents to leave any of the 30 items blank for empirical reasons. We've been testing and retesting the psychometric properties of the LPI for nearly 20 years, and our data tell us that ALL of the 30 LPI items do, in fact, apply to any leader at any level in any organization. Nearly 300 other researchers have also conducted studies using the LPI, and their conclusions are the same as ours. We know that each item accounts for a percentage of the statistical variance in why a leader is successful on a number of dimensions, including productivity, teamwork, employee satisfaction, and leader credibility. We also know that the more frequently a leader engages in each behavior the more positive the outcomes. Therefore, our tests indicate that each one of 30 items that assesses a leadership practice is an appropriate measure.
Additionally, the psychometric properties of each of The Five Practices of Exemplary Leadership are based upon scales that include responses to six statements (not five or four items). We know that the more items used to construct a scale the more reliable it will be-that is, the more likely it measures what it purports to measure. In addition, all of the normative data is based upon responses to all six statements that measure each leadership practice. If an individual did not have a response to one or more statements that comprise a practice we would be less confident in both the reliability and validity of their data.
Second, let's examine the specific question that is being asked on the LPI. When responding to the items on the LPI, the observer is asked: "How frequently does this person (the leader) engage in the behavior described?" The observer is then asked to rate the frequency of each of 30 behaviors on a scale from 1 (almost never) to 10 (almost always). It is very important to keep this question in mind, because the rationale for not including a "not applicable response" is based on the nature of the scale. This is a frequency scale; it is not a rating scale about how satisfied the observer is with the leader or how well the leader displays the behavior. It is about how frequently the observer sees or experiences the behavior. We use the frequency scale because it permits a rating under most conditions.
Third, the instructions we give to LPI administrators and leaders urge them to distribute the instrument only to those "who directly observe" the person in a leadership role. We therefore assume that the observers will have enough exposure to the leader to be able to offer assessments of his/her behavior. If that is not the case, then the LPIs should be distributed to other individuals who have directly observed the leader's actions.
Given these three factors-(1) all 30 items are valid and reliable measures of leadership behavior, (2) behaviors are measured on a frequency scale, and (3) the observer has had direct experience with the leader-a "does not apply" response is not appropriate. If all the conditions are met, then a response from the observer should be possible.
With these things in mind, if an observer says, "I just don't have enough information to respond," or "I don't know if you do this or you don't," or "I don't know if this behavior applies to you," what's the underlying message? It's been our experience that the observer is really saying, "I don't see that behavior so I can't rate that person on this item." In all these instances the observer is providing real information and feedback. The observer has already offered a rating by virtue of that observation. He or she is actually saying, "I don't see or experience you engaging in this behavior." Why waste this response and data?
So, the more critical question is not "Why don't you allow a 'not applicable' or 'no opinion' response," but "Why do you recommend that the respondent use a response code of '3' rather than some other number-say '1' or '2' or even '4'"?
We talked with respondents (primarily observers) about their responses and from these interviews determined that the use of "almost never" (1) and "rarely" (2) should be reserved for situations where the observer wanted to send a strong and meaningful message to the leader about his/her leadership behavior. Those who were unsure about how frequently a person engaged in a particular behavior tended to use the response "seldom" (3) to capture that sentiment. In this latest version of the LPI we now make an explicit statement that if the respondent feels a "statement does not apply," it is probably because they don't see or experience the behavior. That means, we say, "this person does not frequently engage in that behavior, at least around you. In that case, assign a rating of 3 or lower."