Validity and Reliability of the Xyting Insight™ Instrument
How the Instrument Works
The Xyting Insight™ assessment instrument developed by Xyte, Inc. is available online, taken individually, and completed in approximately 10 to 15 minutes. Individuals log on to the survey through the Internet, read instructions, and click on a six-point continuum to select the behaviors that they prefer in response to simple questions. The online format requires the respondent to answer all questions before a cognofile is determined. The computer software analyzes and calculates the responses and if needed, dynamically generates additional questions to verify the cognofile of the respondent.
Results of the Xyting Insight™ assessment are generated immediately after all questions have been answered. The individual is presented with a report on the computer screen describing one appropriate cognofile from sixteen possibilities. The respondent is asked to read the cognofile and determine if it matches with his or her behaviors by selecting a percentage of agreement. This provides self-assessment validity, sometimes called face validity. The cognofile report may also be printed for future reference. An individual who does not agree with his or her cognofile may take the assessment again.
Construction of the Instrument
People long have tried to understand other people and why they act as they do. Why do some people always get things done and on time? Why do others have a sixth sense about people and have amazing interpersonal skills? Who makes a good leader?
The development of personality instruments for understanding and predicting behaviors began with Jung in the 1920s. The Myers-Briggs instrument is one such example that was based on observations of behaviors that were grouped to describe a particular profile.
The Xyting Insight™ assessment is based on a model developed using principles of information theory, brain research, and systems engineering. It was scientifically derived by Xyte, Inc. and reflects a new understanding of how the mind functions rather than mere observations of behaviors grouped into profiles as other instruments have done. Xyting Insight™ reflects innate behaviors based on the way a person thinks and processes information.
Xyte’s instrument identifies sixteen primary cognofiles with four distinct constructs based on the understanding of how people absorb, process, and delineate information into their competencies of actions, words, and thinking.
The application of statistics to theory is called psychometrics. Psychometrics aids in developing a reliable and valid instrument. Reliability is the reproducibility or precision of the measuring instrument that answers the question: If the same instrument is given twice to the same person will it produce the same result? Validity is an assessment of accuracy and answers the question: Are we measuring what we think we are measuring?
The Xyting Insight™ assessment has been proven to be both reliable and valid. Xyte demonstrates the efficacy of the Xyting Insight™ assessment by constantly monitoring the results in terms of reliability and validity.
Xyte determined the reliability of the instrument by a test retest which is the strongest method for determining reliability. In the most recent test retest, ninety-two percent of the respondents received the same cognofile. The reliability also has been periodically reviewed by correlating each item with its corresponding scale score. (A correlation to estimate item discrimination, where each of the items within a scale was added to compute a total scale score.) A correlation matrix is computed so all of the inter-correlations could be observed. It is desirable to have all inter-correlations be low (items are independent) with each item correlated to the scale score. Under most circumstances the creation of a scale score in this method creates linear combinations that artificially inflate the item correlations with the scale score. However, in this case we are interested in item discrimination where we can discern which items are or are not consistent with the overall scale. In each of the subsets all measures of internal consistency were significant beyond the .000 level. Thus, in all instances virtually all items are significantly related to the scale score.
There are different types of validity: Observations or face validity is important to the validation of the instrument, as it represents the initial and personal reaction. As individuals completed the assessment, they had the opportunity to express personal reaction. These reactions document that the assessment provides accurate cognofile interpretation. To record feedback, individuals were asked to indicate the percentage of their agreement with the findings (cognofile) of the instrument.
The review of each question as a histogram is important in determining construct validity in order to detect if there are items that do not reflect a bi-modal distribution. All Xyting Insight™ questions fit a bi-modal distribution. Construct validity was established through a factor analysis, which demonstrated that the items clustered into the four dichotomous categories. In previous analysis, 58.5 percent of the variance was explained.
Content validity is the extent to which the questions logically relate to the concept. Each item was carefully reviewed by the authors of the Xyting Insight™ questionnaire to ensure that the questions accurately reflect the measured concepts.
Criterion validity is the comparison of the results with a known standard. A personal interview was considered the “gold standard” and was compared to the computer-generated cognofile to confirm the criterion validity of the Xyting Insight™ instrument during the initial development of the instrument.
Analysis by: Professor Emeritus Donald N. McIsaac, Ph.D. University of Wisconsin-Madison