alexa JobMatchTalent Construct Validity: Correlations with IPIP Scales

ISSN: 2471-2701

Clinical and Experimental Psychology

Reach Us +44-1753390542

JobMatchTalent Construct Validity: Correlations with IPIP Scales

Bengt Jansson1, Klaus Olsen2 and Trevor Archer1*
1Department of Psychology, University of Gothenburg, S-40530 Gothenburg, Sweden
2JobMatchTalent, JobMatch Sweden AB, Skårs Led 3, 400 20 Göteborg, Sweden
*Corresponding Author: Trevor Archer, Department of Psychology, University of Gothenburg, Gothenburg, Sweden, Tel: +46 31 7864694, Email: [email protected]

Received Date: Jul 11, 2018 / Accepted Date: Aug 30, 2018 / Published Date: Sep 06, 2018

Abstract

The present study that assessed the construct validity of the JobMatchTalent (JMT) instrument indicated that most of the IPIP scales exhibited high correlations often with the corresponding JMT scales. The five scales that did not reach the 'acceptable' level were very close to the requested limit, but with the exception of the Concentration ability (c3) attribute. This scale in JMT correlated to a low degree with both IPIP and NEO-PI-R scales. This shortcoming emphasizes the requirement for further investigation.

Keywords: JMT; IPIP; NEO-PI-R; Correlation; Recruitment

Introduction

Analysis of construct validity is usually based on three types of analyzes: (i) analytical approaches related to latent variables forming the structure of scales, (ii) correlations with external tests, possibly based on MTMM methodology, (iii) differences in test ratings over specific groups [1].

In the present study, construct validity in the inventory JobMatchTalent (JMT) was studied by use of correlations with 'external' tests/scales chosen to resemble the scales in the JMT. Data from a previous study, where NEO-PI-R was used for analysis of concurrent validity in the JMT [2], was also utilized for comparisons in this study. Construct validity in the JMT has also been analyzed with exploratory factor analysis [3]. Moreover, predictive validity of the JMT has been reported [4].

The purpose of this study was to evaluate correlations for construct validity between the JMT scales and external tests/scales in accordance with the EFPA Guidelines [5].

Method and Materials

Design of external tests/scales in order to validate the JMT, and data collection has been performed by a member of JMT's staff. The IPIP resources (see IPIP) has been systematically inventoried to find suitable scales for comparisons.

Data was collected for a couple of weeks during the spring of 2018, using the LinkedIn and Facebook communities for distribution of an internet-based questionnaire. In total, 450 responses were received.

Selection of comparative scales

JMT consists of ten main scales, where each main scale has three subscales (Table 1). JMT's scales have been matched to external scales by evaluating available IPIP scales. For 26 of JMT's scales, suitable IPIP scales were found, but for four scales this was not accomplished. These scales were instead compared to similar scales in NEO-PI-R with data from the concurrent validity study [2].

(A) Work structure  
  (a1) Focus on planing
  (a2) Focus on details
  (a3) Focus on order
(B) Personal drive  
  (b1) Self-motivation
  (b2) Optimism
  (b3) Mood stability
(C) Stress-index  
  (c1) Self-control
  (c2) Resilience
  (c3) Concentration ability
(D) Decision characteristics  
  (d1) Thoughtfulness
  (d2) Willpower
  (d3) Persistence
(E) Activity  
  (e1) Physical energy
  (e2) Mental energy
  (e3) Need for speed
(F) Drive  
  (f1) Winning instinct
  (f2) Vision
  (f3) Development motivation
(G)  Acting  
  (g1) Sphere of influence
  (g2) Power of initiative
  (g3) Risk taking
(H) Tolerance  
  (h1) Concurring image
  (h2) Tolerant attitude
  (h3) Trust in others
(I) Social interest  
  (i1) Displayed consideration
  (i2) Diplomacy
  (i3) Contact creating
(J) Communication  
  (j1) Force in communication
  (j2) Communicativity
  (j3) Openness

Table 1: The JMT scales applied for data-collection with short denotations.

Furthermore, if an IPIP scale contained more than six items, the six most representative items were selected. Thus, all IPIP scales contained six items. The IPIP scales were formed on a 5-graded Likert scale. A sum score was obtained for each IPIP scale. This score was adjusted to the range of a JMT scale in order to facilitate comparisons between the different scales. It should also be noted that the IPIP scales originated in several, often person-oriented tests. A detailed overview of these tests/ scales is found in Table 2 below.

JMT IPIP scales with descriptions CA
a1 Non-Planfulness (CAT-Personality Disorder v1.1) 0.84
a2 Perfectionism (HEXACO: C/Perf) 0.8
a3 Disorder ({oppesite} Flexibility) 0.8
b1 Self-discipline (NEO C5) 0.85
b2 Optimism (Scheier0. Carver0. & Bridges0. 1994) 0.86
b3 Emotional stability (BIG 5: Factor IV) 0.86
c1 Calmness (6FPQ: Even-tempered/AG2) 0.87
c2 Anxiety ({oppesite} 16pf Factor O/Apprehension) 0.8
c3 Non-Perseverance (CAT-Personality Disorder v1.1: D3) 0.83
d1 Impulse Control (HPI: HIC) 0.7
d2 Assertiveness (AB5: I+/III+ vs I-/III) 0.75
e1 Exercise (ORAIS) 0.85)
e2 Activity level (NEO: E4) 0.71
f1 Achievement striving (TCI: P3) 0.78
f3 Love of Learning (VIA: Lov) 0.77
g1 Leadership (ORVIS) n/a
g2 Initiative (TCI: P1) 0.85
g3 Risk-taking (JPI: Risk Taking/Rkt) 0.78
h1 Cooperation (NEO: A4) 0.73
h2 Tolerance (TCI: C1) 0.72
h3 Mistrust (CAT-Personality Disorder v1.1) 0.83
i1 Nurturance (AB5C: II+/V- vs II-/V+) 0.71
i3 Friendliness (NEW NEI: E1) 0.87
j1 Assertiveness (NEO: E3) 0.84
j2 Talkativeness (AB5C: I+/IV- vs I-)/IV+) 0.84
j3 Reserve (16pf: Factor N/Privateness) 0.86

Notations: JMT: short denotations of scales in the JMT; IPIP: The international personality item pool; CA: Cronbach’s alpha. Note: IPIP scales were not available for the JMT scales d30. e30. f20. i2.

Table 2: Descriptions of the IPIP scales with corresponding JMT scales0. and reported values of Cronbach’s alpha.

Acceptable values

As to construct validity, EFPA's limit value for an acceptable correlation has been set at 0.55 for comparison to a selected test (see EFPA.). The corresponding limit for concurrent validity (a criterionbased test) is 0.20.

Statistical processing

Pearson correlations were used measure relationships between the scales. Furthermore, the reliability (Cronbach's alpha) was investigated for the 26 IPIP scales, which often were shortened to six items, and, thus, differed from the original scales/tests. On the other hand, the JMT scales retained their design, and their reliability has rather recently been reported (see JMT Technical Manual). Finally, it should be noted that ANOVA was used to compare the age of men and women. In addition, diagrams were used to illustrate deviations from limits of the values.

Results

The number of questionnaires answered was 448, of which 80% were women (two participants failed to state their gender). The average age was 41.4 years (SD=11.9). There was no difference between the age of men and women (F(1,444)=0.83, p=0.36).

Nine occupational categories emerged such as supervisors, sales staff, administration, but also services/education, teaching and studying, technology/IT. Three categories comprised a small number of people (five or less). See Table 3 below for a detailed overview.

Occupational category Frequency Percent
CEO 4 0.9
Leadership 57 12.7
Sale 25 5.6
Administration 55 12.3
Technology/IT 30 6.7
Crafts/Construction 5 1.1
Service/counseling/Health 51 11.4
Teaching 37 8.3
Student 62 13.8
Jobseekers 42 9.4
Pensioner 5 1.1
Other 75 16.7
Total 448 100

Table 3: Frequencies of occupational categories incorporated in the study (N=448).

Reliability

Reliability (Cronbach's alpha) for the 26 IPIP scales was on average 0.76 (the median was 0.77). Six scales had values just below 0.70 (in the range 0.61-0.68) (Figure 1).

clinical-experimental-psychology-deviations

Figure 1: Cronbach’s alpha values for IPIP tests/scales over corresponding JMT scales, and shown as deviations from alpha=0.70 (N=448).

Correlations between JMT and IPIP scales

The mean of the 26 correlations between the JMT and IPIP scales was 0.65 (the median value was 0.67). Of these correlations, 21 were above the limit 0.55 for acceptable values. However, four of five values were below the limit, although very close (within 1-3 hundredths for the JMT scales (b2) Optimism, (d2) Willpower, (f3) Development motivation, (h2) Tolerant attitude. A significant test showed that these four values did not differ from n 0.55 (0.184<p<0.377). On the other hand, the IPIP scale for c3 (Concentration ability) had a significantly lower value (r=0.43) relative to the limit value (0.55) (Figure 2).

clinical-experimental-psychology-correlations

Figure 2: Pairwise correlations between IPIP and JMT scales shown as deviations from r=0.55 (N=448).

For descriptive purposes, it was investigated whether the frequences of correlations across occupational categories were similar. For the larger categories (n>5) was the distribution of low to high correlations similar over categories. This similarity of patterns was illustrated by a fairly equal proportion of reasonably high (>0.50) correlations (in relation to all 26 IPIP scales) across occupational categories. For nine categories, with a sample size larger than five participants, the proportions of reasonably high correlations were between 0.73 and 0.92 (median value was 0.85).

Furthermore, all correlations between the 30 JMT and the 26 IPIP scales were calculated in order to investigate the convergence and divergence of the relationships, but without a strict statistical testing of such a model.

Instead, the discrimination ability of a particular IPIP scale was studied by taking note of frequencies of correlations below and above a certain limit (r<0.30). Such an approach showed that the IPIP scale was usually correlated low with other JMT scales compared with the target scale. This tentative finding was further supported by counting ‘acceptable’ (>=0.55) and ‘non-acceptable’ (<0.55) correlations. It was found that 32 (out of 780, i.e. 30*26) were larger, or equal to, 0.55. Frequencies for the ‘non-acceptable’ were: 610 (<0.30), 121 (0.30-0.49) and 17 (0.50-0.54). The chosen IPIP scales seemed to be discriminatively related to the corresponding JMT scales. Yet, there were twelve nonexpected correlations equal, or higher, than 0.55. Moreover, half of these consisted of three pairs of reciprocal relationships, i.e. an IPIP scale correlated relatively high with the JMT pair, and conversely for the JMT scale in relation to the corresponding IPIP pair. In addition, a couple of IPIP scales corrrelated with two or three JMT scales. However, it should be emphasized that usually these twelve correlations were lower than the corresponding expected ones (see above).

Correlations between JMT and NEO-PI-R scales

The approach carried out with data from a previous study of JMT and NEO-PI-R relationships, showed the following relationships (according to the limits for concurrent validity). For the five JMT scales associated with IPIP, which were below the 'accep ' limit, it was noted that (b2 Optimism) (d2 Willpower) and (h2 Tolerant attitude) were highly related to similar NEO-PI_R scales, while (c3 Concentration) and (f3 Development motivation) were only 'acceptable' related to similar NEO-PI-R scales. A detailed overview of the relationships is given in Table 4 below.

  Optimism(b2) Concentration ability(c3) willpower(d2) Development motivation(f3) Tolerant attitude(h2)
(O) Actions 0.367 0.040 0.093 0.204 0.266
(O) Ideas 0.114 0.145 0.148 0.224 -0.120
(C) Acheivement Striving 0.025 0.292 0.415 0.102 0.182
(E) Assertiveness 0.230 0.168 0.446 0.009 0.155
(E) Positive emotions 0.461 0.162 0.191 0.150 0.384
(A) Trust 0.207 -0.021 -0.093 0.106 0.394
(A) Compliancs 0.038 -0.115 -0.421 0.054 0.316
(N) Angry Hostility -0.057 -0.114 0.097 -0.009 -0.385

Table 4: Complementary correlations between JMT scales and NEO-PI-R (N=390)0. where the JMT scales correlated less than 0.55 with corresponding IPIP tests/scales.

For the four JMT scales, which lacked the corresponding IPIP scales, a high level of similarity was observed with similar NEO-PI-R scales. The following correlations (=0.35) were noted: Persistence (d3) vs Self-discipline ('C' in NEO-PI-R); Need for speed (e3) vs Activity ('E' in NEO-PI-R); Vision (f2) vs Achievement Striving ('C' in NEO-PI-R); Diplomacy (i2) vs Compliance ('A' in NEO-PI-R) [6,7].

Significant concordance was obtained between the JMT recruitment instrument and the NEO-PI-R clinical instrument for personality pertaining to: vision with achievement striving, ‘persistence’ with ‘self-discipline’, ‘activity’ with ‘need-for-speed’, ‘diplomacy’ with ‘compliance’, respectively, whereas ‘persistence’ was counter-predictive for ‘depression’. See Table 5, below.

  Persistence Need for Speed Vision Diplomacy
(C) Acheviment Striving 0.22 0.25 0.35 -0.21
(C) Self Discipline 0.38 0.00 0.02 -0.04
(E) Activity 0.12 0.55 0.34 -0.23
(A) Compliance -0.09 -0.31 -0.25 0.47
(N) Depression -0.35 0.13 -0.04 0.07

Table 5: Correlations between JMT scales and NEO-PI-R (N=390)0. where no corresponding IPIP tests/scales to the JMT scales were found.

Discussion

The investigation of construct validity at JMT showed that most of the IPIP scales often exhibited high correlations with the corresponding JMT scales. The five scales that did not reach the 'acceptable' level were very close to the requested limit, but with an exception for Concentration ability (c3). This scale in JMT correlated low with both IPIP and NEO-PI-R scales. This shortcoming emphasized the need for further investigation.

Regarding the reliability of the (mostly) corrected IPIP scales, it was consistently acceptable or good, but there were a handful of IPIP scales slightly below the alpha value 0.70. However, there is no simple association, that if the reliability of these scales had been somewhat higher, their correlations with the JMT scales would have been improved. This was a reason why no improvement of their reliability was performed.

The correlational analyses between the JMT and NEO-Pi instruments provide substance for comparisons between the personal attributes self-reported by the participance, such that, in the Neo-PI, “Achievement striving”, “Self-disciple”, “Activity”, “Compliance” showed positive concordances with “Vision”, “Persistence”, “Need-for-Speed”, and “Diplomacy”, in the JMT instrument, whereas “Depression” in the former displayed negative concordance with “Persistence”, in the latter (Tables 4 and 5). In a comparison of the Big Five Aspect Scales and Personality Inventory for DSM-5 (Sample 1) with the NEO-PI-R and Beck Depression Inventory-II (Sample 2), the attributes, “withdrawal”, “industriousness”, and “enthusiasm” interacted to predict/contrapredict depressive tendencies in both samples [8]. All three attributes provide a bearing upon persistence that lend a level pf credence to the present study. Furthermore, in another comparison of the Ten Item Personality Inventory, the Big Five Inventory and the Revised NEO Personality Inventory, resulting data, from 938 adult participants fitted the model adequately implying that moderate to strong significant effects were forthcoming [9]. Finally, in a study of Danish members of parliaments’ personalities using the Big Five Inventory and the Revised NEO Personality Inventory (n=81), several convergent attributes from these two ‘clinically-selective’ instruments underline the multiinstrument utility [10]. In this context, the occupationally-selective aspect of the JMT instrument ought to be recognized as a unique agency for recruitment.

Taken together, the present results gave additional support for an acceptable construct validity in the JMT instrument. Along with previous studies on JMT's factor structure, on predictive and concurrent validity, a pattern in favor of a stable construct validity emerges.

Limitations

There several functions of the attributes in the JMT instrument that may suffer from certain disadvantages arising from the notion of ‘reciprocity’ that may have required certain convergent-divergent tests but were precluded in the interests of parsimony.

References

Citation: Jansson B, Olsen K, Archer T (2018) JobMatchTalent Construct Validity: Correlations with IPIP Scales. Clin Exp Psychol 4: 197. DOI: 10.4172/2471-2701.1000197

Copyright: © 2017 Jansson B, et al. This is an open-access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.

Select your language of interest to view the total content in your interested language

Post Your Comment Citation
Share This Article
Relevant Topics
Article Usage
  • Total views: 182
  • [From(publication date): 0-0 - Dec 15, 2018]
  • Breakdown by view type
  • HTML page views: 161
  • PDF downloads: 21

Post your comment

captcha   Reload  Can't read the image? click here to refresh