Four Layers Approach for Developing System Thinking Assessment Tool for Industrial and Systems Engineers

Purpose: This paper deals with four layers approach for developing a tool for assessing engineering systems thinking. Every engineer and, in particular, industrial and systems engineer needs a systems view that enable to perform systems engineering tasks successfully. The paper’s purpose is to present a tool for assessing systems thinking of industrial and systems engineers. The tool may be used for industrial and systems engineering workplace selection and development, developing industrial and systems engineering curriculum, education, and training programs, as well as a standard tool for assessing industrial and systems engineers’ competencies. Design/methodology/approach: We have to find a way to evaluate a systems view or in other words, a high capacity for engineering systems thinking (CEST) Since there is no known way of directly ‘measuring’ systems thinking in general and CEST, in particular, an indirect method is needed. This paper proposes an idea for developing an indirect means, i.e. a questionnaire for assessing the CEST of industrial and systems engineers. The idea is composed of four logic layers that will be presented in the paper. Findings and originality/value: Eighty-three, which later aggregated to thirty-four competencies, of successful industrial and systems engineers were identified in a previous study. They can be classified into four categories – ten cognitive competencies, eleven abilities, ten individual traits and three dealing with multidisciplinary knowledge and experience. Thus, the content validity of the proposed tool can be achieved by basing its items on the finding in the latter study. Additional studies were done on systems thinking among industrial and systems engineers and among engineers from various disciplines. One of the findings of a previous study is that in order to be a successful industrial and systems engineer, one must have both a will and interest in being an industrial and systems engineer. The three components discussed here – success in an industrial and systems engineering position, interest in industrial and systems engineering positions and CEST – are all interrelated. The will and interest to be an industrial and systems engineer means basically the desire and interest to be involved in job positions that require CEST. In other words, we may hypothesize that there is a high positive correlation between the engineering systems thinking extent (CEST) of an individual and his/her interest in what is required from successful industrial and systems engineers. Research limitations/implications: Owing to time constraints, the research only included a limited sample of industrial and systems engineers. In order to enlarge the external validity of this research, we should ask more subjects from different engineering disciplines and use additional tools such as interviews and observations. Originality/value: The framework of this research is unique in term of its new approach and evaluation processes


Introduction
The end of the 20 th century included significant enhancements to the systems age as the internet and globalization rose to dominance, and nearly limitless access to information and intellectual resources became possible. With increased access comes increased awareness of complexity. Problems are becoming more difficult and situations more chaotic. It is in this context that systems thinking must be applied. Systems thinking provides the perspective and tools to address these challenges; a way to combat the chaos [1].
Systems thinking can be defined as a view, process or approach. Before defining systems thinking lets clarify the term "system. Systems can be placed on one or more of these four categories [2]:

1.
Natural Systems-Systems that have their origin is in the universe and are as they are as a result of forces and processes which characterize the universe. world of innumerable sets of human activities that are more or less consciously ordered in wholes as a result of some underlying purpose or mission.
In the current research we will concentrate on the man-made systems and systems situations that are of importance for individuals as well as for various groups including public and private organizations and their enterprises in developing capabilities for learning to think and act in terms of systems.
We must remember the embedded nature of systems. What is considered a holistic, systems view is considered a reductionist view when the boundaries of the system are redrawn. For example, in social psychology, one may focus work at the individual level of analysis, remembering there are also contributions from the other levels of analysis, such as the group and organizational levels of analysis [3].
System elements are integrated to meeting the need to be provided by the Systems-of-Interest. Each of the system elements can deliver one or more services to the System-of-Interest and can interacting with the other elements of a system product or service instance an effect emerges [2].
According to the general systems theory, synergy exists in all systems. In the literature, it is sometimes mentioned as emergent properties.
Katz et al. [4][5][6][7], claim that a system is more than just a collection of parts. The system properties, capabilities, and behaviors certainly emerged from the system parts, but they also appear from the interactions among these parts. The properties of the system consist of more than the aggregate of the proportion of the individual parts and interactions. The whole is more than the sum of its components. Systems have emergent properties that are not found in their parts. This synergistic effect is one of the central and most important attributes of a system, but it is, at times, hard to identify [8,9]. A system, therefore, is a whole that cannot be divided into independent parts. From this, two of its most important properties derive: every part of a system has properties that it loses when separated from the system, and every system has some properties-its essential ones-that none of its parts do [10].
In the analysis of systems, there have emerged two approaches, labeled as "engineering" and "management". The engineer seeks to understand and design through quantitative, well defined, or "hard" methods while the management analyst leverages more qualitative, collaborative, or "soft" methods [1]. Both approaches, the "hard" and "soft" methods applied systems thinking as an integral part of the industrial and systems engineer's job.
Industrial and systems engineering is an interdisciplinary field of engineering that focuses on how to design and manage complex engineering projects over their life cycles. Systems thinking is what makes industrial and systems engineering different from other kinds of engineering and is the underpinning skill required to do systems engineering [10].
As industrial and management engineer moves from problem definition through the systems thinking approach, multiple viewpoints are essential to understanding the problem situation and solution. The paraphrase "forest thinking" to clarify the concept of systems thinking "forest thinking" involves a "view from 10,000 meters rather than focusing on local trees" and "considering how the system influences systems on the other side of the line and how these latter systems influence the former system." Systems thinking is a continuum of activities which range from the conceptual to the technical. The adoption of systems thinking occurs when we are standing back far enough-in both space and time-to be able to see the underlying web of ongoing, reciprocal relationships which are cycling to produce the patterns of behavior that a system is exhibiting. You are employing a systems perspective when you can see the forest (of relationships), for the trees. You are not employing a systems perspective when you get "trapped in an event".
Systems thinking seeks to address and solve complex problems by understanding the system parts and their interactions within the context of the whole system, rather than in isolation [7]. Systems thinking are a discipline for seeing the whole. It is a framework for seeing interrelationships and repeated events rather than things, patterns of change rather that static 'snapshots.' Systems thinking is the perception of the 'constructs' underlying complex problems [11].
Systems thinking is considered a high-order thinking skill. The simplest thinking skills are learning facts and recall. Higher order skills include critical thinking, creative thinking, analysis, problem-solving, and systems thinking According to Senge [11] a good systems thinker, particularly in an organizational setting, is someone who can see four levels operating simultaneously: events, patterns of behavior, systems, and mental models. It is systems thinking that bring the disciplines of personal mastery, mental models, shared vision and team learning all together.
To perform successfully systems engineering tasks, industrial and systems engineers need a systems view, in other words, a high capacity for engineering systems thinking (CEST). Engineering Systems Thinking is a major high-order thinking skill that enables individuals to perform systems engineering tasks [12,13]. The main characteristic of systems engineers possessing high CEST is the ability to see the whole picture and identify the system emergent properties, capabilities, behaviors, and functions without looking at the detail. It is assumed that CEST is a measurable and consistent quality of personality, and it can be used to distinguish among individual engineers.
The battery for assessing CEST of individuals comprises a set of paper-and-pencil, field, and lab tests. This paper discusses one of the paper-and-pencil tests -an interest inventory -considering rationale, principles, uses, and stages of development, and presents some examples and results of studies aimed at checking its reliability and validity.

Research Methodology
How to Assess Capacity for Engineering Systems Thinking (CEST) by a paper-and-pencil Questionnaire: the idea

The Assessing need
A tool for assessing systems thinking of engineers, once validated, may be used for industrial and systems engineering workforce selection and development, developing industrial and systems engineering curriculum, education, and training programs, as well as a standard tool for assessing industrial and systems engineers' competencies. Such a tool may be used for selection, filtering, and screening of candidates for industrial and systems engineering job positions, and for placing the 'right person in the right job'. The tool may also be used as an instrument for evaluating the effectiveness of industrial and systems engineering curriculum and training programs.
Since there is no known way of directly 'measuring' systems of the TTCT in 1984 eliminated the Flexibility scale from the figural test but added Resistance to Premature Closure and Abstractness of Titles as two new criterion-referenced scores on the figural. To the five normreferenced measures that he now had (fluency, originality, abstractness of titles, elaboration and resistance to premature closure), Torrance added 13 criterion-referenced measures: emotional expressiveness, story-telling articulateness, movement or actions, expressiveness of titles, syntheses of incomplete figures, synthesis of lines and circles, unusual visualization, extending or breaking boundaries, humor, richness of imagery, colorfulness of imagery, and fantasy [16].

The second layer-thinking decomposing
From IQ and creativity assessing approaches, we have learned that the first stage of developing a tool for assessing systems thinking in engineers should be decomposing the 'engineering systems thinking' to factors that can be separately assessed. But how is this done? In order to answer such a question, let us consider one of the approaches taken in Leadership research.
There are a vast number of books, articles, and studies available which attempt to identify the competencies and qualities of effective leaders. For example, Morrison [17] presents a global leadership model, while Jokinen [18] reviews and discusses the main findings of previous leadership studies aimed at identifying characteristics and competencies of effective and successful leaders.
Thus, we can see that a basic approach for understanding the 'leadership phenomenon' is to identify the competencies of successful leaders. It is suggested here that, in the same manner, in order to understand the 'engineering systems thinking phenomenon,' we should try to identify the competencies and characteristics of engineers who are systems, thinkers. But how can we know which engineers can be considered as systems thinkers as the tool for assessing CEST has not been developed yet?
The third layer-successful systems thinkers As mentioned above, to perform successfully industrial and systems engineering tasks, industrial and systems engineers need a systems view, in other words, a high capacity for engineering systems thinking (CEST). Thus, for the sake of this study, we may hypothesize that successful industrial and systems engineers are characterized by high Capacity for Engineering Systems Thinking (CEST). That is to say, successful industrial and systems engineers are systems, thinkers. This hypothesis should be reconsidered when the development of the systems thinking assessment tool has been completed. In order to understand the 'engineering systems thinking phenomenon,' let us try to identify therefore the competencies and characteristics of successful industrial and systems engineers.
In actual fact, this has already been done. Frank [8] aimed at identifying the characteristics of successful industrial and systems engineers. The study included observations of industrial and systems engineers who were evaluated -by at least three peers and two supervisors -as being 'successful'. Eighty-three, which later aggregated to 34 competencies, of successful industrial and systems engineers were identified in this study. These 34 competencies were classified into: • Ten cognitive characteristics -understanding the big picture, interconnections, systems synergy, multiple perspectives, systems without getting stuck on details, implications of proposed change, a new system or concept immediately upon presentation, analogies and parallelism between systems, limits to growth and thinking creatively. All these 10 characteristics are related to systems thinking. thinking in general and CEST, in particular, an indirect method is needed. For example, IQ tests are paper-and-pencil indirect tests for 'measuring' the intelligence of individuals. This paper proposes an idea for developing an indirect means, i.e. a questionnaire for assessing the CEST of industrial and systems engineers. The idea is composed of four logic layers.

The first layer -thinking assessment
As stated above, engineering systems thinking is a major high-order thinking skill that enables individuals to perform systems engineering tasks [12,13]. We are dealing here with a thinking skill, so it is relevant to discuss the common methods of 'measuring' other thinking skills such as intelligence and creativity.
An intelligence quotient, or IQ, is a score derived from one of several standardized tests designed to assess intelligence. One of the common ways for assessing intelligence is by separately assessing abilities such as arithmetic, spatial imagery, reading, vocabulary, memory, and general knowledge and then weighting the individual scores to one general index (the IQ). For example, The WAIS-III version of the Wechsler Adult Intelligence Scale was released in 1997. It provided scores for Verbal IQ, Performance IQ, and Full-Scale IQ, along with four secondary indices -Verbal Comprehension, Working Memory, Perceptual Organization, and Processing Speed. The Verbal Comprehension Index (VCI) included the following tests: Information, Similarities, and Vocabulary. The Working Memory Index (WMI) included: Arithmetic, Digit Span, and Letter-Number Sequencing. The Performance IQ (PIQ) included six tests and also provided two subindexes: perceptual organization and processing speed. The Perceptual Organization Index (POI) included: Block Design, Matrix Reasoning, and Picture Completion. The Processing Speed Index (PSI) included: Digit Symbol-Coding and Symbol Search [14].
The current version of the test, the WAIS-IV, which was released in 2008, is composed of 10 core subtests and five supplemental subtests, with the 10 core subtests comprising the Full-Scale IQ. With the new WAIS-IV, the verbal/performance subscales from previous versions were removed and replaced by the index scores. The General Ability Index (GAI) was included, which consists of the Similarities, Vocabulary, and Information subtests from the Verbal Comprehension Index and the Block Design, Matrix Reasoning, and Visual Puzzles subtests from the Perceptual Reasoning Index. The GAI is clinically useful because it can be used as a measure of cognitive abilities that are less vulnerable to impairments of processing and working memory [15].
There are four index scores representing major components of intelligence: Verbal Comprehension Index (VCI), Perceptual Reasoning Index (PRI), Working Memory Index (WMI), and Processing Speed Index (PSI). Two broad scores are also generated, which can be used to summarize general intellectual abilities: Full Scale IQ (FSIQ), based on the total combined performance of the VCI, PRI, and WMI, and PSI General Ability Index (GAI), based only on the six subtests that the VCI and PRI comprise. Eleven abilities -analyzing the need, analyzing/developing the concept of operations, requirements analysis, conceptualizing the solution, generating the logical solution (functional analysis), generating the physical solution (architecture synthesis), 'seeing' the future, using simulations and industrial and systems engineering tools, optimizing, using systems design considerations, leading trade studies.

•
Ten individual traits -management skills, team leader, building and controlling the work plan, defining boundaries, taking into consideration non-engineering factors, good human relations, team player, good communication skills, good interpersonal skills, autonomous and independent learner, strong learning skills, willing to deal with systems, curious, innovator, initiator, promoter, originator, asks good questions.

•
Three dealing with multidisciplinary knowledge and experience.
Thus, the content validity of the proposed tool can be achieved by basing its items on the finding in the latter study [8]. Each competency of successful industrial and systems engineers, found in that study, may be separately assessed by a single item or several items of the proposed tool and then the individual scores weighted to one general index.

The fourth layer-identifying systems thinkers competencies
One of the findings by Frank [8] is that in order to be a successful industrial and systems engineer, one must have both a will and interest in being an industrial and systems engineer. In addition, as mentioned, successful industrial and systems engineers possess a high capacity for engineering systems thinking (CEST). Thus, the three components discussed here -success in an industrial and systems engineering position, interest in industrial and systems engineering positions and CEST -are all interrelated. The will and interest to be an industrial and systems engineer basically means the desire and interest to be involved in job positions that require CEST. In other words, we may hypothesize that there is a high positive correlation between the engineering systems thinking extent (CEST) of an individual and his/her interest in what is required from successful industrial and systems engineers. Figure 1 is a simple concept map that depicts the relationships between these three components: If this hypothesis is supported, then it enables developing a method for assessing the extent of CEST of individuals. This is because interests may be assessed by an interest inventory which is very common and frequently used to help people choose a profession, and as a selection tool (to determine whether a certain individual is suitable for a certain role) in the recruitment process [19].

Assessing the interest for engineering positions' required capacity for engineering systems thinking (CEST)
Frank [20] introduces a tool for assessing interest for industrial and systems engineering positions and other engineering positions' required capacity for engineering systems thinking (CEST). Usually, the items in interest inventories deal with preferences, specifically likes and dislikes regarding a diverse group of activities, jobs, professions or personality types. Likewise, the items included in the tool discussed in this chapter refer to ranges of likes and dislikes regarding industrial and systems engineering activities, various disciplines and knowledge required from industrial and systems engineers, industrial and systems engineering activities and types of people involved in projects.
In its present version, the tool consists of 40 pairs of statements. For each pair, the examinee has to choose between the two statements according to his/her preference. The examinee checks answer "A" if he/she prefers the first statement or answer "B" if he/she prefers the second statement. In order to improve the questionnaire's reliability, questionnaire items were reorganized, so in some cases "A" represented the systems thinking answer and in other cases "B" represented the systems thinking answer. Each "A" answer receives 2.5 points while each "B" answer receives no point. Thus, the range of the scores is 0-100.

Validating the tool -Results of recent studies
Four types of validity have already been checked in a series of studies of pilot studies -content validity, contrasted groups validity, concurrent validity and construct validity [20]. Here are results of additional study: Koral et al. [21] identified the factors that influence the development of systems thinking among industrial and systems engineers. In this study, contrasted group validity was determined by comparing the grades of two contrasted groups-industrial and systems engineers and particular engineers such as software and hardware engineers. It was found that there is a significant difference between the average scores of CEST among industrial and systems engineers and particular engineers (Sig=0.000).
Moreover, a significant correlation was found between supervisors' ranking in relation to these engineers' systems thinking capabilities and the average score they achieved on Frank's questionnaire (Sig=0.000, r=0.855). This finding is determined the concurrent validity. Construct validity was checked by factor analyzes.
In addition, while they were filling out the questionnaire, the engineers themselves were asked to evaluate their desire to engage in systems-related projects; a significant correlation was found between this evaluation and the results of Frank' questionnaire (Sig=0.000, r=0.763). In contrast to these findings, no correlation was found between the capacity for engineering systems thinking and number of years' employment experience.
The study's findings are in line with those of previous studies, according to which industrial and systems engineers, in particular, and any individual -who can report about himself or others -that they notice details or immediately see the big picture.
In addition to all of the above-mentioned findings, all of the engineers were divided into personality groups according to the MBTI questionnaire (2013). The MBTI personality type test is a method that evaluates personality type using a psychometric questionnaire. The goal of the test is to help people identify their dominant preferences, tendencies, and personality traits. According to the questionnaire, people have four psychological functions through which they experience the world: Energy: Extraversion versus Introversion; Information: Sensing versus Intuition; Decisions: Thinking versus Feeling; Lifestyle: Judging versus Perceiving. For each, one of the four functions is dominant most of the time. The result of this questionnaire is one of the 16 character archetypes. According to Koral et al. [21], 60% of the respondents belong to the STJ (Sensing, Thinking, Judging) group. This finding emphasizes the fact that a large percentage of the engineers belong to particular personality groups with unique traits.
The study findings show, among other things, that: A. High capacity for engineering systems thinking among industrial and systems engineers is a process that develops gradually.

B.
Two main factors may assist in the development of systems thinking among industrial and systems engineers:

1.
Learning in a formal teaching/training framework • It's important to include a practical project as part of the learning process.

•
Systems thinking should be taught within the context of the learners' studies, and not as isolated knowledge.

•
There is currently no existing curriculum that focuses on developing systems thinking among industrial and systems engineers.

Experience
• Experience in different job positions, which enables learning about different aspects of the system, contributes more than experience in only one job position.

•
Working together with a manager who has developed systems thinking capabilities is likely also to contribute to the development of systems thinking.

3.
A high level of systems thinking also relies on personality traits. These traits may be identified in a number of ways: • A questionnaire for assessing the Capacity for Engineering Systems Thinking [20] • MBTI -Myers-Briggs Type Indicator -Personality Type test (2013) [22] • Testimonies of the employees themselves and their employers.

Conclusions
Every enterprise strives to fill positions in the organization with employees who have the best chance to succeed. Employees are also interested in entering positions that fulfill their aspirations. Selection and screening processes can help match the interests of both parties, thus contributing both to the organization and the individual. The selection process for industrial and systems engineering positions should reliably predict those employees who can succeed and reject those who are likely to fail. Out of the employees who can succeed as industrial and systems engineers, it is necessary to choose those who have the highest chance of succeeding.
From the organization's point of view, rejection of candidates who might have succeeded in industrial and systems engineering positions can be critical, especially under conditions of an ever-increasing shortage of industrial and systems engineers. Likewise, placing engineers who later fail in industrial and systems engineering positions is also an expensive error, taking into consideration the necessary training which will be invested and the subsequent damage which might be caused to the projects in which they are involved. Industrial engineers and systems engineers have a very sensitive and crucial role in the organizations. Placing incompetent engineer is an error that may hamper the organization's ability to compete, whereas having the right industrial engineer in the right place may save a fortune.
The tool presented in this paper may be used for selection, filtering, screening of candidates for industrial and systems engineering job positions, and for placing the 'right person in the right job'.

Future Research
The tool for assessing interest for industrial and systems engineering positions required capacity for engineering systems thinking (CEST) should be verified and validated by additional studies. Future research may also discuss the ongoing argument in the literature about whether systems thinking ability is inherited (innate) or learned (acquired), and whether CEST may be improved and acquired through learning.