Ohanyan A and Harutyunyan H*
American University of Armenia, Eurasia International University, Yerevan, Armenia
Received November 26, 2015; Accepted January 30, 2016; Published February 10, 2016
Citation: Ohanyan A, Harutyunyan H (2016) The Role of Internal Audit in Continuous Improvement of Quality Management Systems at Private HE Institutions: A Case Study of Eurasia International University (Armenia). J Bus Fin Aff 5:170. doi:10.4172/2167-0234.1000170
Copyright: © 2016 Ohanyan A, et al. This is an open-access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Visit for more related articles at Journal of Business & Financial Affairs
The recent worldwide expansion of higher education and increase of the enrolment number, emergence of diverse educational providers, as well as various delivery modes continuously provokes the need for development of appropriate internal and external quality assurance mechanisms for different types of higher education providers. Private higher education is one of the most dynamic and fastest growing segments of postsecondary education at the turn of 21st century and, lacking traditions of quality provisions, the private tertiary institutions are challenged to build appropriate mechanisms from scratches. These institutions are urged in developing internal quality assurance systems to address the external controls and external accreditation requirements, as well as to face an increasing competition. The internal quality assurance systems set up by universities vary in scope, employed models (in case if used formal quality models at all), objectives, implementation methods etc. Implementing internal quality management systems for the sake of meeting the requirements of accreditation bodies, inherits the risk of symbolic adaptation of quality management rather than an embedded quality improvement system. Continues improvement is agreed by many experts on quality in education as a central element in achieving improvement. Yet, continuous improvement is observed as lacking in many HEIs. This research is aimed at exploring and examining the importance of internal audit for sustaining the implemented quality management systems and continuous improvement of private higher education institutions. It examines the emerging role of internal audit function (IAF) for quality and the importance of the internal audit for ensuring the continuous improvement of the private higher education institutions.
Quality assurance; Internal audit; Higher education; Quality management systems; QMS; Continuous improvement; IAF
Research proposition (RP)
The internal audit for quality contributes to the continuous improvement of the implemented quality management systems at private higher educational institutions.
The Structure of the paper
In the first section of the paper the research methods and the overview of literature studied is presented. Based on the described research methods a case study of a private university located in a Armenia (Eurasia International University) is later described, which has been in the process of implementing a quality management system during the recent two years and is currently searching for models to ensure not only effective deployment of the implemented system, but also continuous improvement directed to the realization of strategic goals. The landscape of Armenian higher education system is presented further, as an environment, in which Eurasia International University operates.
After the short demonstration of the quality management system implementation, the paper then discusses the major obstacles and challenges that the task forces responsible for the system implementation have encountered. At the end of the paper conclusion and recommendations for further improvement are presented, based on the literature examined and already implemented studies, as well as the current study itself.
To investigate the abovementioned research propositions a qualitative method of action research was adopted. The real experience of Eurasia International University in Armenia (EIU), a private higher education institution was examined, which is a relatively young institution (established in 1996) with lack of quality assurance rigid mechanisms in the past. Taking into account existing elements of quality management, a totally new system was designed and implemented in 2013-2014, which was reviewed by external accreditation committee during the institutional accreditation of the university in 2015 and was evaluated as positive. The major challenge is to ensure the further and sustainable enhancement of the implemented system, which was also noted by the external expert panel.
The action research, having a dual commitment to study a system and concurrently to collaborate with members of the system in reforming it, as well as being a method that enables people to reflect upon and improve their own practice was considered as the best fit for this study . The researchers have not only been examining the system implementation, but have also been actively involved in the reforms of EIU throughout the recent years .
Because of its flexibility, action researchers have the ability to deal with a variety of evidence including direct participation, documents, archival records, interviews, and direct observation and thus, the study was conducted based on the analysis of documentation and data that was accumulated over the system implementation, as well as evidence gathered from unstructured interviews with the work groups responsible for quality management system implementation and strategy revision. The major bases for the analysis were also the personal reflections of the authors (active participants and initiators of the system design) on success and failures during the system implementation, as well as interviews with other stakeholders (management, administrative and academic staff, students, external consultants).
The limitation of the research is that the assumptions, conclusions and findings cannot be confirmed through statistical models (being a qualitative study) and the results may have limited generalizability. This leaves room for further investigation of the mentioned research propositions and of the role of internal audit for sustaining and continuously improving the implemented quality management systems, as well as the relevance of integration of internal audit for quality and financial reporting.
The internal audit and internal audit function (IAF) have been widely explored through different studies, which mostly have been concentrated on public and private non educational entities [3-7]. There are limited studies though on the correlation, as well as the relationships of internal audit and continuous improvement of the implemented quality management system. The studies are especially limited for higher education system in Europe and neighboring countries, which undergo drastic reforms especially under the umbrella of Bologna Process. As reported by European University Association, among the core changes during the first decade of the 21st century was identified the development of internal quality assurance processes at higher education institutions: 60% of the 2010 Trends questionnaire respondents recognized it as a major institutional development “in the past ten years”, which placed it as the top change driver for institutions in Europe .
The literature on implementation of quality management system in higher education and quality assurance in general is vast [9-16]. There is lack of literature though on continuous improvement of the implemented systems and the role of internal audit (IA) and internal control systems for HE and a large research gap exists for further exploration. Among the studies on IA practices in higher education an empirical study conducted by Arena gives an overview of the current state in Italian higher education sector . The study presents the findings of a research carried out among Italian universities and aimed at investigating IA adoption patterns and characteristics of IA units in higher education. The paper illustrations a still limited diffusion of IA among Italian universities and among the analyzed universities operational auditing represents the core of IA activities, though much effort is still dedicated to financial auditing and compliance. Based on the research outcomes, few universities are dedicating increasing attention to risk management too, which is stated to be indicative of the change of the context in which universities compete. As stated by the researcher, the introduction of IA in Italian universities is generally quite recent: only three institutions, among the respondents, have established an IA unit more than ten years ago. No statistically significant differences emerged in relationship to university size and location . Based on the research outcomes 42.86% of respondents have an IA team that performs IA activities and 34.29% have a dedicated IA department.
A research conducted by Faudziah Hanim Fadzil, Hasnah Haron for Malaysian Public Listed Companies lay in the domain of the research propositions and topic of this paper with a concentration on publicly listed companies . The researchers provide empirical evidence whether compliance with internal auditing practices will lead to a better internal control system . This study examined the extent of the internal auditing practices by the internal audit department and the effects of the practices have on the quality of the internal control system. Analyses were done from the Internal Audit Practice Scale and Internal Control System Scale that were sent to all internal auditors and audit committees of the Malaysian Public Listed Companies for the year 2001. The influence of internal auditing practices and the quality of internal control system is studied. As revealed, only the monitoring aspect of the quality of internal control system is positively influenced by the management of the internal audit department. Based on the research outcomes, performance of audit work and audit reporting of the internal auditing practices positively influence the control activities aspect of the quality of the internal control system .
As stated in the Internal Control-Integrated Framework published by Committee of Sponsoring Organizations (COSO) of the Tread way Commission (COSO), internal auditors provide the third line of defense in assessing and reporting on internal control and recommending corrective actions or enhancements for management consideration and implementation. While having an internal audit function is not considered requisite of internal control by COSO, it is seen as a way of enhancing the scope, frequency, and objectivity of reviews and highlighting the areas that need to be improved within the organization. The scope of internal auditing is typically expected to include oversight, risk management, and internal control, and assisting the organization in maintaining effective control by evaluating their effectiveness and efficiency and by promoting continual improvement and amending established systems .
The function of internal audit is considered central in contributing to the effective governance by the Flemming Ruud as well, in particular to assist in the identification of risk factors, to analyze the consequences, as well as to assist the management in the prioritization of risk management and control systems . He also highlights the importance of audit function to assist management and the board by improving risk management and control processes, as well as assume the role of an in-house advisory function that offers analyses and assurance to the board as to the functioning of the risk management and internal control systems .
The Competency Framework for Internal Auditing (CFIA) study, conducted by The Institute of Internal Auditors Research concluded that in the context of burgeoning change and uncertainty, the internal audit function adds value by providing assurances to those charged with organizational governance that organizational risk exposures are well understood, being monitored, and under control . The risk based approach is highlighted in the above mentioned studies and was also highlighted as a core change of ISO 9001:2015 (yet to be approved).
On the overall image of the above discussed literature a contradicting state is revealed in the quantitative study conducted by Arena et al. [17,22]. Within 365 private Italian companies 74% of the responding companies have introduced an internal audit function. The rest not having internal audit function during the time of the research, 30% were planning to introduce, 50 per cent were not willing to introduce an internal audit function in the future and another 20% have removed their internal audit function. With respect to the internal audit agenda, the researchers found that operational audits, compliance audits and the monitoring of the internal control system are the most prevalent activities within the studied sample .
The business environment has experienced rapid and revolutionary change with far reaching consequences for organizations worldwide. Management responses to fierce global competition have included improved quality and risk management initiatives, reengineered structures and processes, and greater accountability — all needing more timely, reliable, and relevant information for decision-making . This statement is very much relevant to the higher education sector, which increasingly becomes competitive and in need of quality and risk management systems, appropriate data for decision making. The requirements of external accreditation also add pressure for higher educational institutions to ensure accountability, for which internal audit, internal control system and quality management systems in general play an integral role.
The higher education sector in Armenia broadly comprises public and private universities, as well as state associated institutes, academies, and conservatories, as defined by Armenian legislation regulating graduate and postgraduate education . Under the Soviet system, the higher education sector was entirely comprised of state institutions and was completely financed by the state. After the dissolution of the Soviet Union in 1991, the major trend in higher education sector was the establishment of private education institutions. Already in 2003 the number of private universities was 73 (with 29.7 % of total HE enrollment) in comparison to only 20 public universities . The external accreditation mechanisms during the period of around 20 years were enacted by the state to only regulate the private educational sector and the public institutions were shielded by the state enjoying number of privileges by law (e.g. the students of public universities where exempted from army service). The public universities didn’t undergo external accreditation until 2011, when a new accreditation process emerged, based on the requirements and obligations within the Bologna Process.
The landscape of the higher education system in Armenia changed tremendously throughout 2010-2015.
Most of the private institutions either didn’t manage to compete or where closed down by the Ministry of Education and Science due to not meeting the minimum license requirements. The number of private universities decreased to 33 as of 2015 . Although the number of private institutions decreased, new players continued entering the market in light of the internationalization of HE system of Armenia, including transnational, international universities and branch campuses. This also added competitive pressure in the system and was partially a wake-up call for the state universities, which were enjoying the prestige gained during the socialist system and where indolent, with limited reform initiatives.
As of 2015 the HE system of RA comprises of 24 public universities , 5 universities based on intergovernmental agreements, 4 branches of foreign state universities and 4 branches of foreign private HE institutions , 33 private HE institutions . Taking into consideration the very limited number of student cohort (85 922 total enrolment in 2014) in Armenia, law level of internationalization and international student inflow (4.25 % of total student body) the need for quality assurance to ensure a firm footage in the marketplace is already unquestionable . Adding to this the new state requirements of re-accreditation based on ESG (European standards and guidelines) standards fosters the HE institutions (already both public and private) to establish internal quality assurance systems and continuously audit the effectiveness.
The previous scheme of external accreditation of higher education institutions (enacted during the period of 2000-2011) was completely different compared to the European system and the new Guidelines, Criteria and Standards for Quality Assurance in the Armenian Tertiary Education (the new guidelines were confirmed by the Government Decree N 959, 30 June 2011).
The previous systems was concentrated more on outcome/output, rather than input and the quality assurance processes. The base for the accreditation rejection could be for instance the percentage of students getting positive mark during the final examinations: according to the Government Decree N 372 (7 July 2000) the percentage of positive graders had to be above 60% .
On the contrast the new standards based on the European system is more concentrated on the quality assurance processes and through a wide range of standards the external accreditors have a role of “assisting higher education institutions in managing and enhancing their quality” (European Association for Quality Assurance in Higher Education, 2009) rather than punishing for low performance . Transparency, accountability are the core of the ESG standards and philosophy, which was not widespread phenomenon in the former system.
Currently, taking into consideration the vast differences between the mentioned two systems of external accreditation, the reaccreditation has become a major challenge for not only private, but also state universities. The implementation of internal quality assurance mechanisms in accordance with the new standards in both types of institutions requires drastic shift of management philosophy and a transformative leadership, as it assumes a wide range of simultaneous reforms and in most of cases transformations.
Eurasia International University is a private institution of higher education in Armenia established after the collapse of the Soviet Union. Nearly twenty years of its existence have been quite productive for the comparatively young institution. EIU already provides educational opportunities in the VI (Bachelor), VII (Master) and VIII (Researcher: Candidate of Science) levels of National Education Qualification Framework of the RA.
In 2002, according to the Decision of the Ministry of Education and Science of the Republic of Armenia (RA), EIU was conferred state accreditation as well as vested with authority to award diplomas of state standard. As the external accreditation processes changed in Armenia, EIU launched the process of redesigning the internal quality assurance system to adhere to the new state standards, as well as to the European Standards and Guidelines of Quality Assurance. In 2015 EIU applied for the external institutional accreditation and based on the external evaluators’ conclusion of the National Centre for Professional Education Quality Assurance Foundation (Armenia) (presented in July 2015) the developed documentation base of quality management system the EIU is able to ensure regulated quality management, which will later lead to formation of quality culture, as well as continuous improvement of processes. The following Criterion “The institution has a set infrastructure for internal quality assurance, which promotes establishment of a quality culture and continual development of the institution” was evaluated as Positive, based on the review and evaluation of the Institutional Self-assessment report, as well as the site visit of the external experts.
Responding to the new state requirements EIU initiated the restructuring and redesign of the internal quality management system, which should satisfy the “Guidelines, Criteria and Standards for Quality Assurance in the Armenian Tertiary Education” (developed based on the European Standards and Guidelines (ESG)).
A self-assessment realized by the task force highlighted the major shortcomings of the university, based on which four separate working groups were gathered to address the issues: (i) a task force responsible for the development of the EIU new strategic plan for the upcoming period, (ii) a task force responsible for the redesign of the internal policies and procedures, as well as the quality manual (iii) a task force responsible for the redesign of work instructions, forms and templates, (iv) departmental groups involving internal and external stakeholders to redesign the existing educational programs (learning outcomes, methods, evaluation and assessment etc.) based on the redesigned requirements, standards and templates. The last work group started working and piloting the new mechanisms during the final stages of the project, when the new mechanisms were already confirmed by the appropriate bodies.
Development of the new strategy, the action plan and internal controls
Redesign of the organizational chart and development of the value chain: The work group responsible for the development of the EIU Strategic Plan started with restructuring of the organizational chart. The major issues highlighted in the self-assessment report were the overlapping functions, extra chains and high structure not relevant for a small institution. All of the mentioned issues where addressed: several units where either closed down or merged/regrouped to ensure a flatter structure.
During the redesign and regrouping of the organizational units the work group explored the Porter's Value Chain, which “provides a systematic way to divide the firm into its discrete activities”. It also underlies the EIU Strategic Plan and directions. Based on the Porter’s model the major primary (educational, PR and Communications, International relations, Research and Development, Career Development) and support activities (infrastructure, human resources, financial management) where highlighted. An additional layer, which is not discussed in the Porter’s value Chain Model was added, including the advisory units/activities (Method Council, Scientific Council, Quality Assurance Center etc.), which are considered as advisory for rector, but report to the Management Board (Figure 1). The organizational chart was clustered in the above-mentioned three major groups, based on developed Value Chain of EIU.
Environmental scanning, risk identification and assessment: Acting in an extremely volatile market with intense competition and ongoing legislative unexpected changes, the EIU needed detailed information and analysis anchored around issues of current concerns both on external and internal levels. An in-depth analysis was carried out through qualitative and quantitative research methods involving internal (students, administrative and academic staff) and external stakeholders (policy makers, employers, NGOs etc.). The external factors affecting EIU activities directly (competitors) and indirectly (political, economic, social, demographic, technological) where explored and where a sound bases for identifying strategies to leverage the current opportunities and strengths, as well as neutralize the weaknesses and threats.
Development of strategic goals, an action plan and internal controls: Another task of the work group when developing the new Strategic plan was to have a clear action plan tagged with the Strategic goals, with concrete measurable indicators and controls. The previous plan was more of a wish list and it was almost unrealistic to monitor and audit the plan neither periodically, nor at the end of the period. The realization of the Strategy was very much dependent on the management and employee will, loyalty and motivation. The task of the work group was to design a plan that would have clear strategic goals, objectives and actions, performance indicators to benchmark and measure the universities’ success both in the short and long run in concrete areas of activity.
Financial planning, monitoring and audit: The lack of financial planning mechanisms, impulsive and hasty financial decisions where the source of strategy under realization in most of the cases before the system implementation. The absence of financial management and audit mechanisms would lead to the complete failure of the new QMS implementation. To address the issue the work groups developed financial planning, management and monitoring provisions, which were already lobbied and implemented.
According to the newly implemented mechanism, the EIU financial planning starts with defining priorities at the beginning of the academic year based on the EIU Strategic Plan.
The defined priorities and directions are then stipulated by chairs and departments in their budgets within a one-month period. The budgets are discussed, revised and the final version is submitted for approval of the Management Board by the end of the year (Figure 2).
Financial monitoring is performed by the EIU Rector together with the Department of Financial management. The latter presents monthly, quarterly and annual statements to the Rector. Financial statements include the following four components:
a. Statements of financial position,
b. Statements of comprehensive income,
c. Statements of cash flows,
d. Statements of changes in equity.
For current financial management, considering given circumstances, the Rector can make budget allocation and revision to the extent that is not more than 20% of the budget approved by the Management Board. For changes in larger proportions, the approval of the Management board is required to proceed further. During the annual meeting of Management Board, a decision is made on conducting internal or external financial audit as of the given year, and responsible persons and timelines are defined.
Development of the QMS documentation base
The major mission of the work groups was to implement a simple, not complicated systems and procedures, without any ambiguity when developing a QMS system for a relatively small institution. For developing the documentation base of the system, a more useful and straightforward approach was ISO 9001:2008 standards and requirements, in comparison to ESG standards, which in some cases, are vague. As the university should pass an external accreditation based on ESG standards, the work group had to continuously compare and ensure meeting the ESG requirements, although as a guiding principle was adopted the ISO 9001:2008.
The ISO 9001:2008 requirements where considered and modified as documentation guidance for the EIU QMS. The following 5 document levels where ensured: (i) documented statements of a quality policy and quality objectives, (ii) the quality manual, (iii) documented regulation and procedures, (iv) documents needed by the organization to ensure the effective planning, operation and control of its processes (work instructions, job descriptions and work plans where considered here by EIU), (v) records and templates (Figure 3).
The EIU quality manual consists of the section on quality policy, quality objectives and principles, the overall scope of the QMS, customer satisfaction, management responsibility, quality planning, management and revision, responsibility, authority distribution and communication channels, resource and infrastructure management, financial management, monitoring and evaluation, data analysis and major procedures, as well as process interactions (Figure 4). Before development of the manual the Quality Policy was developed and communicated to stakeholders to increase the ownership and involvement in quality assurance processes of key personnel.
The development of the Quality Manual in higher educational institutions is relatively new phenomenon in Armenia and the practices vary greatly. In some institutions the manual is designed in a more academic, while in others in a more “business style”. Searching for a straightforward and efficient system, the work groups adopted a more business style of writing the manual ensuing it involves the necessary information and is simple enough to communicate to stakeholders and safeguard the implementation.
In the manual two types of main processes where divided by the work group when mapping the EIU key processes, including (i) managerial, and (ii) service provision. The service provision processes were also divided into three major groups in particular learning, teaching and research. Overall twenty three key processes where identified (Table 1) by the work groups, which was considered the adequate and controllable level of processes to ensure the quality of management and service provision, as well to systematically implement, monitor and revise. All the processes are subject to audit and respective changes are directed towards increasing their efficiency. Feedback ensures operative observation and review of these processes.
|Managerial process||1||Strategic planning, implementation, monitoring and review|
|2||Individual planning, implementation, monitoring and review|
|5||Staff recruitment, selection, evaluation, trainings and development, promotion|
|6||Information assessment, sharing and management|
|7||Internal audit, assessment, self-evaluation|
|8||Communications and feedback|
|9||Purchasing & Supplier Management|
|10||Property and infrastructure management|
|11||IT system management|
|Service provision process||Learning|
|15||Student evaluation and assessment|
|18||Research work preparation|
|20||Educational program development, introduction,monitoring and revision|
|21||Course development, introduction, monitoring and revision|
|22||Organization of research|
|23||Publicizing research results|
Table 1: The EIU key managerial and service provision processes.
Based on the initial self-assessment, presented earlier, a major issue for EIU was (i) the presence of documents (regulations procedures), that are not implemented in practice, and vice versa (ii) presence of processes and good practices, that are not regulated and are implemented only based on the inherent tradition or a personal will of concrete managers. This was considered as the major weakness for the university, as many aspects where very much dependent on concrete persons and not on regulations accepted by the management bodies. This was leaving room for subjective decision making. To escape this reality, the workgroups completely redesigned the existing documents or added new once based on the agreed processes, as well as the external accreditation requirements.
The task force responsible for the redesign of the internal policies and procedures, as well as the quality manual presented the changes to the task force responsible for the redesign of work instructions and job descriptions, which completed the task based on the already revised organizational chart, the new strategic plan, as well as the outlined processes mentioned in the quality manual.
The final stage of the preparation of the EIU QMS documentation base was the development and the redesign, standardization of level V documents (forms and templates). The major templates, that were developed or redesigned included, but where not limited to the following: strategic action plan template; policy, regulation and procedures template; Minutes of Meeting (MOM) template; individual plan and report template; job description template; educational program development template; syllabi and course plan template; project plan template etc.
Document coding and dissemination
The processes and the regulating documents (policies, procedures, etc) where then coded based on the newly developed Document Control Procedure and where uploaded in the internal Document Register. Although the Quality Assurance Centre is responsible for organizing the processes of creating, revising, filing and publicizing the documents and making the required changes in the electronic register of documents in the framework of quality management system, the heads of departments and all of the employees are responsible for notifying and informing about the documents decreasing or hindering (unnecessary ones) the effectiveness of processes.
In the virtual registrar the QA Centre has created a forum, where discussions on required new documents or on the revision of old once are always open for staff and students. All the employees and students who have an access to the electronic register can present their proposals in the forum. The documents (e.g. procedures, templates) may also change based on the (i) results of internal or external audit, (ii) based on external changes (e.g. legal), (iii) in case of a new or amended processes.
The departmental heads and the Quality Assurance center are responsible for the publicizing the document, making sure that the old version is no longer in use and the latest version is uploaded in the electronic registrar. Both bodies are also responsible for initiating meetings and trainings on the changes made. Based on the Document Control procedure employees are responsible for checking the consistency of hard copy documents with the electronic copies in the electronic register, as the latest version filed in the electronic register is considered the valid version of the document.
The feedback mechanisms have been in place right from the establishment of the university, but those where limited to measuring only the student satisfaction of educational programs and support services, faculty quality, as well as teaching and assessment methods. The student surveys have been conducted on a systematic base semiannually. The Exit Survey was also conducted annually and systematically during the last three years. Analyzing the responses of direct customers (students) is of course significant, but the problem here was that the student opinion gained too much importance and could even result in a firing of faculty member (very rarely). To escape this occurrence the Quality Assurance Centre completely changed the faculty evaluation mechanisms and allocated only 25 % of the faculty methodological evaluation1 to the student survey results.
Among the major drawbacks of feedback mechanisms were the channels for observing the external stakeholders’ opinion, which is already regulated and fixed by the Quality Manual. The following directions are already under examination, which are evaluated by the qualitative and quantitative methods based on the best fit and necessity.
a. Satisfaction of internal stakeholders with educational programs (educational content, teaching and learning methods, evaluation means, professionalism of the academic staff, advisory),
b. Satisfaction of internal stakeholders with educational environment (general comfort, technical base, educational and methodological literature, electronic environment),
c. Satisfaction of internal stakeholders with auxiliary services (printing and copying, catering etc.)
d. Satisfaction of external stakeholders with EIU services (surveys among employers, graduates and other stakeholders)
Some of the surveys are already fixed and concrete periodicity of evaluation is defined in the EIU Quality Manual (Table 2). The outcomes and analysis of both qualitative and quantitative studies are then compiled and presented to management, as well as disseminated within internal stakeholders (Table 2).
|Course evaluation||The objective of the research is to courses included in the curriculum:||Annual (at the end of the academic year)|
|Content and availability of respective effective teaching and learning methods|
|Trustworthiness and objectivity of the student evaluation and assessment methods|
|Complexity of the material|
|Conformity of the number of hours, as well as topics with the defined learning outcomes|
|Position of the course in the curriculum|
|Teacher/lecturer evaluation by student||Quality of the material presented||Semi anuual|
|Teaching quality (teaching and assessment methods, objectivity etc.)|
|Exit survey||Studies the opinion of newly graduated students about the following:||Righ after state graduation exams|
|Quality of educational programs;|
|Quality of teaching methods;|
|Quality of the work performed by the academic, administrative and support services staff;|
|Quality of consulting services,|
|Quality of educational environment, resources and infrastructure|
|Quality of centers and departments (e.g. Career Development Centre, International Relations Centre etc.)|
|Employers’ opinion about the EIU graduates||Studies the satisfaction of employers with the knowledge, skills and competences of the EIU graduates, as well as their compliance with employers' requirements||Annual|
|Employment and career development||Studies the employment indicators of the EIU students, the employment status as of date||Annual|
|Comprehensive self-assessment survey (among students and staff)||Studies the quality of the below mentioned items:||Annual|
|Management and administration|
|Services contributing to career development|
|Engagement of students in educational projects and research|
|International programs, availability and information dissemination|
|Information and communication services|
|Catering and other services,|
|Main processes (consulting, student evaluation and assessment etc.)|
|Overall satisfaction from the services provided by EIU|
Table 2: The list of quantitative researches carried out by the EIU, objectives and periodicity.
Previously the quantitative studies were conducted through dissemination of paper surveys. The data where then coded and imputed into computer and then analyzed trough different statistical software programs (mostly SPSS). Aiming at reducing the paperwork, as well as the staff and time needed for conducting the studies, currently all of the quantitative surveys are conducted through electronic platforms. This has almost three times reduced the time; twice the human resources needed to conduct a single study and nullified the paperwork at all. This is a considerable result for the university, taking into consideration the scarce resources and the complex task that has the Quality Assurance Centre.
Redesign of the educational programs
After the redesign of the existing documents and development of new once, including the Quality Manual, the regulations and procedures, work instructions, templates etc., the forth work group launched the redesign of the educational programs. The major change here was the change to learning outcome based educational programs and courses. The pilot was implemented in the Law department, where the programs where modified base on the EU “TUNING” methodology, whit active involvement of faculty members, students, alumni and employers. It is worth mentioning here, that in the previous system, the courses, as well as the earning outcomes where defined and standardized by the state (Ministry of Education and Science in particular). The recent changes and reforms already leave room for universities to benchmark internationally, as well as nationally and develop own educational programs. Some of the courses are still required though by the state (e.g. Armenian Language, History etc.), but the drift is towards more of a decentralization and power to tertiary institutions to develop own programs, based on market needs.
Implementation of management information system of EIU
During the recent years, before the systematic change and implementation of the QMS, The Quality Assurance Centre started and endeavour of implementing and internal information management system (MIS) for automation of most of the educational processes at the university. Data management and decision support system development was gaining continuous importance at the university, which effected the decision making process positively: previously more intuitive decision where now backed up with sound data analysis and automatic reports. During the QMS implementation new modules where added in the system, based on the developed new processes and templates. The previous resistance of using MIS systems is currently replaced with a demand of employees to eliminate as much of the routine tasks (possible to automate) as possible, decrease the paperwork and manual processes. The information is currently not a “power” of concrete managers, but whoever has an access. The adoption of MIS systems decreased the dependence from concrete employees to gain information for analysis and quality assurance processes, which in most of cases was consuming too much time.
QMS implementation and training
Perhaps the most difficult task of the work groups was to communicate the above described massive changes to administrative and academic staff members, most of whom have been working at EIU more than 5 years and are used to the previous management and systems. To escape to much resistance, as well as build a system that is close to reality and is realistic to implement, the workgroups have been closely collaborating with staff. The participative nature of the reforms made it easier the process of implementation. The changes where first presented to key staff members and managers and based on the feedback, appropriate amendments were made. The final versions where presented to administrative and academic staff trough intensive and interactive trainings. As a result, most of the mechanisms have already been implemented and only those requiring financial commitment are still in the process of discussion and planning (e.g. the new regulation on the Faculty Evaluation and Promotion).
During the QMS design and later implementation there were several obstacles identified and overcome by the work group. The leadership and the change management was perhaps the most interesting to observe for the work groups: during the reforms the work groups had change sponsors, including the rector (the most influential figure in the university), but the problem was the way of thinking and the time needed for the “painless” mindset shift of top management. A shift of autocratic management inherent from the Soviet system to a more democratic and participative management was vital for the success. As the project had limited time and financial resources, the mindset shift of the top and middle management, as well as personnel was a major challenge. It was a key for the work groups to ensure people feeling comfortable about the change and work with enthusiasm. Besides of changing systems and mechanisms, the work groups were trying keep the intense communication with and training of management and personnel to ensure the long-run effect of the reforms, which was difficult taking into consideration the mentioned limited resources. Generally, there was very minor resistance, as both management and staff were realizing, that without the reforms, the existence of the university will already be questioned, but they needed time to “digest” and believe in the reforms, change the way they’ve been working for more than 20 years.
The need for mindset shift was not only identified by the system implementers, but was also constantly mentioned by the middle and top managers during the interviews. It was highlighted as the core challenge for themselves, taking into consideration the time scope (around 20 years) they have been working within completely different system and requirements.
Assumptions of an extra workload
The second obstacle was the busy schedule of the middle management and staff to deal with an extra task in parallel with their own day routine. The work group members had to plan and take this limitation into consideration, when working with people, to be able to manage own limited time resources. Besides, with the new system implementation new duties where added and the key message of the work groups here was that although some duties are added, others are cut down and operationalized. As a result, in general, the change didn’t pose extra workload and this message was consistently communicated to management and staff, so a resistance to change didn’t emerge.
Among the core shortcomings of the university was the lack of financial planning mechanisms and traditions, as well as execution based on the plan. As some of the developed mechanisms of the work groups demanded planned financial recourses, the implementation of the tools and the regulations was becoming questionable. The redesigned methodology and regulation of faculty remuneration and performance evaluation for instance would become senseless, unless appropriate financial resources where allocated periodically, every year.
To address this issue, the work group responsible for the Quality Manual design added a section on financial planning and reporting and lobbied the acceptance by the top management, as presented earlier in the study.
During the semi-structured interviews the middle and top managers of EIU have more frequently mentioned three major reasons that have hindered the previous attempts to implement different systems and reforms at EIU including:
a. Vague requirements and expectations on strategic level,
b. Irregular reporting by organizational units and lack of consistent procedure,
c. Lack of monitoring mechanisms, regular and consistent checks.
As described in the presented case study, the first step of system implementation was the redesign of the EIU Strategic plan with clear and measurable objectives. Based on that, the short and long-term action plans with concrete KPIs where developed and communicated to all of the departments and human resources responsible for the implementation. Alongside an internal information management system was implemented with a planning and reporting tool, based on which performance appraisals are conducted and departments are evaluated.
These reforms have addressed the above mentioned first two concerns on vagueness of requirements and weak reporting mechanisms. The further challenge for the organization is to establish and maintain a clear monitoring mechanism that will not only check the variance of the planned and realized results and outcomes, but also the processes and the compliance with regulations and procedures, set up internal controls. Among the major challenges is the deployment and continuous improvement of the established system.
For the sake of ensuring the utilization of the implemented system and continuous improvement, as well as the monitoring outcomes, controls and the processes already clarified during the system implementation, it is further suggested to introduce an independent internal audit function (IAF). As stated in the interviews of EIU staff and management, although previously several controls where in place, the lack of consistent checks made those controls weaker in the course of time. This highlights the importance of the integration of internal control system and monitoring (in this case we suggest the introduction of IAF as a monitoring tool): based on the previous experience, setting only internal control system and restructuring the organization is not enough for sustaining the reforms, as the previous organizational and work culture existing for around 20 years proved to be stronger.
The definition of internal audit itself reflects the major shortcomings of EIU in a sense of an independent, objective assurance of organization's operations, as well as a systematic, disciplined approach to evaluate and improve the effectiveness of risk management, control, and governance processes. As defined and based on the recent researches on internal audit, it plays integral role also in overall risk management of the organization, which is another challenge for EIU taking into account the external environment it is currently acting in.
Based on the survey conducted by Ernst & Young on evolving role of internal audit, the (realized among chief audit executives, board members etc.) 75% stated that their internal audit function has a positive impact on their overall risk management efforts . We suggest utilizing the study, experience and best practices of the industry and embed the IAF in the quality management system of EIU, although this is not a strict requirement by the higher education external accreditation standards (including Armenian National Quality Assurance Criteria and Standards and European Standards and Guidelines). Having an internal audit function is not considered requisite for internal control by industry standards as well (e.g. COSO), but it is seen as a way of enhancing the scope, frequency, and objectivity of reviews and highlighting the areas that need to be improved within the organization.
The introduction of the internal audit function would also largely encroach to the advancement of the newly established internal control system and the internal quality management system. The feedback based on the audit of identified key processes and controls within those processes would be invaluable for further improvements.
As stated by Fadzil et al. if the internal audit function is not available, the management needs to apply other monitoring processes in order to assure itself and the board that the system of internal control is functioning as intended . In the same study the researchers provide empirical evidence whether compliance with internal auditing practices will lead to a better internal control system: based on the outcomes the performance of audit work and audit reporting of the internal auditing practices positively influence the control activities aspect of the quality of the internal control system.
In a nutshell, the introduction of the independent internal audit function is seen as a major tool to increase the effectiveness of risk management, control, and governance processes, as well as continuous improvement of the implemented quality management system at EIU in general. The application of IAF and the support of the function in terms of the organizational governance perspective is also discussed in a number of publications [6,17,18,20,31]. The application of the internal audit function should be realized with caution though, taking into consideration the unique academic culture and should be adapted based on the institutional peculiarities.
The second recommendation is the integration of different types of internal audits, including financial, compliance, operational and information technology audits, coming up with an integrated checklist and audit activities. The recommendation is also based on the previous experience of the university, as well as feedback from the external experts working on the EIU quality management system implementation and the key staff. This approach is rather relevant for EIU and integrative audit would ensure a comparatively simple monitoring mechanism and thus make the implementation more realistic, time and cost effective.
Lastly, the application of integrated internal audit would play a central role in preparation for external audits, checks (e.g. implemented by Ministry of Education and Science, Ministry of Finance) and accreditations (national or international). To this end, the design of the audit checklists based on the external audit, check and accreditation requirements would be advantageous.
Make the best use of Scientific Research and information from our 700 + peer reviewed, Open Access Journals