University Departments Evaluation: A Multivariate Approach
Abstract
Aim of the paper is to present a new model, based on multivariate statistic analyses, allowing to express a synthetic judgement on Departments activities by taking into consideration the whole set of indicators describing them both as aggregations of researchers and as University autonomous organs.
The model, based on Principal Component Analysis and Cluster Analysis, allows both to explain the determinants of Departments performances, and to classify them into homogeneous groups.
The paper shows the results obtained by testing the proposed model on University of Naples “L’Orientale” Departments, using data extracted by the 2007 assessment report to the Ministry of University and Research.
References
[2] CSS - Consiglio Italiano per le Scienze Sociali 2006. La valutazione della ricerca. Libro bianco. Venezia, Italy: Marsilio.
[3] Cugini, A., Michelon, G. 2009. La misurazione della performance nei Dipartimenti universitari. Economia & Management, n° 4 (July/August).
[4] Dehon, C., McCathie, A., Verardi, V. 2009. Uncovering Excellence in Academic Rankings. A case study on the Shanghai Ranking. Working Paper Mimeo.
[5] Dubini, P. 2005. Misurare la competitività dei sistemi Paese: il modello del World Economic Forum. Economia e Management, n°6.
[6] EUA - European University Association 2009. Institutional Evaluation Programme: Guidelines for participating institutions. Belgium, Brussels: EUA Editions.
[7] European Commission, Enterprise Directorate-General 2009, European Innovation Scoreboard (EIS). PRO INNO Europe Paper, n° 15, Bruxelles, Belgium.
[8] Figà, Talamanca, A. 2009, E' lecito usare l'Impact Factor per valutare la qualità della ricerca?, relation presented to the conference The problem of evaluation in Social Sciences, Bergamo University, 25 November 2009, Bergamo, Italy.
[9] Frey, B.S., Rost, K. 2008. Do Rankings Reflect Research Quality?, WP n. 390, Institute for Empirical Research in Economics, University of Zurich.
[10] Gallo, M. 2007. The scaling problems in service quality evaluation. Metodološki Zvezki, vol. 2, 165-176.
[11] Gherghi, M., Lauro, C. 2004. Appunti di Analisi dei Dati Multidimensionali. Metodologia ed esempi. Naples, Italy: RCE Edizioni,.
[12] Guan, J., Wang, J. 2004. Evaluation and interpretation of knowledge production efficiency. Scientometric, n° 59, 131-155.
[13] Hicks, D. 2007. The role of quantitative research evaluation in developing international research excellence, relation presented to the conference organized by the Italian Society of Economists (SIE) on Research Evaluation: experiences and methods, Ancona University, 2-3 April 2007, Ancona, Italy.
[14] Hirsch, E.J. 2005. An index to quantify an individual's scientific research output. PNAS, vol. 102, n° 46, 16569-16572.
[15] King, D.A. 2004. The scientific impact of nations. Nature, 430, 15 July 2004.
[16] Lippi, M., Peracchi, F. 2007. Il primo esercizio italiano di valutazione della ricerca: una prima valutazione, relation presented to the conference organized by the Italian Society of Economists (SIE) on Research Evaluation: experiences and methods, Ancona University, 2-3 April 2007, Ancona, Italy.
[17] Liu, N.C., Cheng, Y. 2005. Academic Ranking of World Universities - Methodologies and Problems, Higher Education in Europe, 30, n°2.
[18] Macerata University 2009. Se questa vi sembra una valutazione, Macerata, Italy: EUM - Edizioni Università di Macerata.
[19] Marcuzzo, M.C., Zacchia, G. 2007. L’econlit e gli strumenti per la valutazione della ricerca economica in Italia. Rivista italiana per gli economisti, n°. 2., 279-310.
[20] MIUR. 1998. Indicazioni per la preparazione delle relazioni dei Nuclei di valutazione interna e insieme minimo di indicatori, doc 11/98.
[21] MIUR. 2009. D.M. 23 settembre 2009 prot. 45/2009. Decreto criteri di ripartizione del Fondo di Finanziamento Ordinario (FFO) delle Università per l'anno 2009.
[22] MIUR CUN. 2010. Valutazione della ricerca. Prot. 612, 30/3/2010.
[23] Moed, H.F. 2008. UK Research Assessment Exercises: Informed judgments on research quality or quantity? Scientometrics, n° 74, 141-149.
[24] Narin, F., Hamilton, K.S. 1996. Bibliometric performance measures. Scientometrics, n° 36, 293-310.
[25] Neary, J.P., Mirrlees, J.A., Tirole, J. 2003. Evaluating Economics Research in Europe: An Introduction. Journal of the European Economic Association, Vol 1 , n° 6, p. 1247.
[26] OECD. 1997. The evaluation of scientific research: selected experiences. Paris.
[27] OECD. 2009. Science Technology and Industry Scoreboard 2009. Paris.
[28] RAE - Research Assessment Exercise. 2006. Assessment panel's criteria and working methods, doc. RAE 01/06, in http://www.rae.ac.uk.
[29] Remler, D.K., Pema, E. 2009. Why do Institutions of Higher Education Reward Research While Selling Education?. NBER Working Paper No. 14974.
[30] Rizzuto, C. 2003. Finanziamenti, progetti, centri di eccellenza, FAR, FISR, distretti tecnologici, in La ricerca universitaria: esperienze, modelli, proposte, eds. Casciotti C. and Mosconi G., p. 96-115, Italy, Rome: CRUI editions.
[31] Seglen, P.O. 1997. Why the impact factor of journals should not be used for evaluating research. BMJ, 314- 497.
[32] Van Leeuwen, T. 2006. Descriptive Versus Evaluative Bibliometrics. Monitoring and Assessing of National R&D Systems, in Handbook of Quantitative Science and Technology Research, part 3, 373-388, Netherlands: Springer.
[33] Van Raan, A. 2005. Challenges in Ranking of Universities, Invited paper for the First International Conference on World Class Universities, Shanghai Jaio Tong University, Shanghai, June 16-18, 2005.
[34] Vinke, P. 2009. University Rankings, in Ranking Universities, eds. Dehon, C., Jacobs, D., Vermandele, C., pp.11-25, Belgium, Bruxelles: Editions de l'Université de Bruxelles.
[35] World Economic Forum. 2009. The Global Competitiveness Report 2009-2010.
The Copyright Transfer Form to ASERS Publishing (The Publisher)
This form refers to the manuscript, which an author(s) was accepted for publication and was signed by all the authors.
The undersigned Author(s) of the above-mentioned Paper here transfer any and all copyright-rights in and to The Paper to The Publisher. The Author(s) warrants that The Paper is based on their original work and that the undersigned has the power and authority to make and execute this assignment. It is the author's responsibility to obtain written permission to quote material that has been previously published in any form. The Publisher recognizes the retained rights noted below and grants to the above authors and employers for whom the work performed royalty-free permission to reuse their materials below. Authors may reuse all or portions of the above Paper in other works, excepting the publication of the paper in the same form. Authors may reproduce or authorize others to reproduce the above Paper for the Author's personal use or for internal company use, provided that the source and The Publisher copyright notice are mentioned, that the copies are not used in any way that implies The Publisher endorsement of a product or service of an employer, and that the copies are not offered for sale as such. Authors are permitted to grant third party requests for reprinting, republishing or other types of reuse. The Authors may make limited distribution of all or portions of the above Paper prior to publication if they inform The Publisher of the nature and extent of such limited distribution prior there to. Authors retain all proprietary rights in any process, procedure, or article of manufacture described in The Paper. This agreement becomes null and void if and only if the above paper is not accepted and published by The Publisher, or is with drawn by the author(s) before acceptance by the Publisher.