Proof.—If the human body is in no way affected by a given external body, then (II. vii.) neither is the idea of the human body, in other words, the human mind, affected in any way by the idea of the existence of the said external body, nor does it in any manner perceive its existence. But, in so far as the human body is affected in any way by a given external body, thus far (II. xvi. and Coroll.) it perceives that external body. Q.E.D.
Proof.—When the human mind regards external bodies through the ideas of the modifications of its own body, we say that it imagines (see II. xvii. note); now the mind can only imagine external bodies as actually existing. Therefore (by II. xxv.), in so far as the mind imagines external bodies, it has not an adequate knowledge of them. Q.E.D.
I have mentioned elsewhere my surprise at colleagues who continue to imagine ‘the [so-called] system’ as a top-down authority that shapes and coerces academic work. I continue to believe that this is a misleading depiction, and that ‘the [so-called] system’ is nothing more than the sum total of bureaucratic, political and academic practices – including individual academics. Case in point: the obsession with performance measurement and bibliometric analyses performed by non-experts.
Recently, Diana Hicks et al. illustrated the point – between 1984 and 2014 mentions of the much maligned ‘journal impact factor’ (the average citations for papers published in the last two years in a journal) increased dramatically in journal articles and editorials; the obsession with this form of debasing performance measurement, which wants to reduce academic work to single digits, hasn’t been driven by contributions from academics and professionals working in bibliometrics/scientometrics/infometrics and research evaluation, but has taken place in the pages of multi-disciplinary, medical and life sciences journals.
I was reminded of this recently while reading through the latest post from the SIGMETRICS mailing list – for those who don’t know, the list is
intended for the exchange of technical information among members of the performance evaluation community. Typical submissions include performance-related questions and announcements of research papers, software, job opportunities, conferences, and calls for papers.
One of the regular features of the list is a contribution from Eugene Garfield including bibliographic details of recent papers mentioning bibliometrics, scientometrics etc. Today, as I read through it struck me that most of the articles listed were a) published in journals outside of the field of bibliometrics/scientometrics/infometrics, b) published by academics with listed affiliations outside of bibliometrics/scientometrics/infometrics disciplines, and c) contained little or no engagement with the academic field of bibliometrics/scientometrics/infometrics. Below are a couple of examples (note – scroll through to the bottom if you want to skip to the rest of this post) – Example 1:
Scientific impact of studies published in temporarily available radiation oncology journals: a citation analysis
Nieder, C; Geinitz, H; Andratschke, NH; Grosu, AL
[Nieder, Carsten] Nordland Hosp, Dept Oncol & Palliat Med, N-8092 Bodo, Norway.
[Nieder, Carsten] Univ Tromso, Fac Hlth Sci, Inst Clin Med, N-9038 Tromso, Norway.
[Geinitz, Hans] Johannes Kepler Univ Linz, Krankenhaus Barmherzigen Schwestern, Dept Radiat Oncol, A-4010 Linz, Austria.
[Geinitz, Hans] Johannes Kepler Univ Linz, Fac Med, A-4010 Linz, Austria.
[Andratschke, Nicolaus H.] Univ Zurich Hosp, Dept Radiat Oncol, CH-8091 Zurich, Switzerland.
[Grosu, Anca L.] Univ Hosp Freiburg, Dept Radiat Oncol, D-79106 Freiburg, Germany.
SPRINGERPLUS, 4 10.1186/s40064-015-0885-y FEB 24 2015
The purpose of this study was to review all articles published in two temporarily available radiation oncology journals (Radiation Oncology Investigations, Journal of Radiosurgery) in order to evaluate their scientific impact. From several potential measures of impact and relevance of research, we selected article citation rate because landmark or practice-changing research is likely to be cited frequently. The citation database Scopus was used to analyse number of citations. During the time period 1996-1999 the journal Radiation Oncology Investigations published 205 articles, which achieved a median number of 6 citations (range 0-116). However, the most frequently cited article in the first 4 volumes achieved only 23 citations. The Journal of Radiosurgery published only 31 articles, all in the year 1999, which achieved a median number of 1 citation (range 0-11). No prospective randomized studies or phase I-II collaborative group trials were published in these journals. Apparently, the Journal of Radiosurgery acquired relatively few manuscripts that were interesting and important enough to impact clinical practice. Radiation Oncology Investigations’ citation pattern was better and closer related to that reported in several previous studies focusing on the field of radiation oncology. The vast majority of articles published in temporarily available radiation oncology journals had limited clinical impact and achieved few citations. Highly influential research was unlikely to be submitted during the initial phase of establishing new radiation oncology journals.
Holliday Emma, 2013, INTERNATIONAL JOURNAL OF RADIATION ONCOLOGY BIOLOGY PHYSICS, V85, P23
Wazer DE, 1999, RADIATION ONCOLOGY INVESTIGATIONS, V7, P111
Solberg TD, 1999, J Radiosurg, V2, P57
Joschko M A, 1997, Radiation oncology investigations, V5, P62
Maire JP, 1999, J Radiosurg, V2, P7
Chaney A W, 1998, Radiation oncology investigations, V6, P264
Epperly MW, 1999, RADIATION ONCOLOGY INVESTIGATIONS, V7, P331
Sanghavi S, 1999, J Radiosurg, V2, P119
Monga U, 1999, RADIATION ONCOLOGY INVESTIGATIONS, V7, P178
Norman A, 1997, Radiation oncology investigations, V5, P8
Kramer B A, 1998, Radiation oncology investigations, V6, P18
Kang S, 1999, RADIATION ONCOLOGY INVESTIGATIONS, V7, P309
Merrick G S, 1998, Radiation oncology investigations, V6, P182
Leborgne F, 1997, Radiation oncology investigations, V5, P289
Seymour C B, 1997, Radiation oncology investigations, V5, P106
Teicher BA, 1996, Radiat Oncol Invest, V4, P221
Garell PC, 1999, J Radiosurg, V2, P1
Nathu R M, 1998, Radiation oncology investigations, V6, P233
Nieder C., 2012, STRAHLENTHERAPIE UND ONKOLOGIE, V188, P865
Durieux Valerie, 2010, RADIOLOGY, V255, P342
Gibon D, 1999, J Radiosurg, V2, P167
Banasiak D, 1999, RADIATION ONCOLOGY INVESTIGATIONS, V7, P77
Smith BD, 1999, RADIATION ONCOLOGY INVESTIGATIONS, V7, P125
Stickle RL, 1999, RADIATION ONCOLOGY INVESTIGATIONS, V7, P204
Schmidt-Ullrich RK, 1999, RADIATION ONCOLOGY INVESTIGATIONS, V7, P321
Nieder C, 2013, J Cancer Sci Ther, V5, P115
Durand R E, 1997, Radiation oncology investigations, V5, P213
Haffty B G, 1997, Radiation oncology investigations, V5, P235
Fernandez-Vicioso E, 1997, Radiation oncology investigations, V5, P31
Peschel RE, 1999, RADIATION ONCOLOGY INVESTIGATIONS, V7, P278
Chidel MA, 1999, RADIATION ONCOLOGY INVESTIGATIONS, V7, P313
Prete J J, 1998, Radiation oncology investigations, V6, P90
Kondziolka Douglas, 2011, STEREOTACTIC AND FUNCTIONAL NEUROSURGERY, V89, P56
Kanaan Ziad, 2011, ANNALS OF SURGERY, V253, P619
Shao Hongfang, 2013, ONCOLOGY REPORTS, V29, P1441
Stringer Michael J., 2010, JOURNAL OF THE AMERICAN SOCIETY FOR INFORMATION SCIENCE AND TECHNOLOGY, V61, P1377
Gieger M, 1997, Radiation oncology investigations, V5, P72
Nieder Carsten, 2013, SpringerPlus, V2, P261
Kulkarni Abhaya V., 2009, JAMA-JOURNAL OF THE AMERICAN MEDICAL ASSOCIATION, V302, P1092
Holliday Emma B., 2014, INTERNATIONAL JOURNAL OF RADIATION ONCOLOGY BIOLOGY PHYSICS, V88, P18
Johnson C R, 1998, Radiation oncology investigations, V6, P52
Desai J, 1998, Radiation oncology investigations, V6, P135
Roach M 3rd, 1997, Radiation oncology investigations, V5, P187
Sheridan MT, 1997, Radiat Oncol Invest, V5, P186
Of the 44 cited references listed, one is in a bibliometrics/scientometrics journal. Example 2:
Highest Impact Articles in Microsurgery: A Citation Analysis
Kim, K; Ibrahim, AMS; Koolen, PGL; Markarian, MK; Lee, BT; Lin, SJ
[Kim, Kuylhee; Ibrahim, Ahmed M. S.; Koolen, Pieter G. L.; Markarian, Mark K.; Lee, Bernard T.; Lin, Samuel J.] Harvard Univ, Div Plast Surg, Beth Israel Deaconess Med Ctr, Sch Med, Boston, MA 02115 USA.
JOURNAL OF RECONSTRUCTIVE MICROSURGERY, 31 (7):527-540; 10.1055/s-0035-1546292 SEP 2015
Background Microsurgery has developed significantly since the inception of the first surgical microscope. There have been few attempts to describe “classic” microsurgery articles. In this study citation analysis was done to identify the most highly cited clinical and basic science articles published in five peer-reviewed plastic surgery journals. Methods Thomson/Reuters web of knowledge was used to identify the most highly cited microsurgery articles from five journals: Plastic and Reconstructive Surgery, Annals of Plastic Surgery, Journal of Plastic, Reconstructive & Aesthetic Surgery, Journal of Reconstructive Microsurgery, and Microsurgery. Articles were identified and sorted based on the number of citations and citations per year. Results The 50 most cited clinical and basic science articles were identified. For clinical articles, number of total citations ranged from 120 to 691 (mean, 212.38) and citations per year ranged from 30.92 to 3.05 (mean, 9.33). The most common defect site was the head and neck (n = 15, 30%), and flaps were perforator and muscle/musculocutaneous flaps (n = 10 each, 20%, respectively). For basic science articles, number of citations ranged from 71 to 332 (mean, 130.82) and citations per year ranged from 2.20 to 11.07 (mean, 5.27). There were 27 animal, 21 cadaveric, and 2 combined studies. Conclusions The most highly cited microsurgery articles are a direct reflection of the educational and clinical trends. Awareness of the most frequently cited articles may serve as a basis for core knowledge in the education of plastic surgery trainees. Level of Evidence III.
GARFIELD E, 1987, JAMA-JOURNAL OF THE AMERICAN MEDICAL ASSOCIATION, V257, P52
Hirsch JE, 2005, PROCEEDINGS OF THE NATIONAL ACADEMY OF SCIENCES OF THE UNITED STATES OF AMERICA, V102, P16569
Rickard Rory F., 2014, ANNALS OF PLASTIC SURGERY, V73, P465
Egghe Leo, 2006, SCIENTOMETRICS, V69, P131
DUBIN D, 1993, ARCHIVES OF DERMATOLOGY, V129, P1121
Baltussen A, 2004, INTENSIVE CARE MEDICINE, V30, P902
Ibrahim George M., 2012, EPILEPSIA, V53, P765
Volgas DA, 2006, Orthop Trauma Dir, V05, P29
Fersht Alan, 2009, PROCEEDINGS OF THE NATIONAL ACADEMY OF SCIENCES OF THE UNITED STATES OF AMERICA, V106, P6883
Masic Izet, 2013, JOURNAL OF RESEARCH IN MEDICAL SCIENCES, V18, P516
Paladugu R, 2002, WORLD JOURNAL OF SURGERY, V26, P1099
Loonen MP, 2008, Plast Reconstr Surg, V121, P320e
KOSHIMA I, 1989, BRITISH JOURNAL OF PLASTIC SURGERY, V42, P645
GODINA M, 1986, PLASTIC AND RECONSTRUCTIVE SURGERY, V78, P285
Hallock Geoffrey G., 2012, PLASTIC AND RECONSTRUCTIVE SURGERY, V130, P769E
ALLEN RJ, 1994, ANNALS OF PLASTIC SURGERY, V32, P32
TAYLOR GI, 1975, PLASTIC AND RECONSTRUCTIVE SURGERY, V56, P243
Garfield E, 2006, JAMA-JOURNAL OF THE AMERICAN MEDICAL ASSOCIATION, V295, P90
Zhang Wen-Jun, 2013, ANNALS OF PLASTIC SURGERY, V71, P103
Wei FC, 2002, PLASTIC AND RECONSTRUCTIVE SURGERY, V109, P2219
Shashikiran N D, 2013, Journal of the Indian Society of Pedodontics and Preventive Dentistry, V31, P133
Fenton JE, 2002, JOURNAL OF LARYNGOLOGY AND OTOLOGY9th Meeting of the British-Society-of-History-of-ENT, SEP, 2001, BIRMINGHAM, ENGLAND, V116, P494
TAYLOR GI, 1975, PLASTIC AND RECONSTRUCTIVE SURGERY, V55, P533
ZAMBONI WA, 1993, PLASTIC AND RECONSTRUCTIVE SURGERY, V91, P1110
ROELANTS G, 1978, BULLETIN OF THE MEDICAL LIBRARY ASSOCIATION, V66, P363
NYLEN C O, 1954, Acta oto-laryngologica. Supplementum, V116, P226
DANIEL RK, 1973, PLASTIC AND RECONSTRUCTIVE SURGERY, V52, P111
Thomson Reuters Web of Science, Institute for Scientific Information (ISI) Journal Citation Reports,
MOON HK, 1988, PLASTIC AND RECONSTRUCTIVE SURGERY, V82, P815
HIDALGO DA, 1989, PLASTIC AND RECONSTRUCTIVE SURGERY, V84, P71
Wang Dashun, 2013, SCIENCE, V342, P127
Celayir S., 2008, EUROPEAN JOURNAL OF PEDIATRIC SURGERY, V18, P160
Wei FC, 2002, PLASTIC AND RECONSTRUCTIVE SURGERY, V109, P2227
WEI FC, 1986, PLASTIC AND RECONSTRUCTIVE SURGERY, V78, P191
BOYD JB, 1984, PLASTIC AND RECONSTRUCTIVE SURGERY, V73, P1
Nam Jason J., 2014, JOURNAL OF BURN CARE & RESEARCH, V35, P176
Does Interdisciplinary Research Lead to Higher Citation Impact? The Different Effect of Proximal and Distal Interdisciplinarity
Yegros-Yegros, A; Rafols, I; D’Este, P
[Yegros-Yegros, Alfredo] Leiden Univ, Ctr Sci & Technol Studies CWTS, Leiden, Netherlands.
[Rafols, Ismael; D’Este, Pablo] Univ Politecn Valencia, Ingenio CSIC UPV, E-46071 Valencia, Spain.
[Rafols, Ismael] Univ Sussex, SPRU Sci & Technol Policy Res, Brighton, E Sussex, England.
[Rafols, Ismael] OST HCERES, Paris, France.
PLOS ONE, 10 (8):10.1371/journal.pone.0135095 AUG 12 2015
This article analyses the effect of degree of interdisciplinarity on the citation impact of individual publications for four different scientific fields. We operationalise interdisciplinarity as disciplinary diversity in the references of a publication, and rather than treating interdisciplinarity as a monodimensional property, we investigate the separate effect of different aspects of diversity on citation impact: i.e. variety, balance and disparity. We use a Tobit regression model to examine the effect of these properties of interdisciplinarity on citation impact, controlling for a range of variables associated with the characteristics of publications. We find that variety has a positive effect on impact, whereas balance and disparity have a negative effect. Our results further qualify the separate effect of these three aspects of diversity by pointing out that all three dimensions of interdisciplinarity display a curvilinear (inverted U-shape) relationship with citation impact. These findings can be interpreted in two different ways. On the one hand, they are consistent with the view that, while combining multiple fields has a positive effect in knowledge creation, successful research is better achieved through research efforts that draw on a relatively proximal range of fields, as distal interdisciplinary research might be too risky and more likely to fail. On the other hand, these results may be interpreted as suggesting that scientific audiences are reluctant to cite heterodox papers that mix highly disparate bodies of knowledge-thus giving less credit to publications that are too groundbreaking or challenging.
Sarewitz D, 2004, ENVIRONMENTAL SCIENCE & POLICY Symposium on the Politicization of Science – Learning from the Lomborg Affair, FEB, 2002, Boston, MA, V7, P385
Page SE, 2007, DIFFERENCE: HOW THE POWER OF DIVERSITY CREATES BETTER GROUPS, FIRMS, SCHOOLS, AND SOCIETIES, P1
Gunn J, 1999, The development of the social sciences in the United States and Canada: the role of Philantrophy, P97
PETERS HPF, 1994, JOURNAL OF THE AMERICAN SOCIETY FOR INFORMATION SCIENCE, V45, P39
Ioannidis John P. A., 2014, NATURE, V514, P561
Levitt Jonathan M., 2008, JOURNAL OF THE AMERICAN SOCIETY FOR INFORMATION SCIENCE AND TECHNOLOGY, V59, P1973
Fleming L, 2001, MANAGEMENT SCIENCE, V47, P117
Porter Alan L., 2006, RESEARCH EVALUATION, V15, P187
Huutoniemi Katri, 2010, RESEARCH POLICY, V39, P79
Lariviere Vincent, 2010, JOURNAL OF THE AMERICAN SOCIETY FOR INFORMATION SCIENCE AND TECHNOLOGY, V61, P126
Abbot A, 2001, Chaos of disciplines,
Zhang L, 2015, Journal of the Association for Information Science and Technology,
Lowe P, 2006, JOURNAL OF AGRICULTURAL ECONOMICS, V57, P165
Boyack KW, 2014, STI 2014 Leiden Conference, P64
Katz S., 1997, Scientometrics, V40, P541
Kiesler S, 2005, Social Studies of Science, V35, P733
Corsi Marcella, 2010, AMERICAN JOURNAL OF ECONOMICS AND SOCIOLOGY, V69, P1495
Morillo F, 2003, JOURNAL OF THE AMERICAN SOCIETY FOR INFORMATION SCIENCE AND TECHNOLOGY, V54, P1237
Bauman Z., 2005, Liquid life,
Gibbons M, 1999, NATURE, V402, PC81
Sarewitz Daniel, 2007, ENVIRONMENTAL SCIENCE & POLICY, V10, P5
Adams J, 2007, Report to the Higher Education Funding Council for England,
Nightingale P., 2007, Science and Public Policy, V34, P543
Rhoten D., 2009, Thesis Eleven, V96, P83
Rinia EJ, 2001, RESEARCH POLICY, V30, P357
Wagner Caroline S., 2011, JOURNAL OF INFORMETRICS, V5, P14
Mallard Gregoire, 2009, SCIENCE TECHNOLOGY & HUMAN VALUES97th Annual Meeting of the American-Sociological-Association, AUG 15-19, 2002, CHICAGO, IL, V34, P573
Lariviere Vincent, 2015, PLOS ONE, V10,
Wang Jian, 2015, PLOS ONE, V10,
Jacobs Jerry A., 2009, ANNUAL REVIEW OF SOCIOLOGY, V35, P43
Stirling Andy, 2007, JOURNAL OF THE ROYAL SOCIETY INTERFACE, V4, P707
Nooteboom Bart, 2007, RESEARCH POLICY, V36, P1016
ERC. ERC Grant Schemes, Guide for Applicants for the Starting Grant 2011 Call,
Hessels LK, 2011, Industry & Higher Education, V25, P347
Waltman Ludo, 2013, JOURNAL OF INFORMETRICS, V7, P833
Hessels Laurens K., 2011, SCIENCE AND PUBLIC POLICY, V38, P555
Leydesdorff Loet, 2013, JOURNAL OF THE AMERICAN SOCIETY FOR INFORMATION SCIENCE AND TECHNOLOGY, V64, P2573
Turner S, 2000, Practising Interdisciplinarity, P46
Steele TW, 2000, JOURNAL OF THE AMERICAN SOCIETY FOR INFORMATION SCIENCE, V51, P476
Hollingsworth R, 2000, Practising Interdisciplinarity, P215
Llerena P, 2004, Science and Innovation Rethinking the Rationales for Funding and Governance, P69
Molas-Gallart J, 2014, Journal of Science Policy and Research Management, V29, P69
Porter Alan L., 2009, SCIENTOMETRICS, V81, P719
Frenken Koen, 2009, JOURNAL OF INFORMETRICS, V3, P222
Uzzi Brian, 2013, SCIENCE, V342, P468
Karim Salim S. Abdool, 2011, NATURE, V474, P29
van Rijnsoever Frank J., 2011, RESEARCH POLICY, V40, P463
Hoffman E, 1999, Letters of transit Reflections on exile, identity, language and loss,
Phillips Nicola, 2009, REVIEW OF INTERNATIONAL POLITICAL ECONOMY, V16, P85
Katz J., 1997, Research Policy, V16, P1
Laudel Grit, 2006, RESEARCH EVALUATION, V15, P2
Willmott Hugh, 2011, ORGANIZATION, V18, P447
van Eck Nees Jan, 2013, PLOS ONE, V8,
Rafols Ismael, 2012, RESEARCH POLICY, V41, P1262
Klein Julie T., 2008, AMERICAN JOURNAL OF PREVENTIVE MEDICINE, V35, PS116
Bruhn JG, 1995, INTEGRATIVE PHYSIOLOGICAL AND BEHAVIORAL SCIENCE, V30, P331
National Academies of Science, 2004, Facilitating Interdisciplinary Research,
Braun T, 2003, SCIENTOMETRICS, V58, P183
Carayol N, 2005, RESEARCH EVALUATION8th International Conference on Science and Technology Indicators, SEP 23-25, 2004, Leiden, NETHERLANDS, V14, P70
Waltman Ludo, 2011, SCIENTOMETRICS, V87, P467
NARIN F, 1991, SCIENTOMETRICSINTERNATIONAL CONF ON OUTPUT INDICATORS FOR EVALUATION OF THE IMPACT OF EUROPEAN COMMUNITY RESEARCH PROGRAM, JUN 14-15, 1990, PARIS, FRANCE, V21, P313
Rafols Ismael, 2007, INNOVATION-THE EUROPEAN JOURNAL OF SOCIAL SCIENCE RESEARCHConference on Converging Science and Technologies – Research Trajectories and Institutional Settings, MAY 14-15, 2007, Vienna, AUSTRIA, V20, P395
Rafols Ismael, 2010, SCIENTOMETRICS, V82, P263
Barry Andrew, 2008, ECONOMY AND SOCIETY, V37, P20
Stirling A, 1998, SPRU Electronic Working Papers, P28
Bruce A, 2004, FUTURES, V36, P457
Salter A, 2002, IPTS Report, P66
Chavarro Diego, 2014, RESEARCH EVALUATION, V23, P195
Example 3 (still not from in a specialised journal) is written by academics working in the bibliometrics/scientometrics/infometrics field (check the affiliations), and the differences are clear – a quick check of the references confirms that it engages that field. In other words, it meets one of the minimum standards for published academic work that our peer review processes are supposed to enforce. As should be obvious from the above comparison, any academic working in the bibliometrics/scientometrics/infometrics field would immediately know that Example 1 and Example 2 do not engage the field, which begs the question, how did these articles make it through the peer review process?
I am not saying that academics in other fields might not have something useful to offer on the subject of bibliometrics/scientometrics/infometrics, and indeed given how some of the ideas from the discipline permeate their professional life academics would do well to be across some of the basic concepts. But imagine if the current situation were reversed – an academic working in the field of bibliometrics downloaded some easily accessed data on cancer outcomes and wrote an article titled something like ‘Radiation, Surgery or Chemotherapy? Effectiveness of treatment for patient outcomes’. Not only that, but imagine that the article contains no references to the field of radiation oncology…and then it gets submitted, peer reviewed and published in a bibliometrics journal! It makes no sense at all.
In my experience working with proprietary citation data, it is complex, requires huge amounts of cleaning and curating, and the data that come from front-end products like Web of Science (WoS) and Scopus look nothing like custom data solutions that funding councils and groups such as CWTS, iFQ and Science-Metrix work with on a regular basis in research evaluation, policy development and research. A quick look at the Scopus Custom Data Documentation surely illustrates that we should be more thoughtful than to simply download some data from Scopus and get on with the analysis.
The problem is hopefully obvious, but the reasons are not. Why is it that when it comes to the specialised discipline of bibliometrics/scientometrics/infometrics, seemingly any academic thinks that they can do it and academic rigour does not apply?
One of the reasons for the above situation is that products like Scopus and WoS have been aggressively marketed as easy solutions to the complex problem of research management. They provide push-button answers to what are, effectively, issues of public policy and industrial relations. Push-button solutions to other policy issues such as economic inequality, aging populations or migration would no doubt likewise find a welcome market.
Partly, it is the fault of those of us working in bibliometrics/scientometrics/infometrics and research evaluation who perhaps should have foreseen these consequences and policed the use of citation data better. As Hicks et al. recount,
As scientometricians, social scientists and research administrators, we have watched with increasing alarm the pervasive misapplication of indicators to the evaluation of scientific performance.
But again, this is only part of the explanation – as Hicks et al. also point out, it is impractical to think that we can be in the room every time there is a discussion about research evaluation within a university, or every time an academic from outside of the field mentions impact factors or h-indexes.
Which brings me back to my point – yet another part of the explanation must be that academics themselves are involved in perpetuating the current misuse of metrics such as ‘impact factors’, as Example 1 and Example 2 above illustrate.
This should be part of how we think about ‘the system’. Thinking about academics at the mercy of ‘the system’ is a one way transaction in which academics are affected by ‘the system’, but in this account academics play no role in creating ‘the system’, perpetuating ‘the system’ or benefiting from ‘the system’. I accept that there are important aspects of university policy and administration that are out side the control of the average academic, like government research priorities, funding council rules, university budgets etc. I also accept that academic work is in many ways hostage to global commercial interests (big publishers, citation data providers etc.). But as in Example 1 and 2 above where academic rigour is clearly compromised – in the name of an easy answer, a quick publication, the inherent competitiveness of academics…I don’t know what – and as recently outlined in Hicks’ et al, and as in a range of other aspects of academic work, academics’ practices sustain ‘the [current] system’. To improve ‘the system’, to make it more open, engaged and democratic, we must understand these complex interactions, accepting that how academics choose to practice academic work plays an important part. Then we have to agree to hold to a higher standard; then we can begin to change what we can change and not rely on simplistic and disingenuous explanations of how ‘the system’ is broken.