Problem and Aim
Within the scope of a pilot project iFQ and IWT will develop the basic principles for a monitoring instrument. This is intended to provide information to the BMBF and its specialist departments about potentially interesting international cooperation partners in any field. This accompanying project within the Competence Centre for Bibliometrics will focus on cardinal methodological questions: With the help of which methods and techniques can complex and emerging fields be bibliographically delineated? How can suitable international cooperation partners on different levels such as countries, institutions and authors be distinguished with the help of bibliometric methods? Which indicators can be used for the identification of interesting cooperation partners?
State of Research
So far for field delineation mainly relevant keywords, classification systems and thesauri have been used. Thus, the classifications offered by the Web of Science (WoS) and Scopus are inappropriate for the delineation of complex and emerging subject fields. In the scope of the project „OAN – Open Access Netzwerk“ (DFG) and „EERQI – European Educational Research Quality Indicators“ (EU), the Institute for Science Networking Oldenburg (ISN) developed a method based on natural language processing (NLP) which allows to distinguish between documents according to their subject field attribution. Potential cooperation partners have so far been mainly identified by bibliometric (based on publication and citation data) as well as by non-bibliometric indicators. A systematic comparison which indicators are applicable for the identification of potential cooperation partners does not exist.
Description of the Project
With the help of the method developed by ISN an attempt is made to identify and extract publications of a defined subject field out of a large document set (WoS/Scopus). The algorithm will be trained with abstracts of a respective amount of publications. Based on this information a fingerprint is calculated. Ideally this fingerprint can clearly be found in the whole document set and can be used for field delineation. Potential cooperation partners will be identified by identical methods on the country, institutional or author level. Approaches that are based on the analysis of cooperation and citation networks or which allow for the evaluation of the time stability of indicators will be tested and enhanced. Scale-free indicators will be developed as well.

Haiko Lietz, Institute for Research Information and Quality Assurance (iFQ),
Phone: +49 (0) 30 / 206 41 77 - 21; E-Mail: lietz@forschungsinfo.de
Jian Wang, Institute for Research Information and Quality Assurance (iFQ),
Phone:+49 (0) 30 / 206 41 77 - 37; E-Mail: wang@forschungsinfo.de

Problem and Aim
Several universities run a project in order to develop a set of indicators for internal benchmarking. Organizational service is provided by HIS. The universities collect data regarding internationality, teaching and research performance as well as gender equality. Within the context of a pilot study, the iFQ will conduct a bibliometric analysis of two subject areas based on publication and citation data in order to assess research performance. A working group consisting of iFQ, chancellors and scientists will be set up to support the study.
The accompanying KB project will address fundamental methodical questions concerning the applicability of bibliometric indicators in university benchmarking, such as the handling of distortions in the comparison of institutions, the alignment of disciplinary and organizational structures and the normalization by personnel data. Within the working group and, if applicable, by means of interviews with scientists, it will be analyzed in which respect indicators correspond to the internal views of scientists. It is the aim of the project to clarify the conditions of a national benchmarking.
State of Research
Bibliometric indicators are used in numerous university rankings. On the one hand, they tend to ignore the methodic problems and more sophisticated indicators discussed in research literature, so that bibliometrics is not used to full capacity; on the other hand, bibliometric indicators are only rarely confronted with the scientists’ inner perspectives – both topics will be accomplished by the project. A set of indicators applicable for benchmarkings will be defined.
As standards of analyzing bibliometric indicators in relation to personnel data hardly exist, the project aims to develop methodic foundations in this respect.
Description of the Project
Based on institutional addresses and personal names in Web of Science, publication data of the two subjects Chemistry and Physics from the years 2005-2010 will be collected. This publication lists set will be handed to the universities for verification. Publication data added by the universities will be matched with Web of Science. Publication indicators for the whole time period and citation data for 2005-2007 will be calculated. If applicable, this analysis will be expanded to non-source items (data records that exit only as references in Web of Science).

Problem and Aim
The goal of this project is to assess the reliability and validity of the data contained in the Web of Science database (WoS) about funding acknowledgements. The answers to these questions will shed light on the utility of such data for bibliometric analysis.
State of Research
To date, studies about the reliability and validity of the information about funding organizations contained in bibliometric databases are quite scarce. The distribution and dynamics of funding acknowledgements concerning specific funding organizations was subject to an analysis based on about 335,000 publications in the CiteSeer database. The authors have shown that the fraction of mentions of specific organizations is stable overall (Giles et al 2004: 17601). Based on the results of a survey among scientists and the analysis of the Research Output Database of the Wellcome Trust it has been ascertained that the mention of the funding organization in publications of a project is directly linked to the share of the financial support contributed by that organization. The quality of the information provided by Thomson Reuters in the Web of Science database concerning funding acknowledgements has only been examined in two case studies. However, research projects that comprehensively assess the completeness and quality of the information on funding acknowledgements in WoS have not been conducted as yet.
Project Description
In order to answer these questions a comparison between the results of a bibliometric analysis of funding acknowledgements in WoS and the final reports of funding organizations is conducted. The objects of the study will be projects funded by the BMBF and the DFG and of which resulting publications are available in the WoS database. Furthermore the results will be validated by short interviews. This will permit to draw conclusions on the motivation for naming funding organizations and on possible distortions caused by the methods applied by Thomson Reuters. Based on the results it will be assessed how reliable and complete the mentioning of funding acknowledgements in Web of Science is.

01.09.2009 - 31.12.2012

Problem and Aim
Standards of methods and error calcula as well as constitutive quality criteria similar to those that are routine in social research are hardly developed in bibliometrics. This is not only a theoretical deficit but also leads to an uncritical utilization of bibliometric indicators which are often regarded as “objective depictions of reality”. This sub project will first of all collect origins of errors or biases from studies mentioned in the research literature, will formulate a theoretically grounded error calculus and provide respective quality criteria.
State of Research
In research literature of science research the methodological problems which are emanating from the context of the generation of bibliometric indicators have been discussed for quite a long time. However, there is hardly any reliable empirical knowledge about the consequences of various methodological procedures on indicators and their interpretation or interpretability. Standards for the construction of bibliometric indicators and the conduct of bibliometric studies and analyses exist only to a limited extent. Furthermore there is a lack of a systematic and comprehensive overview of the problems and their evaluation. Research literature only partly covers and discusses technical questions.
Project Description
As a starting point for this project there is a systematic analysis of research literature. Based on this, an overview on potential errors and problems of interpretation along with their size and consequences will be elaborated and approaches for corrections and quality criteria will be systematized. The quality criteria that will be developed in several modules of the project will be implemented as algorithms into the data base. During the course of the project they will be used in order to calculate indicators in an adequate way and to assess their quality and explanatory power. Furthermore algorithms will be developed, tested and implemented with the help of which errors can be taken into consideration during the construction process of bibliometric indicators.

01.09.2009 - 31.12. 2011

01.01.2010 - 31. 01.2011