Citation Indices: A Review

Nazeer Khan                                                    PhD

To measure the contribution of a scientist is not only an academic exercise. It helps greatly to the promotion committees, funding agencies, and national & international awarding commissions. Many citations indices have been developed to determine the contribution and impact of scientists’ researches. This review article covers 27 citation indices, and computed 23 indices which are not very difficult to compute. Values of these indices are calculated using the author’s citations. To determine the quality of research performance it is suggested to use several indices and indicators, instead of only one. However, there is dire need to develop a better index which should be easy to calculate and reduce the drawbacks of h related indices.
KEYWORDS: Citation indices, h-index, g-index, e-index, s-index, x-index.
HOW TO CITE: Khan N. Citation indices: A review. J Pak Dent Assoc 2020;29(2):87-93.
DOI: https://doi.org/10.25301/JPDA.292.87
Received: 30 December 2019, Accepted: 29 February 2020

INTRODUCTION

The scientists who are well known and have received many academic awards, such as Noble award, Academic of Science award recipients, do not require to assess their impact on scientific literature. However, other scientists and authors do demand to evaluate their contributions and impact on the scientific ecosystem, for their recruitment as faculty members, promotion, award of research grants, academic awards etc. Quantification of the scientific contributions is required for such awards and promotions. However, the process of quantification is quite
opaque, clubby and capricious.
Many information, such as number of publications in impact factor journals (Institute for Scientific Information
(ISI)), publication in journals without any impact factor, number of recent publications, number of publications in national and international journals, total number of citations etc. are being used for above mentioned awards and
promotion. Many criteria have developed for assessing the researchers’ contributions, such as: 1) Total number of
publications (Np), 2) Total number of citations (Nc,tot), 3) Citations per publications, ratio of Nc,tot to Np, 4) Total number of significant publications, number of publications with an arbitrary fixed number (y) of citations, and 5) Number of citations of few most cited publications. However, there are many disadvantages of using these criteria for assessing the contribution of scientists. Criterion number 1 concentrates only on research productivity and does not give the impact of the publications in the literature. Criterion 2 may overestimate the contribution of some highly cited papers, such as review articles. It could also be increased by some comprehensive national study, in which the particular person was just a co-author. Criterion 3 may downgrade the scientists with larger latest productivity than fewer older productivity.
Criterion 4 and 5 depend upon the arbitrary numbers. This review article briefly discusses the methods and techniques that have been introduced after 2005, to measure the research productivity and its impact on the scientific literature, except the ones which are very complicated in computations. The author has used his own publications and citations as an example to show the values of 23 citation methods except four which are very complicated in computations.
Indices
N-index: N-index defines as the total number of publications of a researcher. All the co-authors take the equal
rank. It does not count the citations, means it ignores the impact of the publications. Since it is easy to calculate, therefore it is still being used for academic promotions in Pakistan and other countries.
h-index: To overcome the problems mentioned in the introduction, Jorge E. Hirsch, a physicist from University of California, USA has developed a reasonably better criterion of a scientist’s productivity and citations, known as h-index1,
which is defined as:
A scientist has index h if h of his/her Np articles have at least h citations each and the other (Np-h) articles have
less than h citations each. It is robust (unaffected by many uncited articles, as well as some highly cited articles) and it mingled by publications and citations. Hence, this index supports those scientists who publish a continuous stream of articles with persisting and above average impact.2
A further advantage of h index that any Web of Science can be used to determine this index, because this index does not change by adding low cited articles.2
The minimum number of citations of a scientist with h-index is h2. Of course the total number of citations (Nc,tot)
of the scientist is usually a lot more than h2. If we relate Nc,tot

and h2 by the equation: Nc,tot = ah2, the value of ‘a’ will be somewhere 3 to 5.1
Therefore this index loses a lot of information. In the example mentioned at the end of the article showed that h index only covers 23% of the total citations.
Burrell3 showed that the h-index is approximately linear in career length, log of productivity (publication) rate and log mean citation rate for moderate citation rates, using Poisson distribution, in the stochastic modeling. It means than h-index can be estimated with straight line using the above mentioned predictors. Braun et al4 defined the h-type index for the journals; calculated the h-index of few journals and compared them with the ISI impact factors. Braun et al5 also recommended of using the h index as an alternative to the Impact Factor (IF), because it is robust and combines the effect of quantity and quality (number of publications and citations). Bornmann & Danial2
indicated that the convergent validity of the h index in different research fields indicate that the h index is a valid indicator for research performance at the micro level.
Hirsch indicated that a scientist might be described as “successful” when achieving an h-index of 20 after 20 years,
or “outstanding” when scoring h of 40 over the same period, and an h index of 60 after 20 years, or 90 after 30 years
characterize truly unique individuals.1
Many weaknesses of h index have been identified by different authors. Few of them are as follows: (1) h index could be affected of scientists with the same name if it has been calculated using web of science or other search engine. (2) This index does not distinguish between active and inactive researchers.
Significant works of past of an inactive scientist could give higher h-index as compared to the latest work of a present
active scientist.3 A senior and well-established scientist always has advantage as compared to newcomer.4 Comparisons of researchers of different fields on the basis of h-index are not appropriate.5
This index could be influenced by self-citations and number of co-authors.2 Many indices developed later on and tried to resolve these issues. hm- index: One of the main shortcomings of the hindex is that it does not consider all the citations and publications. In developing the distribution theory regarding h-index, Glanzel6 introduced hm-index, which was little modification of average citation rate discussed earlier and defined as:

h(2)-index: Kosmulski7 , indicated that h-index of a scientist could be mixed up with the other scientists of the
same last name and initials, until and unless the search researcher knows the specialty and interest or complete curriculum vitae of the particular scientist. To overcome this problem, he introduced the following index h(2) index7, as
citation index: h(2) index is defined as the highest natural number such that h(2) most-cited papers received at least [h(2)]2 citations each. h(2) in the example is 11. It means that there are 11 articles which have more than (11)2=121 citations.
Since h(2) is quite smaller than h-index, therefore there would be lot less chance of mixing-up of the authorship as mentioned in the above paragraph. The lower bound of the total number of citations (Nc,tot ) would be [h(2)].3
Therefore the relationship between h(2) and h indices will be: h(2) = h2/3. Kosmulski computed the h index and h(2) index of 19 Chemistry professors of a Polish university, and determined the correlations between those indices. The correlation between h and h(2) indices was quite high with value of 0.9138.
h(I)- index: One of the shortcomings of h-index is that it does not consider how many co-authors have been involved in publishing the articles. Batista et al8 introduced h(I)-index which overcomes this problem and is defined as:

Na(T) is the total number of authors (author multiple occurrences are allowed) in h core. In the given example the first article has 13 authors, second article has 11 authors, third article has 6 authors and the last article of h-core, i.e, 33rd article has 5 authors. Altogether, those 33 articles has 195 authors. Using the above formula h(I) is 5.58.
g-index: Egghe9 pointed out that it makes sense that publications with low citations should have insensitive impact
in the citation index as embraced in h-index. However, this index is also insensitive regarding high citations with the papers which have already been included in the h-core. He indicated that those high citations should somehow be counted in the index. Egghe introduced g-index, as stated below, to overcome this weakness of the h-index. According to the definition, if the articles are arranged in decreasing order of the citations, then g-index is the highest number g of articles that together received g2 or more citations. g-index is unique and always greater or equal to h-index (g > h). It can easily be seen that higher the citations in the top-ranked papers, the higher the g-index.

Ravichandra Rao10 compared the correlations between h-index and g-index of 168 authors and found a very high
correlation of 0.97.
m index: The m-index, proposed by Bornmann et al11, is the median number of citations received by articles in the h-core. m-index is smaller or equal than h index.
A-index: Jin12, a bibliometrician from China tried to solve the problem of not counting upon articles having a larger number of citation indicated by Egghe that high citations should be counted somehow. He tried to solve the problem by taking the mean of the citations of h papers of h-core. Therefore, the A-index is:

Where ci is the citations of ith paper of h- papers arrangedknnnnnn,  in descending order of citations. A-index is equal to h-index if all the cis are equal to h. Jim el al12 showed that A > g > h.
R-index: Jin et al13 defined one more index that depends upon square root instead of sum of citations of h-core. This one is labeled as R-index and defined as:

if all he cis are equal to h.
AR-index: One of the problems of h-index is the lack of sensitivity regarding the performance changes. It never decreases, even the scientist slowdown in publication or stop producing publications. To overcome this problem Jin14 introduced AR-index, which is age dependent. AR-index is defined as:

Where ci is defined earlier and ai is the age of the ith publication. If all cis are equal to h and ai is one year for all i then AR = h. AR decreases as time passes and it shows that the scientist is slowing down or not producing.
m quotient: Burrell3 has discussed different scenarios, such as publication/production rates, citations rates and scientists’ career length. He picked up different statistical distributions and matched them with some possible citations options. In this his paper he defined a citation index known as ‘m quotients’, which is the average h index per year and computed as follows:

where y is the number of years since first publication. The m quotient avoids a bias towards more senior scientists with longer careers and more publications in past, but now slowdown or stop publishing in recent past.
e-index: h-index only consider the information that can be taken from h2 (square of h) citations, and ignores thea remaining citations. To overcome this problem, Chun-Ting

Zhang15, a physicist from China introduced a new index, known as e-index which is complementary to h-index, part of ignored citations and defined as follows:

where e2 are the remaining citations (citations outside of h-core) and ci are the citations received by the ith paper. e2+h2 cover the total citations of the scientist up to the h publications.
hg-index: The h- and the g-index measure different features of a scientist’s citations. h-index closes itself into h2
citation, which is lot less than total citations and insensitive of highly cited articles. However, g-index which is more
sensitive towards highly cited articles. Together, g and h present could present a concise picture of a scientist’s achievements in terms of publications and citations.
Alonso et al16 introduced hg-index by taking the geometric mean of h and g indices and is defined as:

hw-index: This index was introduced by Egghe and Rousseau17 in 2007. This weighted h-index is defined as:

where ro is the largest row index j such that

hT-index: As mentioned earlier, publications with low citations are insensitive in the citation index as embraced
in h-index. However, this index is also insensitive regarding high citations with the papers which have already been included in the index. To include all the citations of the publications, average citations per article was introduced, to make a better index for the citations of the all the publications, Anderson et al.18 introduced hT-index, which
is defined as:

Where, hT (j) is defined as:12q

Where, N is the total number of publications, cj is the number of citations of jth publication, arrange in descendingorder. The main advantage of hT over h index is that it includes all the citations of the articles and involves the duration of publication.
Since the computation to determine h.. needs complex mathematics therefore this index has not been used in the
example.
i×cindex (maxprod index): To cover-up all the publications and citations, Kusmolaski19 introduced an index known as maxprod index. It is defined as: “the highest value among values i×ci, where i denotes the ith article and ci is the number of citations received by the ith article”. maxprod = max (i×ci ).
w-index: In calculation of h-index of an individual researcher, the author considers equal weight in the publications, irrespective of the order in the authorship list. To overcome this problem Zhang20 has introduced a weighed h-index known as w-index in 2009 which depends upon the rank of the corresponding author in the publication. The weight of the first author and the corresponding author are assigned as 1 and the remaining authors are given the rank of

where n is the total number of authors in the article and k is the order of the corresponding author. Except the first
author and the corresponding author, the w-index reduces the h-index of other authors. He also introduces the coefficient of author’s• contribution by dividing the weighted citation with total citations.
f-index: As mentioned earlier, g-index could be more than the number of cited publications. To resolve this problem of g-index Tol21 has introduced f-index using harmonic mean, which is defined as:

f-index is always lesser than the cited publications. Tol21 also introduced another index, based on geometric
mean namely; t-index.

t-index: t-index is defined as:

( is the symbol of multiplication, like is the symbol of summation)
In both f and t indices, the additional citation of a low cited paper counts more than an additional citation of a high
cited paper. However, this effect is stronger in case of the harmonic mean. The following inequality exists between h,
g, f and t as: h < f < t < g.
W-index (Qiang Wu): Qiang Wu22 introduced a citation index in 2008 which is simple and easy to calculate.
W-index is defined as the maximum number of articles a scientist has in which each receives the citation of 10w or
more and the remaining has fewer than 10(w+1) citations. Qiang indicated that in most of the cases, w-index is about one-fourth of h-index. Furthermore, if a scientist has w-index of 10 or more, he/she should be considered as outstanding researcher. Qiang showed that this index has very high correlation with A-index, followed by g-index and then h-index.
j-index: Todeschini23, an Hungarian scholar introduced a new bibliographic citation index known as j-index and
defined as:

j-index follows the same course of ideas like h-index with increments, but overcome the weaknesses of h-index
of degeneracy and insensitivity to citation distribution. Therefore, j-index is the improved version of h-index.
s-index: Silagadze24 introduced a new bibliographic index in 2009, known as the strength or s-index, using a citation entropy and is defined as:

Silagadze showed that s-index gives better results than h-index in some special cases. However, the articles which
do not get citation any more, get poor results than h-index.
i10 index: In 2011 Google has introduced a simple index known as i10 index.25 It defines as, the number of publications with at least 10 citations. It is a simple and straightforward indexing measure and easy to calculate. Google has picked up an arbitrary number 10 for showing some reasonable impact of the articles.
h• -index: Zhang26 has introduced a more comprehensive index in 2016 including all: h-squared, excess and h-tail citations. It is defined as: h = rh,

C-index: Stallings et al27 has introduced this index. The main purpose of this index was to count the contribution
effect in the article. Since the computation process needs heavy mathematical skills and understanding, therefore has not been discussed over here.
x-index: Wan28 has introduced a new bibliographic index which indicates the scientist’s impact emerged from influential articles . It is defined as the largest number of papers with influential citation number > x, where each influential citation comes from a paper for which the average ACNPP (Average Citation Number per Paper) of its authors > x. The ACNPP is the ratio of the total citations to the total number of publications of the author.
X-index is always less or equal to h-index. It emphasizes more on the citation quality than number of citations.
The calculation of x-index needs lengthy computation, therefore this index has not been included in the example.
Graphic index: Neill et al29, developed a graphic method to show the researcher’s productivity. It plots the graph of average citation with impact of the journal and time period of the publication. This method also allows the comparison of total articles as well as impact with his/her peers, published in prominent scientific journals. This method could also extrapolate the trend of early-career scientists with future expected outcomes.

DISCUSSION

To measure the contribution of a scientist for award, nomination or promotion several criterion have been
suggested in the literature. Initially decision makers were using the number of publications as the criterion for promotion and nomination. Later on the factor of effective publications were added for evaluation and hence, number of citations were also included as one of the feature for assessment.
Usually the citations of a scientist are extremely skewed. In the example of this paper, one publication has received 663 citations and first ten publications, about 8% of the total publications, have accounted more than 50% of the total citations. On the other hand, 13 publications showed zero citations. But before the introduction of Hirsch h index, the

criterion for promotion or awards was either the number of publications or citations. Hirsch developed h index which involved both the number of publications and citations. This index eagerly welcomed by the scientists working in information sciences and bibliometric. Within four years of publishing of Hirsch’s work, about 150 papers published related to this article.16 However, scientists have also showed many weaknesses of this index and presented ideas to rectify those problems. Twenty six different variants of h-index are discussed in this article.
The major drawbacks of h-index, such as (1) not considering all the citations (2) insensitiveness of articles with very high citations, (3) same weight given to each authors (4) It never decreases even researcher slowdown and stop publishing etc. To rectify these problems, investigators presented different ideas. Ghanzel (hT-index), Anderson et al (hT-index), Kusmolaski (i x c-index), Qiang (W-index) and Silagadze(s-index) addressed the problem of not considering all the citations. Egghe9 (g-index), Jin12 (A-index), Jin et al13 (R-index), Alonso et al16 (hg-index) and
Tol21 (f-index and t-index) discussed the problem of h-index regarding insensitivity of articles with very high citations. Batista et al8 (h(I)-index) and Zhang20 (w-index) focused on the problem of equal weight given to each author for h-index calculation. Jin et al13 (AR-index) and Burrell3 (m-quotients) tried to rectified the problem of ‘never decreasing phenomena’. However, Rao et al and Jin et al indicated high correlations between h, g and r indices. Bornmann et al30, published a meta-analysis of h-index with 37 other variants of h-index and they also found high correlations between hindex and other variant. Therefore, these advancements have hardly increased any added information to the h-index. In a recent review on indices related with h-index, Waltman31 has discussed the indices along with normalization for field differences and adjustment for co-authors. However, it is quite clear that, h-index underestimates a lot of the impact of a scientist. Therefore, there is dire need to develop a better
index which should be easy to calculate and reduce the drawbacks of h-index. Statisticians could be involved to develop such indices, which should have strong distribution baseline and have desirable statistical properties. To determine the quality of research performance it is suggested to use several indices and indicators, instead of only one, such as number of articles with zero citation, highly cited articles, articles with first authorship and h index. It should also be noted that publication and number of citations differs significantly across the disciplines. The scientists of field of management science, literature, information technology, humanity etc. do not publish as frequent as the scientists, related Physics, Chemistry, Health sciences etc. Consequently, the number of citations in the former fields are lot less than the later ones. To overcome this problem ‘relative, internationally field-normalized impact’ of publications, discussed by van Rann et al32, could be used as one of the additional indicator. Furthermore, as indicated earlier the hindex does not depend upon the time period of the publication. Therefore, if h-index to be used to quantify the scientists’ research output and impact, then it should be comparable within the same discipline and same publication period.

REFERENCES

  1. Hirsch JE. An index to quantify an individual’s scientific research output. Proc Natl AcadSci USA, 2005; 102:16569-572 https://doi.org/10.1073/pnas.0507655102 1a Anonymous (2005). Data Point. Science, 309(5738), 1181 https://doi.org/10.1126/science.309.5738.1181c
  2. Bornmann, L, Daniel, HD. What do we know about the h index? J Am Soc Inform Sc Tech, 2007;58,1381-385
    https://doi.org/10.1002/asi.20609
  3. Burrell QL. Hirsch’s h-index: A stochastic model. J Informetrics 2007; 1:16-25. https://doi.org/10.1016/j.joi.2006.07.001
  4. Braun T, Glänzel W, Schubert A. A Hirsch-type index for journals.Scientometrics 2006; 69:169-73 https://doi.org/10.1007/s11192-006-0147-4
  5. Braun T, Glänzel W, Schubert A. A Hirschtypeindex for journals. The Scientist 2005:19: 8
  6. Glanzel Glanzel, W. (2008) On some new bibliometric applications of statistics related to the h-index. Katholieke Universiteit Leuven. https://lirias.kuleuven.be/bitstream/123456789/198470/1/MSI_0801.pdf. https://doi.org/10.1007/s11192-007-1989-0
  7. Kosmulski M. A new Hirsch-type index saves time and works equally well as the original h-index. Inter Soc Scientometric Informetrics newsletter 2006; 2:4-6.
  8. Batista PD, Campiteli MG, Kinouchi O. Is it possible to compare researchers with different scientific interests? Scientometrics 2006: 68:179189 https://doi.org/10.1007/s11192-006-0090-4
  9. Egghe L. An improvement of the h-index: The g-index. ISSI newsletter. 2006;2:8-9.
  10. Ravichandra Rao, I.K. (2007). Distributions of Hirsch-index and G-index: An empirical study. In D. Torres-Salinas & H.F. Moed (Eds.), Proceedings of the 11th Conference of the International Society for Scientometrics and Informetrics (Vol. 2, pp. 655-658). Madrid, Spain:Spanish Rese Counl (CSIC).
  11. Bornmann L, Mutz R, Daniel HD. Are There Better Indices for Evaluation Purposes than the h Index? A Comparison of Nine Different Variants of the h Index Using Data from Biomedicine. J Am Soc Inform Sc Tec 2008; 59:830-37 https://doi.org/10.1002/asi.20806
  12. Jin B H. H-index: An evaluation indicator proposed by scientist. Science Focus (in Chinese), 2006,1:8-9,
  13. Jin BH, Liang L, Rousseau R, Egghe L. The R-and AR-indices: Complementing the h- index. Chinese science bulletin. 2007;52:855-63 https://doi.org/10.1007/s11434-007-0145-9
  14. Jin BH. The AR-index: Complementing the h-index. ISSI Newsletter, 2007;3:6. https://doi.org/10.1007/s11434-007-0145-9
  15. Zhang CT. The e-Index, Complementing the h-Index for Excess Citations. PLoS ONE9 2007 4:e5429.
    https://doi.org/10.1371/journal.pone.0005429
  16. Alonsos, Cabrerizo FJ, Herrera-Viedma E, Herrera F. hg-index: a new index to characterize the scientific output of researchers based on the h- and g-indices. Scientometrics 2010;82:391-400. https://doi.org/10.1007/s11192-009-0047-5
  17. Egghe L, Rousseau R. An h-index weighted by citation impact. Information Processing Management 2008:44;770-80 https://doi.org/10.1016/j.ipm.2007.05.003
  18. Anderson, TR, Hankin, RKS., & Killworth, P. D. Beyond the Durfee square: Enhancing the h-index to score total publication output. Scientometrics, 2008;76:57-588. https://doi.org/10.1007/s11192-007-2071-2
  19. Kosmulski, M. MAXPROD – A New Index for Assessment of the Scientific Outputof an Individual, and a Comparison with the h-index. Inter J Scientometrics, Informetrics and Bibliometrics, 2007;1: 1-5.
  20. Zhang CT. A proposal for calculating weighted citations based on author rank. EMBO reports. 2009;10:416-17. https://doi.org/10.1038/embor.2009.74
  21. Tol, RSJ. The h-index and its alternatives: An application to the 100 most prolific economists. Scientometrics, 2009;80,317-24 https://doi.org/10.1007/s11192-008-2079-7
  22. Wu Q. The w-index: A significant improvement of the h-index. arXiv preprint arXiv:0805.4650. 2008.
  23. Todeschini R. The j-index: a new bibliometric index and multivariate comparisons between other common indices. Scientometrics 2011: 87:621-639. https://doi.org/10.1007/s11192-011-0346-5
  24. Silagadze ZK. Citation entropy and research impact estimation. Acta Phys. Polon. B 2010; 41: 2325-2333.
  25. Google Scholar Blog (2011). Google Scholar citations open to all. Published 16 November 2011. Retrieved June 20, 2016, from http://googlescholar.blogspot.com/2011/11/google-scholar- citationsopen-to-all.html
  26. Zhang CT. The h’-index, effectively improving the h-index based on the citation distribution. PloS one. 2013;8:e59912 https://doi.org/10.1371/journal.pone.0059912
  27. Stallings J, Vance E, Yang J, Vannier MW, Liang J, Pang L, Dai L, Ye I, and Wang G. Determining scientific impact using a collaboration index. Proceedings of the National Academy of Sciences. 2013;110:9680-85. https://doi.org/10.1073/pnas.1220184110
  28. Wan X. x-index: A Fantastic New Indicator for Quantifying a Scientist’s Scientific Impact. arXiv preprint arXiv 2014:1405.0641.
  29. Neill US, Thompson CB, Gibson DS. Assessing research productivity: A new way of evaluating academics’ research output using easily obtained data. The Scientist 2015; 29(1).
  30. Bornmann L, Mutz R, Hug SE, Daniel HD. A multilevel metaanalysis of studies reporting correlations between the h index and 37 different h index variants. J Informetrics. 2011;5:346-59. https://doi.org/10.1016/j.joi.2011.01.006
  31. Waltman L. A review of the literature on citation impact indicators. J Informetric. 2016;10:365-91. https://doi.org/10.1016/j.joi.2016.02.007
  32. Van Raan AFJ. Comparison of the Hirsch index with standard bibliometric indicators and with peer judgment for 147 chemistry research groups. Scientometrics 2006: 67:491-502 https://doi.org/10.1556/Scient.67.2006.3.10

  1. Professor of Biostatistics, Jinnah Sindh Medical University Karachi.
    Corresponding author: “Dr. Nazeer Khan” < nazeerkhan54@gmail.com >