tag:blogger.com,1999:blog-6763377479629539589.post6636048594694986860..comments2023-10-11T09:41:19.089+02:00Comments on Philosophical Comment: What's This Thing Called Online Automated Bibliometrics and Citation, Anyway? - Scam, Fraud or just Plain Hustling?Christian Munthehttp://www.blogger.com/profile/03373442927438898939noreply@blogger.comBlogger5125tag:blogger.com,1999:blog-6763377479629539589.post-24147617998782118482013-11-11T14:12:15.291+01:002013-11-11T14:12:15.291+01:00Thanks a bunch for this elaboration - and point ta...Thanks a bunch for this elaboration - and point taken re Google Scholar having no skilled staff to do what would take skill to do :) I do understand the lure of the automated approach and I do appreciate and support the openess ideal. However, my point about having to consider functionality in relation to the real context of researchers still hold. I do understand that the disciplinary affiliation classification of these gadgets is something completely different than what we normally perceive as such classification, especially when ranking people (you don't become a brilliant exponent of X through having zillions of brilliant and well-cited papers on Y). Alas, though this insight is (I know from personal experience) not shared by policy makers or funding agencies, although they happily use these services to decide the fates of researchers – the recent REF in the UK being just the latest in a long line of examples. This cannot be ignored when one ponders whether or not to present something as a tool for ranking scholars.Christian Munthehttps://www.blogger.com/profile/03373442927438898939noreply@blogger.comtag:blogger.com,1999:blog-6763377479629539589.post-54762937290974017612013-11-11T13:50:45.985+01:002013-11-11T13:50:45.985+01:00Hi Christian, I'm the journalist who wrote the...Hi Christian, I'm the journalist who wrote the Nature piece. <br /><br />Most of what you say about errors is entirely correct, but it is all already well explained at both the Scholarometer website FAQ, and briefly (I only had 600 words for the entire story) in the online news piece.<br /><br />The first source of error is that the Scholarometer team are crowdsourcing the tags used to classify academics in their rankings. If an academic is classified under 'bioethics' it is because somebody searched for that scholar and chose to put them in that category. Of course this introduces errors - but that is how the automated process works. The Scholarometer team allow multiple tags. The alternative - a team pronouncing on the 'correct' classification for tens of thousands of academics - would be mind-bogglingly time-consuming. Thomson Reuters does it this way - but Thomson Reuters charges a lot of money for its products and does not allow the results to be made public. Some also criticize Thomson Reuters' categorizations (it's a never-ending taxonomic game really).<br /><br />The second source of error is Google Scholar itself, which is built up both automatedly by scraping the web pages of publishers and researchers, and also because academics interact with Google Scholar to correct their personal records. <br /><br />On this point I think you made an error: I do not believe that there are any 'Google Scholar staff' poring over whether someone should be classified as a bioethicist or not! The only staff there work on the technical side. <br /><br />Obviously, this automated/crowdsourcing process also introduces errors. Notably, searches often find it hard to tell apart researchers with the same name (also an issue in Thomson Reuters' databases, and one that ORCID is being introduced, among other unique identifiers, to help solve). But Google Scholar is getting better and better. <br /><br />That is the bet that both the Scholarometer team and Google Scholar have taken: that an automated approach + refinement through online crowdsourcing (because academics are motivated to correct their own scholarly records), will provide reasonable bibliometrics. Most importantly, these bibliometrics are also public and free.<br /><br />The alternative is for universities to pay lots of money to companies that will collect and normalize bibliometrics for them. This is, to be honest, what many universities do anyway. But - as the Scholarometer team note - many academics and heads of department may not be aware of their own records, nor that citation metrics differ so radically across different fields and that normalization is important. This corrected h-index is their first attempt to highlight this. <br /><br />So, take it for what it is: an entertaining story, but one where the metrics probably have errors. <br /><br />As I say, this is all briefly packed into the Nature story (see the last few paragraphs), but there's a longer explanation on the Scholarometer FAQ.<br /><br />cheers<br /><br />Richard.<br />Richardhttps://www.blogger.com/profile/05283119426718130894noreply@blogger.comtag:blogger.com,1999:blog-6763377479629539589.post-71187707048620442022013-11-10T10:55:01.169+01:002013-11-10T10:55:01.169+01:00By the way, I have added a bit to that particular ...By the way, I have added a bit to that particular section to point readers to your comment and to make sure there is no misunderstanding!Christian Munthehttps://www.blogger.com/profile/03373442927438898939noreply@blogger.comtag:blogger.com,1999:blog-6763377479629539589.post-19017538393963609032013-11-10T10:25:14.552+01:002013-11-10T10:25:14.552+01:00Hi, Jonathan and thanks for the connect and heads ...Hi, Jonathan and thanks for the connect and heads up - as for the! I did mention that at least one of the J Kimmelman's picked out by Google Scholar is someone publishing bioethics papers, but there appears to be other JK's around as well, not distinguished by Google. I'm glad that you see the main point of my post being about these metrics and automated gadgets and the role they play in our funding world! Christian Munthehttps://www.blogger.com/profile/03373442927438898939noreply@blogger.comtag:blogger.com,1999:blog-6763377479629539589.post-5096999684786320322013-11-10T02:16:44.930+01:002013-11-10T02:16:44.930+01:00Jonathan Kimmelman here. Thanks for the plug- sort...Jonathan Kimmelman here. Thanks for the plug- sort of. <br /><br />That there are many, many scholars working in bioethics who deserve a higher ranking than me testifies to the fallibility of these metrics- at least in fields like philosophy or bioethics. <br /><br />But I can testify that Scholarometer did get one thing right: I am indeed a REAL bioethicist! Can learn more about my team's work at: translationalethics.comJonathan Kimmelmanhttps://www.blogger.com/profile/04735338453860941287noreply@blogger.com