What's This Thing Called Online Automated Bibliometrics and Citation, Anyway? - Scam, Fraud or just Plain Hustling?

It's been a while since the last substantial post, blaming deadlines, deadlines, deadlines for that. Having submitted a major delivery this Friday, here's a sort of inhale before I jump right into the next leg of the fall semester triathlon – a major research bid that I'll be heading.

So, this week, the distinguished science journal Nature's online news section published an entertaining piece on what the outcome may be when all researchers, regardless of field, are ranked according to citation – how much their work is referred to by other researchers – using open online automated resources, such as Google Scholar, or its special citation section. Using a service called Scholarometer, Nature had this guy, slightly surprisingly to many, coming out on top (wonder who he is? - click the pic!):

Strange, isn't it? Not when considering that they have been using the so-called h-index, a mathematical construct devised to reflect the citation weight (rather than rate) of a scholar (that is, this is h-index as used in Google Scholar, in the more professionally advanced and commercial Web of Knowldge, it is something else, but the purpose is the same), thereby reflecting the value of having more articles with more citations rather than just many concentrated to one publication. They then perform what is referred to as normalisation for different scholars in relation to the size of their respective fields. So, what makes Marx come out on top is that he is a more well-cited historian than what, e.g., Albert Einstein is a well-cited physicist, considering that physics is a very much larger discipline than history is. Now, of course, none of this says anything about quality or influence on the progress of research (no more than what the Billboard chart says regarding music) – it merely measures popularity as an object of citation among fellow scholars. In fact, the notion that citation proves anything over and above that others have taken some sort of interest in one's work is highly contestable – said without denying the no doubt important use that citation and citation tracking has in science and research.

But here's the funny thing. Having been pointed to the Scholarometer toy, I of course couldn't resist checking out my own pet fields! So here's what came out when looking at the h-index ranking in bioethics – the field where much of my most weighty specialisation is located (click the image to view a scaled up version):

I could recognize some names, such as Simo Vehmas who happens to be a good friend, but several others were completely unfamiliar to me.  Now, bioethics broadly conceived is a large field so it need not be surprising that one doesn't know the name of completely decent fellows within it, but the fact that I could not place any of the top four names made me wonder. But then it struck me: wait a second, I do know one of those names, the top one at that, but certainly not in the role of a bioethicist, but as a world-renowned researcher in reproductive genetic medicine and leader of the team that performed the first successful preimplantation genetic diagnosis in the early 1990's. I happened to know this, since I published a book on the ethics in the aftermath of this technological advance in 1999 (available for online reading and download through that link). So Alan H Handyside is a prime medical researcher, which of course is what ups his h-index to such heights, as may be confirmed by inspecting a Google Scholar search on his name - what makes him top name in bioethics is, seeemingly, merely that someone tagged his name with that disciplinary affiliation. So what about A Pandiella? Same story it appears, this is a cell-biologist with a no doubt impressive citation count and, I'm certain, many important results up the sleeve. Moving on to R Frydman it's almost the same story, as the bulk of the publications are here in reproductive biomedicine, but it's more complicated as it appears that there is also another R Frydman, who is publishing in the field of health policy/economics, but these persons are treated as one! Next one, J Kimmelman is likely to be a similar story, since there is one with a good number of publications clearly in bioethics [retrospective note added after publication of this post: this person, Jonathan Kimmelman has added a comment below and clarified his affiliation, which is indeed in bioethics] and another publishing in very specialised biomedical science that has attracted vast numbers of citations (I checked some of the respective author affiliations in this case and they don't seem to match either). Last, before we get to my friend Simo, we have F Olivennes, who again seems to be a purely biomedical researcher in the field of reproductive medicine and embryology, who for some reason has been tagged as belonging to bioethics.

These, then are the top researchers of my field according to Scholarometer - no wonder I never heard of them in that role. And, in fact, it seems that the problem appears already at the Google Scholar source, for checking the top name of the straight citation ranking for bioethics, we meet this guy – yup, yet another biomedical researcher classified as a bioethicist. Number two is this guy, whoever he is, same story all over again, and then come some names I'm familiar with and respect in the way one would expect of people ranked to be at the top of one's field. Just to twist the knife some extra turns, I also did a quick check for medical ethics; same story, this is the top guy, apparently, and this is no. three I hear (number two in this ranking actually is a well-known bioethicist who happens to also be a medical researcher, so that kind of animal does exist).

So, what we may conclude is that for these fields, attempts at measuring citation has been severely corrupted by failures of disciplinary/field classification that swamp rankings with citation counts of no relevance for the field at all. I haven't looked through the entire publication lists of the people mentioned, but many of them appear to have basically no output belonging to ethics of any sort. They might, of course, have tagged along on a few ethics papers led by others as clinical/scientific experts (which is fine), but this does not make them highly cited bioethicists, it makes them medical researchers whose medical citation counts look impressive in the context of a field-normalisation to bioethics rather than medicine. In addition, we have seen an obvious identity problem, where the automated online citation counters are unable to distinguish people with similar surname plus initial – makes for quite a lot of error, I would say.

But what is the root of the classification errors with regard to field-normalised/specific citation measures? There are several (possibly overlapping) possibilities. One is, of course, that authors misclassify themselves, as may happen in Google Scholar Citation, where you as author decide what fields to belong to. For example, I could myself have made a strategic choice to pass myself off as belonging to the philosophy of medicine field, which would not exactly be lie albeit bending it a bit, and with my current total citation of 408 ended up in a handsome 6th place, rather than the less impressive placings I enjoy as bio- or medical ethicist or just ethicist. But not all authors are in this system, as you have to actively join it and manage it a bit for it to work (thus your responsibility for how you classify yourself), so the problem might also come from the classification done by the Google Scholar staff; I wouldn't be surprised if several of the strange things described earlier are due to Google's experts confusing "bioethics" with "biometrics" or "biotechnical", for example. The qualification of this staff for doing what they are doing is completely blacked out to me, as I suspect it is to most other scholars, and still many us – like the team behind Scholarometer – take it rather serious. Now, with regard to Scholarometer, there may certainly be error sources located there as well, since one may require of an academically construed automation tool that it is checked for serious error of the sort I have been displaying – which has apparently not occurred to or engaged the team at Indiana University Bloomington responsible for the product.

But wait a second! Wouldn't that mean to, sort of, making the automated citation counter, sort of, not automated? Yes indeed, that is what it means! And hence the title of this little peak into the fascinating games sometimes played in the world of academia to no apparent use for anyone. Alas, though, through the way in which governments and other funders of research are increasingly using bibliometrics and citation as quality indicators to determine the allocation of funds, preferably in an as automated way as possible (partly because of the hype represented by Scholarometer and the article in Nature), thus falling prey to the sort of weirdness here described, this sad example of pretending to have a technology that works when one hasn't, is actually putting fellow scholars and researchers at risk of losing funds and other resources, miss jobs and promotions, et cetera, for no good reason at all.

My plea to Nature and other journals, Scholarometer and Google Scholar is simply this: stop pretending that there's something there that is actually not in evidence. Those who provide these services: make them work as they should or shut them down. Scholarly media: ignore them until they have something to show for real and not merely for fancy.

See you soon!
 

A Bunch of Posters and Presentations on the Ethics of Person Centred Care and Shared Decision Making in Adolescent Diabetes now Online

Yea, what he said! These things are all outcomes of ongoing research in the cross-disciplinary project Organizing Person Centred Care in Pediatric Diabetes: Communication, Decision-making, Ethics and Health, where myself and some younger philosophers/ethicists collaborate with psychologists, linguists, care and organisation researchers and medical specialists. The project's original plan can be seen here. We are now a few years in and the last week has been a hectic one of delivery both at the yearly international conference of pediatric and adolescent diabetes specialists, ISPAD 2013, that was held in my town last week, and an ethics seminar at the University of Groningen that I visited this thursday.

Here's what came out:

1) An oral presentation (slides) of the main results so far of our entire project, delivered by one of the senior researchers, Marianne Törner.

2) A poster by my Ph.D. student Thomas Hartvigsson and myself, presenting results from an ongoing analysis of videotaped meetings between doctors, patients and parents on the possible value of assessing decision competence as part of a person centred consultation session. The title is Respecting the Adolescent Diabetes Patient as an Autonomous Person - What Does it Imply? Assessing and Managing Decision Capacity for Care Decisions and Self-care

3) Another poster, this time by my post doc in the project, Anders Herlitz, and myself likewise presenting results from an ongoing analysis of videotaped meetings between doctors, patients and parents, this time in the form of a general claim with regard to how person centred care and shared decision making needs to be adapted to fit patients with weak or vulnerable capacities for responsibility and conditions requiring care to be mostly self-managed. The title is The Moral Psychology of Person Centred Adolescent Diabetes Care: two potentially conflicting ethical dimensions of shared decision making for sustainable self-care

4) An oral presentation (slides) that I gave in Groningen more or less at the same time as Anders presented the one above that develops our argument that standard models of shared decision making and person centredness of care lacks a crucial virtue ethical elelment that needs to be added and that this complicates the ethical assessment of the organisation of such care. The title is the same as above.

Currently me and Anders and Thomas and the rest of the team are working hard on, first, two articles, that we hope to submit shortly, and then another batch of two following up more complications of person centred acre and shared decision making that we have discovered through our video-studies. Stay tuned!

Why Southern Sweden's Police Should Immediately Create a Registry of Police Officers and their Families

So here's the not so very tasty news from my country today, broken by our largest daily, Dagens Nyheter: The police force of southern Sweden has been running a registry, based on biological/genetic/familial/genealogical principles worthy of any classic nazi or racial biologist, of roma or "traveller" people, comprising of over 4000 persons, among which are 1000 plus children – and also some long deceased ancestors. Read English reports here, here and here (some other Swedish reports are here, here, here, here). One of the family-tree structures look like this (click it for more examples):

The police of south Sweden has made a number of increasingly pathetic attempts to dodge this obviously undodgeable shitstorm, ranging from first denying the existence of any registry, then admitting that but claiming that it is not organised by any "ethnic" principles (the title of the registry is "traveler abouts" – duh!), then possibly sensing this is not going to go away, starting to shift the blame to the classic sort of patsy: the lone, slightly crazed police officer acting "on his own" and excessing a little bit in his/her otherwise admirarable sense of duty. This nice composition of headlines from today sums up how totally feeble and cowardly this whole thing is being handled by those responsible locally:

 Right. And at the same time the registry, for legally it is one, is reported to have been in wide use, not only in the south, but also in my own quarters of Western Sweden, for instance. Ok, so maybe not a lone gunman after all....

By now the national chief of police, Bengt Svensson, reacted strongly and critically, to say the least, and ordered all regional chiefs to probe if anything similar is going on anywhere else (you hear those shredders working all through the nights, and all those hard-drives being dismounted and crushed, folks?) and, in consequence, the police has reported itself to.... the police and the relevant prosecutor has already said that an investigation will be opened regarding possible crimes against the law on registration of personal information (PUL, which is pretty strict), and possibly other laws as well. A bit late, but still, our minister of Justice, Beatrice Ask – otherwise famous for condoning police racial profiling, or at least pooh-poohing reports about it – also has made a statement. Should that fail, the Council of Europe's commissioner for human rights, Nils Muznieks, has already stated his view that the whole thing is in violation of the European Convention on Human Rights (since Sweden is an EU member state, also a part of my country's constitution), so that would take the case to a probable loss in the European Court of said human rights, which would then force a legal change, if necessary. In parallel, also today, UN's committee against racial discrimination is once again mounting criticism against Sweden, among other things for its lack of action against discrimination and stigmatisation of people of roma origin.

This leads into the funny bit, which motivates this post:

It is namely the last line of defense (offered by deputy chief, Petra Stenkula, and picked up by this shady excuse for a legal commentator, to mention one of a bunch – nice company you keep, deaputy chief!), apparently, of the south Sweden police, that the registry (which is not a registry, but a "list") of roma or travelers (which is not that but a list of "traveler abouts") is (at least initially), in fact a completely legitimate part of a police investigation of some crime connected to a family feud some years back – let alone, it might have gone a bit astray over the years. Right, that's why they need those long deceased people born in the 1800's and the two year olds in there - stupid me, I should have known!

But wait a bit, let's not be too harsh!! Let's in fact accept this explanation - it is actually quite apt by the police to set up registries of this sort, if only it pertains to an ongoing criminal investigation. Let's now chew really well on exactly that:

As we saw, there's a new criminal investigation that has just been opened! Apparently it's about police officers in south Sweden creating and using possibly illegal and unlawfully racist registries of people. So, let's accept the just given explanation and the principle that it implies and act really thoroughly on that: Let's have that registry of all those police officers in south Sweden who are potentially implicated for setting up or using (the Swedish law relevant here may actually ban mere use) the registry – quite possibly that'll be all of them. And be very sure to have deputy chief Stenkula's name as the very first entry, as the feeble attempt at whitewashing that she has offered obviously makes her a prime suspect. And while you're at it, let's put all their family members in there too - including small children of about two and up and a few long dead ones as well, just for good measure. 

You just act on your own fine principles and do just that! Savor how good it feels! Do it tomorrow, do it now, do it yesterday and enjoy!